Chapter: Table of Contents Page 1 | 66 Program-Level Mid-Term Evaluation „Powering Agriculture – An Energy Grand Challenge for Development (PAEGC)” Evaluation Report August 2016 Chapter: Table of Contents Page 2 | 66 Program-Level Mid-Term Evaluation „Powering Agriculture – An Energy Grand Challenge for Development (PAEGC)” Evaluation Report August 2016 Syspons GmbH Prinzenstraße 84, Aufgang 1 10969 Berlin Authors: Lennart Raetzell, Motje Seidler © Syspons 2016. All rights reserved. Chapter: Table of Contents Page 3 | 66 Table of Contents Table of Contents ....................................................................................... 3 List of Abbreviations .................................................................................. 4 1 Management Summary ......................................................................... 5 2 Introduction ......................................................................................... 8 3 Methodological Approach ..................................................................... 9 3.1 Methodological Concept .......................................................................................... 9 3.2 Methodological Challenges ...................................................................................11 3.3 Research Design ...................................................................................................11 4 The Powering Agriculture Initiative at a Glance ................................. 13 5 Evaluation Findings ............................................................................ 15 5.1 Relevance of PAEGC ..............................................................................................15 5.2 Effectiveness of PAEGC’s Structures and Processes..............................................21 5.3 Effectiveness of the Four Components ..................................................................27 5.4 Efficiency of PAEGC ..............................................................................................31 5.5 Impact of PAEGC ..................................................................................................32 5.6 Sustainability .......................................................................................................33 6 Conclusions and Recommendations ................................................... 36 6.1 Conclusions ..........................................................................................................36 6.2 Recommendations ................................................................................................37 Annex ....................................................................................................... 42 Table of Figures Figure 1 – Phases of the Mid-Term Evaluation ..............................................................................................11 Figure 2 – Four Technical Components of PAEGC ..........................................................................................14 Figure 3 – Applications in 1st and 2nd Call by Organization ..............................................................................17 Figure 4 – Applications in 1st and 2nd Call by Developing vs. Developed Countries .............................................17 Figure 5 – Applicants of the 2nd Call that have Received Previous Funding from Partners ...................................18 Figure 6 – Developing vs. Developed World Awardees from 2nd Call ................................................................18 Figure 7 – Regional Distribution of Awardees from 2nd Call ............................................................................ 19 Figure 8 – Awardees from 2nd Call by Organization ........................................................................................19 Figure 9 – PASTO Services Utilised by Innovators .........................................................................................25 Figure 10 – Rating of AOR Services .............................................................................................................25 Figure 11 – 2013 Innovators and Their Innovation Stages as of Spring 2016 ...................................................28 Figure 12 – Overview of Achieved Impacts of the 2013 Innovators ................................................................. 28 Figure 13 – Options for PAEGC's new strategic set-up .……………………………………………………………………………………………… 41 Annex I. Bibliography ....................................................................................................................................42 II. Overview of all Conducted Interviews ................................................................................................. 45 III. List of Analysed Challenge Funds ........................................................................................................47 IV. Terms of Reference ..........................................................................................................................50 V. Assessment Grid ..............................................................................................................................56 VI. PASTO Mid-Term Innovators Assessment August 2016…………………………………………………………………………………….65 NOTE: This report was funded by “GIZ Sustainable Energy for Food – Powering Agriculture” on behalf of the PAEGC Partners. The findings and conclusions are solely attributed to Sys￾pons and not necessarily endorsed or adopted by the PAEGC Founding Partners. Chapter: List of Abbreviations Page 4 | 66 List of Abbreviations AECF AOR BMZ CES DFID EnDev FAO GCD GIZ IADB M&E MOOC OPIC PAEGC PASTO PAX PPP SDG SIDA USAID African Enterprise Challenge Fund Agreement Officer’s Representative German Federal Ministry for Economic Cooperation and Development Clean Energy Solutions Department for International Development of the United Kingdom Energising Development Food and Agriculture Organization of the United Nations Grand Challenges for Development Deutsche Gesellschaft für Internationale Zusammenarbeit Inter-American Development Bank Monitoring & Evaluation Massive Open Online Course Overseas Private Investment Corporation Powering Agriculture: An Energy Grand Challenge for Development Powering Agriculture Support Task Order Powering Agriculture Xcelerator Public-Private Partnership Sustainable Development Goal Swedish International Development Cooperation Agency United States Agency for International Development WebMo Powering Agriculture Online Monitoring Platform Chapter: Management Summary Page 5 | 66 1 Management Summary 1. Background Global challenge funds offer a new approach to international cooperation as they support high-risk, high-potential solutions to global development challenges. This means that challenge funds do not only take a high financial risk, but also adopt a highly innovative bottom-up approach, as they facilitate ideas that could potentially transform the lives of people affected by poverty worldwide. This is especially relevant in the context of rising food insecurity and the adverse consequences of climate change, which are inextricably linked. As agriculture is still mostly based on fossil fuels, prices of agricultural products exhibit a close relationship with energy prices, which leads to increasing price volatility on the global food market. Together with the projected population growth in the coming years, this trend will likely contribute to greater food insecurity for people living in developing countries. At the same time, an intensification of agricultural production will lead to an increase in greenhouse gas emissions and thus contribute significantly to global warming and climate change worldwide. Hence, while agricultural pro￾duction needs to be intensified to feed a larger amount of people, it needs to be done in a more energy￾efficient way, decoupling it from the use of fossil fuels. In light of these developments, USAID, SIDA and BMZ initiated the “Powering Agriculture: An Energy Grand Challenge for Development Initiative (PAEGC)” to address this developmental challenge and the need for an integrated approach on the international level. The BMZ contribution to PAEGC is imple￾mented by GIZ. The main objective of PAEGC is to support new and sustainable approaches to acceler￾ating the development and deployment of clean energy solutions1 for increasing agricultural productivity and/or value in developing countries. As such, the initiative targets farmers and agribusinesses, as well as consumers threatened by poverty in developing countries. PAEGC follows the energy-agriculture nexus approach, acknowledging the complex interactions between these sectors. The initiative has been recently extended to run through 2019 and involves four components: Technology and Business Model Innovation, Commercial Financing, Mainstreaming and Acceleration, as well as Knowledge Management. The latter two components were added in June 2014. 2. Purpose of the Mid-Term Evaluation At the time of this program-level mid-term evaluation the Powering Agriculture Initiative had already existed for four years and implemented two global innovation calls. As a consequence, PAEGC commis￾sioned an evaluation team from Syspons GmbH to conduct a mid-term evaluation of the initiative’s structures, processes, and impacts on program-level. The objective was to measure the aggregated contributions of PAEGC‘s interventions in achieving the program‘s goal. The results shall enable the partners to take informed decisions to adjust the structure, strategy, and management of the fund for the future, as well as to facilitate mutual learning among the partners. The mid-term evaluation was undertaken from April 2016 until June 2016. Within the given time frame the evaluation team conducted an in-depth analysis of relevant documents and data, in-depth interviews as well as an analysis of the results from a previous impact evaluation on project level. Based on the mid-term evaluation’s findings, the evaluation team developed recommendations for the future strategy, structure, and implementation processes of the initiative. An explanation on methodological limitations of the study can be found in section 3.2 of the report. 3. Key Findings and Conclusions PAEGC’s strength is that it is a unique challenge fund that addresses highly relevant global develop￾mental challenges such as food insecurity and climate change with an integrated approach. None of the other 50 challenge funds analysed have adopted a nexus approach integrating agriculture and energy, 1 Clean energy is defined as: Usable energy (i.e. electricity, illumination, heating/refrigeration, mechanization) that is derived from renewable sources and supports a reduction in fossil fuel use, increase in efficiency, and/or limitation of greenhouse gas emissions. Clean energy sources include – solar, hydro, wind, geothermal, sustainably harvested biomass, and biogas. The term “clean energy solution” is defined as: A combination of appropriate technology and a business model that addresses the clean energy demands of a select market. Chapter: Management Summary Page 6 | 66 while working on a global scale2 . At the same time, the initiative’s set-up allows it to reach new and innovative actors from the private sectors in developing countries and emerging markets, thereby bring￾ing new perspectives into the field of international cooperation. Another strength is the initiative’s ef￾fective and efficient cooperation and steering structure, which is characterised by a mode of co-produc￾tion and the usage of comparative advantages of the partners’ competences. The partnership thus allows the optimal use of each other's assets and resources in order to achieve its results in an effective and efficient way. However, the Powering Agriculture Initiative also exhibits some weaknesses. To date, PAEGC has not reached its objective of establishing a financing mechanism for scaling-up. In addition, it has so far been unable to initiate a public-private partnership facility, which limits the opportunities to leverage further private-sector investment and accelerate the funded innovations. Another weakness is the lack of a clear concept for measuring the developmental impact during PAEGC’s implementation. This is in con￾trast to the partners’ high expectations in this regard, as they need developmental impact data on PAEGC’s performance to justify the investment of public funds. In addition, the funded innovations are currently not thoroughly linked to the bilateral structures and programmes of the partners3 . Even though PAEGC shares information and data with the partners’ bilateral programs, there is mostly no systematic integration of the funded projects into the bilateral structures. One the one hand, this limits the oppor￾tunities to further mainstream PAEGC’s approach into bilateral programs, and on the other hand pre￾sents a missed opportunity to further support to the awardees and the scaling-up of the funded inno￾vations through the creation of synergies with these bilateral programs. While systematic support by the bilateral structures was not included in the in the original program planning, the results of the evaluation show that this would be a beneficial addition to the program. As a result of these weaknesses the Powering Agriculture Initiative currently faces some threats to its successful implementation. The lack of an effective concept for collecting data on the developmental impact might undermine PAEGC’s credibility vis-á-vis the political decision makers in the donors’ home countries and limit the potential for further international support. Furthermore, while the initiative pos￾sesses an overall coherent strategy that is shared by all partners, there exists a lack of clarity concerning the strategic approach to mainstreaming4 of the nexus approach, as not all partners view this as a priority. This might limit PAEGC’s chances to advance the energy-agriculture nexus on the international agenda. Despite these weaknesses and threats, it can however be concluded that PAEGC has the opportunity to strengthen its position as a unique challenge fund in the energy-agriculture nexus. 4. Recommendations Based on the findings of this mid-term evaluation the following eight recommendations are put forward. They are divided into three categories: recommendations on the general strategic level, recommenda￾tions on the strategic set-up of PAEGC, and recommendations on the operational level. General strategic level: 1. Recommendation: The Powering Agriculture Initiative should be continued as it is a unique challenge fund that addresses valid developmental challenges in the energy-agriculture nexus. 2. Recommendation: The partners should try to win new partners from the private sector to increase the leverage ratio of PAEGC. For this purpose they should clarify their expectations regarding the private sector’s role and the added value the private sector partner can gain from its participation in PAEGC. 2 The “energy-agriculture nexus approach” is defined here as a “conceptual approach to better understand and systematically analyse the interactions between the natural environment and human activities, and to work towards a more coordinated management and use of natural resources across sectors and scales.” (FAO, 2014, p.3) 3 It needs to be noted here that the linkage between innovators and bilateral structures of the partners is rather seen as an additional service than a promised support by the partners. 4 ‘Mainstreaming’ here refers to the incorporation of ideas and approaches into organisations, institutions and networks. In this context we predominantly refer to the mainstreaming of the energy-agriculture nexus. Chapter: Management Summary Page 7 | 66 3. Recommendation: The partners should develop a system to measure developmental impact by using “the last hard number” concept and independent reviews for high impact projects. Strategic set-up of PAEGC: 4. Recommendation: The partners should develop a new strategic set-up to further increase PAEGC’s effectiveness. Operational Level: 5. Recommendation: In future calls the partners should put a stronger emphasis on the analy￾sis of local markets and demands as a pre-requisite for applicants. Moreover, the assessment of the quality of these analyses as well as the conceptual feasibility of the proposed business model should be a central selection criteria in possible future calls (especially for scaling-up windows). The selection process should be structured in such a way that innovators can al￾ready receive business models and market analysis support during the selection phase. 6. Recommendation: In order to further increase clarity the partners should clearly communi￾cate the roles of PASTO and AORs towards the innovators. 7. Recommendation: The partners should extend the support offered by PAX to the innovators for 3 more years, specifically to high impact innovators, to increase the likelihood of their fu￾ture success and PAEGC’s effectiveness. 8. Recommendation: The partners should move forward to integrate gender not only as “gen￾der equity” but as “gender as a social construct” in PAEGC’s selection and M&E processes5 . The newly hired gender consultant at PASTO can be used for this purpose. 5 This could be done for example by conducting gender sensitivity analyses, which not only look at gender equity but also on how the projects might affect gender roles and norms in society. For “gender as a social construct” see for example Marecek, J., Crawford, M., & Popp, D. (2004). On the Construction of Gender, Sex, and Sexualities". In A.H. Eagly, A.E. Beall, & R.J. Sternberg. Chapter: Introduction Page 8 | 66 2 Introduction The Deutsche Gesellschaft für Internationale Zusammenarbeit GmbH (GIZ) commissioned Syspons GmbH on behalf of all PAEGC to conduct a program-level mid-term evaluation of the “Powering Agricul￾ture: An Energy Grand Challenge for Development Initiative (PAEGC)”. The initiative is a multilateral effort of the United States Agency for International Development (USAID), the Government of Sweden, the Government of Germany, Duke Energy Corporation, and the Overseas Private Investment Corpora￾tion (OPIC). The objective of the mid-term evaluation is to measure the aggregated contributions of PAEGC‘s interventions in achieving the program‘s goal. Hereby the evaluation seeks – on the basis of recommendations – to enable the partners to take informed decisions for improving the structure and strategy of the fund and fund management for the future. Furthermore, the evaluation aims to facilitate mutual learning among the partners. The main user of this evaluation is the management team of the initiative’s partners, consisting of BMZ/GIZ, USAID, and SIDA. The results will be shared with the private partners OPIC and Duke Energy as well. On request the mid-term evaluation will further be shared by the partners with other relevant stakeholders The mid-term evaluation was undertaken from April 2016 until June 2016. Within the given timeframe the evaluation team conducted an in-depth analysis of all relevant documents and data, in-depth inter￾views as well as an analysis of the results from a previous impact evaluation on project level. Based on the mid-term evaluation’s findings, Syspons developed recommendations for the future strategy, struc￾ture, and implementation processes of the initiative. A detailed description of the applied methodology can be found in chapter 3. The following report presents the results of this mid-term evaluation. It is structured as follows:  Chapter 3 explains the methodological approach chosen for this mid-term evaluation.  Chapter 4 provides an overview of PAEGC.  Chapter 5 presents the results of this mid-term evaluation. It analyses the initiative’s relevance, the effectiveness of its structures and processes as well as its four components. Furthermore, it assesses the initiative’s efficiency, impact, sustainability and the integration of gender aspects in its processes and structures.  Chapter 6 draws conclusions and gives recommendations for the initiative’s future strategy.  The annex includes the bibliography, an overview of the conducted interviews, a list of analysed challenge funds, the Terms of Reference, and the evaluation’s assessment grid. Chapter: Methodological Approach Page 9 | 66 3 Methodological Approach The following chapter details the methodological approach of this program-level mid-term evaluation. It illustrates the methodological steps that were taken and research instruments that were employed. Thereby, it clarifies what sources the conclusions are based on and how the conclusions were reached. 3.1 Methodological Concept Global challenge funds offer a new approach to development cooperation as they support high-risk, high-potential solutions to global development challenges. This means that challenge funds do not only take a high financial risk, but also adopt a highly innovative bottom-up approach, as they facilitate ideas that could potentially transform the lives of people affected by poverty worldwide. Given these particular characteristics, donor organizations of challenge funds have a vital interest in assessing the potential effectiveness, relevance and sustainability of their interventions while implementing their activities and funded projects. Evidence of the initiative’s potential effectiveness and relevance as well as its efficient implementation is especially critical for ensuring that the initiative is on track and does not have unde￾sired consequences. Furthermore, this kind of evidence can inform the partners of the potential of achieving the envisioned developmental impact, on which basis decisions regarding the initiative’s man￾agement and strategic outlook can be taken. To generate this needed evidence, mid-term evaluations are used to inform and guide the decision-making process of donor agencies and program managers. As such, mid-term evaluations are situated between process and impact evaluations as they focus on strategy and implementation issues, but also take a wider perspective to consider the first effects and potential sustainability of an intervention (DANIDA, 2006, pp. 9-10). Hence, the purpose of a mid-term evaluation is to help identify which features are working well and which features are not (Morra-Imas & Rist, 2009, pp. 9-10). As a consequence, this program-level mid-term evaluation assessed PAEGC along the OECD-DAC crite￾ria6 as well as its implementation structure to analyse whether the current implementation structure is sufficient to reach the intended objective of PAEGC. For this purpose a contribution analysis was chosen to assess PAEGC’s effectiveness and impact as well as to identify influencing factors. A contribution analysis is an approach that assesses the performance of policies and programmes towards an outcome or outcomes. This type of analysis was developed by John Mayne for situations where designing an ‘experiment’ to test cause and effect is impractical. A contribution analysis attempts to address this by focusing on questions of “contribution”, specifically to what extent observed results (whether positive or negative) are the consequence of the policy, programme or in this case PAEGC (Mayne, 2001). Thus, contribution analysis provides an alternative way of thinking about the problem of attribution to the traditional positivist approach. The traditional approach tries to prove causality via a counterfactual applied in experimental or quasi-experimental designs. A contribution analysis in contrast does not attempt to prove that one factor – PAEGC – “caused” the desired outcome, but rather to explore the contribution PAEGC is making to the observed results. By using the existing results framework (“theory of change”) of PAEGC and collecting evidence from various sources to test this theory, the aim is to build a credible “performance story”. This can demonstrate whether PAEGC or other factors were an important influencing factor in driving the observed change (Ibid.). For simplification purposes, the implementation of a contribution analysis as articulated by John Mayne can be broken down into six steps: 1. Set out the attribution problem to be addressed: In the case of PAEGC this is the contribution the funded innovations make to e.g. the increase in agricultural production and/or value among farmers and agribusinesses 6 Relevance, effectiveness, impact, efficiency and sustainability Chapter: Methodological Approach Page 10 | 66 2. Develop a theory of change/results framework: The theory of change7 for PAEGC was developed by the partners and documented in the M&E plan. It sets out the specific steps of PAEGC, how it is expected to bring about change and clarifies the expected short, medium and long-term outcomes. 3. Populate the model with existing data and evidence: Data to populate the model has been collected from interviews and secondary data (see chapter 3.3). 4. Assemble and assess the “performance story”: The assembled performance story of PAEGC was assembled and critically assessed in an iterative process during the data collec￾tion phase. It was discussed and analysed with different stakeholders (see chapter 3.3). 5. Seek out additional evidence: New data and evidence was gathered during the data col￾lection phase to fill-in evidence gaps and to test impact hypotheses. For this purpose further analytical work was carried out (see chapter 3.3). 6. Revise the “performance story”: In an iterative process the “performance story” was refined and is elaborated upon in this report. Next to the contribution analysis the implementation of PAEGC was evaluated to identify internal influ￾encing factors on PAEGC’s effectiveness and impact. Hereby the cooperation system as such and the processes of the cooperation were analysed by using an analytical approach structured along the fol￾lowing five key dimensions: strategy, cooperation, steering structure, processes as well as learning and innovation (see assessment grid in annex V). Moreover, the relevance of PAEGC was assessed by comparing the initiative with 50 similar challenge funds to identify the unique features of PAEGC. In addition it analysed the demand for such an instru￾ment on the international political agenda as well as the relevance of the funded innovations in the local context. Regarding the evaluation criteria of efficiency, the mid-term evaluation focused in consultation with the partners solely on the leverage ratio of private capital and benchmarked this leverage ratio with other challenge funds. The production and allocation efficiency of PAEGC was not addressed as relevant data was not available due to PAEGC’s current stage of implementation. Furthermore, within the criteria of sustainability, the following three analytical dimensions were exam￾ined to assess the potential sustainability of PAEGC:  Financial sustainability was defined as the ability of the innovators to market their innovations independently from subsidies.  Technical sustainability was defined as the degree to which the innovations can be maintained by the beneficiaries without external technical assistance.  Environmental sustainability was defined as the rates of renewable resource harvest, pollution creation, and non-renewable resource depletion that can be continued indefinitely. If they can￾not be continued indefinitely then they are not sustainable (Daly, 1990, pp. 1-6). In addition to the OECD-DAC criteria the mid-term evaluation also evaluated the integration of gender into the processes and structures of PAEGC. Hereby, the structure and processes of PAEGC were ana￾lysed with regard to their gender sensitivity and gender mainstreaming. 7 PAEGC’s theory of change can be summarized as follows: “By engaging and mobilizing diverse, global solver communities, USAID and its Partners can source, select, incubate, test, and scale up science and technology innovations that will overcome critical barriers to development and accelerate the pace at which the world’s most pressing development prob-lems can be addressed” Chapter: Methodological Approach Page 11 | 66 3.2 Methodological Challenges In the course of the mid-term evaluation some methodological challenges were encountered, which could not – despite the efforts undertaken by the evaluation and PAEGC management team – be rem￾edied. These must be viewed as limitations to the evaluation’s methodology.  The analysis of the developmental effectiveness and impact of PAEGC could solely be based on a secondary data analysis of the monitoring data that has been collected by PASTO and the online platform WebMo. Only part of this data had been validated by PASTO through field visits or interviews with direct beneficiaries. Moreover, the existing monitoring data was often incom￾plete and hence does not provide a comprehensive picture of the developmental effectiveness and impact of PAEGC. As a result only anecdotal evidence regarding PAEGC’s effectiveness and impact on the innovator level can be presented in this report.  Moreover, most of the funded projects under PAEGC had not reached the stage of wide scale implementation at the time this mid-term evaluation was conducted as most of them only started field testing. As a consequence most funded projects could not yet have an impact on e.g., the increase of agriculture productivity on the level of the beneficiaries.  Additional data collection on the innovator level (1st and 2nd call) to substantiate findings re￾garding PAEGC’s structure was not possible due to time and legal constraints. While the evalu￾ation team discussed the possibility of field visits with the partners, eventually it was agreed that no additional data should be collected on innovator level in order to be able to complete the evaluation at the time of the partners meeting in June 2016. Furthermore, it was deemed important not to duplicate PASTO’ previous mid-term evaluation on innovator level. Hence, the evaluation team based its analysis solely on the results of the innovator-level mid-term assess￾ment that was conducted by PASTO. It should be mentioned though that field visits will be done as part of an impact evaluation in 2018/2019 since the partners acknowledge the importance of data collection at the level of awardees and beneficiaries. 3.3 Research Design The program-level mid-term evaluation consisted of three phases: Figure 1 – Phases of the Mid-Term Evaluation 3.3.1 Desk Study & Explorative Interviews At the beginning of the mid-term evaluation, a constitutive coordination meeting was held via phone on 7th April 2016 between GIZ and Syspons in order to discuss the details of the assignment and to adjust the organisation and planning of the mid-term evaluation. The starting point of the first phase was a desk research which included an analysis of relevant docu￾ments of the initiative, academic and “grey” literature on the nexus approach as well as documents from other challenge funds. Its objective was to provide a description of the initiative’s origin, objectives, Theory of Change, organisation and implementation structure. Furthermore, it allowed PAEGC to be located within the context of the international development agenda and in relation to other initiatives. A complete list of all analysed documents can be found in the annex. Chapter: Methodological Approach Page 12 | 66 In addition to the desk research, the evaluation team conducted explorative interviews with all three partners (GIZ, SIDA, and USAID) during the first phase. The aim of these interviews was to gain a deeper understanding of the strengths and weaknesses of the initiative, as well as to find out what the partners expect from the mid-term evaluation. Building on the previously undertaken analytical steps and the terms of reference, the evaluation team then developed a methodological approach for the mid-term evaluation (see chapter 3.1). Further￾more, as part of this methodological design, an assessment grid was created (see annex). In this as￾sessment grid the collected information of the prior analysis was summarised and systemised in the form of central questions and analytical aspects relevant to this mid-term evaluation. The evaluation team then assigned indicators and/or descriptors to every aspect. In a last step during the first phase the assessment grid and the methodological approach was discussed and finalised in two telephone conferences with GIZ, SIDA, and USAID. 3.3.2 Secondary Data Analysis & In-depth Interviews In the second phase of the mid-term evaluation, the evaluation team conducted a secondary data analysis on the basis of the data provided by the initiative. The documents that were analysed included a sample of the applications from the first call for proposals in 2012, the yearly reports of the 2013 cohort of innovators and the monitoring data uploaded on WebMo, as well as the raw data from the innovator-level mid-term assessment conducted by PASTO. The purpose of this analysis was to evaluate what results had been achieved so far on the innovator-level, to populate the Theory of Change with data and to assess to what extent PAEGC’s envisioned Theory of Change works. Moreover, the evaluation team conducted 27 semi-structured in-depth interviews with several stake￾holders from the partners’ organizations, PASTO, external experts, and representatives of other chal￾lenge funds. The following interview partners provided their input:  Partners (USAID, BMZ/GIZ and SIDA): 21 interviews  External experts and other challenge funds (incl. PASTO): 6 interviews The objective of the interviews was to collect data and to seek out additional evidence to examine the Theory of Change of PAEGC. Hence, the interviews focused on the relevance of PEAGC as well as on the effectiveness of its processes and structures. Moreover, it was discussed to what extent other factors influenced the observed changes and whether these observed changes can be attributed to PAEGC. Furthermore, the interviews served to validate the observed results from the desk research and the secondary data analysis. A complete list of all interview partners can be found in the annex. 3.3.3 Synthesis and Reporting At the beginning of the synthesis and reporting phase the collected data was analysed, triangulated8 and synthesised on the basis of the assessment grid. Afterwards, the preliminary results were presented at the Partners’ Meeting in Washington D.C. on June 2, 2016. On the basis of this presentation and the subsequent discussion with participants from GIZ, SIDA and USAID, a report was drafted that summa￾rised the mid-term evaluation’s conclusions and recommendations. The draft report was presented to GIZ, SIDA and USAID. The feedback received was incorporated into the final report by the evaluation team, which was submitted to the partners at the beginning of August 2016. 8 Data, method and researcher triangulation was used. Chapter: The Powering Agriculture Initiative at a Glance Page 13 | 66 4 The Powering Agriculture Initiative at a Glance “Powering Agriculture: An Energy Grand Challenge for Development (PAEGC)” is a partnership that was launched in 2012, when the United States Agency for International Development (USAID), the Govern￾ment of Sweden (SIDA), the Government of Germany (BMZ), Duke Energy Corporation and the United States Overseas Private Investment Corporation (OPIC) combined resources to create the PAEGC initi￾ative.9 The overall goal of Powering Agriculture is “to support new and sustainable approaches to accel￾erate the development and deployment of clean energy solutions10 for increasing agriculture productiv￾ity and/or value in developing countries” (Annual Report FY 2015, p. 1). As such, the initiative’s target group is composed of farmers and agribusinesses, and ultimately populations affected by poverty that shall benefit from the agricultural innovations. The initiative is set up to run through 2019. The Powering Agriculture initiative follows the energy-agriculture nexus approach, acknowledging the complex interactions between these sectors. This conceptual approach has been introduced in the global debate on development in order to better understand and systematically analyse the interactions be￾tween the natural environment and human activities, and to contribute to a more coordinated manage￾ment and use of resources in the field of energy and agriculture (FAO, 2014). At the same time, the Powering Agriculture initiative is based on the Grand Challenges for Development (GCD) model of USAID, which pursues to define problems, identify constraints, and provide evidence￾based analysis for several issues in international development. The model is based on two fundamental assumptions about international development: First, it is rooted in the belief that science and technology can have an important transformational impact when applied appropriately, and second, that it is crucial to engage stakeholders worldwide in the search for solutions in order to achieve progress. In order to solve the developmental challenges within the energy-agriculture nexus, the Powering Agri￾culture initiative pursues the following objectives: 1. support clean energy technology and business model innovations for agriculture 2. ensure that financial intermediaries obtain the necessary financial resources to help organisa￾tions scale their innovations and reach the farmers and agribusinesses that need these technol￾ogies 3. develop partnerships with public and private sector entities that want to support PAEGC’s goals 4. serve as an information resource on clean energy and agriculture for stakeholders worldwide Based on these four objectives, PAEGCs programmatic interventions are divided into four Technical Components (also see figure 2):  Technology and Business Model Innovation Component: Under this component the initi￾ative provides financial support mainly in the form of grant funding to private, non-profit, and academic entities to design, pilot, and market innovative technologies and business models. These technologies and business models shall further integrate clean energy solutions within agricultural supply chains in developing and emerging countries. Until now, PAEGC has launched two separate calls for proposals in 2013 and 2015, resulting in the provision of 24 grants to so￾called innovators or awardees.  Commercial Financing Component: This component seeks to encourage private sector equity and debt investments within the clean energy-agriculture nexus by providing grant funding, 9 They are collectively referred to as the ‘partners’. 10 Clean energy is defined as: Usable energy (i.e. electricity, illumination, heating/refrigeration, mechanization) that is derived from renewable sources and supports a reduction in fossil fuel use, increase in efficiency, and/or limitation of greenhouse gas emissions. Clean energy sources include – solar, hydro, wind, geothermal, sustainably harvested biomass, and biogas. The term “clean energy solution” is defined as: A combination of appropriate technology and a business model that addresses the clean energy demands of a select market. Chapter: The Powering Agriculture Initiative at a Glance Page 14 | 66 guarantees, and other incentives to financial intermediaries. To facilitate this process, it is en￾visioned to establish a “Global Financing Facility”.  Mainstreaming and Acceleration Component: This component comprises public-private partnerships, training, as well as integration with agricultural and energy programs of the part￾ners. As such, it includes activities to catalyse private-sector investment as well as technical assistance and training in order to enable the scaling-up of proven clean energy solutions for enhancing agricultural productivity and/or value. The idea is to pursue an agriculture energy ‘nexus’ approach in supporting the PAEGC Partners’ ongoing or planned energy and agricultural programs.  Knowledge Management Component: Under this technical component PAEGC seeks to col￾lect, analyse, and disseminate knowledge that has been gained through the implementation of the initiative’s interventions and other activities related to the energy-agriculture nexus. As such, it includes a results-based monitoring and evaluation system, research and analysis, as well as networking activities. While the initiative started at first with the two components Technology and Business Innovation and Commercial Financing, the latter two components were added in mid-2014. The partners share respon￾sibilities with regard to the four components. While USAID is in charge of the general fund management, as well as the call for proposals and the award management, SIDA is responsible for the commercial financing component and the incorporation of gender aspects. The responsibility of GIZ lies with the knowledge management component. The mainstreaming and acceleration component is a joint effort by USAID, Sida, and GIZ. Figure 2 – Four Technical Components of PAEGC Source: PAEGC 2016a, adapted by Syspons 2016 Chapter: Evaluation Findings Page 15 | 66 5 Evaluation Findings The analytical approach of this mid-term evaluation was based upon a contribution analysis in combi￾nation with a process analysis of PAEGC’s implementation structure. To carry out this mid-term evalu￾ation, the evaluators conducted interviews with 30 stakeholders from USAID, SIDA, and GIZ as well as PASTO, external experts, and representatives of other challenge funds. Furthermore, a desk study and a secondary data analysis were used to collect qualitative and quantitative data about the performance of the PAEGC Initiative. On the basis of the collected data and the design of the mid-term evaluation (see chapter 3), valid conclusions about the performance of the Powering Agriculture Initiative can be drawn. 5.1 Relevance of PAEGC 5.1.1 Developmental Relevance of PAEGC Historically and globally, agricultural intensification has been largely accompanied by the increased use of energy inputs – particularly petroleum based products for electricity generation and liquid fuel – to produce, process and distribute agricultural products. As a consequence, food prices on the global mar￾ket have become extremely volatile, as the cost of agricultural production exhibits a close relationship with energy prices. This price volatility in turn can adversely affect food insecure populations – particu￾larly those located in emerging economies and developing countries. In many of these countries, farmers and agribusiness also face food losses due to lacking access to technologies for cooling and other post￾harvest processes. In addition, these populations are also negatively affected by the projected global population growth for the coming years as agriculture is still the most important source of livelihood for most people in these countries. By 2050 the United Nations Food and Agriculture Organisation (FAO) projects that an increase in 70% of 2005-2007 food production levels will be needed to meet the demand of a growing world population, which is expected to reach 9.6 billion people (FAO, 2009). If these trends continue and are not addressed, the agricultural intensification will lead to an even higher demand for fossil fuels worldwide. This development could contribute to greater food insecurity for vulnerable pop￾ulations in developing countries and lead to an increase in greenhouse gas emissions within the agricul￾tural sector, thus contributing significantly to global warming and climate change worldwide (USAID, 2014). At the same time, however, there is currently a lack of instruments to address these challenges in an inter-sectoral way on the international level, according to the interviewed external experts. These in￾terviewed external experts stated that most existing instruments follow a sectoral approach – e.g., either targeting the energy, the agriculture or climate sector – and do not address this developmental challenge with an integrated approach. As a result an increasing need for integrated solutions and/or instruments can be observed on the international level, according to the interviewed experts. This need will in their opinion grow even more in the future as the Sustainable Development Goals (SDGs) can only be reached by following an inter-sectoral and thus integrated approach. For instance SDGs such as SDG 1 (No Poverty), SDG 2 (Zero Hunger/Sustainable Agriculture), SDG 5 (Gender Equality), SDG 7 (Affordable and Clean Energy), SDG 8 (Economic Growth), SDG 9 (Innovation), SDG 13 (Climate Action) and SDG 17 (Partnerships for the Goals), to which PAEGC also makes a contribution according to the interviewees, can only be reached if holistic and integrated developmental instruments are adopted by the donor community and other relevant stakeholders. Consequently USAID, SIDA and GIZ (the latter on behalf of BMZ) initiated the Powering Agriculture Initiative (PAEGC) to address this developmental challenge and the need for an integrated approach on the international level. PAEGC thereby supports energy- and climate-smart agriculture in developing and emerging countries by means of a challenge fund to foster innovative solutions for the above de￾scribed developmental challenge (see chapter 4). This is done by combining activities in four different components, as described in chapter 4. According to all interview partners the main assumption of the Chapter: Evaluation Findings Page 16 | 66 initiative is that improved access to clean energy solutions will allow farmers and agribusinesses to mechanize their value chains, making mechanized value chains more efficient and sustainable, add value during the processing of primary commodities and/or increase the possibilities for processing and storing of fresh produce. This in turn should lead to an increase in the income of farmers, reduce the dependency of the agricultural sector on fossil fuels, and increase the amount of food supplies in the market to satisfy the demand of a growing population; ultimately contributing to an increase in food security in vulnerable populations and a reduction in Greenhouse Gas Emissions.. For this purpose PAEGC targets – according to the interviewed stakeholders – the most important bar￾riers to the integration of clean energy technologies in the agricultural sector. These are, among others: lack of awareness, financial means, demand for and access to clean energy technologies on the side of the farmers, as well as a limited client base, and lack of access to credit and opportunities for scaling￾up on the side of technological enterprises. Furthermore, a comparative study of 50 challenge funds conducted by the evaluation team confirmed the relevance of PAEGC on the international level. It showed that PAEGC currently is the only global challenge fund which addresses this developmental challenge using an integrated approach by funding and upscaling innovative solutions at the interface of the energy and agriculture sector (through the four different components). Only the African Enterprise Challenge Fund (AECF) and one challenge fund funded by DFID also addressed both sectors. However, AECF is focused on Africa, while challenge fund by DFID only targeted Vietnam and has already been closed down (see list of analysed challenge funds in annex III). Hence, PAEGC is the only fund with a global outreach operating in the energy-agriculture nexus. 5.1.2 Developmental Relevance of the Target Group PAEGC aims at attracting a wide range of organizations from industrialized and developing countries that have not worked in the field of development cooperation before. These should develop innovative solutions in the energy-agriculture nexus that benefit vulnerable and poor populations affected by food insecurity (USAID, 2012). For the interviewed partners it is thereby important that the funded organi￾zations not only work with “traditional actors”, but also those that have not worked in the field of development cooperation, as they wanted to attract new and fresh knowledge to solve the developmen￾tal challenges in the agriculture energy nexus. PAEGC also serves for some partners as a vehicle to work closely together with the private sector. In this regard the analysis of the received applications for the first and second call demonstrates that PAEGC was able to attract a diverse range of organizations and could increase this diversity from the first to the second call (see figure 3). While in the first call applicants hailed mainly from the for-profit (45%) and non-profit sector (33%), the second call applicants showed a more diverse range of organi￾zations from the private sector (43%), NGOs (19%), not-for-profit (6%), foundations (3%) and other organizations (10%). Moreover, the total number of applications received increased by 84% from the first call (473 applications received) to the second call (871 applications received). Chapter: Evaluation Findings Page 17 | 66 Figure 3 – Applications in 1st and 2nd Call by Organization Source: PAEGC, 2013c; PAEGC, 2015c, adopted by Syspons 2016 Simultaneously, the number of countries by applicant increased by 30% from the first call (76 countries) to the second call (99 countries). The share of applicants from developing countries was thereby strengthened. In the first call 55% of the applicants came from developing countries, while in the second call 62% of the applicants were from developing countries (see figure 4). Figure 4 – Applications in 1st and 2nd Call by Developing vs. Developed Countries Source: PAEGC, 2013c; PAEGC, 2015c, adopted by Syspons 2016 Chapter: Evaluation Findings Page 18 | 66 Moreover, an analysis of the applicants of the second call highlights that the majority of applicants (ca. 66%) in the second call had not received funding from one of the partners before (see figure 5). Figure 5 – Applicants of the 2nd Call that have Received Previous Funding from Partners Source: PAEGC, 2015c, adopted by Syspons 2016 The diversity of the applicants is also reflected in the group of awardees in the second call. The analysis demonstrates that in the second call 13 finalists from nine countries were awarded a grant. The majority of these awardees (62%) are from developing countries and other developed countries (23%) (see figure 6). Figure 6 – Developing vs. Developed World Awardees from 2nd Call Source: PAEGC, 2015c, adopted by Syspons 2016 Chapter: Evaluation Findings Page 19 | 66 In terms of regional distribution, the biggest group (39%) of awardees comes from Africa, followed by North America (23%) Asia (15%) and Europe/ Eurasia (15%) (see figure 7). Figure 7 – Regional Distribution of Awardees from 2nd Call Source: PAEGC, 2015c, adopted by Syspons 2016 Of these 13 awardees 46% constitute a private enterprise, while 16% are non-governmental organiza￾tions (see figure 8). A direct comparison of the diversity of the second call awardee with the first call awardees is not feasible, as the respective data was not collected for the first call. Figure 8 – Awardees from 2nd Call by Organization Source: PAEGC, 2015c, adopted by Syspons 2016 An analysis of the reached target group on beneficiary level could not be conducted at the time of the evaluation as most funded projects were still in a piloting stage. Thus, data on the beneficiary level was not available. 5.1.3 Developmental Relevance of the Innovations Besides the general relevance of PAEGC, it is important to assess the relevance of the funded innovations in the local context to ensure that they address relevant challenges in the energy-agriculture nexus. For this purpose the evaluation team analysed the submitted applications and yearly reports of the innova￾tors, as most funded innovations were field testing their innovations at the time of the evaluation. As a consequence no empirical data was available to examine the relevance and local ownership of the funded innovations on the beneficiary level. Chapter: Evaluation Findings Page 20 | 66 However, the analysis of the submitted applications and quarterly reports highlighted that most funded organisations did not conduct a market analysis to identify the demand for their products. This is par￾ticularly the case for the organisations of the first call, in which a market analysis was not an application requirement. Regarding the 2015 cohort, some innovators have conducted a separate market analysis. However, also in the second call most funded organisations focused on the supply side of their business, by describing what their products could offer to the market and how the technologies would benefit the agricultural farmers and agribusinesses. As such, the funded organisations exhibit a good knowledge of the local context in terms of the target group and their living conditions in their applications, while demonstrating a lack of knowledge with regard to the local market and demand for their product. Information on the relevance of the innovations within the local context is also missing from the sub￾mitted reports, as most innovators focused on the technical aspects of their innovation instead of the envisioned developmental impact of their innovation in the local context. This might also be the result of a missing predefined reporting structure for the yearly reports as these vary widely in structure and content. While PAEGC has introduced a standardized reporting template for all 2015 innovators, this was not the case for the 2013 cohort that was assessed in this evaluation. In contrast the interviewed experts consider the funded clean energy technologies by PAEGC innovative and relevant in the given local context. At the same time the interviewed experts also stress that the assessment of the technologies’ quality is difficult at this point of time as most funded technologies are in an early stage of development. This makes it difficult to evaluate their innovative potential, as they have not entered the market yet. In addition they point out that there are no objective criteria formu￾lated by PAEGC to deem a technology as either innovative or not innovative. While PAEGC has formu￾lated a definition of “Clean Energy Technology Innovation” in its first call for proposals in 2012, this definition remains very vague. It simply states that “PAEGC support is available for development and/or adaptation of affordable technologies focused on clean energy generation, storage, and more efficient end-use within the agriculture sectors of developing countries that have potential for achieving com￾mercial scale. This support may be for new technologies that are still in the incubation/demonstration stage, and/or existing technologies that are modified to respond to the demands of a specific target market.” (USAID, 2012, p. 11) In contrast, other challenge funds such as the African Enterprise Chal￾lenge Fund (AECF) have developed their own, more precise innovation scales, which allows the fund to assess the innovative potential of the technology on a scale from zero to six (AECF, 2014). 5.1.4 Assessment of the Relevance Based on these results the evaluation team comes to the conclusion that PAEGC is a unique and relevant challenge fund. None of the other 50 analysed challenge funds have adopted a nexus approach inte￾grating agriculture and energy, and work on a global scale. Furthermore, it became obvious that PAEGC addresses global developmental challenges such as food insecurity and climate change that will gain in importance in the future. In this context, PAEGC also addresses a need on the international global agenda for integrated ap￾proaches that address developmental challenges in an inter-sectoral manner. These kinds of instruments will gain further importance in light of the Sustainable Development Goals, which can only be reached by using holistic and integrated approaches. PAEGC thereby has so far already contributed to eight of the 17 SDGs. At the same time, PAEGC has been successful in reaching a wide variety of organisations, which are mainly located in developing countries and from the private sector. Moreover, the majority of these organisations have not worked in the field of development cooperation before, thereby bringing new perspectives into the field of development cooperation. In contrast, PAEGC’s success on the level of the beneficiaries with regard to vulnerable and/or poor target populations cannot be assessed, as most funded innovations were in the pilot phase at the time of this evaluation. As a consequence robust data on the beneficiary level was not yet available. Chapter: Evaluation Findings Page 21 | 66 The developmental relevance of the funded innovations must also be assessed cautiously. Although most interviewed stakeholders view the innovations as relevant for the local context and most of the funded organisations possess good knowledge of their target group, the majority of the innovators do not possess thorough knowledge regarding the expected demand for their technologies on the local market. This is also reflected in the quarterly reports, in which most organisations report on the tech￾nological features of their innovation instead of its developmental relevance11. This is also due to the fact that most of the innovators are still in an early stage of development, thus focusing on technological aspects of the solution instead of potential benefits for the target group. Nonetheless, as a consequence it remains be seen to what extent the funded innovations will experience local ownership and demand in the local context in the future. 5.2 Effectiveness of PAEGC’s Structures and Processes 5.2.1 Strategy, Cooperation and Steering Structure In order to be effective, any initiative in the field of development cooperation must design a strategy that allows actors to orient all their efforts and activities toward those objectives that are crucial to success. Such a strategy should be based on a holistic logic, upon which the initiative’s partners can act and respond in individual cases. Expressed in other words, the partners need to share a logic that goes beyond the aspects of strategic orientation and coordination, a logic that establishes a shared horizon of meaning and a culture of cooperation. The evaluation showed that the partners have developed and currently share such a common strategic vision for PAEGC. It was defined as follows: “PAEGC seeks to identify and support new and sustainable approaches to accelerate the development and deployment of clean energy solutions for increasing agriculture productivity and/or value in devel￾oping countries focusing on pro poor.” Furthermore, the interviews demonstrated that all three governmental partners (USAID, SIDA, and BMZ/GIZ) have a common understanding of the initiative’s objectives and strategy. However, in the interviews it also became obvious that the partners vary in their interpretation of different aspects of the strategy. Although it is not mentioned in the strategy or in the initiative’s Theory of Change, two partners also consider the promotion of the energy-agriculture nexus approach on the international development agenda as one of the initiative’s main objectives, while the other partner does not see this as a priority. Moreover, these two partners also see it as their task to mainstream this approach into their bilateral portfolio once it has been proven to be effective. Furthermore, different views exist regarding the thematic and regional focus of the initiative in relation to the envisioned developmental effectiveness. While all interview partners confirmed that the regional and/or thematic approach has an influence on the effectiveness of the initiative, several interview part￾ners mentioned that setting a thematic and/or geographic focus instead of fostering a broad range of innovations worldwide would increase PAEGC’s developmental effectiveness. As PAEGC currently sup￾ports innovative Clean Energy Solutions for various agricultural value chains in many different develop￾ing and emerging countries, there is only limited potential to create synergies and build on lessons learned across countries and projects. Other challenge funds such as the Multilateral Investment Fund by the Inter-American Development Bank (IADB), which focuses on projects in Latin America and the Caribbean, can generate synergies more effectively as developed technologies can be transferred into other settings more easily. This is due to the fact that political, cultural, climatic and infrastructural contexts are more similar within a region than across regions. Thus, by focusing on a particular agricul￾tural value chain or region, PAEGC could build up und use the technological expertise more efficiently 11 It needs to be noted that an elaboration of the developmental relevance of the innovation is not a specific requirement in the quarterly reports. Chapter: Evaluation Findings Page 22 | 66 to overcome developmental problems, according to the interviewed stakeholders. However, other inter￾view partners see the broad thematic and global call for proposals as an advantage, as it increases the chance of finding innovations with a high potential for developmental impact. Moreover, several interview partners noted that the need for developmental impact on the side of the partners does not correspond with the kind of innovations that PAEGC is currently funding. As the initi￾ative mostly fosters early-stage innovations that entail a high risk of not proving viable, the partners’ general high expectations with regard to their developmental impact cannot be satisfied in the short run. The strategic vision is thus rather designed for later stage innovations that can be scaled up easily. In addition, PAEGC’s grant structure is designed to give support for three years, which is commonly seen by the interviewed external experts as too short for the funded innovations to develop an observ￾able developmental impact. For this reason other challenge funds such as the African Enterprise Chal￾lenge Fund (AECF) extended their grant period to six years, in order to be able to show developmental impact to its donors. PAEGC in contrast opted for opening a second window in its second call to especially fund the scaling-up of late stage clean energy innovations, while the first window funds – as in the first call – early-stage innovations. Despite these differences, all interviewed partners agreed that the overall shared strategy enables an effective cooperation and steering structure within the initiative. PAEGC’s cooperation structure is char￾acterised by collaboration and sharing of responsibilities among the partners. This is evident in the division of tasks in relation to the four components. While USAID is responsible for the general fund management as well as the call for proposals and the award management, SIDA focusses on the com￾mercial financing component and incorporation of gender aspects. GIZ is in charge of the knowledge management component, and jointly, the Partners support mainstreaming and acceleration. According to the interviewed partners this division of responsibilities reflects the financial and human resources available at each partner organization, as well as the main area of expertise and competences. The partners’ description of their work moreover shows that the allocated time and resources at the respec￾tive partner organizations are sufficient to carry out the assigned tasks. In general, all interviewed stakeholders on the partner side stated that all PAEGC partners are actively involved in the initiative and consider the cooperation structure to be effective and efficient. They especially highlighted the synergies that can be drawn from using the comparative advantages of each partner and the added value that is created through co-production. Additionally, most interviewed representatives of the partner organizations stated that the roles within the initiative are assigned clearly and that each partner is aware of their own as well as other partners’ tasks. However, this only holds true for the governmental partners, as the roles of Duke Energy and OPIC as private sector partners is not clear to all other members. Several interviewees mentioned that they have not had any contact with these organisations in the context of the initiative so far and there￾fore are not aware of how these organisations are actually involved. Some interviewed stakeholders characterised these partners as “silent partners” that have not played an active part in the initiative for a long time. The cooperation structure is further supported by a clear steering structure on the strategic and opera￾tional level. On the strategic level the partners hold annual meetings, in which they discuss the results of the past year, plan upcoming activities, and take strategic decisions for the next year. On the oper￾ational level the management team, consisting of the partner’s representatives, comes together every six weeks in a telephone conference to talk about current issues and challenges. In addition to these formal meetings, there is a frequent informal exchange via various communication channels (email, telephone, WebMo), which all partners describe as effective and efficient. 5.2.2 Processes Every challenge fund uses a wide variety of processes to achieve its goals. They consist of work flows – sequences of activities that are undergone for specific periods in order to achieve a specific objective. For the Powering Agriculture Initiative, two processes are especially relevant, which were analysed in Chapter: Evaluation Findings Page 23 | 66 terms of their effectiveness. This is on the one hand the selection process of innovators, and on the other hand the award management process. Concerning the selection process of innovators, the data of the PASTO mid-term evaluation revealed that the majority of innovators are satisfied with the process. 81% of the innovators from the first call rated the selection process as good or excellent. They perceived the process to be collaborative and partner-like, rather than bureaucratic and overly rigid. Innovators who had previously gone through other application procedures at USAID or elsewhere were especially likely to rate the process as logical and user-friendly. Nonetheless, some also mentioned shortcomings, such as a lack of clarity on the principal stages of the process and timeline, and a general lack of communication and feedback during the application process. In addition, several innovators experienced the process as slow and criticized the long waiting time between application and the final announcement of the awards. For the second call, there is no data available on the innovator-level, as the PASTO mid-term evaluation only addressed the innovators from the first round. However, the analysis of the documents and inter￾views revealed that most of the shortcomings of the first call were remedied in the second call. Changes included among others the following:  A broad agency announcement with a more detailed and more explicit description of the appli￾cant process and timeline.  A simplified first stage of the application process that requires the submission of a 600-word concept note that succinctly describes the CES and why it is appropriate for PAEGC funding.  The introduction of a second funding window that is designed to support clean energy technol￾ogies and business models that have already been successfully piloted in a developing coun￾try/emerging market on a small scale, and are now ready for market expansion and scaling-up  A stronger emphasis on the analysis of the business case after the first stage of the application process.  The inclusion of external international experts to evaluate the technological and contextual as￾pects of the proposed submissions.  A reduction of the procurement and award process from 12 months to 10 months. However, despite the stronger emphasis on the business case in the second call, the focus on the feasibility of the business models was not sufficient in both calls according to the interviewed experts. They noted that the assessment of the business model did not occupy a central role during the selection process and was not rigorous enough. Similarly, the partners were neither completely satisfied with the market analyses received by the applicants. At the same time, all interview partners they emphasized the importance of a functioning business model for the successful development and scaling-up of the technology. The current design of the selection process does not lead to a sufficient business model assessment, which according to the interview partners may thus reduce the potential to successfully scale the innovations at a later stage and ultimately achieve developmental impact. Furthermore, they stressed that the concept of “innovation” is not clearly defined by PAEGC12 (also see section 5.1.3). This leads to uncertainties regarding the requirements of the technology in the selection process. The African Enterprise Challenge Fund in contrast provides an example of a more precise ap￾proach to innovation. While following a similar approach to PAEGC in funding innovative agricultural technologies, it uses an innovation scale that includes clear definitions of different levels of innovation. Next to the selection process the grant management process is the second relevant process within PAEGC. It is characterized by a division of responsibilities: While USAID holds the general responsibility for the process and is in charge of granting awards, financial management as well as milestone revisions, 12 In the 2012 call for proposals, PAEGC defines Clean Energy Technology Innovation as “development and/or adaptation of affordable technologies focused on clean energy generation, storage, and more efficient end-use within the agriculture sectors of developing countries that have potential for achieving commercial scale. This support may be for new technologies that are still in the incubation/demonstration stage, and/or existing technologies that are modified to respond to the demands of a specific target market.” (USAID, 2012, p. 11) Chapter: Evaluation Findings Page 24 | 66 it has subcontracted a consultancy to support the operational tasks of USAID in the context of the award management. USAID is, however, the responsible administrator of the award management. The con￾sultancy is organised as the Powering Agriculture Support Task Order (PASTO), which has the following seven tasks: 1. Advising on and strengthening PAEGC awardees' ability to comply with USAID policies and pro￾cedures 2. Monitoring awardee performance and assisting awardees with specific needs 3. Designing and conducting performance and impact evaluations 4. Organising and facilitating PAEGC events and travel 5. Facilitating outreach and communication 6. Developing and transferring PAEGC knowledge 7. Supporting future PAEGC procurements This organisational model is viewed by most interviewed stakeholders as effective and efficient. It thereby also reflects a general trend among challenge funds that over the last decades have subcon￾tracted parts or all of their fund management for cost effectiveness reasons to the private sector. The interviewed stakeholders in particular view the chosen organisational model for PAEGC as advantageous for the following reasons: Within the model the financial responsibility rests with USAID. This minimises the risk of embezzlement of funds. At the same time PASTO provides PAEGC with expertise in different fields that could not be provided by the partners. In addition, other tasks such as monitoring and the organisation of events can be provided at a lower cost by outsourcing them to the private sector. During the interviews, however, some disadvantages of this organisational model could also be identified. These concern mainly the clarity of roles between PASTO and USAID. Due to the necessity of working closely together in this model, the roles and responsibilities between USAID and PASTO are not always clear to the innovators. At the same time, the interview partners from GIZ and SIDA acknowledged that USAID is very transparent on the division of responsibilities in the award management process and on PASTO’s role and budget. They emphasized that USAID is always willing to share information on PASTO when requested. On the side of the innovators PASTO’s support to USAID grant management is rated by 81% as good or excellent. The innovators mainly utilise PASTO services for M&E, milestone review, milestone modi￾fication and business acceleration (see figure 9). The latter was realised through the Powering Agricul￾ture Xcelerator (PAX) program and was viewed by the innovators as particularly favourable for their business development. Chapter: Evaluation Findings Page 25 | 66 Figure 9 – PASTO Services Utilised by Innovators Source: PASTO, 2016, adopted by Syspons 2016 With regard to the USAID award managers (AOR), a mixed picture emerges. The innovators rank the performance of the AOR very differently depending on the person, with an average score of 4.0 out of 5. While three AOR score high to excellent marks (on a scale from 1 (poor or non-existent) to 5 (excel￾lent), two AOR are rated at the lower end of the scale (see figure 10). The latter can be explained by a lack of or slow responsiveness to questions or approval of documents submitted, primarily in reference to projects’ award revision requests according to PASTO’s mid-term review. However, this has been remedied in the second call by assigning just one AOR for all 13 innovators of the second call. This arrangement enables PAEGC to identify recurring issues and shorten the time for responses. Figure 10 – Rating of AOR Services Source: PASTO, 2016, adopted by Syspons 2016 In addition the innovators requested to receive more support from their AOR in terms of leverage of local field offices and missions of the partners as well as in identifying local partners and additional funding. 5.2.3 Learning and Innovation Learning and innovation are key features of every challenge fund. To capture lessons learned, recognise good practices or discover challenges in the fund’s implementation, M&E systems are used to inform the fund’s management. The collected information can then be used by the fund manager to initiate Chapter: Evaluation Findings Page 26 | 66 learning and innovation within the fund and among its partners. Recently, more and more challenge funds also appoint so-called learning managers in the form of external organisations that are tasked with reviewing the performance of the fund independently and reporting their results to the fund man￾ager. A prerequisite for the assessment of the performance of a challenge fund is an established Theory of Change in order to analyse if the envisioned impact hypothesis hold true. Moreover, an effective moni￾toring system has to be established, which collects data regularly and feeds it back to the fund manager. Within PAEGC the partners have agreed on a coherent Theory of Change and have established a moni￾toring system that allows data collection on a daily basis to keep track of the innovators’ performance. However, not all objectives of PAEGC are reflected within the Theory of Change. As mentioned in chapter 5.2.1 some partners also see the mainstreaming of the energy-agriculture nexus approach as one of PAEGC’s objectives, though it is not stated as an objective in the current Theory of Change. In addition, the monitoring system is only viewed by 50% of the innovators of the first call as useful, because most see it as not intuitive and confusing to use. As a consequence, data entries by the innovators are inco￾herent and infrequent. Moreover, the current M&E system in place has not developed a coherent concept for assessing the developmental impact of PAEGC. Although the M&E plan foresees an impact evaluation at the end of the current funding period of PAEGC, there is no concept in place to measure developmental impact during the funding period of PAEGC. This is however important to satisfy the needs for developmental results on a political level and to rally support for PAEGC on an international level. Other analysed challenge funds solved this challenge of collecting impact data on the innovator level by using the con￾cept of the “last hard number” as well as by commissioning independent impact reviews for high impact projects identified by this concept. The concept of the “last hard number” uses the number that the business is able to tell the challenge fund from its own records without recording additional data. It will be the closest logical connection to the net benefit per household and/or the number of households benefitting13. Once this number is known, the fund manager can identify what kind of data must be collected in order to validate the developmental impact. After the developmental impact has been ap￾proximated through the necessary monitoring tools, it can be decided whether this project is a high impact project and should be analysed in-depth by an independent impact review. 5.2.4 Assessment of PAEGC’s Structures and Processes Based on these findings the evaluation team assesses the structure and processes of PAEGC positively. In general PAEGC possesses a coherent strategy that is shared by its partners. Moreover, it is supported by an effective and efficient cooperation and steering structure, which is characterised by a mode of co￾production and the usage of comparative advantages of the partners’ strengths. Its selection and award management processes also function effectively and shortcomings in these processes have been ad￾dressed effectively over time. Minor shortcomings could be identified in the strategic outlook of PAEGC. Here the objective of promot￾ing the energy-agriculture nexus approach on the international agenda is not shared by all partners. Moreover, the global thematic approach of the initiative has a potentially limiting influence on its devel￾opmental effectiveness as lessons learned as well as synergies cannot be easily generated due to the wide variety of funded innovations. In addition the adopted strategy to generate developmental impact does not go hand in hand with the funding of early stage innovations under PAEGC’S first and second call as those need time to reach the stage of “developmental impact”. As a consequence, the partners do not receive the needed developmental impact data to satisfy their political demands within their respective countries. 13 For example, an out-grower scheme is likely to be able to tell how many farmers it worked with, how much money it paid to each, and often what their costs are. This means that the “last hard number” is actually extremly close to the developmental impact. Only the baseline information is missing in order to develop a full beneficary model. A supplier project, by contrast, may only know how much produce it sold – but will not know how many people bought it, how they used it, or what the benefit was. This means that their “last hard number” is some distance from the final developmental impact, and more primary and secondary research is likely to be needed in order to complete the benificiary model. Chapter: Evaluation Findings Page 27 | 66 Other areas for improvement are the award management process and the current M&E system. Regard￾ing the former, roles and responsibilities concerning AOR and PASTO have been opaque for some of the innovators, especially at the beginning of the initiative. USAID has made several efforts to clarify the roles by documenting and communicating the award management responsibilities to the innovators, for example through several “Powering Agriculture Requests and Reminders Bulletins” (PARRBs). Nonethe￾less, PASTO’s assessment on innovator level as well as the interviews conducted for this evaluation show that some of the innovators still perceive a lack of clarity in this regard. Therefore, efforts to clearly communicate roles and responsibilities in the award management process should continue. Con￾cerning the communication among the PAEGC partners, the results from the interviews show that there is a transparent and proactive communication by USAID on the award management process. Monitoring and learning can be improved with regard to the measurement of developmental impact, for which PAEGC is currently lacking an effective concept. This makes it difficult to fulfil the needs of the partners to deliver impact data on the political level in their respective countries. Here the concept of the “last hard number” and independent impact reviews of high impact projects as well as a more user￾friendly monitoring tool might remedy this challenge. However, it is has to be noted positively that PAEGC has established a Theory of Change and a monitoring system that collects data on the innovator level on a daily basis. This is a prerequisite for learning and innovation and is not a usual feature of challenge funds. 5.3 Effectiveness of the Four Components 5.3.1 Component 1: Technology and Business Model Innovation In PAEGC’s first component, the Technology and Business Model Innovation Component, the initiative’s objective is to increase farmers’ and agribusinesses’ access to and/or use of clean energy solutions in order to increase agricultural production and/or value among farmers and agribusinesses. Moreover, it strives to increase support for low carbon economic growth within the agricultural sector. In order to achieve these objectives, this component provides financial grants to private, non-profit, and academic entities, which should enable them to design, pilot, and market innovative technologies and business models. These technologies and business models shall further integrate clean energy so￾lutions within agricultural supply chains in developing and emerging countries. To date, PAEGC has launched two global calls for proposals in 2012 and 2014, resulting in the provision of 24 grants to so￾called innovators or awardees. The mid-term evaluation focused on the first cohort of awardees, as there is no data available yet for the second cohort that just started in 201514 . The innovators’ progress is supported by the Powering Agriculture Xcelerator programme (PAX), which is part of PASTO. While the PAX programme provides assistance to the innovators in terms of their business strategy and acceleration, PASTO assists them with gender integration, compliance with legal regulations, milestone revision as well as monitoring (among other tasks). Although PAX support is limited to three years and thus to the grant period, the interviewed innovators and partners alike see this program as a success factor for advancing and accelerating the innovations. In contrast, a limiting factor that was noted by many innovators as well as other stakeholders in the interviews was the lack of local support via the bilateral structures of the partners. Most innovators would like to be more strongly connected to and receive more assistance from the local in-country missions or field offices of USAID, SIDA, and GIZ. Several interview partners confirmed this view by highlighting the synergies that could potentially be realised through a stronger collaboration with the partner offices in the field, e.g., by integrating the innovator’s technologies as pilots into bilateral pro￾grammes of the partners. PAEGC already reacted to this demand by including mainstreaming activities (see component 3). 14 The second call for proposals was divided into two windows. Next to clean energy solutions in an early design stage, a second window was opend that focused more on upscaling, offering support to the testing of business models. Chapter: Evaluation Findings Page 28 | 66 Looking at the results of the first global innovation call, it can be stated that 12 innovations have been sourced out of 473 applicants, disbursing ca. $ 13 million. The funding provided by PAEGC has thereby enabled the innovators to further develop their technologies, even though progress has not been as fast as initially expected by the partners according to PASTO’s mid-term review. As one award was can￾celled15, there are currently 11 active awards. At the moment of this mid-term evaluation, nine innova￾tors started field testing of their technology (see figure 11). Figure 11 – 2013 Innovators and Their Innovation Stages as of Spring 2016 Source: PASTO, 2016, adapted by Syspons 2016 Even though most of the innovations have not entered the market yet, first empirical data from the field testing shows that 3219 beneficiaries could be reached, of which 1022 are women. Furthermore, 158 wholesalers, retailers, or maintenance professionals are accessible to beneficiaries for selling and ser￾vicing clean energy solutions. In addition, 1145 persons, of which 265 are women, have been trained or participated in demonstrations on CES technology. As a result 0.17 MW of clean energy capacity has been installed so far through the funded innovations (see figure 12). Figure 12 – Overview of Achieved Impacts of the 2013 Innovators Source: PASTO, 2016, adapted by Syspons 2016 However, the reported data has to be treated cautiously as it mainly relies on the self-reporting of the innovators and has so far only been partially verified by PASTO. In addition, with regard to the other indicators of PAEGC’s result framework, no results can be reported as only anecdotal evidence exists at 15 The cancelled award refers to Experience International in Indonesia- Chapter: Evaluation Findings Page 29 | 66 this moment. This is due to the fact that most innovators have just started field testing their technologies (see figure 11). 5.3.2 Component 2: Commercial Financing PAEGC’s second component, the Commercial Financing Component, has the objective to encourage private sector equity and debt investments within the energy-agriculture nexus by providing grant fund￾ing, guarantees, and other incentives to financial intermediaries. To facilitate this process, the initiative sought to establish a “Global Financing Facility” (LIFT) that brings the innovators either together with potential investors or provides them with loans from banks in order to allow a scaling-up of their tech￾nologies. According to the interviews conducted, PAEGC has engaged in numerous approaches and dis￾cussions with potential cooperation partners, but has so far been unable to establish a financing mech￾anism for scaling-up. The main reason for this was legal constraints, which inhibited the conclusion of a cooperation agreement with financial intermediaries. Currently SIDA is working on identifying potential partners and solutions for the commercial financing component. However, the interview partners agreed that the process is rather complicated and tedious, so that the success of this component remains uncertain according to their perspective. 5.3.3 Component 3: Mainstreaming and Acceleration The third component of PAEGC, the Mainstreaming and Acceleration Component, was included in mid￾2014 and has two primary objectives. The first purpose is to catalyse private-sector investment as well as technical assistance and training in order to enable a scaling-up of proven clean energy solutions for enhancing agricultural productivity and/or value. The second purpose is to mainstream the energy￾agriculture nexus into the bilateral structures of the partners. The idea hereby is to promote the inte￾gration of the energy-agriculture nexus approach into the PAEGC Partners’ ongoing or planned bilateral energy and agricultural programs. To contribute to the achievement of these two objectives, the initiative implemented a “Massive Open Online Course” (MOOC) with 1354 active participants, of which 35% were women. Of the 1354 active participants 238 (18%) received a certificate at the end of the course. This is a significantly higher rate than other MOOC courses, which on average have a rate of 9%. Moreover, it reached a variety of stakeholders who mostly came from Sub-Saharan Africa (40%), followed by Europe (26%) and Asia (15%). They mainly worked in international organisations (23%) and NGOs (23%), followed by research organisations (16%), the private sector (13%) and governmental organisations (4%). As a result the MOOC was regarded as a success by all involved interview partners, as it allowed experiences in the energy-agriculture nexus to be spread within a large group of relevant actors and generally raised awareness for the nexus approach. With regard to the mainstreaming of the nexus approach within the three partner organizations, the partners follow different approaches, as the interviews revealed. GIZ actively pursues the integration of good practices and innovations in its bilateral structures, mostly through pilot projects and public-private partnerships. For example, the nexus approach was integrated in two bilateral pilot projects on energy efficiency measures in tea factories in Kenya and solar-powered milk chilling in Tunisia. In these public￾private partnership projects, synergies with PAEGC were used to mainstream innovative clean energy solutions into GIZ’s own bilateral programmes. In addition, GIZ has implemented a training competition in East Africa that allowed training institutes to receive funding and support for trainings on the energy￾agriculture nexus. The conducted trainings resulted in the establishment of valuable partnerships with the training institutions and other actors in the field. Furthermore, GIZ has launched a regional hub for East Africa in Nairobi, Kenya in January 2016.The hub provides support to bilateral and regional projects of GIZ and links them to PAEGC innovators. There are currently 15 sites in East Africa where innovators are implementing projects. The hub offers the possibility to intensify PAEGC’s connection to local struc￾tures and to facilitate exchange among innovators and further relevant stakeholders through events and workshops. The hub’s activities thereby contribute particularly to the PAEGC components Chapter: Evaluation Findings Page 30 | 66 “Knowledge Management” and “Mainstreaming and Acceleration”. It is meant to function as an acceler￾ator for regional as well as supra-regional knowledge exchange and will therefore potentially play an important part in future knowledge management activities. At USAID, plans for mainstreaming exist, but currently do not exceed mostly informal knowledge sharing on the nexus approach. At the moment USAID is trying to strengthen the incorporation of innovations in the energy-agriculture nexus into its bilateral portfolio. For instance, USAID is experimenting with a business incentive model that nudges bilateral USAID programmes to work with innovations that have been previously funded through USAID challenge funds. Furthermore, the agency is working on a kind of procurement catalogue for innovations that will facilitate the process of finding and applying innova￾tive technologies that have proven successful. The information on funded innovations is currently avail￾able online on various websites, such as globalinnovationexchange or the PACE initiative16 . The idea is to make information on innovations available to various donors in order to strengthen the global ex￾change on innovations in the energy-agriculture nexus. In contrast to GIZ and USAID, SIDA currently does not have established mechanisms to mainstream the nexus approach into its programmes accord￾ing to the interview partners. The different approaches to mainstreaming described by the partners are reflected in the assessment of PAEGC’s strategy as well, which found that the promotion of the nexus approach is not a priority objective for all partners (see 5.2.1). Hence, the information obtained in the interviews indicates that there is not a common and systematic strategy to mainstreaming that is pursued by all partners. Another limiting factor that has been identified by the interview partners is the non-existence of the public-private partnership (PPP) component that was initially planned. While GIZ offered to take over the management of the PPP component, this could not be realised due to legal constraints. As such, public-private partnerships have not been systematically implemented by the Powering Agriculture Ini￾tiative so far, and have only been realised through GIZ’s bilateral structures. Several interview partners see this as a missed opportunity to catalyse further private-sector investment and mainstream the nexus approach more broadly. Nonetheless, the partners still plan to realise the PPP component by using the remaining funds of the initiative in the current phase. 5.3.4 Component 4: Knowledge Management The Knowledge Management Component within PAEGC, included in mid-2014, follows the objective to collect, analyse, and disseminate knowledge that has been acquired through the implementation of the initiative’s interventions and other activities related to the energy-agriculture nexus. This includes a results-based monitoring and evaluation system, research and analysis, as well as networking activities. As such, the component allows the initiative also to analyse and present lessons learnt and hence to disseminate the value it has added to the work of the innovators. As part of the knowledge management component, PAEGC has implemented two research studies with the Food and Agricultural Organization of the United Nations (FAO) as a strategic partner on clean energy solutions in agricultural value chains and associated business models. These studies have been supplemented by several workshops and conferences on the topic, such as a joint workshop with FAO on “prospects for solar-powered irrigation systems” (SPIS) 2015 in Rome17. Moreover, the initiative has established four digital media outlets to disseminate its knowledge and experiences in the energy-agri￾culture nexus to the wider public. These include the initiative’s own website poweringag.org, which obtains ca. 3000 direct visits a week, a PoweringAg portal on the online platform Energypedia with around 38,000 visits up to June 2016, to which PAEGC contributes articles on the energy-agriculture nexus, a digital newsletter, and a Twitter account with 2.287 followers. In addition to these permanent digital platforms, PAEGC has launched a press campaign with main articles in The Guardian and National Geographic, which have the potential to reach 78.58 million readers. 16 See www.globalinnovationexchange.org and https://www.usaid.gov/PACE 17 This workshop was based on a study conducted by GIZ. Chapter: Evaluation Findings Page 31 | 66 Despite these activities, the interviewed partners and external stakeholders expressed that there is a lack of clarity with regard to the strategic purpose of this component. To them it was not clear to what extent knowledge management should close a gap in research, cater to a demand for information in this field or to whom this information should be provided (policy-makers, general public, academia, etc.). As a result they identified room for improvement this component with regard to sharpening the strategic purpose, target groups and subsequently focusing the knowledge management activities. 5.3.5 Assessment of the Four Components’ Effectiveness Based on these findings the evaluation team comes to mixed conclusions regarding the effectiveness of PAEGC’s four components. The Technology and Business Model Innovation Component can generally be assessed positively, as PAEGC has sourced a number of Clean Energy Solutions and has provided rele￾vant support for the innovators to develop their technologies. At the same time though, the support structures for innovators in this component could be strengthened by connecting them more thoroughly to the bilateral structures and programmes of the partners. With regard to the achieved outcomes and impacts of this component, only preliminary conclusions can be drawn. At the time of this evaluation, most funded innovators just had begun field testing their innovations. As a consequence only preliminary data regarding for instance the increase in farmers’ and agribusinesses access to and/or use of clean energy solutions or the increase in agricultural production and/or value could be collected. On the basis of this data it can be concluded that the impact of this component looks promising, but that it has to be substantiated by an impact assessment at a later stage. With regard to the Commercial Financing Component, it has to be stated that the objective to establish a financing mechanism for scaling-up could not be reached so far. The evaluation has shown, however, that such a mechanism would add great value to the initiative, as it would facilitate the scaling-up of the innovations. In contrast, the Mainstreaming and Acceleration Component was successful in implementing training courses on the energy-agriculture nexus as well as in mainstreaming it into the bilateral portfolio of GIZ in India, Tunisia and East Africa. Moreover, USAID has plans to start mainstreaming the energy-agri￾culture nexus into its portfolio. Despite these successes however, this component is also marked by a lack of clarity concerning the strategic approach to mainstreaming, as not all partners view this as a priority. Further, it should be considered that this component was not included from the start but added in mid-2014. In addition, a public-private partnership facility could not be initiated so far, which limits the opportunities to leverage further private sector investment and accelerate the funded innovations. The lack of clarity concerning a common strategic outlook is also a limiting factor for the Knowledge Management Component – which was also added in mid-2014 to support the upscaling of innovations. In this component it has not yet been clarified between the partners for what purpose knowledge man￾agement should take place, i.e. whether the knowledge management activities are designed to close a knowledge gap or to pursue agenda setting on the energy-agriculture nexus. Despite this missing stra￾tegic outlook however, the assessment of this component can be cautiously positive, as the initiative has successfully implemented various activities such as a landscape study in collaboration with FAO and a communication strategy to disseminate its knowledge and experiences across a wide range of actors. 5.4 Efficiency of PAEGC In the general literature on challenge funds one of the criteria for the success of challenge funds in terms of efficiency is their leverage ratio. This ratio indicates to what extent the challenge fund has been able to draw investment from the private sector for achieving its developmental goals. The underlying assumption is that the money obtained by the private sector will multiply the impact that can be reached by public funding alone. Hence, the efficiency of the Powering Agriculture Initiative can be analysed through the level of additional funding raised. As indicated in the methodological approach, it is not possible in this case to analyse the initiative in terms of its allocation and production efficiency, as most of the funded innovations have not entered the production phase yet (see chapter 3.1). Chapter: Evaluation Findings Page 32 | 66 For the first funding round in 2013, PAEGC has raised more than USD $4.131 million of additional funding (contributions by the innovators themselves and by Duke Energy). For the second call, the amount of additional funding could be increased to USD $14.491 million.18 That means the leverage ratio of PAEGC was 1.16 in 2013, and subsequently grew to 1.40 in 2015. Despite this growth, the leverage ratio is still below that of other challenge funds, which lies around 2.0. The Energising Devel￾opment programme for example has a leverage ratio of 1.73, while the African Enterprise Challenge Fund stands out with a leverage ratio of 4.21. In the conducted interviews the representative of the partners all agreed that an increase of the leverage ratio would not only be beneficial for the efficiency of PAEGC but also for its effectiveness in the long run. Suggested ways to possibly increase PAEGC’s leverage ratio included cooperation with impact/in￾vestment funds, the integration of additional private partners, or a higher cost share of the awardees in future calls. For the latter, it needs to be acknowledged that the cost share has already been raised in the second call. Furthermore, the responses given by the interview partners also indicated that increasing PAEGC’s effi￾ciency is a tool to enhance the political impact of the initiative. Catalysing the developmental impact through the leveraging of money from the private sector will help to position PAEGC successfully vis-á￾vis the donor governments and within the international development community. 5.4.1 Assessment of PAEGC’s Efficiency Based on the evaluation’s findings, it can be concluded that the leverage ratio of the Powering Agricul￾ture Initiative is below that of other challenge funds. However, when comparing the leverage ratio to those of other challenge funds, it needs to be taken into account that PAEGC is a relatively young challenge fund that has only existed for four years. Nonetheless, considering the strategic outlook of the initiative beyond 2019, it would be beneficial for PAEGC to find new ways of attracting additional funding from the private sector. This would help the initiative to enhance its developmental impact and thereby to increase its political credibility vis-á-vis donor governments. 5.5 Impact of PAEGC 5.5.1 Impact on Innovators-Level According to PAEGC’s results framework, the envisioned developmental impact of the initiative is to increase agricultural productivity and/or value in developing countries through the adoption of innova￾tors’ clean energy solutions. Furthermore, farmers, agribusinesses as well as customers shall benefit from the CES through improved energy services, an increased income, and the reduction of greenhouse gases. However, as most innovators from the first round are currently still in the pilot stage, there is no devel￾opmental impact data available yet on the beneficiaries’ level (see figure 11). In addition to this meth￾odological challenge, the current M&E system in place does not contain a coherent concept to assess the developmental impact of PAEGC. As already described in chapter 5.2.3, the M&E plan only foresees an impact evaluation at the end of the current funding period, but not during the funding period of PAEGC. It can be noted, however, that the lacking concept does not correspond with the high expectations regarding the developmental impact mentioned by the partners during the interviews. Most interviewees expect a scaling-up rate of 25% to 35% of the funded innovations, while most other challenge funds aim at 15% (e.g., African Enterprise Challenge Fund). This reflects a discrepancy between expectations and the realistic potential of a challenge fund. While the partners are trying to satisfy these expectation by delivering impact data on PAEGC’s performance to their governments, this is currently difficult in 18 The additional funding for the 2015 cohort includes the cost share of the innovators as well as USD $1.5 million by Duke Energy Corporation. The latter was also included in the calculation for the 2013 cohort, as it was not clear from the data how much of Duke’s contribution was disbursed in which year. Chapter: Evaluation Findings Page 33 | 66 light of the missing concept to measure PAEGC’s developmental impact during the implementation (see chapter 5.2.3) 5.5.2 Assessment of PAEGC’s Impact Due to the current stage of development of the funded innovations, a final assessment of PAEGC’s developmental impact on innovators’ level is not feasible at this point. The findings demonstrate, how￾ever, that the initiative is currently lacking an effective concept for collecting data on the developmental impact during PAEGC’s implementation. This is in contrast with the partners’ high expectations in this regard as they need developmental impact data on PAEGC’s performance for their respective govern￾ments. 5.6 Sustainability 5.6.1 Sustainability on Innovator Level In the inception phase of this evaluation it was jointly decided with the partners to assess PAEGC’s sustainability along the following three dimensions:  Financial sustainability was defined as the ability of the innovators to market their innovations independently from subsidies.  Technical sustainability was defined as the degree to which the innovations can be maintained by the beneficiaries without external technical assistance19 .  Environmental sustainability was defined as the rates of renewable resource harvest, pollution creation, and non-renewable resource depletion that can be continued indefinitely. If they can￾not be continued indefinitely then they are not sustainable. First of all, it has to be noted that most of the funded innovations are still in an early stage of develop￾ment and have not entered the market until this day, which means that this evaluation cannot assess the results in terms of sustainability yet. Nonetheless, it can be acknowledged that the initiative has taken several steps in order to achieve its goals concerning sustainability. With regard to environmental sustainability, it can be noted that this aspect is one of the core objectives of the initiative. This is reflected for instance in the call for proposals and annual reports, which explicitly state that the initiative aims to fund Clean Energy Solutions that decrease post-harvest losses, increase energy efficiency and stimulate low carbon economic growth. Furthermore, some of the interview part￾ners also emphasized that the goal of environmental sustainability is a main driver of the initiative. As such, environmental sustainability is also one of the criteria in PAEGC’s selection process, in that the applicants have to elaborate on the associated environmental risks of their technology and undergo an initial environmental examination. Through this process, the partners want to ensure that the initiative does not fund innovations that reinforce harmful environmental practices (see call for proposal from 2012). In PAEGC’s monitoring and evaluation system, the aspect of environmental sustainability is represented by an indicator on the reduction of Greenhouse Gas Emissions (GHG). However, while the M&E system allows to collect data on this aspect in order to analyse the innovations’ impact on environmental sus￾tainability, most innovators are currently still in the piloting phase and thus cannot report on their innovation’s environmental sustainability in terms of the reduction of GHG. As a result, the evaluation team was unable to assess this dimension of sustainability. With regard to technical and financial sustainability, PAEGC’s selection criteria seek to ensure that only those CES are funded that have a realistic potential to survive on the market without subsidies, and to be maintained by the beneficiaries themselves in the future. In the calls for proposals as well as the 19 Technical assistance in this context refers to external assistance by development organizations or PAEGC. It does not include technical maintenance by service providers that may be needed on a regular basis. Chapter: Evaluation Findings Page 34 | 66 initiative’s annual reports it is furthermore stated that PAEGC pursues the objective of sourcing market￾oriented Clean Energy Solutions, hence emphasizing the goal of financial sustainability. Despite these clearly stated objectives, the evaluation was not able to assess to what extent technical and financial sustainability have been achieved by PAEGC so far, because most of the funded innovations have not reached the distribution and marketing phase yet (see figure 11). The same holds true for the mid-term evaluation on innovator’s level conducted by PASTO. 5.6.2 Assessment of the Sustainability As in the previous chapter on impact, the lack of data does not allow for a final assessment of PAEGC’s sustainability on the innovator level. This is due to the fact that most of the innovations are still in an early stage of development; therefore no results concerning their financial, technical and environmental sustainability - as defined above - exist. The importance of environmental sustainability for the initiative is reflected in the selection process though, as it involves an environmental examination of the innova￾tions. Furthermore, PAEGC’s monitoring system is designed to measure the reduction in Greenhouse Gas Emissions that can be attributed to the innovation. As there is no final assessment possible at this point, it remains to be seen in how far PAEGC’s objectives in terms of sustainability are achieved in the future. 5.6.3 Integration of Gender in PAEGC’s Concept and Processes All three partners possess an organizational strategy when it comes to the integration of gender into their programmes. The gender strategies of SIDA, USAID, and GIZ demonstrate that the topic plays an important role for all partners, as they define specific institutions, mechanisms and roles for the incor￾poration of gender in their work. Moreover, these strategies are based on a broad understanding of gender, taking into account not only disparities between men and women, but also the social context in which gender is constructed as a concept. PAEGC’s work on gender has been greatly promoted by SIDA, whose policies contain three main approaches to gender mainstreaming: The integration of gender equality in general, the targeting of specific groups or issues through special interventions, and dialogue with partners on gender sensitive issues. Looking at PAEGC’s Theory of Change, however, it becomes clear that on the program-level of the initiative, gender was not conceptualized as an essential feature from the beginning, as it does not appear in the results framework. As the interview partners noted, this has led to reporting gaps on gender, because there are no indicators on gender included in the monitoring system. The same holds true for the selection process: As gender was not an obvious selection criterion in the first global call for innovations, gender aspects were only considered to some minor extent in the proposals. This has been remedied in the second call though, as PAEGC incorporated gender into the selection criteria and hired a gender consultant, who assessed the applications and supported the awardees with regard to their gender awareness. Furthermore, the interviews demonstrated that PAEGC’s approach to gender focuses on creating an equitable benefit of the CES for both men and women, for instance by integrating women into the production chain or as users of the innovation. As such, the interview partners all confirmed that gender is conceptually integrated into PAEGC’s processes as „gender equity“, rather than as a social construct. At the same time, the interviewed stakeholders stated that the integration of gender on the innovator level poses a challenge due to substantial knowledge gaps of the innovators. While most of the awardees are generally open and willing to consider gender aspects in their work, they often do not know how to implement gender awareness in practice. For this reason, the Powering Agriculture Initiative has ex￾panded its support by providing tailored advice on gender integration and awareness raising to the innovators through a special gender consultant at PASTO. Apart from providing implementation support for all new PAEGC awardees, the gender consultant is tasked with following all gender activities of ongoing PAEGC projects and monitoring the progress of the innovators in terms of their work on gender integration. As such, all interview partners agreed that PAEGC has strengthened its efforts on gender integration in recent years. Chapter: Evaluation Findings Page 35 | 66 5.6.4 Assessment of Gender Based on these findings, the integration of gender in PAEGC’s concept and processes can be assessed positively by the evaluation team. Especially since the second global call for innovations, the initiative has incorporated gender more thoroughly in its work – on the one hand by including gender in the selection criteria in the call for proposals, and on the other hand by hiring a gender consultant who monitors and provides support to the innovators with regard to their gender activities. However, the results of the evaluation also show that the gender concept applied by PAEGC is limited to “gender equity”, instead of seeing gender as a social construct. The disadvantage of this limitation is that the concept of “gender equity” does not necessarily take into account the socially constructed gender roles, norms and habits within a specific social context, but simply strives for equality between men and women regardless of the consequences this may have for the local society. This does not correspond to the partners’ own gender strategies with a broader understanding of gender, and potentially limits the impact of PAEGC’s interventions on gender mainstreaming. The evaluation team therefore concludes that the initiative could leverage its impact in this regard by incorporating gender as a social construct20 in their interventions. This could be done for instance by an expansion of the M&E system to include a gender analysis during field visits that analyses the projects’ influence on gender norms and structures in the local context. 20 For “gender as a social construct” see for example Marecek, J., Crawford, M., & Popp, D. (2004). On the Construction of Gender, Sex, and Sexualities". In A.H. Eagly, A.E. Beall, & R.J. Sternberg. Chapter: Conclusions and Recommendations Page 36 | 66 6 Conclusions and Recommendations 6.1 Conclusions PAEGC is a unique and highly relevant challenge fund, as none of the other 50 analysed challenge funds has adopted a nexus approach integrating agriculture and energy and is working on a global scale. PAEGC contributes to overcoming global developmental challenges such as food insecurity or climate change and addresses the need on the international agenda for holistic and integrated approaches. At the same time, PAEGC reaches a relevant target group, as it mostly funds organizations from developing and emerging countries and from the private sector that pursue innovative ideas. Among these are many that have not worked in the field of development cooperation before, so the innovators may bring new perspectives into the field of development cooperation. At the same time PAEGC’s structures and processes are effective. It could be demonstrated that PAEGC possesses an overall coherent strategy that is shared by its partners, as well as an effective and efficient cooperation and steering structure. Furthermore, the selection and award management processes func￾tion effectively, as these processes have been refined in the second call. An area of improvement con￾cerning the award management process is the clarity of the roles and responsibilities between PASTO and the AORs for the innovators. Despite extensive communication by USAID on roles and responsibil￾ities, for instance through the “Powering Agriculture Requests and Reminders Bulletin”, innovators still ask for more clarity on the award management process. At the same time, this lack of clarity is not perceived by the PAEGC partners, as USAID has made the division of roles and responsibilities with PASTO in the award management process transparent to the partners. In addition, a limiting factor to the effectiveness is the lack of a common view among the partners on promoting the energy-agriculture nexus on the international agenda. PAEGC supports a wide range of different technologies in different regions of the world. Hence, the broad global thematic approach of the initiative potentially limits its effectiveness as lessons learned and synergies cannot be used efficiently due to the wide variety of funded innovations. Another discrepancy that has been found in the strategy is the focus on early-stage innovations, which does not correspond to the partners’ high expectations for achieving a developmental impact. It needs to be acknowledged though, that the second call also opened a window for innovations that are further advanced in their development stage. In regards to learning and innovation, it can be concluded that PAEGC possesses the necessary prerequisites, as it has a Theory of Change and a mon￾itoring system. A weakness of this monitoring system is, however, that it does not entail an effective concept on how to measure developmental impact during the implementation phase. Concerning the effectiveness of PAEGC’s four components, mixed conclusions have to be drawn, partly due to the fact that components 3 and 4 have been established later. With regard to the Tech￾nology and Business Innovation Model Component a positive picture emerges as it has been successful in sourcing a variety of Clean Energy Solutions. At the same time it was successful in providing relevant support to the innovators in terms of business development and project implementation. However, this support can be further strengthened by establishing a stronger link to the partners’ bilateral structures and programmes in order to generate synergies between the partners’ bilateral portfolio and the funded innovations. Regarding the Commercial Financing Component, it has to be concluded that the objective to establish a financing mechanism for scaling-up has yet to be achieved. The findings demonstrate though, that such a mechanism would be very useful for facilitating the scaling-up of the funded inno￾vations. In contrast hereto the Mainstreaming and Acceleration Component has been effective in imple￾menting trainings courses on the energy-agriculture nexus as well as in mainstreaming it into the GIZ’s bilateral portfolio. A shortcoming of this component is, however, that there is no clear strategic approach to mainstreaming as it is not a priority objective for all partners. Another weakness is the fact that a public-private partnership facility could not be established so far, which limits the opportunities for attracting additional private sector investment that could be useful for scaling-up. The lack of a common strategic outlook is also a shortcoming in the Knowledge Management Component, as the partners have Chapter: Conclusions and Recommendations Page 37 | 66 not clarified among themselves for which purpose knowledge management should take place. Despite this lack of clarity in the strategic outlook however, it can be concluded that the initiative has been successful in implementing various activities to disseminate its knowledge and experiences globally. In terms of efficiency, it can be concluded that the leverage ratio of PAEGC is below that of other challenge funds assessed. However, when comparing the leverage ratio to other challenge funds, it needs to be taken into account that PAEGC is a relatively young challenge fund that has only existed for four years. Nonetheless, considering the strategic outlook of the initiative beyond 2019, it would be beneficial for PAEGC to find new ways of attracting additional funding from the private sector. This would help the initiative to enhance its developmental impact and thereby to increase its political credibility vis-á-vis donor governments. In this regard it has to be noted that PAEGC’s developmental impact and sustainability on the innovator level could not be assessed by this evaluation as most innovators only started field testing their innovations. Moreover, PAEGC is currently lacking an effective concept in its M&E system for meas￾uring impact during PAEGC’s implementation phase. This poses a challenge for the partners, as the impact data is necessary to rally support for the initiative among the donor governments as well as the international community. The integration of gender in PAEGC’s concept and processes can in turn be assessed positively. Espe￾cially since the second global call for innovations, the initiative has incorporated gender more thoroughly in their work by hiring a gender consultant and making gender an explicit selection criterion. However, it also became obvious that the gender concept applied by PAEGC is limited to “gender equity”, instead of seeing gender as a social construct. This does not correspond to the partners’ own gender strategies with a broader understanding of gender, and potentially limits the impact of PAEGC’s interventions on gender mainstreaming. 6.2 Recommendations Based on the findings of this mid-term evaluation the following eight recommendations are put forward. They are divided into three categories: recommendations on the general strategic level, recommenda￾tions on the strategic set-up of PAEGC and recommendations on the operational level. 6.2.1 General Strategic Level Globally, PAEGC is a unique challenge fund which addresses valid developmental challenges in the en￾ergy-agriculture nexus. However, improvements can be made regarding its leverage ratio and its meas￾urement of developmental impact. 1. Recommendation: The Powering Agriculture Initiative should be continued as it is a unique challenge fund, which addresses valid developmental challenges in the energy-agriculture nexus. PAEGC is a unique challenge fund in the energy-agriculture nexus. Through its inter-sectoral ap￾proach, its focus on innovative technologies, and its multilateral partnership structure, it possesses a unique selling point vis-à-vis other challenge funds in the field of development cooperation. As such, PAEGC has attracted a high number of applicants in the past, as well as great interest from other cooperation partners. Given this demand, it should be continued and if possible scaled-up in the long-run. 2. Recommendation: The partners should try to win new partners from the private sector to increase the leverage ratio of PAEGC. For this purpose they should clarify their expectations regarding the private sector’s role and the added value the private sector partner can gain from its participation in PAEGC. PAEGC could enhance its efficiency and effectiveness by increasing its leverage ratio. This could be done using two different approaches: Chapter: Conclusions and Recommendations Page 38 | 66 o In order to attract additional funding, PAEGC could seek to establish new partnerships on a strategic level with private companies that are interested in collaborating and investing in the field of energy and agriculture.21 Before reaching out to new private partners though, it is essential that USAID, SIDA and BMZ/GIZ discuss and agree on a suitable type of partner￾ship. This involves not only a consultation on the type, size and location of a potential private sector partner, but most importantly also on the role the new partner should play in the initiative. o Furthermore, PAEGC could further increase the cost share in future calls to leverage more private funds from their awardees. This can be either done by simply increasing the cost share to a higher percentage or by adopting a model in which the cost share increases during the grant period. Another option is to open a call for public-private partnerships, attracting innovators that are already cooperating with a private company as a financial resource. The latter model was adopted for example by the Global Challenge for Develop￾ment “Securing Water for Food” in which the awardees have to provide 25% matching funds in the first year and 50% matching funds in the second and third year of their grant. o As a third option, PAEGC could cooperate with private social impact investors or social im￾pact funds, such as the Universal Green Energy Access Program (UGEAP) of Deutsche Bank. This type of investor aims to support projects with a developmental and social impact by scaling up the available capital from local markets, and may therefore provide a suitable source of additional funding for PAEGC. 3. Recommendation: The partners should develop a system to measure developmental impact by using “the last hard number” concept and conduct independent reviews for high impact projects. The results of the analysis highlight the need for PAEGC to develop an effective concept for data collection on developmental impact. This is necessary in order to satisfy the partners’ need for developmental results on the political level and to rally support for the initiative on the international level. As a result, for future calls PAEGC should adopt the concept of the “last hard number”.22 The concept of the “last hard number” uses the number that the business is able to tell the challenge fund from its own records without recording additional data. It will be the closest logical connection to the net benefit per household and/or the number of households benefitting. Once this number is known, the fund manager knows what kind of data has to be collected in order to validate the developmental impact. After the developmental impact has been approximated through the neces￾sary monitoring tools, it can be decided whether this project is a high impact project and should be analysed in-depth by an independent impact review. In addition, PAEGC should identify high impact projects based on the collected data of its monitoring system. Based on this data PAEGC should commission independent impact reviews for these projects to prove and substantiate their develop￾mental impact. So in summary, PAEGC can use two approaches (simultaneously) in order to measure developmen￾tal impact: First, based on the data of the existing projects, PAEGC can identify high impact projects and commission independent impact reviews for those. Second, in future calls PAEGC can not only identify high impact projects, but also apply the concept of the “last hard number” on top of that. In both cases the gathered data on the developmental impact can be used to position PAEGC on the international level and to satisfy the needs for developmental impact data on the political level. 21 These partnerships could take a similar form as with Duke Energy Corporation and OPIC. 22 For more information on the “last hard number concept”, please read Richardson, L., Smith, D., Blundy, C., 2015. Specialist Evaluation and Quality Assurance Service. Sheffield: International Organisation Development (IOD PARC), p. 12 Chapter: Conclusions and Recommendations Page 39 | 66 6.2.2 Strategic Set-Up of PAEGC In order to be more effective in the future, PAEGC should adopt a new strategic set-up to scale up successful funded innovations. For this purpose different options can be adopted by the partners. 4. Recommendation: The partners should develop a new strategic set-up to further increase PAEGC’s effectiveness. The findings of the mid-term evaluation highlight that PAEGC has mostly funded early-stage inno￾vations through its first call as well as the second call’s first window. As a result most innovations only started field testing at the time of the evaluation and were not able to demonstrate develop￾mental impact yet. In order to scale-up the funded innovations, PAEGC has to develop mechanisms to bring these innovations to commercialisation. Only then will it be possible for PAEGC to report developmental impact on a large scale. Furthermore, it became obvious from the evaluation that a regional and/ or thematic focus as well as the integration of the funded innovations into the bilateral structures of the partners will increase the effectiveness of PAEGC. Consequently, PAEGC should develop a strategic set-up in which these dimensions are taken into account. For the re-conceptualisation of PAEGC’s strategic set-up the following options are put forward. The options are designed to highlight differences in possible strategic set-ups. In reality, however, these options or particular features of these options could be combined to develop a distinct strategic set￾up for PAEGC. o Option 1 – Simply Scaling Up: Within this option the strategic focus and structure of PAEGC remains the same. However, in addition to already existing mechanisms and structures, a PPP facility is developed with a regional or thematic focus that helps to commercialise late stage innovations. Via the PPP facility previously funded innovations from earlier calls or new late-stage innovations, which need scaling-up, could be funded. The advantages of this option are a strengthened profile for PAEGC, through its regional or thematic specialization, and a focus on the support for innovators. At the same time, the transaction costs for the foreseen adjustments in the initiative are kept low. The main disadvantage of this option is however, that it only slightly strengthens PAEGC’s effectiveness as it does not integrate the funded innovations into the bilateral structures of the partners. As a result, mainstreaming, as in the current structure of PAEGC, is not further pursued as a priority. Even though mainstreaming could also be pursued through the promotion of exchanges, trainings and roundtables, a stronger linkage with the partners’ bilateral structure would be the most effective way to achieve this. o Option 2 – Scaling it by Decentralising it: Also in the second option the strategic focus and structure of PAEGC remains the same. What makes this option different from the first one is that PAEGC establishes cooperation with an investment/impact fund, through which a new funding window for scaling previously funded as well as new innovations is opened. By choosing this option, the leverage ratio can be increased and private sector involvement can be strengthened. Furthermore, risky investment in innovations could be mitigated through the selection criteria of the investment fund. However, the selection process for a suitable fund might prove to be difficult as the results for component 2 have shown in this evaluation. In addition, the funding of high impact projects through the chosen fund might not be guar￾anteed, as the selection of innovations by the fund might be predominantly based on eco￾nomic feasibility and not on an assessment of the potential for developmental impact. More￾over, PAEGC’s effectiveness will most likely not be increased through this model as PAEGC will not adopt a thematic and/or regional focus. Additionally, the innovations will also not be mainstreamed into the bilateral programmes of the partners as their funding will depend on the assessment of the fund and not on the assessment of their compatibility with the part￾ners’ bilateral portfolio. o Option 3 – Scaling it by Mainstreaming it: In this option the mainstreaming of the energy￾agriculture nexus approach into the partners’ bilateral structures becomes one of PAEGC’s strategic objectives. As a result the Theory of Change is adapted accordingly. In addition, a Chapter: Conclusions and Recommendations Page 40 | 66 PPP facility is developed with a regional or thematic focus to scale up already funded and new late stage innovations, which need support in commercialisation. Furthermore, it would be a prerequisite for funding these innovations that they could be integrated into the bilat￾eral portfolio of the partners. This could, for example, take the form of a pilot, in which the funded innovation functions as a test case while the objective of the bilateral programme would be to set-up the needed framework conditions for the scaling-up of the innovation (e.g., adjustments of legal regulations or development of a market). The advantage of this option would be that the energy-agriculture nexus approach is strengthened, while at the same time the innovators receive more support for scaling-up via their integration into the partners’ bilateral structures. As a result, PAEGC’s effectiveness would be strengthened most. In addition the approach and the accomplished results would be easy to communicate. A potential disadvantage of this option is, however, the loss in diversity of innovations. If PAEGC decides to focus only on certain regions or agricultural value chains, the variety of funded potential solutions to the developmental challenges will automatically be reduced. Figure 13 – Options for PAEGC’s new strategic set-up 6.2.3 Operational Level PAEGC possesses efficient and effective structures and processes to implement its call for innovations. However, to further improve PAEGC’s processes and structures, adjustments with regard to the selection process, gender, and the clarity of roles and responsibilities in the award management should be im￾plemented. 5. Recommendation: In future calls the partners should put a stronger emphasis on the analy￾sis of local markets and demands as a pre-requisite for applicants. Moreover, the assessment of the quality of these analyses as well as the conceptual feasibility of the proposed business model should play a central role in future calls (especially for scaling-up windows). The selec￾tion process should be structured in such a way that innovators can already receive business models and market analysis support during the selection phase. The feasibility of the business model as well as the suitability of the innovation for the local market are key factors for a successful commercialisation of the innovations. Hence, more importance should be given to market analysis and business model analysis as a selection criterion in future selection processes. Although an analysis of the local market was included as a selection criterion in the second call for proposals, the applicants did not provide analyses that were rigorous enough to ensure a successful commercialisation of the innovations. For this reason the structure of future Chapter: Conclusions and Recommendations Page 41 | 66 calls should be adapted in such a way that the innovators can already receive business models and market analysis support during the selection process. This kind of support could be given after the innovators have been selected on the basis of their full proposal and prior to their presentation to the evaluation board. By doing this, the number of innovators receiving business development sup￾port is restricted. Additionally, the innovators still would have the opportunity to adapt their inno￾vation based on the received support. The changes could then be presented before the evaluation board and form a selection criteria for the final selection. While the support for applicants with early￾stage innovations should focus on market analysis, support for later-stage innovations should em￾phasise commercialisation and scaling-up. A possible vehicle for rolling out this kind of support could be the PAX programme. 6. Recommendation: In order to increase clarity the partners should clearly communicate the roles of PASTO and AORs towards the innovators. In order to ensure an effective award management, the roles of PASTO and the AORs should be clearly defined and communicated towards the innovators. Despite the fact that USAID has already offered extensive information on roles and responsibilities, it should continue to pursue a proactive communication policy concerning the award management process towards the innovators. At the same time the partners should continue to proactively use the regular six-week calls on the opera￾tional level to clarify ambiguities and uncertainties if necessary. 7. Recommendation: The partners should extend the support offered by PAX to the innovators for 3 more years to increase the likelihood of their future success and PAEGC’s effectiveness. The findings of the present analysis, including the results of PASTO’s evaluation, highlighted the importance of providing continuous support to the innovators in terms of their business development and scaling-up. Therefore, the PAX program should be extended for three more years beyond the funding period of the innovators. In order to focus the support, the program should only be extended to those innovators who are most promising in terms of commercialisation and scaling-up. The process for the selection of these innovators should be based on clear criteria and be based on an agreement between the partners. 8. Recommendation: The partners should move forward to integrate gender not only as “gen￾der equity” but as “gender as a social construct” in PAEGC’s selection and M&E processes. For this purpose the newly hired gender consultant at PASTO can be used. The gender concept applied by PAEGC is limited to “gender equity”, instead of seeing gender as a social construct. This does not correspond to the partners’ own gender strategies with a broader understanding of gender, and potentially limits the impact of PAEGC’s interventions on gender main￾streaming. As a consequence, PAEGC should integrate a gender sensitivity analysis into their future selection processes, which not only looks at gender equity but also on how the projects might affect gender roles and norms in society. In addition the M&E concept should be expanded to include a gender analysis during field visits that analyses the projects’ influence on gender norms and struc￾tures in the local context, especially in the targeted value chains. For the development and imple￾mentation of these instruments the newly hired gender consultant at PASTO should be used. Chapter: Annex Page 42 | 66 Annex I. Bibliography AECF, 2015. Impact Report 2014. Lusaka: African Enterprise Challenge Fund (AEFC). Retrieved in May 15th 2016 from http://www.aecfafrica.org/downloads/AECF-2014-Impact-Report.pdf African Bamboo Plc, 2013a. Technical Solution Narrative. Development of an integrated solution for the thermal treatment of agricultural goods based on a demand driven and energy-efficient biomass combustion unit: Final Revised Application (Internal Document). African Bamboo Plc, 2013b. Branding and Marking Plan. Development of an integrated solution for the thermal treatment of agricultural goods based on a demand driven and energy-efficient biomass com￾bustion unit: Final Revised Application (Internal Document). African Bamboo Plc, 2013c. Budget Narrative. Development of an integrated solution for the thermal treatment of agricultural goods based on a demand driven and energy-efficient biomass combustion unit: Final Revised Application (Internal Document). Brandi, C., Richerzhagen, C. and Stepping, K., 2013. Post 2015: Warum ist der Wasser-Energie-Land￾Nexus für die künftige Entwicklungsagenda wichtig? Analysen und Stellungnahmen, Bonn: German Development Institute (DIE). Retrieved on May 7th 2016 from https://www.die-gdi.de/uploads/me￾dia/AuS_8.2013.pdf Daly, H. E., 1990. Toward some operational principles of sustainable development. Ecological Econom￾ics 2:1–6. Danish Ministry of Foreign Affairs (DANIDA), 2006. Evaluation Guidelines. Copenhagen: DANIDA. Dombronwsky, I., 2011. Wasser-Energie-Ernährung – Brauchen wir eine Nexus-Perspektive? Die Bon￾ner Nexus Konferenz. Die aktuelle Kolumne. Bonn: German Development Institute (DIE). Retrieved on May 7th 2016 from https://www.die-gdi.de/uploads/media/Kolumne_Dombrowsky.14.11.2011.pdf FAO, 2009: Global Agriculture towards 2050. High-Level Expert Forum - How to Feed the World 2050, 12-13 October 2009. Retrieved on May 10th 2016 from http://www.fao.org/fileadmin/tem￾plates/wsfs/docs/expert_paper/How_to_Feed_the_World_in_2050.pdf FAO, 2014. The Water-Energy-Food Nexus. A new approach in support of food security and sustaina￾ble agriculture. Rome: Food and Agriculture Organization of the United Nations (FAO). Retrieved on May 7th 2016 from http://www.fao.org/nr/water/docs/FAO_nexus_concept.pdf Fosseli Olsen, E., Fones Sundell, M., Braten, Y., 2016. Gender in the challenge funds Securing Water for Food and Powering Agriculture – Overview, challenges, lessons learned and recommendations. Oslo: International Law and Policy Institute (ILPI). GIZ, 2011. Bonn2011 Conference: The Water, Energy and Food Security Nexus – Solutions for the Green Economy. Eschborn: Konferenz-Sekretariat Internationale Wasser-Politik und Infrastruktur GIZ. GIZ, 2015. Programmvorschlag für ein IZR-Vorhaben. Powering Agriculture - Nachhaltige Energie für Ernährung, PN 2013.6254.0 (Internal Document). GIZ, 2016. Fortschrittsbericht IZR Vorhaben. Nachhaltige Energie für Ernährung, PN 2013.6254.0 (In￾ternal Document). International Development Enterprises, 2013a. Branding Strategy and Marking Plan. Clean Irrigation Solutions (CIS) for Increased Agricultural Productivity: Final Revised Application (FFA) (Internal Doc.) International Development Enterprises, 2013b. Budget Narrative. Clean Irrigation Solutions (CIS) for Increased Agricultural Productivity: Final Revised Application (FFA) (Internal Document). Chapter: Annex Page 43 | 66 International Development Enterprises, 2013c. Response to Technical Clarifying Questions. Clean Irri￾gation Solutions (CIS) for Increased Agricultural Productivity: Final Revised Application (FFA). Mayne, J., 2001. Addressing attribution through contribution analysis: using performance measures sensibly. Canadian Journal of Programme Evaluation, 16, 1-24. Menke, C., Helbig, F., Berdel, F., 2009. IDEAS – Energy Innovation Contest Evaluation of the first round (2009) of winning projects: Extended Executive Summary Report. Washington, DC: Inter-Amer￾ican Development Bank. Morra-Imas, L. & Rist, R. (2009). The Road to Results: Designing and Conducting Effective Develop￾ment Evaluations. Washington D.C., USA. O´Riordan, A.-M., 2014. Triple Line´s Approach to Enhancing Value for Money in Challenge Funds. London: Triple Line Consulting. O´Riordan, A.-M., 2015. Realising the Potential of International Development Challenge Funds: Les￾sons from Practitioners. London: Triple Line Consulting and University of Bath Centre for Development Studies. O´Riordan, A.-M., Copestake, J., Seibold, J., Smith, D., 2014a. Practice Brief 1: What is a Challenge Fund and When Should it be Used? London: Triple Line Consulting and University of Bath Centre for Development Studies. Retrieved on May 24th 2016 from http://www.tripleline.com/news/up￾load/2016/Triple%20Line%20Consulting_Practice%20Brief%201.pdf O´Riordan, A.-M., Seibold, J., and Smith D., 2014b. Practice Brief 2. Designing a Challenge Fund: Im￾portant considerations. London: Triple Line Consulting and University of Bath Centre for Development Studies. Retrieved on May 24th 2016 from http://www.tripleline.com/news/upload/2016/Tri￾ple%20Line%20Consulting_Practice%20Brief%202.pdf PAEGC, 2013a. Memorandum of Understanding. Powering Agriculture: An Energy Grand Challenge for Development (Internal Document). PAEGC, 2013b. Amendment to Memorandum of Understanding (Internal Document). PAEGC, 2013c. Component 1 - Application Results (Internal Document). PAEGC, 2014. Powering Agriculture: An Energy Grand Challenge for Development Work Plan (Internal Document). PAEGC, 2015a. Annual Report Financial Year 2014. Washington, DC: Powering Agriculture: An Energy Grand Challenge for Development (Internal Document). PAEGC, 2015b. PAEGC Partners’ Meeting Main Points, Key Decisions, & Unresolved Issues – Partners Meeting June 18th, 2015 (Internal Document). PAEGC, 2015c. Global Innovation Call Round 2 (GICR2) Application Results (Internal Document). PAEGC, 2016a. Annual Report Financial Year 2015. Washington, DC: Powering Agriculture: An Energy Grand Challenge for Development (Internal Document). PAEGC, 2016b. PoweringAg MOOC “Sustainable Energy for Food” – Evaluation (Internal Document). PAEGC, n.d. Scope of Work: Gender Integration Specialist - Powering Agriculture Support Task Order (Internal Document). PASTO, 2015a. PASTO Activity Tracker as of April 30, 2015 (Internal Document). PASTO, 2015b. Pax/Pais Summary Report. Arlington, VA, 16th –19th November 2015 and Washing￾ton, DC, 20 November 2015 (Internal Document). PASTO, 2016. Powering Agriculture: An Energy Grand Challenge For Development. Mid-Term Innova￾tors´Assessment (Internal document). Chapter: Annex Page 44 | 66 Promethean Power Systems, 2013a. Branding and Marking Strategy. Reducing Milk Spoilage through Solar-Powered Chilling: Final Revised Application (FFA) (Internal Document). Promethean Power Systems, 2013b. Budget Narrative. Reducing Milk Spoilage through Solar-Powered Chilling: Final Revised Application (FFA) (Internal Document). Promethean Power Systems, 2013c. Response to Technical Clarifying Questions. Reducing Milk Spoil￾age through Solar-Powered Chilling: Final Revised Application (FFA) (Internal Document). Richardson, L., Smith, D., Blundy, C., 2015. Specialist Evaluation and Quality Assurance Service. Sheffield: International Organisation Development (IOD PARC). Retrieved on May 25th 2016 from http://www.tripleline.com/downloads/publications/Review-Umbrella-Fund-Evaluation-Challenge￾Funds.pdf SIDA, n.d. Presentation on Gender in Challenge Funds: Examples from Powering Agriculture Energy Grand Challenge (PAEGC) and Securing Water for Food (SWFF) (Internal Document). SunDanzer Refrigeration, 2013a. Budget Narrative. Solar-Powered Refrigeration for Dairy Farms: Final Revised Application (FFA) (Internal Document). SunDanzer Refrigeration, 2013b. Technical Solution Narrative. Solar-Powered Refrigeration for Dairy Farms: Final Revised Application (FFA) (Internal Document). Triple Line Consulting and Crown Agents, 2015a. Learning from the Civil Society Challenge Fund: Pre￾dictive Modelling. London: Triple Line Consulting and Crown Agents. Retrieved on May 24th 2016 from https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/497363/learning￾brief-predictive-modelling.pdf Triple Line Consulting and Crown Agents, 2015b. Learning from the Civil Society Challenge Fund: Nat￾ural Resource Governance and Climate Change. London: Triple Line Consulting and Crown Agents. Re￾trieved on May 24th 2016 from https://www.gov.uk/government/uploads/system/uploads/attach￾ment_data/file/497355/learning-brief-beneficiary-natural-resource-governance-climate-change.pdf Triple Line Consulting and Crown Agents, 2015c. Learning from the Civil Society Challenge Fund: Ben￾eficiary Feedback Mechanisms. London: Triple Line Consulting and Crown Agents. Retrieved on May 24th 2016 from https://www.gov.uk/government/uploads/system/uploads/attach￾ment_data/file/497356/learning-brief-beneficiary-feedback-mechanisms.pdf University of Georgia Research Foundation (UGARF), 2013a. Branding Strategy and Marking Plan. Bio￾gas-Powered Evaporative Cooling for the Dairy Industry: Final Revised Application (FFA) (Internal Document). University of Georgia Research Foundation (UGARF), 2013b. Budget Narrative. Biogas-Powered Evap￾orative Cooling for the Dairy Industry: Final Revised Application (FFA) (Internal Document). University of Georgia Research Foundation (UGARF), 2013c. Response to Clarifying Questions. Biogas￾Powered Evaporative Cooling for the Dairy Industry: Final Revised Application (FFA) (Internal Docu￾ment). USAID, 2012. Broad Agency Announcement (BAA) for Powering Agriculture: An Energy Grand Chal￾lenge for Development. USAID, 2014. Broad Agency Announcement (BAA) for Powering Agriculture: An Energy Grand Chal￾lenge for Development (PAEGC), Second Global Innovation Call. USAID, 2015a. Cam Report No. 02-015.020 (Internal Document). USAID, 2015b. Powering Agriculture Support Task Order. Third Year Work Plan: October 2015 to Sep￾tember 2016. Washington, DC: US Agency International Development (USAID) (Internal Document). USAID, n.d. End of Communication Campaign Report (Internal Document). Chapter: Annex Page 45 | 66 II. Overview of all Conducted Interviews Name Institution Function/Position Date Shelby, Ryan USAID Fund Manager 27.04.2016 / 25.05.2016 Foster, Jeremy USAID Energy Advisor/PAEGC AOR 23.05.2016 Clark, Emily USAID Energy Advisor/PAEGC AOR 25.05.2016 Hansen, Lexine USAID Senior Policy Advisor Global Climate Change Initiative 23.05.2016 Ferguson, David USAID U.S. Global Development Lab 20.05.2016 McMahan, Ku USAID SWFF Program Manager; U.S. Global Development Lab 24.05.2016 Bonnell, Alexis USAID U.S. Global Development Lab 27.05.2016 Macleod, Robert USAID Office of Engineering 24.05.2016 Ngugi, Moffatt K. USAID Bureau for Food Security 31.05.2016 Beeler, Amy USAID Feed the Future 03.06.2016 Iyob, Biniam USAID Water and Irrigation Advisor 27.05.2016 Blanchard, Jeannelle PASTO Chief of Party 25.05.2016 Rodhe, Frida Sida Programme Officer, Department for International Organisations and Policy Support 25.04.2016 / 27.05.2016 Jonsson Cissé, Kerstin Sida Head of Unit, Global Economy and Environment 23.05.2016 Garman, Roger Sida Coordinator, Loans & Guarantee Unit; former programme officer 27.05.2016 Wahlen, Gisela BMZ Desk Officer 17.05.2016 Weitz, Maria GIZ Head of Project Sustainable Energy for Food – Powering Agriculture 21.04.2016 / 27.05.2016 Fleischer, Gerd GIZ Head of Unit Agricultural Innovations, Sustainability Standards 17.05.2016 Jakobeit, Laura GIZ Advisor Sustainable Energy for Food – Powering Agriculture 25.05.2016 Ghose, Nilanjan GIZ Technical Expert Indo-German Energy Programme, GIZ India 23.05.2016 Chapter: Annex Page 46 | 66 Name Institution Function/Position Date Meder, Katharina GIZ Coordinator Regional Hub East Africa Sustainable Energy for Food – Powering Agriculture 19.05.2016 Messinger, Christoph GIZ/EnDev Coordinator/Country Manager 18.05.2016 Zymla, Bernhard GIZ Leader of Competence Center Energy and Transport 24.05.2016 Flammini, Alessandro FAO Desk Officer for Natural Resources 31.03.2016 Richardson, Lydia Triple Line Director 20.05.2016 Admetlla, Ignacio Fernandez IADB Multilateral Investment Fund 26.05.2016 Freeman, Katie World Bank Agriculture Economist at Department for Global Agriculture Practice 08.06.2016 Chapter: Annex Page 47 | 66 III. List of Analysed Challenge Funds Fund name Sector Funding Partners Website / Affiliated blog Fund Manager Comments Additional Support to Civil Society Organisations Civil Society United Kingdom Department for International Development (DfID) no information available Triple Line Consulting Ltd., Crown Agents Fund no longer active Afghanistan Business Innovation Fund (ABIF) Private Sector Development, Agriculture United Kingdom Department for International Development (DfID), Australia's Department of Foreign Affairs and Trade (DFAT) http://www .itad.com/projects/fund-manager￾of-the-afghanistan-business-innovation-fund￾abif/ itad, Landel Mills Ltd. Fund no longer active Africa Enterprise Challenge fund Agri-business, Financial services, Information services, Climate change adaptation, Renewable energy United Kingdom Department for International Development (DfID), Australia's Department of Foreign Affairs and Trade (DFAT), Ministry of Foreign Affairs of the Netherlands (MFA), International Fund for Agricultural Development (IFAD), Consultative Group to Assist the Poor (CGAP), Danish International Development Agency (Danida), Swedish International Development Cooperation Agency (Sida) http://www .aecfafrica.org/ KPMG International Development Advisory Services Africa Agricultural Technology Transfer Research Challenge Fund Agricultural Technology United Kingdom Department for International Development (DfID) https://agritt.landellmillsprojects.com/ Landel Mills Ltd. Arab Partnership Economic Facility Economic Governance, Private Sector Development, Social Entrepreneurship United Kingdom Department for International Development (DfID), Foreign & Commonwealth Office (FCO) https://www .gov.uk/guidance/arab￾partnership DFID Arab Partnership Participation Fund Democratic Governance United Kingdom Department for International Development (DFID), Foreign & Commonwealth Office (FCO) https://www .gov.uk/guidance/arab￾partnership no information available Ashden International Awards Energy Christian Aid, Citibank, Eurostar, Impax, The World Bank, United Kingdom Department for International Development (DfID) http://www .ashden.org/ no information available no current information available Bangladesh Health Innovation Challenge Fund Healthcare United Kingdom Department for International Development (DfID) http://www .hicfund.org.uk/ GRM International Bangladesh Rights and Goverance Challenge Fund Civil Society, Basic needs United Kingdom Department for International Development (DfID) https://devtracker.dfid.gov.uk/projects/GB-1- 113976 Manusher Jonno Foundation Fund no longer active Business Innovation facility Private Sector Development United Kingdom Department for International Development (DfID) http://www .bifprogramme.org/ no information available Fund no longer active Business Sector Advocacy Challenge Fund (BUSAC) Private Sector Development, Agri-business Danish International Development Agency (Danida), United States Agency for International Development (USAID), European Union (EU) http://www .busac.org/ Mr. Nicolas Jørgensen Gebara (BUSAC) Canada Fund for African Climate Resilience Environment, Adapting to climate change Canadian International Development Agency (CIDA) http://www .international.gc.ca/development￾developpement/partners-partenaires/calls￾appels/climate-resilience￾climatique.aspx?lang=eng no information available Children and Violence Evaluation Challenge Fund Evaluation of violence prevention / child protection Network of European Foundations (NEF) http://www .evaluationchallenge.org/ no information available Civil society challenge fund (CSCF) Civil Society United Kingdom Department for International Development (DfID) https://www .gov.uk/guidance/civil-society￾challenge-fund Fund no longer active Civil Society Support Programme Civil Society Irish Aid, Canadian Department for Foreign Affairs, Trade and Development, Ministry of Foreign Affairs of the Netherlands (MFA), Norwegian Agency for Development Cooperation (NORAD), Swedish International Development Cooperation Agency (Sida), United Kingdom Department for International Development (DfID), British Council, International NGO Training and Reasearch Centre (INTRAC) http://cssp-et.org/ British Council, INTRAC Climate and Development Knowledge Network Innovation Fund Climate change, Disaster risk management United Kingdom Department for International Development (DfID), Ministry of Foreign Affairs of the Netherlands (MFA) http://cdkn.org/about/who-we-are/innovation￾fund/?loclang=en_gb no information available climDev Special Fund (CDSF) Climate resilient development African Development Bank (AfDB), Swedish International Development Cooperation Agency (Sida), African Union Commission, UN Economic Commission for Africa http://www .climdev-africa.org/The-ClimDev￾Special-Fund African Development Bank (AfDB) Construction Ideas Fund Construction, Real estates United Kingdom Department for International Development (DfID) https://www .gov.uk/guidance/construction￾ideas-fund Coffey International Development Chapter: Annex Page 48 | 66 Fund name Sector Funding Partners Website / Affiliated blog Fund Manager Comments Cooperative Facility for Africa Challenge Fund Environment, social development, employment International Labour Organisation (ILO) http://www .ilo.org/public/english/employment /ent/coop/africa/areas/challenge.htm COOPAfrica Fund no longer active Development Awareness Fund Civil Society United Kingdom Department for International Development (DfID) no information available no information available Fund no longer active Development Innovation Ventures (DIV) Universal (17 sectors: "Development Challanges") United States Agency for International Development (USAID) https://www .usaid.gov/div/about no information available Emprender Paz Private Sector Development, Peacebuilding Swedish International Development Cooperation Agency (Sida) http://emprenderpaz.org/ no information available Enterprise Challenge Fund Private Sector Development Australia's Department of Foreign Affairs and Trade (DFAT) http://www .enterprisechallengefund.org/ Coffey International Development Fund no longer active Enterprise Innovation Challenge Fund Private Sector Development Inter-American Development Bank (IDB), United Kingdom Department for International Development (DfID), Canadian International Development Agency (CIDA) http://competecaribbean.org/program/enterp rise-innovation-challenge-fund/ Compete Caribbean Financial Education Fund Financial Education United Kingdom Department for International Development (DfID) no information available Genesis Ansalytics and Cardno Emerging Markets group Food Retail Industry Challenge Fund (FRICH) Agri-business, Food Export United Kingdom Department for International Development (DfID) https://www .gov.uk/guidance/food-retail￾industry-challenge-fund-frich Nathan Associates London Girls' Education Challenge Fund Education United Kingdom Department for International Development (DfID) https://www .gov.uk/guidance/girls-education￾challenge PwC Global Poverty Action Fund (GPAF) Poverty Reduction United Kingdom Department for International Development (DfID) https://www .gov.uk/guidance/global-poverty￾action-fund-gpaf Triple Line Consulting Ltd.,Crown Agents Fund no longer active Governace and Transperancy Fund human rights, natural resource management United Kingdom Department for International Development (DfID) https://www .gov.uk/guidance/governance￾and-transparency-fund-gtf KPMG Development Services Ltd Fund no longer active Grand Challenges Canada Health, Economy Government of Canada http://www .grandchallenges.ca/ no information available Health Enterprise Fund Health United States Agency for International Development (USAID), United Kingdom Department for International Development (DfID) http://www .healthenterprisefund.org/ Abt Associates (April Warren) Human Development Innovation Fund for Tanzania Health, Education, Water and Sanitation United Kingdom Department for International Development (DfID) http://www .hdif-tz.org/ KPMG International Development Advisory Services Africa Humanitarian Innovation Fund Humanitarian Assistance (Water and sanitation, Gender Based Violence) United Kingdom Department for International Development (DfID), Dutch Ministry of Foreign Affairs, Swedish International Development Cooperation Agency (Sida) http://www .elrha.org/hif/home/ Elrha Humanitarian Innovation Initiative Humanitarian Assistance (Health, Disaster Risk Management, Water and Sanitation, Settlements) United States Agency for International Development (USAID), United Kingdom Department for International Development (DfID) https://www .usaid.gov/div/humanitarian￾initiative no information available Initiative supports winners of the Development Innovation Ventures (DIV) competition Innovation Against Poverty Poverty reduction, Private Sector Development Swedish International Development Cooperation Agency (SIDA) http://www .sida.se/iap no information available Innovation Fund for the Americas Energy, Climate Change, Education, Democracy and Rule of Law , Health, Education United States Agency for International Development (USAID), Bill and Melinda Gates Foundation https://www .usaid.gov/news-information/fact￾sheets/innovation-fund-americas no information available part of USAID's Development Innovation Ventures (DIV) JP1 and JP5- CoopAfrica Challenge Funds Civil Society, Rural Development, Agriculture UN Interagency Selection Committees composed of FAO, ILO, UNIDO, UNIFEM http://www .ilo.org/public/english/employment /ent/coop/africa/areas/jps.htm no information available Land: Enhancing Governance for Economic Development (LEGEND) Land Governance United Kingdom Department for International Development (DfID) http://www .landportal.info/partners/legend/k pmg-challenge-fund KPMG Latin America Impact Economy Innovations Fund Social Entrepreneurship, Energy , Extractive Industries Fundación Avina, Avina Americas, Omidyar Network, Rockefeller Foundation no information available no information available no current information available Making all Voices Count - a grand Challenge for Developmemt Governance, Civil Society, Health, ICT United Kingdom Department for International Development (DfID), the Government of Sweden, United States Agency for International Development (USAID), Omidyar Network http://www .makingallvoicescount.org/ no information available Malawi innovation challenge fund (MICF) Agriculture, Manufacturing United Nations Development Programme (UNDP), United Kingdom Department for International Development (DfID) http://micf.mw/content/home Navin Kumar (MICF) Chapter: Annex Page 49 | 66 Fund name Sector Funding Partners Website / Affiliated blog Fund Manager Comments mFarmer Initiative Challenge Fund Agriculture, ICT Bill & Melinda Gate Foundation, United States Agency for International Development (USAID) http://www .gsma.com/mobilefordevelopment /faq/mfarmer-initiative Coffey International Development mWomen Innovation Fund Technology, womens right (social development) United States Agency for International Development (USAID), Australia's Department of Foreign Affairs and Trade (DFAT), Visa http://www .gsma.com/mobilefordevelopment /wp-content/uploads/2014/06/GSMA- mWomen-Grant-summary-EXTERNAL-VERSION￾Updated4July2014.pdf Coffey International Development Responsible and Accountable Garment Sector (RAGS) Challenge Fund Working conditions in garment sector United Kingdom Department for International Development (DfID) https://www .gov.uk/international￾development-funding/responsible-and￾accountable-garment-sector-challenge-fund Maxwell Stamp PLC Fund no longer active Rights, Democracy and Inclusion Fund (RDIF) Governance, Civil Society United Kingdom Department for International Development (DfID), Australian Agency for International Development (AusAID), Danish International Development Agency (Danida), HUGOU, United Kingdom Department for International Development (DfID), Swiss Agency for Development and Cooperation (SDC) no information available DFID Nepal Fund no longer active Sawaed Programme Private Sector Development, ICT Mohammed bin Rashid Al Maktoum Foundation no information available no information available no current information available Securing Water for Food (SWFF) Water, Food USAID, Swedish International Development Cooperation Agency (SIDA), The Minstry of Foreign Affairs of the Netherlands, Department of Science and Technology of Sout Africa http://securingwaterforfood.org/ The Kaizen Company Shell Springboard Low-carbon economy, Energy Shell http://www .shellspringboard.org/ no information available Shiree/Economic empowerment of the poorest Programme (EEP) Poverty reduction United Kingdom Department for International Development (DfID), Swiss Development Cooperation (SDC) http://r4d.dfid.gov.uk/Project/60869/ ECORYS Fund no longer active Sightsavers Innovation Fund Health, Education Sightsavers, United Kingdom Department for International Development (DfID) http://blog.sightsavers.org/subject/innovatio n-fund/ no information available Vietnam Business Challenge Fund (VBCF) Inclusive Business, Agriculture, Low Carbon Growth, Basic Needs United Kingdom Department for International Development (DfID) http://www .vbcf.org.vn/ SNV Chapter: Annex Page 50 | 66 IV. Terms of Reference 1. Background of PAEGC The Powering Agriculture: An Energy Grand Challenge for Development (PAEGC) is a partnership be￾tween USAID, the Government of Sweden, the Government of Germany, Duke Energy Corporation and the Overseas Private Investment Corporation. The goal of PAEGC is to support new and sustainable approaches to accelerate the development and deployment of clean energy solutions23 for increasing agriculture production and/or value in developing countries. Expected outcomes of PAEGC: • Farmers’ and agribusinesses’ access to and/or use of clean energy solutions have increase. • Agricultural production and/or value among farmers and agribusinesses have increased. • Support for low carbon economic growth within the agriculture sector has increased. Impact indicators of PAEGC: • Type and number of clean energy solutions developed • Number/type of beneficiaries (farmers/agribusinesses/ customers) with improved energy services due to PAEGC assistance • Type and number of wholesalers/retailers/ maintenance professional accessible to benefi￾ciaries for selling/servicing clean energy solutions • Clean energy generation capacity installed or rehabilitated as a result of PAEGC assistance • Number of persons attending trainings/demonstrations on CES technology • Change in agriculture production attributed to use of PAEGC Innovators’ clean energy solu￾tion • Change in income attributed to use of PAEGC Innovators’ clean energy solution • Expected life of project savings from energy efficiency or energy conservation, as a result of PAEGC assistance • Amount of investment mobilised in U.S. dollars, from private and public sources, for climate change as a result of PAEGC assistance • Greenhouse Gas (GHG) emissions, estimated in metric tons of CO2e, reduced, sequestered, and/or avoided as a result of PAEGC assistance 2. Purpose of the Midterm Evaluation The PAEGC Monitoring and Evaluation Plan stated that PAEGC would undergo a performance evaluation and at the 2015 PAEGC Partners meeting the Partners agreed to move forward on a midterm perfor￾mance evaluation. It is desirable to undertake a midterm evaluation at this point, as a second set of innovators have been chosen who will be initiating their activities. The mid-term evaluation will also be used to take informed decisions for improving the structure and strategy of the fund and fund manage￾ment for the future. The midterm performance evaluation will be conducted in two stages: the Program-level stage and the Innovator-level stage. • Program-level: measures the aggregated contributions of PAEGC’s interventions in achieving the program’s Goal. This is also referred to as the GCD-level. 23 Clean energy is defined as: Usable energy (i.e. electricity, illumination, heating/refrigeration, mechanization) that is derived from renewable sources and supports a reduction in fossil fuel use, increase in efficiency, and/or limitation of greenhouse gas emissions. Clean energy sources include – solar, hydro, wind, geothermal, sustainably harvested biomass, and biogas. The term “clean energy solution” is defined as: A combination of appropriate technology and a business model that addresses the clean energy demands of a select market. Chapter: Annex Page 51 | 66 • Innovator-level: measures the progress and impact of PAEGC’s individual Innovators, and individual contributions to achieving the PAEGC goal. The Program-level evaluation will be conducted by an external consultant while the Innovator-level will be conducted by the M&E Specialist of the Powering Agriculture Support Task Order (PASTO) who will review data, interview the 2013 innovators and PASTO staff, draw conclusions and set forth recommen￾dations and lessons learned. This evaluation has begun in February, 2016. Results of both evaluations will feed in to the meta-evaluation. 3. Program-Level Evaluation Based on the Innovator-level evaluation the Program- and Meta-level evaluation will be conducted by an external consultant, with the purpose to analyse • the achievement of PAEGC to accelerate innovative CES for increasing agricultural produc￾tions and/or value in terms of the OECD DAC criteria (relevance, effectiveness, impact, efficiency and sustainability) • the effectiveness of PAEGC’s 4 components (Technology and Business Model Innovation Component; Commercial Financing Component; Mainstreaming and Scaling Component; Knowledge Management Component) to accelerate innovative CES for increasing agricul￾tural productions and/or value; • the potential of “call for proposals” /contest to support innovations; • the future potential for PAEGC in regards to the international agenda (climate financing, SDGs). The information gathered will be beneficial in improving the ongoing program’s implementation, con￾tributing to evidence-based decision making for future intervention design, and identifying where and how resources can most effectively be allocated to reach PAEGC’s goal. Further, it will inform the PAEGC Partners about effectiveness of the Grand Challenge model and develop recommendations for future management models, including opportunities provided by the international agenda. A second aspect of the midterm evaluation concerns the potential of PAEGC as a Grand Challenge for Development to address gender equality and to strengthen gender mainstreaming. In this regard, the evaluation should investigate how well PAEGC has integrated gender aspects on the program level in its calls for proposals, its management of the awards, and its support to the innovators. The evaluation should also include technical aspects, mainly based on the Innovator-level evaluation conducted by PASTO, regarding • the evaluation of procurement and award process by PAEGC (from the viewpoint of the Founding Partners and the Innovators), • the evaluation of award management by PAEGC during implementation, and • the evaluation of Innovators’ achievements. 4. Methodology The Program-level evaluation is a desk study, including interviews with PAEGC Partners, Innovators, PASTO, as well as analysis of the data from the field visits and evaluation on Innovator-level. In the following the most important questions are outlined as well as data sources and suggestions for data analysis methods. 4.1 Data Collection Methods - Document Review - Data Analysis/Indicator Review - Interviews with USAID Program/AORs Chapter: Annex Page 52 | 66 - Interview with PASTO CES - Interviews with innovators (grantees) - Annual Performance Indicator review 4.2 Data Sources - Project documents - USAID Program Managers overview/orientation - GIZ Program Managers - Sida Program Manager/s - PAEGC Partner Representatives - AORs - PASTO staff - Innovator interviews - Annual Performance Indicator reports 4.3 Data Analysis Methods - Qualitative analysis of interview responses, document review findings - Quantitative analysis of indicator results - Structured survey results, statistical data - Quantitative analysis of structured survey results, statistical data 4.4 Questions for the Program-Level Evaluation In connection with the overall evaluation (based on interviews and document review) the external con￾sultant will answer the following questions based on his/her work: A. Overall Impact 1. What is the magnitude of the effect that the program has had so far in achieving the intended goal of the program? (Incl. evaluation of achievements in regards to the three Intermediate Results stated above) 2. How much of the measured change in outcome or result indicator can be attributed to Im￾plementing Partner financial support or other PAEGC interventions? That is, what portion of the result is not explained by the intervention examined by the impact evaluation? 3. To what extent did the impact evaluation find differences between the planned intervention and what was actually delivered? 4. What were some of the unintended outcomes (positive and negative) from the GCD? Were any of these external to the GCD? 5. Were Clean Energy Solutions (CES) adopted by farmers and agribusinesses and utilised at scale? 6. How many viable CES have been sourced? How many have progressed through the innova￾tion pipeline? (steps of 1. piloting, 2. small-scale adoption, and 3. wide-scale adoption) 7. How many consumers/households/others benefitted adopting PAEGC-supported solutions? 8. Focus on poor: Do the innovators have adequate knowledge of the local situation, livelihoods (m/f), envisaged benefits to the target group (m/f) and sustainability of these benefits? 9. To what degree is there demand and local ownership for the innovations? Chapter: Annex Page 53 | 66 10. To what degree do vulnerable groups in societies benefit (pro-poor focus, inclusiveness, gender) (income, employment, water)? 11. How many of the Innovators have measurable positive change in results addressing the specific development challenge? 12. To what degree are the innovations likely to be sustainable? 13. To what degree is environmental sustainability of interventions ensured and followed-up? 14. Was the PGAEC model effective in making progress toward overcoming critical barriers to development problems? What was the development impact? 15. Were there some broad implementation barriers evident in the PAEGC? 16. What were some questions that should have been asked during the program that were not? B. PAEGC Structure 17. Was PAEGC integrated enough with the GCD to maximise synergies and lessons learned? 18. What is the level of success in deploying activities under the Four Technical Components? 19. Is PAEGC structured in an efficient manner to give appropriate support to awardees, take informed strategic decisions and deliver on the expected results? 20. Did PAEGC take appropriate steps to source high-risk, high reward solutions? C. Call for Proposals 21. 21. Did PAEGC select appropriate Innovators? a. Should existing application criteria be prioritised? b. Should additional application criteria be employed or some criteria be excluded? 22. What are the lessons learned with regards to the call? 23. Does the Innovators’ overall progress toward viable CESs reflect the selection process and Innovators chosen? D. Assistance to Innovators 24. What are some of the broad implementation barriers that Innovators have faced? Has PAEGC been able to support overcoming these barriers? 25. What are the reasons for the challenges or failures? 26. Has PAEGC been able to support overcoming these barriers? 27. What were some of the external barriers faced by the Innovators that were not foreseen or addressed by the GCD program? 28. Have the PAEGC Partners leveraged their in-country presence to assist and complement the activities of the Innovators? How was this done? F. Gender 29. Did PAEGC take appropriate steps to integrate gender aspects in the program? 30. Have the Innovators implemented any specific gender activities? Has PAEGC supported them in this regard? Additional questions may be introduced by the consultant, which may generate a more complete un￾derstanding of PAEGC’s performance. Chapter: Annex Page 54 | 66 5. Assignment The assignment will run from April 15 to June 10, 2016. This evaluation will be carried out over 35 person-days, including data collection and analysis as well as drafting a final report and finalizing it. The first draft will be submitted on May 25, 2016 and once comments are received from the PAEGC Partners, the consultant will finalize the report and submit the final by June 10, 2016. Two meetings should be scheduled: 1) A kick-off meeting in Bonn at GIZ mid-April, and 2) a presentation at the PAEGC Partners Meeting in Washington D.C. between June 1st and 3rd. General Requirements For the project evaluation, the consultant must ensure that the following required inputs are in place: • sectoral (content-specific) skills; • evaluation expertise; • methodological expertise (strategy development, moderation skills, the ability to deal with different expectations and reconcile diverse interests); • country specific/region specific expertise. Final Report Contents The final report will set forth important findings, conclusions, and recommendations. It will include an executive summary and be no more than 30 pages, not including tables, charts and annexes. It will contain the following sections: A. Introduction B. Program-level findings (covering all 4 PAEGC components) C. Conclusions D. Lessons Learned E. Recommendations - for the management of PAEGC post 2019 - for the process and design of further calls for proposals - for opportunities to link up to the international agenda (climate financing, SDGs etc.) Contact: Laura Jakobeit (laura.jakobeit@giz.de) Advisor, Powering Agriculture – Sustainable Energy for Food Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbH Chapter: Annex Page 55 | 66 Annex: Quality Requirements for Project/Programme Evaluations In order to safeguard quality at GIZ, it is important that analyses, evaluations and recommendations from evaluations are based on reliable and transparent methods. You must therefore comply with the following standards: Plausible Attribution of Results to the Programme The overarching quality requirement of any evaluation is that it delivers reliable findings with regard to how the changes that occur can be attributed to the project's measures (interventions). What is the relationship between the two? You must distinguish between (a) results brought about by the program, (b) results to which the program contributed and (c) results that were achieved without the program’s involvement. This means that it is important not to interpret all connections observed between measures and results as causal links. Systematic Analysis A high-quality evaluation is based on a precisely defined, clearly delineated object of evaluation. This delineation is reflected in the corresponding excerpt of the results logic for the project being evalu￾ated/appraised. It must be clear what indicators are used to assess the effectiveness and the overarch￾ing development results (impact). In order to ensure that it is possible to evaluate the project, the indicators must be Specific, Measurable, Achievable, Relevant and Time-bound (SMART). If they are multidimensional (not specific), for example, you will need to reformulate them, making sure that the requirements of the project's objectives system are not increased or decreased in the process. All assessments are based on an existing description and previous analysis. The description and the analysis are underpinned with relevant facts and figures. Sources are to be provided for all facts and figures. The assessment of the project must be fair. In other words, its strengths and weaknesses must be documented as completely as possible and presented in a balanced manner. Recommendations are derived from the analysis, addressing key users in whose area of responsibility they can be implemented. Appropriateness of the Evaluation Design and of Data Collection and Assessment The chosen design should be tailored to the programme to be evaluated and to the evaluation questions. Reasons for its selection are to be provided. It should allow only the data that are actually required to be collected, in line with the object of evaluation and the evaluation questions. Each method has strengths and weaknesses. These are to be explained, and ways of handling the weaknesses are to be described. Discussion partners and random samples are to be selected using clearly defined, transparent criteria. Robust Findings Data/information on the same situation are/is collected by involving different actors (data triangulation). Information on the same situation is collected using different data collection methods (method triangu￾lation). Chapter: Annex Page 56 | 66 V. Assessment Grid Desk Research Interviews with Partners Interviews with other Initiatives & Nexus Experts Analysis of Data provided by PASTO & WebMo To what extent do the chosen innovators have adequate knowledge of the local situation, and livelihoods of the target group (m/f) Innovators' knowledge of local context 1. Number of innovators who can describe the target group with regard to a) gender ratio b) average income level c) sources of livelihood x x Local demand for innovations 1. Number of innovators who have conducted a needs assessment/market analysis x x Local ownership of innovations 1. Number of field report of the innovators which reflect upon the local ownership of their beneficiaries 2. Qualitative assessment of the importance of local ownership by the partners for the initiative x x x To what extent was the desired target group reached? Target group 1. Number of innovators differentiated by a) applicants from industrial countries b) applicants from developing countries c) for-profit organisations d) not-for-profit organisations 2. Qualitative assessment of the poverty level of the reached target group by the innovators 3. Number of vulnerable target groups reached that have benefitted from the innovations a) women b) others x x To what extent do the innovative Clean Energy Solutions meet the current state-of-the-art within the relevant country or development context? Scientific Quality of Clean Energy Solutions 1. Number of interviewed experts that deem the developed CES as innovative in the context in which it is developed or implemented x x x x What is the initiative's significance in the field of development cooperation? Relation to other development initiatives 1. Qualitative description and assessment of the initiatives' objectives and portfolio 2. Qualitative description and assessment of the initiatives' contribution to the GCD-Level 3. Qualitative description and assessment of the initiatives' uniqueness vis-á-vis similar initiatives: a) African Enterprise Fund b) Global Innovation Fund c) African Agriculture and Trade Investment Fund 4. Qualitative Assessment of the initiative's contribution to the Sustainable Development Goals (SDGs) x x x x Indicator or Descriptor Sources of Verification Evaluation Criteria Evaluation Questions Level of Analysis To what degree is there demand and local ownership for the innovations? Relevance Chapter: Annex Page 57 | 66 Desk Research Interviews with Partners Interviews with other Initiatives & Nexus Experts Analysis of Data provided by PASTO & WebMo Increase in farmers and agribusinesses' access to and/or use of CES 1. Number of innovators that consider the following support measures of the initiative as crucial for achieving the respective result a) financial support b) advice on business model c) training d) linkages and partnerships c) exchange with other projects on provided communication platforms d) support by in-country representatives of the initiative 2. Qualitative assessment of the contribution of the knowledge management, mainstreaming and acceleration as well as commercial financing components to increase the access of farmers to CES (see line 19-21) x x x Increase in agricultural production and/or value among farmers and agribusinesses 1. Number of innovators that consider the following support measures of the initiative as crucial for achieving the respective result a) financial support b) advice on business model c) training d) linkages and partnerships c) exchange with other projects on provided communication platforms d) support by in-country representatives of the initiative 2. Qualitative assessment of the contribution of the knowledge management, mainstreaming and acceleration as well as commercial financing components to increase the agricultural production and/ or value among farmers and agribusiness (see line 19-21) x x x Increase in support for low carbon economic growth within the agriculture sector 1. Number of innovators that consider the following support measures of the initiative as crucial for achieving the respective result a) financial support b) advice on business model c) training d) linkages and partnerships c) exchange with other projects on provided communication platforms d) support by in-country representatives of the initiative 2. Qualitative assessment of the contribution of the knowledge management, mainstreaming and acceleration as well as commercial financing components to increase support for low carbon economic growth (see line 19-21) x x x To what extent are their differences between the planned intervention on the level of the initiative and what was actually delivered? Differences in Theory of Change vs. actual results 1. Qualitative comparison of the theory of change and the a) implemented activities b) achieved outputs c) achieved outcomes x x Indicator or Descriptor Sources of Verification Evaluation Criteria Evaluation Questions Level of Analysis How much of the measured change in outcome or result indicator can be attributed to Implementing Partner financial support or other PAEGC interventions? Effectiveness Chapter: Annex Page 58 | 66 Desk Research Interviews with Partners Interviews with other Initiatives & Nexus Experts Analysis of Data provided by PASTO & WebMo How many viable CES have been sourced? 1. Type and number of clean energy solutions developed x x x How many CES have progressed through the innovation pipeline? 1. Number of CES in each step within the innovation pipeline a) piloting b) small-scale adoption c) wide-scale adoption x x x Food Security 1. Change in agriculture production attributed to use of PAEGC Innovators' clean energy solution 2. Qualitative assessment of the contribution of the knowledge management, mainstreaming and acceleration as well as commercial financing to overcome developmental challenges in the field of food security (see line 19-21) x x x x Climate Change 1. Expected life of project savings from energy efficiency or energy conservation, as a result of PAEGC assistance 2. Greenhouse Gas (GHG) emissions, estimated in metric tons of CO2, reduced, sequestered, and/or avoided as a result of PAEGC assistance 3. Qualitative assessment of the contribution of the knowledge management, mainstreaming and acceleration as well as commercial financing components to overcome developmental challenges in the field of climate change (see line 19-21) x x x x What were some of the unintended outcomes (positive and negative) from the GCD? Indicator or Descriptor Sources of Verification Evaluation Criteria Evaluation Questions Level of Analysis Developed Clean Energy Solutions (CES) To what extent was the PAEGC model effective in making progress toward overcoming critical barriers to development problems? Effectiveness This question will be answered through the synthesis of the results Chapter: Annex Page 59 | 66 Desk Research Interviews with Partners Interviews with other Initiatives & Nexus Experts Analysis of Data provided by PASTO & WebMo Technology and business model innovation 1. Number of applicants in relation to funded innovations 2. Amount of grant funding dispersed 3. Number of innovations funded 4. Ratio of innovators that have requested advice and received it in comparison to innovators that have requested advice and not received it, with regard to a) milestones b) business models c) referrals, linkages, partnerships, connections d) award modifications e) compliance with USAID policies and procedures x x x Commercial financing 1. Existence of a financing mechanism to scale up innovations (Global Financing Facility - LIFT) 2. Number of cooperation discussions with private sector institutions to host the facility x x Mainstreaming and acceleration 1. Existence of PPP component 2. Number of persons trained a) by innovators b) by PAEGC (e.g. the MOOC , training calls by GIZ, etc.) x x x Knowledge management 1. Experiences from Innovators shared at events in form of a) concepts b) technologies c) business models 2. Experiences from Innovators disseminated by partner organizations in developing and emerging countries 3. Number of visits/followers/members on the communication platforms a) poweringag.org b) Energypedia Wiki c) @Poweringag Twitter Account 4. Level of dissemination of research studies and the number of times a study was downloaded or given to people x x x Efficiency How much public and private investment was the initiative able to leverage? Investment 1. Amount of investment mobilized in U.S. Dollars, from private and public sources, for climate change as a result of PAEGC assistance 2. Leverage ratio of the initiative in comparison to other initiatives: a) African Enterprise Fund b) Global Innovation Fund c) Energising Development d) African Agriculture and Trade Investment Fund x x x Indicator or Descriptor Sources of Verification Evaluation Criteria Evaluation Questions Level of Analysis What is the level of success in deploying activities under the Four Technical Components? Effectiveness Chapter: Annex Page 60 | 66 Desk Research Interviews with Partners Interviews with other Initiatives & Nexus Experts Analysis of Data provided by PASTO & WebMo To what extent were Clean Energy Solutions (CES) developed by the 2013 cohort of innovators adopted by farmers and agribusinesses and utilized at scale? Level of adoption of CES on beneficiaries' level 1. Number of beneficiaries with improved energy services due to PAEGC assistance a) farmers b) agribusinesses c) customers 2. Number of professionals accessible to beneficiaries for selling/servicing clean energy solutions a) wholesalers b) retailers c) maintenance professionals 3. Clean energy generation capacity installed or rehabilitated as a result of PAEGC assistance 4. Number of persons attending trainings/demonstrations on CES technology x x How many consumers/households/others benefitted by adopting PAEGC - supported solutions of the 2013 cohort of innovators? Scope of benefitting target group 1. Number of beneficiaries with improved energy services due to PAEGC assistance a) farmers b) agribusinesses c) customers 2. Change in income attributed to use of PAEGC Innovators' clean energy solution x x How many of the 2013 Innovators have measurable positive change in results addressing the specific development challenge? Impact of innovations on beneficiaries 1. Number of innovators who have contributed at the level of the beneficiaries to a) improved energy services (installed capacity) b) change in income of the beneficiaries c) reduction of greenhouse gases (metric tons) x x To what degree are the innovations likely to be sustainable? Technical and financial sustainability of CES 1. Numbers of innovators that can market their innovation without subsidies (financial sustainability) 2. Number of innovations that can be maintained by the beneficiaries without external technical assistance (technical sustainability) x x To what degree is environmental sustainability of interventions ensured and followed-up? Environmental sustainability of CES 1. Number of innovators that follow-up on the environmental sustainability of their innovations 2. Number of innovations in which the rate of harvest does not exceed the rate of regeneration (if applicable) 3. Number of innovations in which the waste generation does not exceed the assimilative capacity of the environment (if applicable) 4. Number of innovations in which the depletion of non￾renewable resources is accompanied by the development of comparable renewable substitutes (if applicable) x x x Indicator or Descriptor Sources of Verification Evaluation Criteria Evaluation Questions Level of Analysis Impact Sustainability Chapter: Annex Page 61 | 66 Desk Research Interviews with Partners Interviews with other Initiatives & Nexus Experts Analysis of Data provided by PASTO & WebMo Strategy 1. Qualitative description and assessment of options/scenarios a) assessment of their strengths, weaknesses, threats and opportunities b) assessment of the partners' willingness to embrace the options/scenarios c) assessment of their leverage for scaling- up d) assessment of their visibility 2. Qualitative categorization of an ideal partner in terms of a) sharing the same vision for development b) managing for results c) flexibility and innovation d) contractual loyalty x x Cooperation 1. Qualitative description and assessment of the cooperation in terms of a) rule- based power through norms, methods, and roles b) diffusion of power among many actors c) relationship- based power through relationships and dependencies d) concentration of power with individual actors 2. Qualitative description of factors influencing the cooperation a) time to conduct joint processes b) degree of shared understanding of objectives c) communication (transparent, intransparent) d) prioritization of initiative vis- á- vis other tasks e) individual workload 3. Qualitative description of the perceived rules in terms of a) co- production (actors utilize comparative advantages and act on the basis of coordinated strategies and plans) b) strategic alliance (actors act autonomously but in some areas together as a result of expected synergies) c) coordination (actors act autonomously in the knowledge of what the others are doing and anticipate added value) x x Steering Structure 1. Qualitative description and assessment of the used communication channels a) telephone b) regular meetings c) workshops d) ad- hoc exchanges e) web- based tools x x Processes 1. Qualitative description and assessment of the partners in the core processes of the initiative a) application and awarding process b) advice and assistance to the innovators c) evaluation and monitoring process d) dissemination of knowledge e) scaling- up process f) raising of public awareness x x Learning and innovation 1. Qualitative description and assessment of the M&E system a) consistency of the theory of change b) objectives c) indicators d) data collection instruments e) processes to analyse data and share lessons learned x x Struc ture & Imple me nta tion Proc e ss Indicator or Descriptor Sources of Verification Evaluation Criteria Evaluation Questions Level of Analysis Is PAEGC structured in an efficient manner to give appropriate support to awardees, take informed strategic decisions and deliver on the expected results? Chapter: Annex Page 62 | 66 Desk Research Interviews with Partners Interviews with other Initiatives & Nexus Experts Analysis of Data provided by PASTO & WebMo Realized synergies 1. Number of informal information exchanges between PAEGC and other Grand Challenges for Development initiatives 2. Number of meetings between PAEGC and other Grand Challenges for Development initiatives 3. Number of joint realized projects between PAEGC and other GCD initiatives 3. Qualitative assessment of the conceptual integration of PAEGC into the GCD results framework x x x x Lessons learned Barriers with regard to donor cooperation Barriers on the innovators' level What are the reasons for the challenges or failures? To what extent have the PAEGC Partners leveraged their in-country presence to assist and complement the activities of the Innovators? How was this done? PAEGC Partners' in-country presence 1. Number of innovators that were assisted by the partners' in-country presence? 2. Qualitative description and assessment of the partners' in-country assistance that was delivered to the innovators x x What are the advantages and disadvantages of the chosen structure for the PAEGC initiative? Strategic outlook 1. Comparison of the PAEGC structure with the structure of other initiatives a) African Enterprise Fund b) Global Innovation Fund c) Energising Development d) African Agriculture and Trade Investment Fund x x x x Structure & Implementation Process Indicator or Descriptor Sources of Verification Evaluation Criteria Evaluation Questions Level of Analysis To what extent was PAEGC integrated enough with the GCD to maximize synergies and lessons learned? What kind of implementation barriers were encountered by the initiative? This questions will be answered through the synthesis of the results This questions will be answered through the synthesis of the results This questions will be answered through the synthesis of the results This question will be answered through the synthesis of the results Chapter: Annex Page 63 | 66 Desk Research Interviews with Partners Interviews with other Initiatives & Nexus Experts Analysis of Data provided by PASTO & WebMo To what extent is the cooperation with innovators suitable to establish new partnerships for development? Cooperation with innovators 1. Qualitative description and assessment of administrative requirements for the innovators in terms of their a) efficiency b) suitability for the intended objective c) level of specificity 2. Qualitative description and assessment of common objectives between the innovators and partners in terms of a) value added for development cooperation b) profitability for business x x x Advising on and strengthening PAEGC awardees' ability to comply with USAID policies and procedures 1. Number of compliance guides developed 2. Qualitative assessment of developed clauses in subawards and subcontracts 3. Qualitative assessment of PASTO's support to the innovators x x Monitoring awardee performance and assisting awardees with specific needs 1. Qualitative assessment of the reviews on quarterly reports, milestones and work plans of the innovators 2. Qualitative assessment of the management of information on WebMo 3. Qualitative assessment on the organisation and implementation of acceleration activities x x Designing and conducting performance and impact evaluations 1. Qualitative assessment of the reviews on the individual monitoring and evaluation plans of the innovators 2. Qualitative assessment of the conducted evaluation activities by the partners x x Organising and facilitating PAEGC events and travel 1. Qualitative assessment of the provided logistical support to the implemented events x x Facilitating outreach and communication 1. Qualitative assessment of the management of the internet platforms 2. Qualitative assessment of communication material (annual report, strategy, etc.) x x Developing and transferring PAEGC knowledge 1. Qualitative assessment of the Bio-Energy Session of the MOOC 2. Qualitative assessment of the support to studies, technical briefs or thematic groups x x Supporting future PAEGC procurements This question will be answered through the synthesis of the results x x What are the main barriers hindering PASTO's ability to fulfil the 7 tasks in their work plan? What are the areas of improvement that PASTO should focus on? Structure & Implementation Process Indicator or Descriptor Sources of Verification Evaluation Criteria Evaluation Questions Level of Analysis How well has PASTO accomplished the 7 tasks listed in their workplan? This question will be answered through the synthesis of the results This question will be answered through the synthesis of the results Chapter: Annex Page 64 | 66 Desk Research Interviews with Partners Interviews with other Initiatives & Nexus Experts Analysis of Data provided by PASTO & WebMo Did PAEGC take appropriate steps to source high-risk, high reward solutions in both its conducted calls? Selection process 1. Ratio between money invested and beneficiaries adopting the technology 2. Number of innovations that have been adopted a) in the project region b) outside the project region, but within a defined region of the country (state, province, etc.) c) countrywide d) beyond the country x x x Did PAEGC select appropriate Innovators in both calls? Selection process 1. Number of innovators that a) could achieve scale b) were adoptable within the context of emerging markets c) were sustainable, and can ultimately progress without outside assistance d) utilized modern, evidence-based science and technology approaches x x x Should existing application criteria be prioritized? Should additional application criteria be employed or some criteria be excluded? What are the lessons learned with regards to the calls? Theory of change 1. Number of objectives which relate to gender 2. Number of activities by the initiative related to gender 3. Qualitative assessment of the conceptual framework with regard to gender aspects x x x Selection process 1. Number of selection criteria that relate to gender 2. Importance of the gender criteria vis-á-vis other selection criteria x x x To what extent have the Innovators implemented specific gender activities? Innovators' activities 1. Number of gender activities implemented by the innovators 2. Number of innovators implementing gender activities x x To what extent has PAEGC supported the Innovators in regards to gender activities? Support mechanisms of PAEGC 1. Qualitative description and assessment of implemented gender activities by the initiative 2. Number of innovators that have requested and received support for gender activities x x x Indicator or Descriptor Sources of Verification Evaluation Criteria Evaluation Questions Level of Analysis To what extent did PAEGC take appropriate steps to integrate gender aspects in the program? Gender Call for Proposals This question will be answered through the synthesis of the results This question will be answered through the synthesis of the results This question will be answered through the synthesis of the results Powering Agriculture: An Energy Grand Challenge for Development Mid-Term Innovators’ Assessment August 2016 ii PAEGC Mid-Term Innovators’ Assessment Contents Acronyms / Abbreviations ....................................................................................................... iii 1. Introduction and Background ...............................................................................................4 2. Assessment Methodology ....................................................................................................7 3. The Procurement and Award Process ..................................................................................8 3.1 Call for Proposals Process .................................................................................9 3.2 Selection Criteria .............................................................................................10 3.3 Conclusions and Recommendations on Procurement and Award Process ...........10 4. The Award Management Process ......................................................................................12 4.1 Effectiveness of the AORs ................................................................................12 4.2 Effectiveness of PASTO ...................................................................................13 4.3 Effectiveness of WebMo..................................................................................14 4.4 Conclusions and Recommendations on Award Management ..............................14 5. Innovators’ Progress and Impact ........................................................................................17 5.1 Progress of Innovators .....................................................................................17 5.2 Benefits and Drawbacks of Funded Technologies ..............................................19 5.3 Impact on Gender ............................................................................................21 5.4 Scalability and Commercialization of Funded Innovators ....................................23 5.5 Conclusions and Recommendations on Innovators’ Progress .............................28 6. Final Observations and Recommendations .........................................................................30 Appendix A: Mid-Term Evaluation Scope of Work ..................................................................32 Appendix B: Performance Indicator Data, Fiscal Year 2015 ....................................................37 Appendix C: Raw Survey Data..............................................................................................39 Tables Table 1: Winners of the First Global Innovation Call ..................................................................5 Table 2: Funding Windows in PAEGC Second Global Innovation Call ......................................11 Table 3: Implementation Status of the 2013 Innovator Cohort ..................................................18 Table 4: Benefits and Drawbacks of First Call Technologies Funded Under PAEGC .................19 Table 5: First Call Innovators’ Responses on Gender Activities ................................................22 Table 6: Estimated unit costs of clean energy technologies of the 2013 innovators ...................24 Table 7: Progress on Scaling Up and Commercialization .........................................................25 Figures Figure 1: Global Innovation Call Round 1 Timeline and Process ................................................8 Figure 2: Rating of Call of Proposals ........................................................................................9 Figure 3: Application Process ..................................................................................................9 Figure 4: State of CES upon Grant Award ..............................................................................10 Figure 5: Rating of AOR Services ..........................................................................................12 Figure 6: Rating of PASTO Services ......................................................................................13 Figure 7: PASTO Services Utilized by Innovators ....................................................................13 Figure 8: Is WebMo Useful? ..................................................................................................14 Figure 9: 2013 Innovators and Their Innovation Stages as of Spring 2016 ................................18 Figure 10: Involvement of Women in PAEGC Projects ............................................................21 Figure 11: Progress Towards Innovator Initial Goal ................................................................. 28 iii PAEGC Mid-Term Innovators’ Assessment Acronyms / Abbreviations Acronyms Definition AFDB African Development Bank AOR Agreement Officer’s Representative BAA Broad Agency Announcement BMZ Bundesministerium Für Wirtschaftliche Zusammenarbeit (German Federal Ministry for Economic Development Cooperation) CES Clean Energy Solution COP Chief of Party DARPA The Defense Advanced Research Projects Agency GHG Greenhouse Gases GIZ Deutsche Gesellschaft für Internationale Zusammenarbeit (German Society for International Cooperation) M&E Monitoring and Evaluation NGO Non-governmental organization O&M Operation & Maintenance PAEGC The Powering Agriculture: An Energy Grand Challenge for Development PASTO Powering Agriculture Support Task Order PAX Powering Agriculture Xcelerator R&D Research and Development SIDA Swedish International Development Cooperation Agency UGARF University of Georgia Research Foundation USAID United States Agency for International Development VIP Village Industrial Power 4 PAEGC Mid-Term Innovators’ Assessment 1. Introduction and Background This assessment has been conducted over December 2015 to May 2016 under the Powering Agriculture Support Task Order (PASTO). PASTO is funded by United States Agency of International Development (USAID) and implemented by Tetra Tech ES, Inc. PASTO provides support services to the Powering Agriculture: An Energy Grand Challenge for Development (PAEGC) and its Founding Partners to enable their effective management, monitoring and evaluation of the program. The views expressed in this information product are those of the authors 1 and do not necessarily reflect the views or policies of the PAEGC Founding Partners. The Powering Agriculture: An Energy Grand Challenge for Development is a partnership between United States Agency for International Development, the Government of Sweden, the Government of Germany, Duke Energy Corporation and the Overseas Private Investment Corporation (the Founding Partners). The goal of PAEGC is to support new and sustainable approaches to accelerate the development and deployment of clean energy solutions 2 for increasing agriculture production and/or value in developing countries. PAEGC utilizes the financial and technical resources of its Founding Partners to support organizations’ implementation of clean energy technologies and business models that: i. Enhance agricultural yields/productivity; ii. Decrease post-harvest loss; ii. Improve farmer and agribusiness income generating opportunities and revenues; and/or iv. Increase energy efficiency and associated savings within the operations of farms and agribusinesses - while stimulating low carbon economic growth within the agriculture sector of developing countries and emerging regions. Powering Agriculture has selected 12 innovators 3 and 13 innovators during its first and second global innovation call for proposals that took place in 2013 and 2015 respectively. PAEGC conducts monitoring and evaluation (M&E) activities at the following three levels detailed in the program’s Monitoring and Evaluation Plan.  Meta-level - to measure and evaluate PAEGC’s contributions to the overarching Grand Challenges for Development Goal  Program-level - to measure and evaluate the aggregated contributions of PAEGC’s interventions in achieving the program’s goal. This is also referred to as the GCD-level 1 The assessment has been written by the following PASTO personnel: Daria Mashnik -Renewable Energy Specialist, Ron Ivey- M&E Specialist and Jeannelle Blanchard- Chief of Party with inputs from other PASTO team members. 2 In the PAEGC context, the term “clean energy” is defined as: Usable energy (i.e. electricity, illumination, heating/refrigeration, mechanization) that is derived from renewable sources and supports a reduction in fossil fuel use, increase in efficiency, and/or limitation of greenhouse gas emiss ions. Clean energy sources include – solar, hydro, wind, geothermal, sustainably harvested biomass, and biogas. The term “clean energy solution” is defined as: A combination of appropriate technology and a business model that addresses the clean energy demands of a select market. 3 One award was cancelled in early 2014. 5 PAEGC Mid-Term Innovators’ Assessment  Innovator-level - to measure and evaluate the progress and impact of PAEGC’s individual innovators, and individual contributions to achieving the PAEGC goal While the M&E Plan stated that PAEGC would undergo a mid-term performance evaluation, the Partners agreed to move forward on an effort that would be divided into two parts: (1) an assessment of the 2013 innovators to determine their progress to date to be conducted by PASTO and (2) an externa program-level evaluation incorporating the innovator-level assessment to be conducted by SYSPONS GmbH –contracted by PAEGC Partner–GIZ. The report’s findings, conclusions and recommendations will enable the PAEGC Partners to make any necessary mid-course adjustments to improve the effectiveness of the PAEGC program and to accelerate the development and deployment of clean energy solutions of PAEGC’s innovators. This document describes the results of the mid-term performance assessment of the innovators selected and funded as a result of PAEGC first global innovation call in 2013. Table 1: Winners of the First Global Innovation Call Innovator Project Name Country of Implementation Award Value 1 African Bamboo Biomass-Powered Thermal Processing of Bamboo Ethiopia $1,041,145 2 CAMCO Building Markets for Efficient Biomass Power Provision Benin, Tanzania $999,805 3 The Earth Institute at Columbia University Micro-Solar Utilities for Small￾Scale Irrigation Senegal $1,082,161 4 EarthSpark International Smart Grid on Main Street: Electricity and Value-added Processing for Agricultural Goods Haiti $1,091,315 5 ECO Consult A Hydroponic Green Farming Initiative Jordan $1,149,707 6 iDE Solar-Powered Pumps for Improved Irrigation Honduras, Nepal, Zambia $1,499,831 7 Motivo Engineering Hybrid Vehicle with Exportable Power for Community-Based Agriculture Mechanization India $861,158 8 Promethean Power Systems Reducing Milk Spoilage Through Solar-Powered Chilling India $992,980 9 Rebound Technologies SunChill: Solar Cooling for Horticultural Preservation Mozambique $1,1375,853 10 SunDanzer Solar-Powered Refrigeration for Dairy Farms Kenya $1,041,145 11 University of Georgia Research Foundation Biogas-Powered Evaporative Cooling for the Dairy Industry Uganda $1,000,000 12 Experience International4 Solar-Powered Cold Storage and Ice Making Facilities for Fishing Communities in Eastern Indonesia Indonesia $1,000,000 4 This award was cancelled. 6 PAEGC Mid-Term Innovators’ Assessment The assessment is organized to generate insights on PAEGC from the viewpoint of the innovator by asking the following questions: A. How well did PAEGC execute the procurement and award process? B. How well did PAEGC manage the awards? C. What are the impacts of the funded projects to date, including impact on farmers’ agricultural production, gender impacts and climate change mitigation? D. What is the likelihood of scalability and commercialization of the selected innovations? The document presents findings, conclusions and recommendations for each of these questions and ends with final observations and recommendations. It also contains four appendices that include the mid-term evaluation scope of work (appendix A), a summary of the performance indicator data (appendix B) and the raw survey data (appendix C). 7 PAEGC Mid-Term Innovators’ Assessment 2. Assessment Methodology This assessment was carried out through the following methods:  Administration of a survey via the phone/ web conference addressing the four main assessment questions (see Appendix A)  Data collection through check-in virtual meetings with the 2013 innovators  Document review of award documentation, milestones, deliverables and progress reports  Data verification and stakeholder interviews during project site visits  Analysis of reported performance indicator data The PASTO M&E Specialist and Renewable Energy Specialist conducted calls via web conference or telephone with the 2013 innovators from December 2015 to May 2016 during which  The innovators were asked to provide a description of their technology and an upd ate of progress to date.  The survey was administered. Innovators were informed that their responses to questions on the procurement and award process and the award management process would not be attributable to individual innovators in order to encourage a frank discussion.  Each performance indicator and the corresponding data submitted by the innovator was reviewed. The calls ranged from one to three hours, and in some cases, one to two follow-up calls were required for the performance indicator data review. Subsequently, the quantifiable responses were tabulated and all other comments that resulted from asking specific questions were collated, sorted and analyzed. Not every question asked proved to be useful and only those with relevant insights were included in the findings. 8 PAEGC Mid-Term Innovators’ Assessment 3. The Procurement and Award Process Powering Agriculture launched its first global innovation call on December 18, 2012 by issuing a Broad Agency Announcement (BAA). The call resulted in the submission of 473 proposal summaries from applicants representing 76 countries. There were a series of evaluative steps, portrayed in Figure 1 below and in December 2013, 12 winners were selected and announced. The procurement and award process lasted twelve months with the timeline illustrated below. Figure 1: Global Innovation Call Round 1 Timeline and Process The review process consisted of t wo stages as outlined below. Only those proposals that passed the Innovation Screening and satisfied the eligibility requirements had their Technical Solution Narratives evaluated. Stage One:  Innovation Screening: was based on the submission of responses to the following two questions: Question One: To what extent does the proposed solution accelerate the development and deployment of clean energy solutions for increasing agriculture productivity and/or value in low and middle income development countries? Questions Two: How is the proposed solution different from currently available technologies and/or practices in the proposed area of need(s) to qualify as innovative?  Technical Solution Evaluation: evaluation of proposals undertaken by a panel of internal evaluators Stage Two:  Past Performance Review: the evaluation of feedback received from references provided by the applicants  Cost Review: review of proposed budget All evaluation criteria were weighted equally with the exception of the Innovation Screening which was counted as double the value of the other three individual criteria. The PAEGC BAA defined innovation as “the extent to which the proposed solution demonstrated an unconventional and/or creative approach to applying clean energy technology to enhance agricultural productivity and/or 9 PAEGC Mid-Term Innovators’ Assessment value, in a manner that clearly differentiates from alternative approaches and remarkably improves upon existing practices in the target area of operation”. 3.1 Call for Proposals Process As part of the assessment, the 2013 cohort of innovators was asked a series of survey questions regarding how well PAEGC executed the procurement and award process and if the process was straightforward and logical. The responses below demonstrate that majority of the innovators rated the call for proposals as quite good (4) or excellent (5) and found the process logical. Three of the 10 innovators 5 complimented the PAEGC process mechanics in comparison to other awards that they have applied to in the past. The positive and negative feedback on the procurement and award process is highlighted below. The raw responses to the questionnaire collected during the phone interviews are included in Appendix C of this report. Process shortcomings:  Lack of clarity on key stages of the process and timeline  Lack of communication and feedback during the application process  Confusion during the final stages as to whether the innovator has been selected by PAEGC or not  Slow procurement and award process and long waiting time until final announcement Process strengths:  A collaborative and partner-like approach, rather than being bureaucratic and overly rigid  One of the best application processes in the experience of a few innovators For many of the innovators the lack of communication and clarity on key stages of the process left them uncertain about where they were in the process at any point in time. Three of the 8 innovators noted the 12 month waiting time between the call and the announcement of the finalists was too long, particularly in the case of start -up organizations that have limited cash flow. 5 The interviews were conducted with 10 innovators and one sub-awardee who was the actual technology developer and was the one interacting with the AOR, PASTO. Figure 2: Rating of Call of Proposals Figure 3: Application Process 10 PAEGC Mid-Term Innovators’ Assessment One of the innovators recommended a two-stage application process where the applicants submit a short expression of interest during the initial stage and a full application during the subsequent stage to reduce innovator’s upfront time commitment. 3.2 Selection Criteria During the first global call the PAEGC Partners funded innovators at different stages of the innovation process, with the primary focus on early stages of research and development (R&D) and projects that are difficult to finance commercially. Half of the funded clean energy technologies were in the concept stage upon issuance of the awards, as seen from the innovators’ self￾assessment shown on the right. Nine of the ten innovators defined their technologies as “high risk; high-reward” explaining that if their technology proved scalable and commercializable, their CES would deliver energy in pioneering and life-changing ways to many end-users in emerging markets. Some of the innovators were unfamiliar with the selection criteria in the BAA, a few thought it was vague, and others thought it was well defined and suited their CES very well. One of the innovators was thankful that private investor investment was not a selection criterion in the call for proposals which enabled them to apply as an NGO with no private sector investment. The principal recommendation from the innovators on selection criteria to improve the procurement process was to introduce more stage-specific selection criteria and requirements that would differentiate between the early stage of R&D innovators and those who have already demonstrated technical feasibility and/or market acceptance. This is in recognition that at different stages of development, prospective applicants’ proposed solutions will have different levels of risk, technical and financial requirements, and data on which to base their potential for impact and scale. 3.3 Conclusions and Recommendations on Procurement and Award Process The PAEGC award and procurement process of the first global innovation call that took place from December 2012 through December 2013 was assessed as “quite good” and “excellent” by the 2013 cohort of the innovators, 62% of whom found the process logical. The feedback received during the phone interviews varied greatly based on the individual innovator’s previous experience with other award programs. Most of the process shortcomings referred to in Sections 3.1 and 3.2 above were addressed by PAEGC in its second global call for innovation launched in November 2014. Based on the lessons learned from the first global innovation call, the Partners made some changes to the selection process, criteria and requirements, namely:  Released the BAA with more detailed and more explicit description of the applicant selection process and timeline Figure 4: State of CES upon Grant Award 11 PAEGC Mid-Term Innovators’ Assessment  Simplified the first stage of the application process by requiring the submission of a 600- word concept note that succinctly described the CES and why it is appropriate for PAEGC funding  Included external international experts, such as renewable energy technology-specific experts and members of in-country USAID, SIDA and GIZ missions and field offices, to evaluate the technological and contextual aspects of proposal submissions  Reduced the procurement and award process from 12 months to 10 months  Divided the available funding into the following two windows, reflecting that the financial and non-financial needs of prospective applications vary greatly according to the current state of development of their CES. Table 2: Funding Windows in PAEGC Second Global Innovation Call Funding Window Award Value Cost-Share Window 1: Clean Energy Solution - Design Up to $500,000 Up to 15% of award value Window 2: Clean Energy Solution Scaling Up/Commercial Growth $500,000 - $2,000,000 35% of award value (required) One key finding from the interviews with the innovators was the critical importance of an operational local presence in the country/ies in which the innovators work for them to succeed. This was further confirmed during site visits. Rebound Technologies, one of the earliest R&D stage innovators and the highest 2013 recipient of funding, identified their lack of in-country partnerships and local expertise to take their CES from the lab to the target market as the main reason the organization has started to actively explore its CES as an “open-source” technology and allow other players to scale the technology6 . Local presence is vital for establishing the ownership, operation and maintenance (O&M) mechanisms for the in-country installations as well as for building local private partnerships to enable scaling up and commercialization of the technologies. The BAA included “local presence” as one of the required features of all applicants’ CES. Other recommendations that could be implemented in future call for proposals are 1. Introduce more stage-specific selection criteria and requirements that would differentiate between early stage of R&D innovators and those who have already demonstrated technical feasibility and/or market acceptance. 2. Further streamline the applicant selection process and to reduce the duration to no more than six months. 6 In this context Rebound Technology may make the technical design specifications of the CES freely available to the public. They may also never commercialize the CES. 12 PAEGC Mid-Term Innovators’ Assessment 4. The Award Management Process 4.1 Effectiveness of the AORs Each PAEGC innovator was assigned an Agreement Officer’s Representative (AOR), who is a full-time development professional working at USAID is its Energy and Infrastructure Office within its Bureau for Economic Growth, Education and Environment. In many cases, the AORs were changed repeatedly based on personnel decisions within USAID. Out of the 10 innovators interviewed, 5 had AOR changes—2 permanent and 3 temporary. The survey questions measured the quality of the relationship between each innovator and their respective AOR. The innovators rated three of the AORs towards the high end of the five-point scale, and two scored towards the middle to low end, as illustrated above. This results in an average rating of 4.0 out of 5 for AOR services. It is possible that the AOR ratings may have an upward bias given that these responses would be reviewed by the AORs. An attempt to mitigate that bias was made by informing the innovators that their responses would be not be attributable to individual innovators. The feedback on the capabilities of specific AORs ranged from unfavorable to very positive. Several of the AORs were described as responsive and proactive, non -bureaucratic and helpful with milestone revisions and handling of issues. The recurring negative feedback was on the lack of feedback or slow responsiveness to questions or requests for approval of documents submitted, primarily projects’ award revisions requests. Seven out of 10 interviewed innovators have had their award agreements modified at least once since their award’s inceptions including a revision of milestones, deliverables and/or targets. This explains why the lack of responsiveness in the review process was the most common criticism. There were also delays in the approval of milestones which affected disbursement of funds and impacted small organizations especially given their limited cash flow needed to pay for equipment, materials and labor. One of the innovators indicated that have had a pending milestone revision since June 2015, lasting more than 10 months as of April 2016. Conversely, several of the AORs were described as responsive and proactive, non-bureaucratic and helpful with milestone revisions and handling project issues. Three innovators also discussed the importance of the AOR to critically review milestones, question the viability of certain aspects, provide in-depth technical feedback and identify essential milestones for inclusion in award tables such as a subcontract agreement between an innovator and its sub-awardee. One AOR was identified as adding beneficial value in examining and questioning one innovator’s technical approach. The innovators also indicated that they would have liked more support from AORs in the following areas:  Leverage of local USAID Missions, SIDA and GIZ field offices  Assistance with identifying local partners Figure 5: Rating of AOR Services 13 PAEGC Mid-Term Innovators’ Assessment  Assistance with securing additional funding  Assistance with additional deployment opportunities and product commercialization When asked if any of the PAEGC Partners used their in-country presence to help the innovators, only 1 out of 10 innovators answered positively, referencing the local USAID Mission’s assistance with customs clearance for equipment. 4.2 Effectiveness of PASTO The innovators rated their experience with PASTO and identified which of its services they have utilized to date. A majority of the innovators scored PASTO as excellent (5) and quite good (4), as shown in the graph to the right, with an average score of 3.9 out of 5. It is possible that this rating may have an upward bias given that PASTO asked respondents about its own activities. The sole innovator who scored PASTO as “poor” explained that until October-November 2015 their relationship with PASTO had been non-existent. Two innovators noted that their interaction with the former PASTO Chief of Party (COP) were neither helpful nor productive and subsequently discouraged them from approaching PASTO. PASTO changed its COP in May 2015. The services provided by PASTO that were utilized by the innovators were as follows: During the phone interviews, several of the innovators expressed a lack of knowledge of PASTO’s services prior to Powering Agriculture Xcelerator (PAX) workshop which took place in November 2015. A few innovators wished they had made more use of PASTO services and one innovator wanted a clearer distinction between the role of the AOR and PASTO. While the innovators utilized PASTO to address M&E, project milestone and business acceleration questions, many innovators emphasized the usefulness and importance of business Figure 6: Rating of PASTO Services Figure 7: PASTO Services Utilized by Innovators 14 PAEGC Mid-Term Innovators’ Assessment acceleration support, in particular as a means to assist with scale-up and commercialization of their solutions. The innovators indicated that inclusion of these services from the initiation of the awards would have been most useful. This is the case with the 2015 cohort of innovators where PASTO has been involved from the start. 4.3 Effectiveness of WebMo WebMo is the Powering Agriculture online monitoring platform which enables the innovators to input their M&E performance indicator data and upload deliverables, reports and information substantiating completion of their respective milestones. Concurrently, it allows PAEGC Partners and PASTO to monitor the progress of the innovators. All innovators indicated that they had used WebMo. However, the innovators were evenly split regarding the web-based platform’s usability. The responses varied depending on the individual innovator’s level of usage and the innovators who had used WebMo the most were the ones who reported more feedback. WebMo shortcomings:  Not intuitive and confusing to use  Controls are small and some fields, like the disaggregates, are easy to miss.  Difficulty with registration WebMo strengths:  Provides document backup  Central place for all data and documents  Straightforward and intuitive to use One of the innovators suggested that WebMo should be enabled in a way that would allow innovators to look at each other’s data. Another innovator recommended National Science Foundation’s online website, FastLane, as a better alternative to the WebMo. 4.4 Conclusions and Recommendations on Award Management PAEGC’s management of the awards of the 2013 innovator cohort was mixed, but there were some common threads in most of innovators’ responses. 1. Despite the award administrative requirements placed upon the innovators by PAEGC, none of the innovators expressed that these requirements were unnecessary or that they hindered project implementation. Some of the innovators indicated that even though PAEGC was promoted as less of a traditional development model, the award requirements still followed the standard USAID award requirements that are inherently bureaucratic. Figure 8: Is WebMo Useful? 15 PAEGC Mid-Term Innovators’ Assessment 2. One of the shared opinions among the innovators is the wish for additional programmatic support to help scale up and commercialize their innovative CES. While PAEGC is seen as a good seeding mechanism that encourages ground-breaking clean energy developments in the agricultural arena, it did not provide clear and systematic guidance for the innovators on how to get from one development stage to the next. This supports the utility of the Powering Agriculture Xcelerator and the need for the launch of the financing facility that was originally envisioned under the commercial financing component of PAEGC. The utility of PAX was underscored by some of innovators who would have liked it in place at the start of their award since some of the 2013 cohort awards will end this year. 3. Many innovators highlighted the wish to ultimately be connected to local USAID Missions, SIDA and GIZ field offices. While many CES are too early-stage to reach a large number of beneficiaries, these connections can help link the innovators to key potential local partners and stakeholders who can assist in disseminating and financing the technologies. The Missions and field offices can also provide an understanding of the local development context that some of the innovators lack. 4. The responsiveness of the AORs on milestone approval and award modification requests was a major issue and was further compounded by the fact that many of the innovators had never worked with USAID regulations and that the original award documents were poorly structured and not consistently written. In PASTO’s experience assisting with milestone reviews and award modifications, the milestones a) were not in a logical order based on activities to be implemented in the field, so the innovators completed milestones out of order, and that complicated their payments and b) were not as significant as the money associated with their completion, which resulted in overpayment by PAEGC for the level of effort and impact. This was further confirmed during the milestone review site visits, when the innovators indicated that they were unclear as to how to develop their milestone table and even what some of the milestones in their agreement meant. Some of the findings and conclusions above have been addressed for the 2015 cohort of innovators by actions that PAEGC has already taken, namely 1) having just 1 AOR for all 13 innovators, who can identify recurring issues, 2) having PASTO more involved in the development of the milestone tables in the original award agreement, 3) introducing PASTO and its services (including PAX) to the innovators from the beginning of the award and 4) the creation of an award modification guide and template as well as other contractual guidance. The following are additional recommendations to improve the management of the awards;  Improve the continuity of the AOR by having a formal handover meeting with the innovator, and the old and new AORs to ensure consistency  Ensure AORs respond to innovators’ questions, approve milestones, deliverables and award modifications in a timely manner, and perform more of a validity check on milestones rather than a compliance sign-off  Link the innovators from the beginning of award to local USAID Missions, SIDA and GIZ field offices to help the innovators with identifying local partners, additional deployment opportunities and financing mechanisms.  Continue to utilize PASTO to provide assistance to the innovators 16 PAEGC Mid-Term Innovators’ Assessment  Conduct a survey of innovators on items that could make WebMo more intuitive, user￾friendly and encouraging for the innovator’s continued use.  Allow the 2013 cohort, whose awards have ended in 2016, to continue to access PAX services for at least one year. 17 PAEGC Mid-Term Innovators’ Assessment 5. Innovators’ Progress and Impact This chapter outlines the primary findings from the assessment questions that focused on progress and impact since the inception of PAEGC. It also presents data on each innovators’ current status that PASTO systematically gathered information during eight project site visits conducted over November 2015 to May 2016. 5.1 Progress of Innovators The current status of each innovator funded in 2013 is summarized in the table below. It shows that six out of nine innovators interviewed and/or visited have conducted field testing of their technologies. One of them, University of Georgia Research Foundation (UGA), just started the field testing process in March 2016. Motivo Engineering plans to start the process in late 2016 due to a six to eight month customs delay. The awards of two innovators ended in the spring of 2016: Earth Institute at Columbia University and Rebound Technologies. The latter never fully completed the development or field testing of its prototype, and is in the process of exploring its CES as an open-source technology in order to allow other innovators to complete testing and to scale the technology. One innovator, CAMCO, has stalled in its implementation efforts due to lack of agreement with its sub-awardee. However, their sub-awardee and the developer of the CES, Village Industrial Power (VIP), is continuing to do field testing of a third generation of units on their own. EarthSpark International, has been field testing its solar powered micro-grid to provide electricity access to residential customers. While the innovator’s agricultural activities were identified from the beginning, the organization has just recently started to pilot and evaluate their appropriateness, profitability and effectiveness. One innovator out of the 2013 cohort, Promethean Power Systems, has reached commercialization of their CES with sale of more than 162 units. However, this success is not in line with the original proposal for the PAEGC award. Originally the CES units outlined in the proposal were solar powered, however due to changes described in more detail in this chapter, the units installed are instead energy efficient due to their innovative cooling design and only 6 out of 162 units have a solar component. Earth Institute and EarthSpark also have paying customers as they charge tariffs per hour of pumping and per kWh of electricity, respectively; however, the tariffs are not cost-reflective. 18 PAEGC Mid-Term Innovators’ Assessment Table 3: Implementation Status of the 2013 Innovator Cohort Innovator Project Name Country Implementation Status 1 African Bamboo Biomass-Powered Thermal Processing of Bamboo Ethiopia Testing production process on bamboo in Germany and Italy. 2 CAMCO Building Markets for Efficient Biomass Power Provision Benin, Tanzania Implementation has stalled; VIP, sub-awardee and CES developer, continues with field testing outside of PAEGC. 3 The Earth Institute at Columbia U. Micro-Solar Utilities for Small-Scale Irrigation Senegal Award finished; completed a long period of field testing. 4 EarthSpark International Smart Grid on Main Street: Electricity and Value-added Processing for Agricultural Goods Haiti Field testing the residential micro-grid; just starting to evaluate and test agricultural applications. 5 ECO Consult A Hydroponic Green Farming Initiative Jordan Not Interviewed as administered by USAID Jordan. 6 iDE Solar-Powered Pumps for Improved Irrigation Honduras, Nepal, Zambia Completed a long period field testing; transitioning to scale up 7 Motivo Engineering Hybrid Vehicle with Exportable Power for Community-Based Agriculture Mechanization India Just shipped two test unit; will start field testing in later 2016. 8 Promethean Power Systems Reducing Milk Spoilage Through Solar-Powered Chilling India Over 160 units sold; ramping up sales of units. 9 Rebound Technologies SunChill: Solar Cooling for Horticultural Preservation Mozambiqu e Award finished; prototype and field testing incomplete. 10 SunDanzer Solar-Powered Refrigeration for Dairy Farms Kenya Completed a year of field testing; transitioning to scale up 11 U. of Georgia Research Foundation Biogas-Powered Evaporative Cooling for the Dairy Industry Uganda Started field testing at in March with two units. To summarize the status of the innovators’ progress, the figure below categorizes them along the innovation timeline. Figure 9: 2013 Innovators and Their Innovation Stages as of Spring 2016 19 PAEGC Mid-Term Innovators’ Assessment 5.2 Benefits and Drawbacks of Funded Technologies Each innovator provided feedback concerning the advantages and potential drawbacks of their CES, which are summarized individually below. One of the recurring issues referenced by the innovators was the need for a certain level of technical competence for the operation of their units and the need for more thorough training for the operators. One of the innovators noted that they should have selected a different country as the point of entry for their technology due to the lack of technical capacity and personnel capable of operating and maintaining the prototype units. Four out of 10 innovators, Earth Institute at Columbia University, iDE, Promethean Power Systems and SunDanzer stated that their technologies save beneficiaries money based on the their field testing activities to date. In most cases, the innovators needed to collect more in-field data to quantify the savings. Promethean Power Systems and SunDanzer said that their field testing demonstrated an increase in agricultural production and improved agricultural quality, which were attributable to the CES. Earth Institute reported improvements in produce quality due to the reduction of diesel fumes from conventionally used diesel pumps. However, the quality improvements have not contributed to farmers’ increased revenue because the vegetable prices are not quality-based but set by the market. Two of the innovators reported that their technologies resulted in time savings for the beneficiaries: Earth Institute and Promethean Power Systems. Earth Institute’s pilot units saved on irrigation time, particularly for those farmers who irrigated their land manually prior to project implementation. Promethean Power Systems’ milk chilling technology significantly decreased the time travelled by dairy farmers to reach the nearest collection center with milk cooling capabilities. The individual innovator responses highlighting the benefits and drawbacks of their respective technologies are as follows. More innovator-specific information on their progress is offered in Table 7. Table 4: Benefits and Drawbacks of First Call Technologies Funded Under PAEGC Innovator Benefits Drawbacks African Bamboo “Our bamboo production process reduced the energy consumption considerably in comparison to the traditional process carried out in China.” none reported The Earth Institute at Columbia University “We have developed a shared system with no battery storage, which results in a higher utilization rate of solar panels. The shared photovoltaic (PV) systems removes the burden of upfront costs with the farmers making small payments to access the pump water every day.” “The large system requires an operator who needs to be trained and paid, which adds to the operating costs, plus the size of the system has become an issue.” EarthSpark International “Homes in the community have electricity and save about 6.5% of their income by not using candles and kerosene for lighting.” “Despite low consumer prices ($1 to $2 per month), 100% adoption by all residents has proved difficult.” iDE “The ease of use is a clear advantage, “The pump is heavy and we’re 20 PAEGC Mid-Term Innovators’ Assessment Innovator Benefits Drawbacks as the design is based on a treadle pump, cutting reliance on fuel and energy. This pump can be used with shallow water levels. Also, it eliminates the bus rides to obtain fuel. Agricultural production is not changing, but the advantage is that little by little the farmers are replacing their diesel pumps”. working on reducing the weight, therefore increasing the mobility of the unit and decreasing the cost in the process. Another drawback is that the pump moves a much lower volume of water than a diesel powered pump. Also, it takes a while for the farmers to adopt the technology and be okay with relying solely on solar power.” Motivo Engineering “Operational units are projected to save money over conventional fossil fuels.” none reported Promethean Power Systems “The milk cooler is resulting in sales increases for milk farmers. The farmers are also not carrying milk to market, allowing more time for agricultural production. The CES is reducing diesel consumption which would be powering the pumps moving the milk from the collection center to the tanker.” none reported Rebound Technologies “When the innovation is completed, we will be able to improve the quality of produce through cooling. We estimate that this technology will reduce the energy costs to 1/10th as compared to a diesel or grid-powered cool storage unit. All materials can be sourced locally.” none reported SunDanzer “The ability of the dairy farmer to market 10 to 40 liters of evening milk to a processor can raise the family income substantially and result in the payback of the purchase price in less than one year.” “The array needs to be grounded to protect the unit from lightning strikes.” U. of Georgia Research Foundation “Milk cooler concept fits well into the farmers’ ecosystem”. “The CES users require training to operate the technology”. All of the clean energy solutions, with the exception of one, contribute to reducing greenhouse gas (GHG) emissions and mitigating climate change. Six clean energy technologies are replacing diesel consumption, which would otherwise be used to pump water, generate electricity or fuel tractors. UGARF’s technology reduces methane emissions which have a 25 times greater negative impact on climate change than carbon dioxide (CO2) 7. EarthSpark International reduces emissions associated with burning kerosene and charcoal. SunDanzer is the sole innovator with a project that does not reduce fossil fuel use because no diesel powered milk coolers are used by individual farmers in Kenya. The innovators working on solar-powered irrigation solutions explained that the complete transition from diesel pumps to solar pumps sometimes takes a long 7 EPA (2010). Methane and Nitrous Oxide Emissions from Natural Sources. 21 PAEGC Mid-Term Innovators’ Assessment time for many farmers. This was confirmed by PASTO during project site visits where some farmers continued using their diesel pumps as a back-up. Therefore, this results in the reduction of GHG emissions associated with these pumping solutions from being fully realized at this time. 5.3 Impact on Gender The graph below illustrates how the 2013 innovators rated their level of women’s involvement and promotion of gender equity in their projects. The innovators who had not started their field testing at the time of the interviews were excluded from this question due to lack of input on the subject 8 . The scoring ranged from 1 (poor) to quite good (4), with an average of 2.6 out of 5. None of the innovators were collecting any additional gender-related impact data in addition to the disaggregates of the required PAEGC performance indicators; however, many welcomed suggestions from PASTO. There was a clear increase in innovators’ awareness on the importance of gender inclusivity since the Powering Agriculture Innovator Showcase in November 2015, however, many did not know how to actually integrate gender inclusion activities into their ongoing in-country field testing. While the degree to which women are involved in PAEGC projects was rated “poor” during the phone interviews, the project site visits revealed many examples of women directly benefitting from the CES installations in the communities. Several innovators including Earth Institute, iDE and EarthSpark are working with female cooperatives or other unofficial, women-led groups as their direct beneficiaries. Earth Institute has agreed to provide a loan to the women’s collective to cover the initial capital investment for cultivating land and to make use of their available solar pump. EarthSpark has trained 8 out of 3000 women in a cooperative with whom they are partnering, Association of Women of Les Anglais, on the use of the electric corn thresher and breadfruit fryer as pilots for productive uses of electricity that their mini-grid is providing. Another direct benefit, mentioned among EarthSpark’s mini-grid users related to women and children, was the greatly improved indoor air quality due to reduction of fumes from kerosene lamps. SunDanzer is another innovator having a notable impact on female farmers that was observed during PASTO’s field visit interviews in February 2016. The women use the additional income from increased milk sales as a result of the CES milk chilling to pay for their children’s books and school fees, and to invest in their farms through other means such as purchasing extra cow feed and purchasing additional and higher producing dairy cows. One female beneficiary, who 8 Africa Bamboo, Motivo Engineering and University of Georgia Research Foundation 3 1 4 3 2 3.5 2 SUNDANZER REBOUND PROMETHEAN IDE CAMCO/VIP EARTHSPARK EARTH INSTITUTE Involvement of Women in PAEGC Projects (self-assessed) Figure 10: Involvement of Women in PAEGC Projects 22 PAEGC Mid-Term Innovators’ Assessment operated a male-owned farm, attributed an increase in her salary due to the farmer’s additional earnings from the evening milk production. While traditional cultural roles and gender segregated labor markets limit women’s participation in many of PAEGC innovator activities, PASTO observed fluidity in gender roles in some of project site communities visited. As husbands migrate abroad for unskilled labor opportunities, women remain to run the farms and make the essential decisions as the heads of the households. This situation was noted during project site visits to Kenya, Uganda and Nepal where as a result of the additional profits from the CES, the female farmers had savings and spending money apart from what their husbands were bringing in. Below are innovators’ responses to how gender equity was being addressed within their activities: Table 5: First Call Innovators’ Responses on Gender Activities Innovator Responses on Gender African Bamboo “Our intention is to employ 40% of women in our factory which has a target operation start time of June 2017. Considering the current situation in Ethiopia, where you don't find women employed in the industry, we would give a rating of 4 out 5, once we’re operational.” The Earth Institute at Columbia University “We are working with an existing women’s collective and have agreed to provide them a loan to help them purchase the equipment they need to cultivate their shared plot of land. In Senegal women do not own much property and traditional cultural roles limit women’s participation” EarthSpark International “Fifty-seven percent of the signed contracts by consumers connected to the grid had been done by women. Our mini-grid ambassador in Haiti is a woman.” iDE “Most of the users of our pumps are heads of the households which are men. There is a group of women, a cooperative that is growing vegetables and irrigates with our pump. In our experience women are better at keeping track of operational information and finances.” Motivo Engineering “Our technology, a tractor, is gender neutral. At present, as field testing has not been initiated in India, no women have been involved. As the tractor will be used on family farms, it is anticipated that women will benefit from the tractors and we plan to collect data on usage pattern differences by gender to inform the next generation design.” Promethean Power Systems “Women are involved in all dairy farm activities. However, given the culture and dominance of men, women are difficult to involve and we are hampered because the real interface with the villages is the milk processors. We did have one NGO-run chilling center where all the members were women.” Rebound Technologies “We coordinated with an NGO to create a little exposure for women, however we only showed them various components of our technology.” SunDanzer “We did not originally ask for dairy cooperatives with whom we are working to target female farmers. After PASTO’s visit in February 2016, we have been giving women participation more emphasis. One third of the dairy farms involved in the field testing is owned by women and one half of the operators of the chillers is women. There is a woman in Kenya key to our field testing effort and we employ a woman as #2 under our general manager in Kenya.” U. of Georgia Research Foundation “We anticipate half of the beneficiaries to be women. We plan to involve women through the existing women’s networks in the communities and use women to demonstrate the CES units to others. We anticipate that women will be using the milk chiller and will become responsible for making the time payments.” 23 PAEGC Mid-Term Innovators’ Assessment 5.4 Scalability and Commercialization of Funded Innovators Out of the ten innovators reviewed in this assessment only one demonstrated commercial viability, that is, having paying customers that are purchasing their CES. However, that one innovator’s CES did not align with the original proposal at the time of the PAEGC award. Promethean Power Systems initially envisioned having their milk cooling units charged by a PV solar array for off-grid regions in India, however soon after the start of the award it became apparent that the primarily solar solution was not viable due to the following reasons:  The large size of the solar array (4kW capacity) needed to power the refrigeration unit, the compressor in particular, makes the siting of the system in the villages difficult.  The dairy processors purchasing the CES equipment do not benefit from installing the milk collection centers in very remote and off-grid areas due to the large distances to the central chilling centers.  The high cost of solar PV makes the CES less attractive to invest for the dairy processors. As a result, the innovator shifted its focus to develop and implement a robust, highly energy efficient grid-powered milk chilling solution (with cooling capacity of 500L to 2,000L) that can be tailored to each customer’s projected utilization. The customer can add a 1kW solar array component to power the unit’s controls and the pump that transfers milk from the chiller to the truck when grid electricity is unavailable. Demand for this product by processors has been brisk and Promethean is currently producing 25 units to meet an order from a dairy processor. [Information here has been removed from the public version as it is considered Sensitive But Unclassified (SBU), and contains sensitive financial and/or technical information that could damage the competitive advantages of the company/ies involved.] In addition to their work in India, they have partnered with Fonterra, the dairy co-operative behind Anchor milk brand, to introduce their milk chilling technology to Sri Lanka. The reason for Promethean’s success vis-à-vis the other innovators is that they were already selling units at the start of their PAEGC award and they knew their market and its entry points. Two other innovators, iDE and SunDanzer, claimed that they too successfully demonstrated the commercial viability of their CES. However, both organizations have provided their CES at no cost, are still engaged in figuring out customer financing and securing additional funding before they can begin actual sales. Through a set of questions designed to assess the innovators’ progress in scaling up or commercialization, the innovators identified the following main steps required to move their CES closer to commercialization:  Acquiring additional funding to address any remaining field testing, O&M or local training activities  Development of the CES supply chain  Identification of best private sector partners, ideally with high market penetration  Evaluation of potential customer financing mechanisms  Development of marketing strategies to achieve sale targets Below are the main obstacles defined by the innovators in achieving commercialization:  Lack of funding, particularly low-cost capital 24 PAEGC Mid-Term Innovators’ Assessment  Lack of sales talent needed to market the technologies  Slow adoption of CES technologies by the farmers  Inadequate training on new technologies for operators, system users and maintenance professionals  High customs and duty fees on renewable energy and agriculture equipment imports In the process of commercialization, the unit cost of the technology and its affordability directly impacts the willingness of farmers and agribusinesses to purchase and use the CES. The table below lists the CES unit retail costs based on innovators’ estimates. Two innovators, African Bamboo and EarthSpark International, involve large investments. African’s Bamboo’s PAEGC award represents less than 1/15th of the total cost of the bamboo flooring producing factory. Other donors have stepped in to support its investment: GIZ at $1.4 million, African Development Bank at $750,000 and Dutch organization at $842,000. [Information here has been removed from the public version as it is considered SBU, and contains sensitive financial and/or technical information that could damage the competitive advantages of the company/ies involved.] Table 6: Estimated unit costs of clean energy technologies of the 2013 innovators Innovator CES Unit Description Unit Retail Cost* 1 African Bamboo Bamboo flooring producing factory, runs on biomass waste; [Redacted] 2 CAMCO Micro steam combined heat and power (CHP) plant, runs on biomass and bamboo waste; [Redacted] 3 The Earth Institute at Columbia U. Shared battery-less solar irrigation system, powers seven pumps; [Redacted] 4 EarthSpark International Solar-powered mini-grid, electrifies households and a few agricultural activities; [Redacted] 5 iDE Solar powered irrigation pump; [Redacted] 6 Motivo Engineering Hybrid solar tractor with electricity storage; [Redacted] 7 Promethean Power Systems Grid-powered energy efficient milk chilling unit with thermal storage (500L – 2,000L in scale); solar PV component available. [Redacted] 8 Rebound Technologies Solar-thermal cooling horticulture system; [Redacted] 9 SunDanzer Solar powered refrigeration unit for dairy cooling (farmer scale); [Redacted] 10 U. of Georgia Research Foundation Biogas powered evaporative cooling unit for dairy farmers. [Redacted] *estimated by innovators All innovators stated they are working on ways to reduce the cost of their CES units by evaluating one or more of the following options:  Scaling down of the system’s size  Assembling CES locally  Manufacturing CES components locally 25 PAEGC Mid-Term Innovators’ Assessment  Using more affordable system components, whether by outsourcing specific parts or using locally sourced materials Below are summaries of each innovator’s progress with scaling up or commercialization of their CES based on phone survey responses and data collected during PASTO’s site visits: Table 7: Progress on Scaling Up and Commercialization Innovator Scaling up and commercialization progress African Bamboo This innovator is in the middle of testing various production processes in Germany and Italy with local bamboo feedstock to establish a consistent quality bamboo flooring product. There are two individuals working on scalability efforts, and a marketing specialist was just hired to look at potential markets and develop marketing strategies. The African Development Bank (AFDB) is assisting them in developing a more elaborate business plan and a financial model. [Information here has been removed from the public version as it is considered SBU, and contains sensitive financial and/or technical information that could damage the competitive advantages of the company/ies involved.] The concept of wholesalers and retailers is inappropriate for this operation, at least at the country level CAMCO This innovator’s implementation efforts have stalled. However, their sub￾awardee and technology developer, VIP, continues implementation on their own. They are continuing the field testing of the four prototypes installed under PAEGC with funding from FactorE, which provides them scaling-up support to initiate commercialization. VIP states that they are ready to sell 15 to 20 units this year and plan to make 75 sales the following year. [Information here has been removed from the public version as it is considered SBU, and contains sensitive financial and/or technical information that could damage the competitive advantages of the company/ies involved.] VIP claims to have a business model which has been difficult to implement due to lack of financing. Since it is a new technology and they have no proven record, it has been difficult to attract investors. To secure additional funding they have done a pitch presentation to Shell Foundation in March 2015. The high import duties and taxes on their CES, which adds to an already high capital, has been another challenge. The Earth Institute at Columbia University This innovator’s main objective was to prove the workability of a shared solar pump system that uses no battery storage, and they have achieved that. However, they are not at a point where they can approach an investor and have a large number of units deployed, but have a good idea on how to get much closer to that. They have not developed a full business model but have looked at the cost of the system and potential payback periods. They are collaborating with one of the rural electrification organizations that is interested in testing their system to irrigate a large banana plantation. [Information here has been removed from the public version as it is considered SBU, and contains sensitive financial and/or technical information that could damage the competitive advantages of the company/ies involved.] Their original payment plan included using mobile money, however because the communities have set up a local way to pay for the services and it seems to be working, mobile money is no longer a priority. The organization is currently waiting to hear back from additional funding from the USAID Senegal Mission that would allow them to demonstrate their CES technical viability in another location with different soil conditions. EarthSpark International This innovator has not demonstrated commercial viability for their mini -grid but they are charging an electricity tariff to their customers. They are adjusting the business model as they progress along with implementation based on lessons 26 PAEGC Mid-Term Innovators’ Assessment Innovator Scaling up and commercialization progress learned. Their ultimate goal is to have 80 mini-grids, each servicing 500 households, which eventually pay for themselves. They just finished a U.S. Trade and Development Agency funded study where they identified 94 other towns in Haiti with demand for similar mini-grids. [Information here has been removed from the public version as it is considered SBU, and contains sensitive financial and/or technical information that could damage the competitive advantages of the company/ies involved.] Two years ago they spoke with the Ministry of Energy Security and Ministry of Public Works about the grid O&M and are presently looking for more local partners. EarthSpark’s field testing has allowed their technology partners SparkMeter, a smart meter developer, and Zero Base Energy, a mini-grid developer, to spin off and effectively commercialize their products. iDE This NGO had not reac hed the original scale of targeted installations but has built and developed all the components, partnered with a pump manufacturing company (Futurepump) and identified in-country retailers and distributors. Their field testing has also allowed their technology partner and PAEGC 2015 innovator, Futurepump, to start commercialization. One of the strengths of this organization, validated by the project site visits, is their extensive network of local stakeholders. All of their pilot units have been donated an d they are currently working on developing a financial approach. Recently they submitted a challenge to a “Poverty Hackathon” event for people to develop an innovative payment scheme for their CES. [Information here has been removed from the public version as it is considered SBU, and contains sensitive financial and/or technical information that could damage the competitive advantages of the company/ies involved.] Their next steps for commercialization include identifying the right private-sector partner and building their capacity to offer the “after-sale service”, their most important criteria. They are also looking to secure additional funding. Motivo Engineering This organization’s innovation is a shareable tractor with a battery pack that can be rented “on-demand” through text messaging. Motivo Engineering had estimated that 100,000 small tractors are sold annually in India at a cost of about $4,000 each, which is their market. They experienced very long shipping and custom delays and as a result plan to start field testing in late 2016. [Information here has been removed from the public version as it is considered SBU, and contains sensitive financial and/or technical information that could damage the competitive advantages of the company/ies involved.] They have a draft business plan which involves iteration of the technology, establishment of a manufacturing plant and a sales organization, and setting up Motivo Engineering as the R&D unit. They are not working with any wholesalers and retailers at this point and foresee their next challenges to be product adoption and training of involved partners and end users. Promethean Power Systems This company have sold 162 grid-powered milk chillers to dairy processors in India. Six of them have a solar component, which was proposed as the main source of power originally in its proposal for the award. Promethean estimates that 230,000 communities could use village-level milk chillers across India. [Information here has been removed from the public version as it is considered SBU, and contains sensitive financial and/or technical information that could damage the competitive advantages of the company/ies involved.] They are looking to lower the system cost by outsourcing the steel tank production. By purchasing greater volumes of higher quality milk from smallholder dairy farmers, the dairy processors can recapture their investment faster. The company’s biggest hurdle is finding sales talent to market the technologies. 27 PAEGC Mid-Term Innovators’ Assessment Innovator Scaling up and commercialization progress Rebound Technologies This organization has not fully developed its prototype and has only partially field tested it. The field testing has not included any actual pre-cooling of agricultural produce because although two main technical components have been demonstrated, a fully integrated system capable of pre-cooling produce has not been fully realized. The engineering aspect of the technology is 95% complete and the pilot project is 5% complete. [Information here has been removed from the public version as it is considered SBU, and contains sensitive financial and/or technical information that could damage the competitive advantages of the company/ies involved.] They have not developed a business model. The innovator stated that they never anticipated selling their system directly to the farmer. At present they have not contacted wholesalers and retailers or any potential partners who could provide maintenance services to the technology. Their plan is to patent the technology as open-source in order to allow other innovators to complete testing and to scale the technology. They have also incorporated the lessons learned into their IcePoint™ technology and recently completed testing a demonstration unit at a Whole Foods near Denver, Colorado. SunDanzer This organization have tested 40 dairy chillers in the field with farmers but have no actual sales. [Information here has been removed from the public version as it is considered SBU, and contains sensitive financial and/or technical information that could damage the competitive advantages of the company/ies involved.] Their next steps towards commercialization include increasing the cost-share from the farmers, figuring out customer financing, affordability of their technology, and developing the supply chain around the innovation. They are working with the national credit association, who have recently received a Development Credit Authority guarantee from USAID, to set up a program to enable the purchase of the units. The credit association plans to run a pilot lending program with 10 farmers. [Information here has been removed from the public version as it is considered SBU, and contains sensitive financial and/or technical information that could damage the competitive advantages of the company/ies involved.] The company anticipates that out of the five million dairy farmers present in East Africa, a potential of 850,000 customers may opt to upgrade their operations with a solar chiller. They are looking for a partner with high market penetration who can help them enter this market. U. of Georgia Research Foundation This organization started testing its two CES units in March 2015 and they have applied for a grant extension until July 2017 due to the initial target number of units being unrealistic. They stated their main focus is to create a good product. It is too early for them to know if there is intrinsic value demonstrated through the prototype testing, but they imagine there will be demand in countries like Rwanda, Burundi and Malawi. [Information here has been removed from the public version as it is considered SBU, and contains sensitive financial and/or technical information that could damage the competitive advantages of the company/ies involved.] Currently there is no marking plan but UGARF intends to market the units themselves with the sales being set up in incremental payments because generally dairy farmers do not have liquidity. They envision owning the units while the female farmers use the chillers and gain revenue to make small payments. They plan to train women to collect grass for the units’ energy source, which runs on biomass, so they can operate the units during the dry seasons when the milk prices are most profitable. They have not contacted any wholesalers or retailers, as they are too early stage, but have made connections with potential partners who can provide O&M services. 28 PAEGC Mid-Term Innovators’ Assessment 5.5 Conclusions and Recommendations on Innovators’ Progress All the 2013 innovators have made some progress towards their initial objectives. Below is a graph illustrating how the innovators assessed their progress towards their initial goals, which were outlined in their work plans, compared to PASTO’s assessment based on the conducted site visits. All of the innovators have advanced along the innovation ladder as a result of PAEGC support, however the majority of the cohort did not meet their targets as initially planned. The reasons are multifaceted, and include unforeseen equipment delays, changes to the manufacturing processes, prototype optimization to address specific technical shortcomings, absence of well￾researched financial approaches specifically applicable to the local context, and inadequate financing to make a leap from the initial piloting stage to the adoption and market growth stage. In addition, lack of local partnerships with market access prevent the innovators from finding the most effective market entry points. Only Promethean Power Systems had actually achieved commercialization because the company had already demonstrated technical feasibility, market acceptance and had sales channels set in place at the time of PAEGC’s initiation. For greater level of commercialization among its innovators, PAEGC could focus on funding later stage organizations in the future. Part of the challenge of scaling up and commercialization, which was confirmed during the site visits, was the innovators being too far removed from the CES end -users and the lack of streamlined communication between the innovators and their on-the-ground partners. PASTO observed that many of the innovators had not conducted detailed interviews with their end-users before, as well as after, the implementation of their innovations, resulting in a non-user centered design. Furthermore, for local users to come to trust the innovative technologies and change their usual behavior requires comprehensive and often continual training, as well as time. To avoid some of the obstacles many of the innovators have faced during implementation, PAEGC could consider additional intensive, up-front review of proposal documentation by external technical experts who have deep understanding of in-country context and are capable of providing expertise on the viability of the proposed clean energy solutions, business models and financial mechanisms. These experts may also provide technical assistance on whether the proposed project timelines and targets are realistic, and can help the innovators assess the local Figure 11: Progress Towards Innovator Initial Goal 29 PAEGC Mid-Term Innovators’ Assessment policy environment, which could benefit the scaling up efforts in the form of subsidies or duty-free exemption of certain types of equipment. Interestingly, even though many of the innovators are in the initial stages of their scaling up efforts, some of their partners have benefited from the implementation of their PAEGC award and have reached early adoption or commercialization of their technologies. These partners include SparkMeter and Zero Base Energy partnered with EarthSpark International, Futurepump of iDE, and VIP, the sub-awardee of CAMCO. Rebound Technologies incorporated the lessons learned from their R&D efforts into development of their IcePoint™ technology. *Motivo’s assessment is based on the phone interview, as no project site visit has taken place due to the delays in shipping of the CES equipment. 30 PAEGC Mid-Term Innovators’ Assessment 6. Final Observations and Recommendations The progress and likely outcomes for PAEGC 2013 cohort are consistent with other grand challenges or open competitions that source and develop high potential solutions to overcome critical barriers to development through the application of scientific, technolo gical and engineering methods. One of the oldest such competitions, which has been in existence since 1958, is the U.S. Federal Government Defense Advanced Research Projects Agency (DARPA) which funds “high risk/high gain” initiatives. This implies that there is an expectation that some of the funded initiatives can fail and in fact, DARPA’s success rate is only 10% or less. Despite such a low success rate, DARPA has laid the basis for many important innovations, such as the internet, and the U.S. Government considers this approach so important to basic scientific research that it funds the DARPA activity at about $2.9 billion annually. Another way to look at such research is through the eyes of venture capital investors, who based on a study of 2000 companies that received investments of $1 million each during the 2004-2010 period only had a success rate of 25%9 . The innovations selected by PAEGC could also be considered high risk/high gain due to their potential to be transformative. Therefore, the most important factor in assessing PAEGC is not whether the individual clean energy solutions fail or do not achieve their original goals; rather, it is whether the grand challenge is supporting truly innovative research and testing, and that those activities that are successful, or have a high likelihood of success, receive the support necessary to test, adapt and validate new models of clean energy generation for agriculture. If the above examples provide the parameters for such an effort, the Partners should expect a success rate of 10% to 25% and must take steps to support these potentially successful innovations to get from one step to the next. The 2013 innovators are generally engineers or technical specialists, and that has resulted in a heavy focus on technology to the exclusion of the customer/end user and the business model. Therefore, the following should continue to be the areas of PAEGC support to the innovators to enhance the chances of success: 1. Monitoring and evaluation. The innovators need to measure the effectiveness of their clean energy solution, document the performance over time and substantiate their results with evidence in order to communicate those results to potential impact investors. Many of the 2013 innovators still do not understand the value of collecting data during the field￾testing phase and are missing this opportunity. The site visits conducted by PASTO presented an opportunity to collect substantial information that was not always available to or collected by some innovators, particularly those situated outside of the country of implementation, or those without strong local partners. 2. Business acceleration. Many of the 2013 innovators are only now starting to identify their business model and need to think through key partners, value propositions, market demand, customer segments, willingness of customers to pay, delivery channels, cost structure and revenue streams. 3. Gender Integration. PAEGC has been building the awareness of the importance of the role of gender dynamics the development and deployment of the clean energy solution. 9 The Venture Capital Secret: 3 Out of 4 Start-ups Fail, Wall Street Journal, September 20, 2012 31 PAEGC Mid-Term Innovators’ Assessment Innovators now need assistance to integrate specific gender-mainstreaming actions into their individual projects. 4. Partnerships. The innovators that have made the most progress have excellent in-country networks. Leveraging of the in-country connections of the PAEGC Partners is critical to successful implementation and scale-up. Based on the findings from the assessment, the recommendation with regard to PAEGC support to specific innovators is as follows:  Intensify business acceleration support—SunDanzer, iDE and Promethean  Support the documentation of field testing results through M&E support—UGA, Motivo, African Bamboo  Cease support—EarthSpark, Earth Institute and Rebound In closing, the lessons learned from the assessment are as follows:  The survey should have been tested in order to eliminate redundancies, add clarity and precision to questions, and establish a more logical order.  Decoupling the questionnaire and the M&E data verification into two separate calls would have been less confusing to the Innovators.  There is a limitation to an assessment done virtually. It would be better if the survey had been administered during the site visits.  The assessment would have been enhanced by first completing all the site visits and have a fuller picture of the on the ground activities.  PASTO is not the best mechanism to undertake the next assessment if it needs to be truly independent and impartial given Tetra Tech’s role as a USAID contractor and hence PASTO’s close relationship with the innovators. 32 PAEGC Mid-Term Innovators’ Assessment Appendix A: Mid-Term Evaluation Scope of Work PAEGC MIDTERM EVALUATION SCOPE OF WORK March 1, 2016 Purpose of the Midterm Evaluation The Powering Agriculture: An Energy Grand Challenge for Development (PAEGC) is a partnership between USAID, the Government of Sweden, the Government of Germany, Duke Energy Corporation and the Overseas Private Investment Corporation. The goal of PAEGC is to support new and sustainable approaches to accelerate the development and deployment of clean energy solutions for sustainable approaches to accelerate the development and deployment of clean energy solutions 10 for increasing agriculture production and/or value in developing countries. The PAEGC Monitoring and Evaluation Plan stated that PAEGC would undergo a performance evaluation and at the 2015 PAEGC Partners meeting the Partners agreed to move for ward on a midterm performance evaluation. It is desirable to undertake a midterm evaluation at this point, as a second set of innovators has been chosen who will be initiating their activities. The evaluation results and recommendations will enable the PAEGC Partners to make mid-course adjustments to improve the results created by both the initial 11 innovators and the subsequent group of 13 innovators. The midterm performance evaluation of PAEGC will follow evaluation guidelines listed in its PAEGC Monitoring and Evaluation Plan. The purpose of the evaluation, based on technical direction from the PAEGC Program Manager, is to determine:  How well PAEGC executed the procurement and award process, from the viewpoint of the innovators  After the award, how well PAEGC managed the awards  Based on their CES application, what the innovators have achieved since the award  How well the innovators addressed gender equity through their innovations and subsequent award activities. Methodology For PAEGC, Monitoring Activities are conducted at three different levels for different purposes:  Program-level: to measure the aggregated contributions of PAEGC’s interventions in achieving the program’s Goal. This is also referred to as the GCD-level.  Innovator-level: to measure the progress and impact of PAEGC’s individual innovators, and individual contributions to achieving the PAEGC goal. 1010 Clean energy is defined as: Usable energy (i.e. electricity, illumination, heating/refrigeration, mechanization) that is derived from renewable sources and supports a reduction in fossil fuel use, increase in efficiency, and/or limitation of greenhouse gas emissions. Clean energy sources include – solar, hydro, wind, geothermal, sustainably harvested biomass, and biogas. The term “clean energy solution” is defined as: A combination of appropriate technology and a business model that addresses the clean energy demands of a select market. 33 PAEGC Mid-Term Innovators’ Assessment The midterm performance evaluation will be conducted in two stages:  Stage 1 will focus on the on the program-level and will be conducted by an external consultant to be contracted by PAEGC Founding Partner BMZ/GIZ. It will conducted from April 2016 to June 2016.  Stage 2 will focus on the innovator level and will be conducted by the Monitoring and Evaluation (M&E) Specialist of the USAID funded Powering Agricultural Support Task Order (PASTO). It will be conducted from January to April 2016. Stage 2: Innovator Level Performance Status Report The innovator level performance status report will focus on the assessing the performance of the 2013 innovators. It will be primarily qualitative in nature and will entail analysis of interview responses and reported performance indicator data. The evaluation will consist of:  An analysis of reported performance indicators  Trip reports of visits to innovator project sites  Document reviews of innovator proposals, award documents, milestones, deliverables and reports  Interviews with the innovators, PASTO personnel, and USAID Agreement Officer Representatives (AORs) The interviews will take place in conjunction with a data verification process that will also be undertaken by PASTO over January to April 2016. The interview responses will be confidential and not attributable to the interviewees. Specific evaluation questions were previously developed as a part of the PAEGC Monitoring and Evaluation Plan. Innovator Level Performance Status Questions Questions on Procurement, Award Process and the Management of the Award 1. In your opinion, how well did you feel the call for proposals was executed by PAEGC? Rate execution by PAEGC as 5-excellent; 4-quite good; 3-good; 2-not so good; and 1-poor. Please explain. 2. Were appropriate selection criteria used by PAEGC? Y/N/Maybe Please explain. 3. What additional selection criteria might have been used? 4. Was the application process straightforward and logical or was it confusing? Y/N If the process was confusing, what would have improved the process? Please explain. 5. Were there significant barriers/delays caused by PAEGC prior to your organization receiving an award? Y/N/Maybe If yes, what were the barriers/delays that your organization faced? 6. Were these barriers/delays addressed by PAEGC? Y/N 7. Were any barriers/delays not adequately addressed by PAEGC? Y/N Please explain. 8. Were the annual milestones that you set for your organization realistic or not? Y/N Please explain. 9. To what extent was your CES developed and implementable upon award of the award? Rate it 5 to 1 (5-development completed; 4-almost fully developed; 3-somewhat developed; 2-minimally developed; 1-concept stage only) 10. Have any of the PAEGC Partners used their in-country presence to assist and complement your activities? Y/N If yes, what did they do? 11. Would you consider your CES to be high-risk, high-reward? Y/N 34 PAEGC Mid-Term Innovators’ Assessment Please explain. 12. What is the name of your AOR? 13. How would you describe your relationship with your AOR? Rate on a scale of 5 to 1 as 5-excellent; 4-highly positive; 3-good; 2-not so good; and 1-poor or non-existent. 14. What else could your AOR do to be of assistance to your organization’s activities in the Powering Agriculture program? 15. Were there unnecessary or undue requirements placed upon the innovators by PAEGC that hindered implementation? Y/N/Maybe Please specifically identify such requirements and tell why they were unnecessary or burdensome. 16. Do you have any specific suggestions that would improve PAEGC’s award management practices? Y/N Please explain. 17. How would you rate the assistance PASTO has provided you on a scale of 5 to 1? 5-excellent ___; 4-highly positive; ___; 3-good___; 2-not so good; 1-poor or non-existent 18. What areas has PASTO best assisted your needs? (1-USAID Contractual Compliance; 2- USAID Environmental Compliance; 3-Milestones review; 4-Milestone Modifications ; 5-Communications ;6-Assistance with Events ;7-M&E ;8- Business Acceleration; 9-Other(explain)) 19. Have you used WebMo? Y/N Is this a useful tool for reporting your results? Y/N Please explain. Questions on Performance under the Intermediate Results The Monitoring and Evaluation work carried out by the innovators includes self-reporting on a series of indicators under three Intermediate Results:  IR1: Increase in farmers and agribusinesses’ access to and/or use of clean energy solutions  IR2: Increase in agricultural production and/or value among famers and agribusinesses  IR3: Increase in support for low carbon economic growth within the agricultural sector Data associated with these indicators are entered into WebMo, an online monitoring platform, by the innovators and reviewed by the USAID and PASTO. These data will be analyzed by the M&E Specialist as part of the evaluation and conclusions will be developed regarding innovator performance to date. Other questions regarding the innovator’s performance on the three Intermediate Results (IR) indicators, similar to those shown below. IR1: Increase in farmers and agribusinesses’ access to and/ or use of clean energy 1. In your opinion, was your organization able to successfully demonstrate your CES’s commercial viability? Y/N/Not yet Please explain. 2. In the country/ies in which your organization is working, what is the total estimated demand for the CES supported by PAEGC? No. of units _________ $ sales _________ Don’t know______ 3. Have wholesalers and retailers undertaken to sell your CES to farmers or agribusinesses? Y/N 4. Are other service providers leasing your CES to users? Y/N 35 PAEGC Mid-Term Innovators’ Assessment 5. What is the CES retail or prototype price in relation to farmer and agribusiness average annual income? ___% Do you consider that the product is affordable? Y/N Please explain. 6. Is your organization making additional changes to achieve affordability? Y/N If yes, what changes are you undertaking? 7. So far, have there been actual CES sales to farmers and agribusinesses? Y/N How many units? _____ 8. If no sales have been achieved, what further steps are you making to bring effective clean energy solutions to commercial scale within the agriculture sector of developing countries? Please explain. 9. If you are at prototype and not sales level, have you been able to test to your CES prototype successfully? Y/N/Not yet 10. If you are at prototype and not sales level, has your CES prototype been tested with actual farmers and agribusinesses? Y/N/Not Yet 11. How many women have access to the CES on your farm or agribusiness? IR2: Increase in agricultural production and/or value among farmers and agribusinesses 1. Have you been able to demonstrate through your field testing or product roll-out that your CES increases either agricultural production or value, or both? Agricultural production Y ___ N ___ Product value Y ___ N ___. Please provide quantities of production or value if you know them: Agricultural production increases (%)____ Product value increases (%)___ 2. What is the level of women’s involvement in the use of your CES? 5-very high;; 4-high; 3-medium; 2-not so high; 1-low or nonexistent 3. If your users are agribusinesses, what total volume (in standard units used as the Indicator) and sales value of farm products was handled by them while utilizing your CES? Provide total volume and sales volume statistics: production units_____ sales (US$)______ 4. Have the volumes and sales value increased/remained the same/decreased? By what percentage? Volume _________% Sales (US$) _______% 5. According to your knowledge, are there other similar CES technologies or systems utilized in the countries where you are working result in increased agricultural production or decreases in post-harvest losses? Y/N Please give the country and CES type IR3: Increase in support for low carbon economic growth within the agriculture sector 1. Have there been increases in country-level investments supporting production of your CES? Y/N/Not yet Please elaborate, if yes. 2. In your opinion, based on your organization’s activities has there been investment in other similar CES devices for agricultural or agribusiness applications? Y/N/ Please explain. 3. Based on your analysis, will your CES have impact on mitigating the adverse effects of global climate change (GCC) within the agriculture sectors of country/countries in which your organization is working? Y/N Please explain. 36 PAEGC Mid-Term Innovators’ Assessment Innovator Overall Performance: In your opinion, how would you rate your progress towards your initial goal when you applied for the grand challenge? Rate execution by PAEGC as 5-excellent; 4-quite good; 3-good; 2-not so good; and 1-poor. Please explain your answer Final Interview Question: Is there some other comment that you would like to make regarding the effectiveness of the PAEGC program Additional questions may be introduced by the M&E Specialist during the conduct of the interview, which may generate a more complete understanding of innovator performance. Duration The midterm innovator level performance status report will be carried out over 40 person-days, spread over January to April 2016 including time spent on interviews, data collection, analysis and drafting of the report. The PASTO M&E Specialist will not undertake field work, but rather the data collection will be carried out by direct personal interviews when the person is located in Washington, D.C. and virtually for those interviewees located elsewhere in conjunction with the data verification process. The first draft will be submitted on April 15, 2016 and once comments are received from the PAEGC Partners, the M&E Specialist will finalize the report and submit the final to the Program Manager by May 15, 2016. Report Contents The output shall be an innovator performance status report which will form part of the larger evaluation. The innovator performance status report will set forth important findings, conclusions, and recommendations. It will include an executive summary and be no more than 30 pages, not including tables, charts and annexes. It will contain the following sections: A. Introduction B. Procurement and Award Process C. Award Management Process D. innovator Performance (by IR1, IR2 and IR3) E. Conclusions F. Lessons Learned G. Recommendations 37 PAEGC Mid-Term Innovators’ Assessment Appendix B: Performance Indicator Data, Fiscal Year 2015 Below is the M&E indicator data gathered from the 2013 cohort of innovators based on PASTO’s site visits, interviews with the local partners and phone calls to verify the data submitted on WebMo. This table reflects the field testing activities up to the end of September 2015. Since the majority of the innovators have started substantially field testing their units this year, their impact is not reflected in the table below. Some data, as indicated by the color coding is still in the process of being collected, analyzed or updated by the innovators. Verified by a site visit Under review by PASTO with the innovator Verified by phone Not verified No data reported The PAEGC M&E plan defines the performance indicators as follows: IR 1.1: Type and number of clean energy solutions developed (and state of development) IR 1.2: Type and number of beneficiaries (farms/agribusinesses/customers) IR 1.3: Type and number of wholesalers/retailers/maintenance professionals accessible to beneficiaries for selling/servicing clean energy solutions IR 1.1 Developmen t Stage Units Deployed IR 1.2 # of women IR 1.3 # new IR 1.4 MW IR 1.5 # of women IR 2.1 % change IR 2.2 Savings in FY15 $ per beneficiary IR 2.2 Annual Savings $ per beneficiary IR 2.3 Savings in FY15 MJ per beneficiary IR 2.3 Savings in FY15 MJ IRR 2.3 Annual Energy Savings MJ per beneficiary IR 3.1 $ IR 3.2 tCO2 eq. African Bamboo 1 field testing 0 0 0 3 0 0 0 0 0 0 0 0 0 $3,251,447 0 CAMCO 1 fiel testing + R&D 5 15 0 3 3 0.035 9 1 0 0 0 0 0 0 0 400,000 $ 0 Earth Institute 1 field testing 3 4 0 8 3 0.0164 36 0 1.3 t/ha 4.3% $ 128 $ 256.16 1,014.2 4,056.7 3,381 $0 1.16 Earth Spark 1 commercializ ation 1 408 233 5 5 0.093 890 233 0 0 0 0 42,830.0 $480,000 987 iDE 1 field testing 26 26 10 13 0 0.00208 42 10 $0 14.5 Motivo Engineering 2 R&D 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 $0 0 Promethean 1 field testing 51 2720 773 124 2 0.002 73 1 1.37 -13 $0 Rebound 0 R&D 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 SunDanzer 1 field testing 40 46 6 2 0 0.0204 94 20 17 L/day 20.0% $ 918 $ 2,203 -972 -38,880 -2,333 $0 0 UGA 1 field testing 0 0 0 0 0 0 0 0 0 0 0 0 0 0 $0 0 38 PAEGC Mid-Term Innovators’ Assessment IR 1.4: Clean energy generation capacity installed or rehabilitated (in MW) IR 1.5: Number of persons attending trainings/demonstrations on CES technology IR 2.1: Change in agriculture production attributed to use of PAEGC Innovators’ clean energy solution IR 2.2: Change in income attributed to use of PAEGC Innovators’ clean energy solution IR 2.3: Expected life of project savings from energy efficiency or energy conservation (annual) IR 3.1: US Dollar amount of investment mobilized, from public or private sources, for climate change IR 3.2: Greenhouse Gas (GHG) emissions, estimated in metric tons of CO2e, reduced, sequestered, and/or avoided (annual tCO2 eq.) 39 PAEGC Mid-Term Innovators’ Assessment Appendix C: Raw Survey Data [Innovator responses have been removed from the public version as it is considered SBU, and contains sensitive financial and/or technical information that could damage the competitive advantages of the companies involved.] The Procurement and Award Process: 1. In your opinion, how well did you feel the call for proposals was executed by PAEGC? Rate execution by PAEGC as 5-excellent; 4-quite good; 3-good; 2-not so good; and 1-poor. Please explain. 2. Were appropriate selection criteria used by PAEGC? Y/N/Maybe. Please explain. 3. What additional selection criteria might have been used? Y/N. Please explain. 4. Was the application process straightforward and logical or was it confusing? Y/N. If confusing, what would have improved the process? 5. Were there significant barriers/delays caused by PAEGC prior to your organization receiving an award? Y/N/Maybe, If yes, what were the barriers/delays that your organization faced? 6. Were these barriers/delays addressed by PAEGC? Y/N 7. Were any barriers/delays not adequately addressed by PAEGC? Y/N. Please explain. 8. Have you used WebMo? (Y/N). 8b. Is this a useful tool for reporting your results? Y/N. Please explain. 9. Were the annual targets that you set for your organization realistic or not? Y/N. Please explain. 10. To what extent was your CES developed and implementable upon award of the award? Rate 5 to 1 (5-development completed; 4-almost fully developed; 3-somewhat developed; 2- minimally developed; 1-concept stage only). 11. Have any of the PAEGC Partners used their in-country presence to assist and complement your activities? Y/N. If yes, what did they do? 12. Would you consider your CES to be high-risk, high-reward? Y/N. Please explain. Award Management Process: 1. What is the name of your AOR? 40 PAEGC Mid-Term Innovators’ Assessment 2. How would you describe your relationship with your AOR? (5 to 1; 5 –excellent, 1 – not so good) 3. What else could your AOR do to be of assistance to your organization’s activities in the Powering Agriculture program? 4. Were there unnecessary or undue requirements placed upon the innovators by PAEGC that hindered implementation? Y/N/Maybe. Please specifically identify such requirements and tell why they were unnecessary or burdensome. 5. Do you have any specific suggestions that would improve PAEGC’s award management practices? Y/N. Please explain. 6. How would you rate the assistance PASTO has provided you on a scale of 1 to 5? (5 to 1) 7. What areas has PASTO best assisted your needs? (1 – USAID Contractual Compliance, 2- USAID Environmental Compliance, 3-Milestone review, 4-Milestone Modifications, 5- Communications, 6-Assistance with Events, 7-M&E, 8-Business Acceleration, 8-Other). Please explain. Innovator Overall Performance: 1. Now that you’ve answered these questions, how do you rate your progress towards your initial goal when you applied for the grand challenge? Rate execution by PAEGC 5 (high) to 1 (low) 2. Is there some other comment you would like to make regarding the effectiveness of the PAEGC program? Innovator & CES performance: 1. In your opinion, how would you rate your progress towards your initial goal when you applied for the grand challenge? Rate 1(low) to 5(high). Please explain. 2. What have you observed to be advantages and disadvantages of your CES? 3. Have you been able to demonstrate through your field testing or product roll-out that your CES saves the beneficiaries money, as compared to the traditional technology? Y/N. 3b. What is the source of savings? 3c. What are the savings? ($____________________, %____________________) 4. Have you been able to demonstrate through your field testing or product roll-out that your CES increases agricultural production? Y/N/Don’t know. What is the agricultural production increase? (Production unit________; Volume %________, Sales %_______) 41 PAEGC Mid-Term Innovators’ Assessment 5. Have you been able to demonstrate through your field testing or product roll-out that your CES improves quality of agricultural production? Y/N/Don’t know. 5b. If yes, do the beneficiaries earn more income from selling higher quality goods on the market? Y/N. Please explain. 5c. What is the product value increase? ($_____________, %___________________) 6. Have you been able to demonstrate through your field testing or product roll-out that your CES result in time savings when compared to the traditional technology? Y/N. 6b. If yes, how many hours per day/week? 6c. What do the beneficiaries do with the gained free time? 7. If your users are agribusinesses, what total volume and sales value of farm products was handled by them while utilizing your CES? 7b. Provide total volume and sales volume (production units______, $______). 8. According to your knowledge, are there other similar CES technologies or systems utilized in the countries were you are working that result in increased agricultural production or decrease in post-harvest losses? Y/N 8b. Please describe CES and note the country. Additional questions for irrigation projects: 1. Does your CES utilize an innovative payment scheme? Y/N. Please explain. 2. How does the CES affect the ground water table in the area? How do you know? 3. Is the payment scheme based on water consumption or solely on electricity consumption? Scaling up & commercialization of the CES: 1. Are there personnel within your organization who are responsible for your scaling up activities? Y/N 1b. How many? _________) 2. In the country(ies) in which your organization is working, what is the total estimated demand for the CES supported by PAEGC? 3. Has your organization developed a business model? Y/N. If yes, please describe. 42 PAEGC Mid-Term Innovators’ Assessment 4. In your opinion, was your organization able to successfully demonstrate your CES’s commercial viability (i.e. it can be sold in the local market? Y/N. Please explain. 5. So far, have there been actual CES sales to farmers and agribusinesses? Y/N. 5b. How many units? _______ 6. Have wholesalers and retailers undertaken to sell your CES to farmers or agribusinesses? Y/N. 7. Are other service providers leasing your CES to users? Y/N. 8. What is the actual CES retail price? ($______________, NA) 9. Or what is the estimated prototype price? ($_________, NA) 10. What is the CES retail or prototype price in relation to farmer and agribusiness average annual income? (%_____________) 10b.Do you consider that the product is affordable? Y/N. Please explain. 11. Is your organization making additional changes to achieve affordability? (Y/N) 11b. If yes, what changes are you undertaking? 12. If you are at prototype and not commercialization stage, have you been able to test your CES prototype successfully? Y/N/Not yet. 13. If you are at prototype and not commercialization stage, has your CES prototype been tested with actual farmers and agribusinesses? Y/N/Not yet. 14. If no sales have been achieved, what further steps are you taking to achieve commercialization? 15. What other hurdles is your organization facing in scaling up of your CES? 16. Are you receiving support for scaling up your CES? 16b. If yes, what type of support are you receiving and from whom? 16c. Are you satisfied with the level and quality of these services? (Y/N) 17. Have you found any potential partners or stakeholders to support the operation and maintenance of your CES installations and to scale up the CES in the region or support its adoption elsewhere? (Y/N) 43 PAEGC Mid-Term Innovators’ Assessment 17b. How did you identify these partnerships? What is their operational/financial capacity? Impact on gender: 1. What is the level of women’s involvement in the use of your CES? Rate 1 (low) to 5 (high). Please explain. 2. Does your organization collect any gender-related impact data in addition to the required disaggregates? 3. Does the project have any additional plans to further incorporate the equitable participation of women and men in the field activities? (Y/N) 3b. If yes, how? 4. At what stage in the value chain where you CES is being applied are women involved (e.g. management, carrying our training or demonstrations, selling of the units, repair)? 5. What are the barriers in the way of women benefiting more from the CES in the community? 5b. If it is lack of financial resources, does the project or local partners have any plans to facilitate overcoming this barrier? Y/N. Please explain. Impact on low carbon economic growth: 1. Based on your analysis, will your CES have impact on mitigating the adverse effects of global climate change within the agriculture sectors of country/countries in which your organization is working? (Y/N, explain___________________) Chapter: Annex Page 66 | 66 Syspons GmbH Prinzenstraße 84, Aufgang 1 10969 Berlin Lennart Raetzell Manager Telefon: 0151/26 460 483 E-Mail: lennart.raetzell@syspons.com © Syspons 2016. All rights reserved.