WORKERS AT SEWING MACHINES AT A GARMENT FACTORY IN ACCRA, GHANA. CREDIT: MACDONALD ACQUAH, MSI EVALUATION MID-TERM EVALUATION OF THE INVEST MECHANISM October 8, 2020 This publication was produced at the request of the United States Agency for International Development for the E3 Analytics and Evaluation Project. It was prepared independently by Management Systems International, A Tetra Tech Company. (THIS PAGE INTENTIONALLY LEFT BLANK) MID-TERM EVALUATION OF THE INVEST MECHANISM Contracted under AID-OAA-M-13-00017 E3 Analytics and Evaluation Project Prepared by: Rebecca Batts, Team Lead (MSI) Jacob Patterson-Stein, Evaluation Specialist (MSI) DISCLAIMER The authors’ views expressed in this report do not necessarily reflect the views of the United States Agency for International Development or the United States Government. (THIS PAGE INTENTIONALLY LEFT BLANK) ABSTRACT This mid-term performance evaluation examined USAID’s INVEST mechanism, which Agency offices and missions can buy in to fund finance-related activities. Members of an INVEST-facilitated partner network can then submit proposals to work on INVEST buy-ins. This report answered three main questions: 1. How has INVEST operationalized the mobilization of private capital approach through its network of participating firms? 2. How has INVEST captured and validated lessons learned? 3. How has INVEST communicated to USAID and other stakeholders about its mobilization of private capital approach? The evaluation team conducted 64 key informant interviews with representatives of USAID, INVEST, and partner firms that have worked on INVEST activities. The team also reviewed available documentation and data, conducted an online survey of partner network firms, and conducted two case studies of buy-ins for the Southern Africa Regional and Vietnam Missions. The evaluation found that INVEST was valued as administrative and procedural support for USAID missions and offices to procure technical activities faster than traditional procurement processes. More than a third of the USAID and partner firm stakeholders reported they had not engaged with any INVEST learning materials or documents. Communication gaps about the INVEST process created confusion and distrust among some partner firms and USAID stakeholders. Evaluation recommendations focused on ways to improve communication and transparency around INVEST procurement processes; capture, validate, and act on learning; enhance access to and the utility of communications and technical products; and clarify standards around intellectual property and firm relationships. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM ii CONTENTS ABSTRACT i ACRONYMS v ACKNOWLEDGEMENTS vi EXECUTIVE SUMMARY vii INTRODUCTION 1 INVEST BACKGROUND 1 IMPLEMENTATION DEVELOPMENT HYPOTHESIS 2 EVALUATION PURPOSE, AUDIENCES, AND INTENDED USES 2 PURPOSE 3 AUDIENCES 3 INTENDED USES 3 EVALUATION DESIGN 3 EVALUATION QUESTIONS 3 DATA COLLECTION APPROACH 4 SAMPLE 5 ANALYSIS APPROACH 7 THEMATIC/CONTENT ANALYSIS 7 DESCRIPTIVE STATISTICS 8 ANALYTICAL REVIEW OF INVEST REPORTS AND SUPPORTING DOCUMENTATION 8 TRIANGULATION 8 GENDER ASPECTS OF EVALUATION QUESTIONS 8 STRENGTHS AND LIMITATIONS OF THE EVALUATION DESIGN 9 STRENGTHS 9 LIMITATIONS 9 EVALUATION TEAM 10 FINDINGS AND CONCLUSIONS 11 EQ1: HOW HAS INVEST OPERATIONALIZED THE MOBILIZATION OF PRIVATE CAPITAL APPROACH THROUGH ITS NETWORK OF PARTICIPATING FIRMS? 11 SUMMARY RESULTS FOR EQ1 11 INVEST OPERATIONS 11 ADMINISTRATIVE AND PROCEDURAL SUPPORT 15 EQ1A: HOW HAS INVEST FACILITATED THE PROCUREMENT PROCESS FOR MISSION AND OPERATING UNIT BUY-INS TO THE ACTIVITY? 18 MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM iii EQ1B: HOW HAS THE NETWORK OF PARTICIPATING FIRMS UNDER INVEST EXPANDED AND CHANGED OVER TIME? 21 EQ 2: HOW HAS INVEST CAPTURED AND VALIDATED LESSONS LEARNED? 24 SUMMARY RESULTS FOR EQ2 24 CAPTURING AND VALIDATING LESSONS LEARNED 25 EQ2A: HOW HAS INVEST ACTED UPON CAPTURED LEARNING AND RESEARCH? 27 EQ2B: HOW HAVE MISSIONS, OPERATING UNITS AND NETWORK MEMBER FIRMS BENEFITED FROM INVEST LEARNING AND RESEARCH? 29 EQ3: HOW HAS INVEST COMMUNICATED TO USAID AND OTHER STAKEHOLDERS ABOUT ITS MOBILIZATION OF PRIVATE CAPITAL APPROACH? 29 SUMMARY RESULTS FOR EQ3 29 COMMUNICATION ABOUT THE MOBILIZATION OF PRIVATE CAPITAL APPROACH 30 COMMUNICATION ABOUT INVEST PROCUREMENT PROCESSES 31 TRANSPARENCY AND COMMUNICATION 34 INTELLECTUAL PROPERTY 36 OVERSIGHT 37 EQ3A: HOW HAVE OTHER DONORS, HOST COUNTRY COUNTERPARTS, TARGETED BENEFICIARIES AND OTHERS LEARNED ABOUT INVEST’S MOBILIZATION OF PRIVATE CAPITAL APPROACH? 38 EQ3B: HOW HAVE MISSIONS AND OPERATING UNITS MODIFIED THEIR PROGRAMMATIC/STRATEGIC EFFORTS TO INTEGRATED MOBILIZATION OF PRIVATE CAPITAL MODELS, PARTICULARLY THE LEVEL OF BUY-INS, THE NUMBER OF FOLLOW-ON BUY-INS AND FUTURE EXPRESSIONS OF INTEREST? 38 RECOMMENDATIONS 40 RECOMMENDATIONS FOR INVEST 40 RECOMMENDATIONS FOR USAID AND INVEST 40 RECOMMENDATIONS FOR USAID/E3/PCM 40 ANNEX A: EVALUATION STATEMENT OF WORK 41 ANNEX B: INVEST BUY-INS FY17-19 49 ANNEX C: GETTING TO ANSWERS MATRIX 50 ANNEX D: ANONYMIZED RESPONDENT LIST 51 ANNEX E: KEY INFORMANT INTERVIEW GUIDES 53 USAID/E3/PCM INTERVIEW GUIDE 53 NETWORK PARTNER INTERVIEW GUIDE 57 INVEST STAFF INTERVIEW GUIDE 60 BUY-IN CLIENT INTERVIEW GUIDE 64 MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM iv ANNEX F: DOCUMENTS REVIEWED 67 ANNEX G: ONLINE SURVEY INSTRUMENT 75 ANNEX H: SUPPLEMENTAL ONLINE SURVEY FINDINGS 79 ANNEX I: CASE STUDIES 82 ANNEX J: EXAMPLE OF DATA TRIANGULATION 95 ANNEX K: CROSS-WALK OF FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS BY EVALUATION QUESTION 97 ANNEX L: STATEMENT OF DIFFERENCES 100 MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM v ACRONYMS BAA Broad Agency Announcement CBO Community-Based Organization E3 Bureau for Economic Growth, Education, and Environment (USAID) EQ Evaluation Question GenDev Office of Gender Equality and Women’s Empowerment (USAID/E3) GVN Government of Vietnam KII Key Informant Interview MBIO Mission, Bureau, and Independent Office MCEDF Municipal Clean Energy Development Facility MSI Management Systems International NDA Non-Disclosure Agreement NPT National Power Transmission Corporation OAA Office of Acquisition and Assistance (USAID/Bureau for Management) PCM Office of Private Capital and Microenterprise (USAID/E3) PIVOT Practical InnoVative On-the-Job Training PPP Public-Private Partnership PSE Private Sector Engagement RFP Request for Proposals SAR Southern Africa Regional SE Social Enterprise SOO Statement of Objectives SOW Statement of Work USAID United States Agency for International Development WE4 Water and Energy for Food MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM vi ACKNOWLEDGEMENTS The evaluation team thanks all interviewees who took time to share their perspectives and experiences with the team while adjusting to work and life during a pandemic. The team could not have implemented the evaluation without the ongoing and patient cooperation of INVEST staff, particularly Stephanie Fenner. The team appreciates the technical review and support for this report from MSI’s E3 Analytics and Evaluation Project team including Jeremy Gans, Doug Krieger, Bethany Little, and Amanda Janczak. The team also thanks Francis Higdon and Thomas Rojas (USAID/E3/PCM activity managers for this evaluation) and Cameron Khosrowshahi (USAID/E3/PCM COR for INVEST), as well as the E3 Analytics and Evaluation Project COR Dr. Bhavani Pathak, for their continued support of this study. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM vii EXECUTIVE SUMMARY This report presents the results of a mid-term performance evaluation of the United States Agency for International Development’s (USAID’s) INVEST mechanism. USAID’s Office of Private Capital and Microenterprise in the Bureau for Economic Growth, Education, and Environment (E3/PCM) commissioned the evaluation. USAID’s E3 Analytics and Evaluation Project designed and implemented the evaluation. INVEST BACKGROUND In September 2017, USAID issued the INVEST prime contract to DAI. This award followed a co￾creation process under the Catalyzing Private Investment broad agency announcement (BAA). The INVEST mechanism runs from September 2017 through September 2022. As of the end of year two, the contract funding consisted of core funding totaling around $30 million and a buy-in ceiling of $137 million, for a total contract ceiling of $167 million.1 INVEST is intended to develop and test a model built around blended finance (i.e., the strategic use of philanthropic or public resources to catalyze private capital flows in pursuit of development results and financial returns) that will enable USAID to quickly engage new partners and achieve development objectives. USAID missions and offices buy into INVEST to fund finance-related activities. Under this arrangement, DAI provides knowledge management; monitoring, evaluation, and learning; strategic communications support; and facilitates the sharing of information and best practices from buy-ins and subcontracts. Buy-ins are typically competed via separate requests for proposals (RFPs) or awarded through task orders to INVEST’s core partners and the network of partner firms that DAI monitors and manages. Over INVEST’s first two years, the network expanded significantly, from 27 to more than 205 organizations. Partner firms have responded to 32 buy-in opportunities, formed teaming arrangements, shared expertise and learning, and developed an integrated network of specialized partners. EVALUATION PURPOSE AND DESIGN The purpose of this evaluation is to summarize what has worked well and what could have been improved over INVEST’s first two years and provide recommendations for the remainder of the mechanism’s performance period. The evaluation took a mixed-methods approach and used three main data collection methods: • A desk review of INVEST performance reports, statements of objectives, RFPs and other procurement documents, field reports, promotional materials, communication and learning documents, and other available INVEST and partner firm documents and data; • Sixty-four key informant interviews with 60 different respondents covering all but 3 buy-ins, including staff from partner firms that have and have not worked on INVEST buy-ins, USAID buy-in stakeholders, USAID staff unrelated to any specific buy-in but who were familiar with or have worked with INVEST, and implementing partner staff; and • An online survey of partner network firms to understand how they have engaged with and used INVEST learning materials. 1 INVEST Year 2 Annual Report with Financials, slide 55. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM viii KEY FINDINGS AND CONCLUSIONS EQ1: HOW HAS INVEST OPERATIONALIZED THE MOBILIZATION OF PRIVATE CAPITAL APPROACH THROUGH ITS NETWORK OF PARTICIPATING FIRMS? The INVEST mechanism was intended to support and enhance USAID actions to mobilize private capital in support of development goals. INVEST’s annual and quarterly reports described a range of achievements including how INVEST operationalized its approach by identifying and assembling necessary private-sector expertise to assess market conditions, develop and structure financial solutions to development challenges, and utilize transaction advisory services to increase the availability of private capital to address sustainable solutions. Over its first two years, INVEST has operationalized the mobilization of private capital through its partner network by supporting timely, targeted procurement through subcontracts. The network of firms that participate in INVEST buy-ins has expanded steadily over time, although it is difficult to know exactly how based on the data INVEST collects on the network. The mobilization of private capital was not a major feature of INVEST’s first two years, although additional blended finance transactions are anticipated during subsequent phases of its operations. INVEST’s overall research goal is to develop and test a model that enables USAID to quickly engage new partners and mobilize private capital. DAI’s proposal further explained INVEST’s research goal as the generation and aggregation of field-tested evidence on what works in blended finance and why, to create a foundation of knowledge upon which USAID can build new partnerships, programs, and approaches. To this end, respondents highly valued INVEST’s effectiveness in working with USAID missions, bureaus, and independent offices (MBIOs) to quickly facilitate procurement, find relevant partners, and support buy-ins. USAID staff also praised INVEST’s flexible, tailored approach to address the challenges MBIOs faced and the strong project management skills of INVEST staff and cited these as reasons to seek additional buy-ins. Respondents valued how INVEST functioned as administrative and procedural support for USAID MBIO development initiatives, regardless of their connection to the mobilization of private capital. Network partners appreciated INVEST’s simplified procurement format and the speed of its streamlined procurement approach. EQ2: HOW HAS INVEST CAPTURED AND VALIDATED LESSONS LEARNED? INVEST has just begun to capture and validate the broad lessons to be learned as it executes its action research project. Some MBIOs and network partner firms viewed participation in INVEST as an experiment in finding new ways to conduct development activities. Because many of its efforts are at an early stage, INVEST has been limited in its ability to generate and disseminate learning about what works in blended finance and why. The limited generation of formal learning with respect to past buy-ins presents a risk that lessons will be lost over time. While having DAI as the primary facilitating partner can create collective knowledge across buy-ins, absent formal, documented learning processes there is little to mitigate staff changes and fading memories. It may be difficult or impossible to validate research after the fact if careful plans have not been made to collect and analyze information and materials during the research process. INVEST risks losing much of the benefit of its efforts if intentional and careful plans are not made to collect and validate its action research as it unfolds. Stakeholders, including staff from USAID and network partner firms, have not made wide use of the learning materials INVEST has shared to date. This is in part because of limited awareness of how to access learning products, especially technical materials, and stakeholders sometimes describing available learning products as not particularly relevant to their needs or not technically useful. USAID staff reported they would value having greater access to INVEST’s various learning documents, exemplars, templates, and studies. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM ix EQ3: HOW HAS INVEST COMMUNICATED TO USAID AND OTHER STAKEHOLDERS ABOUT ITS MOBILIZATION OF PRIVATE CAPITAL APPROACH? INVEST had not effectively communicated critical information about aspects of its mobilization of private capital approach to partner firms and some USAID staff. Processes from the INVEST BAA that made it possible to quickly award task orders to core partners have not been communicated, which has led to partner network firms questioning the fairness and transparency of procurement processes. This lack of transparency and limited communication has also fostered concerns related to how awards are selected generally and what oversight mechanisms exist. USAID staff valued working closely with INVEST staff, particularly one-on-one communications over media-based (e.g., internet, email, social media) efforts. Other donors, host-country partners, and targeted beneficiaries have not been the primary focus of INVEST’s communication in its first two years. INVEST’s events, workshops, and periodic reports have been useful for broadening the reach of its communication. INVEST has helped show some MBIO staff what is possible through a creative approach to procurement. Although at least four missions have multiple INVEST activity buy-ins, most USAID stakeholders felt their options were limited for integrating the mobilization of private capital into their strategic efforts due to bureaucratic challenges in the Agency. RECOMMENDATIONS RECOMMENDATIONS FOR INVEST • Seek feedback from unsuccessful bidders, especially new and underutilized partners, to understand how they perceive the debriefing materials. Enhance the quality and effectiveness of debriefings as warranted. • Develop and implement a methodology to capture and validate learning materials in real time, ensuring retention of important documents and memorialization of key decisions, including rationales. • Capture and act on lessons learned, even if they are not success stories, by creating a formal, documented process that can be tracked showing the challenge/solution, actions taken/not taken, etc. • Learn how stakeholders access and utilize learning and communication materials to better target MBIOs and partner network firms. • Develop and distribute a policy or set of standards/values/norms around intellectual property. Convene a webinar or other interactive session about the issue of working collaboratively with commercial competitors, and carefully consider the impact of relationships between firms within the partner network. RECOMMENDATIONS FOR USAID AND INVEST • Make all INVEST deliverables available for use in planning and future research efforts, unless there is a compelling and articulable reason not to do so. • Develop and publish a clear explanation of how work is distributed via the INVEST mechanism, including a high-level explanation of the procurement and contracting process and why some work may not be competed via the RFP process. Make a listing of all issued RFPs available for network partners’ easy reference. • Determine where the responsibility lies for protecting all parties to investment transactions from inappropriate or conflicting duties or roles. Locate the “bold line” regarding INVEST practices and MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM x ensure that activity designs reflect good practices in management control. RECOMMENDATIONS FOR USAID/E3/PCM • Conduct an assessment of the technical quality and utility of INVEST’s technical products, considering factors such as the quality, depth, and precision of content and the utility of such products to technically sophisticated users in USAID and the development or financial/investment communities. This review should assess the extent to which blended finance knowledge is conveyed and made accessible, as well as how Agency staff are utilizing such knowledge. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 1 INTRODUCTION This document describes the design, research methodology, and detailed findings, conclusions, and recommendations for a mid-term performance evaluation of the United States Agency for International Development’s (USAID’s) INVEST mechanism. USAID’s Office of Private Capital and Microenterprise in the Bureau for Economic Growth, Education, and Environment (E3/PCM) commissioned the evaluation. USAID’s E3 Analytics and Evaluation Project2 designed and implemented the evaluation. Annex A provides USAID’s approved statement of work (SOW) for this evaluation. INVEST BACKGROUND In September 2017, USAID issued the INVEST prime contract to DAI under contract AID￾OAA-C-17-00090. This award followed a co￾creation process under the Catalyzing Private Investment broad agency announcement (BAA). The INVEST mechanism runs from September 2017 through September 2022. As of the end of year two, total funding consisted of a core ceiling of around $30 million and a buy-in ceiling of $137 million, for a total contract ceiling of $167 million.3 INVEST is intended to develop and test a model built around blended finance that will enable USAID to quickly engage new partners, test and scale up approaches, improve programming, and mobilize capital to achieve development objectives. INVEST was designed to be a global, demand-driven mechanism. USAID missions, bureaus, and independent offices (MBIOs) can buy into INVEST to carry out finance-related activities. To implement buy-in activities, DAI can subcontract technical work either to CrossBoundary, Convergence, and Tideline – which are “core” INVEST subcontractors – or to other partner firms that are part of a broader and growing network created through the BAA process. While other contract mechanisms might involve USAID issuing awards to implement a set of activities over multiple years, this buy-in approach can allow for greater flexibility in what services are provided and over what period since buy￾in clients can choose to fund specific activities. The limit is simply the mechanism’s ceiling, above which INVEST cannot accept additional funds. INVEST was structured so DAI, as the prime contractor, would not be viewed as being in direct competition with partner firms for work. Under this contractual arrangement, DAI’s role is to provide knowledge management; monitoring, evaluation, and learning; strategic communications support, and facilitate the sharing of information and best practices from mechanism buy-ins and subcontracts. Contracts awarded under buy-ins are typically competed via separate requests for proposals (RFPs) or awarded through task orders to INVEST’s core partners and firms in the partner network. 2 Management Systems International (MSI), a Tetra Tech Company, implements the E3 Analytics and Evaluation Project in partnership with Palladium and NORC at the University of Chicago. 3 INVEST Year 2 Annual Report, slide 55. What is Blended Finance? Organizations define “blended finance” in different ways, but it generally refers to a financial transaction that combines concessional capital (i.e., money that is provided on favorable terms such as low or no interest) with commercial finance or other development finance or assistance. Some organizations focus on the mix of concessional and commercial capital, while others take a broader view that includes various de-risking strategies that might be non-financial. Despite the differences in approach and emphasis, the general idea is to “crowd-in” investment to achieve development goals. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 2 Over INVEST’s first two years, the network has expanded significantly, from 27 to more than 205 organizations.4 These network members have responded to 32 buy-in opportunities,5 formed teaming arrangements, shared expertise and learning, and developed an integrated network of specialized partners. Firms in INVEST’s partner network provide technical assistance to research blended finance approaches to lower transaction costs, addressing information barriers, mitigating risk/reward imbalances, providing expertise, facilitating partnerships, and creating opportunities for investors to connect. These activities are intended to increase the flow of capital into investment opportunities that drive measurable and sustainable social, economic, and environmental impacts. INVEST aims to provide: • Accelerated procurement tools to help USAID align to private-sector timelines; • Flexible instruments to periodically expand USAID partners’ specialized expertise; • Collaborative design for tailored support, rapid iteration, and learning for MBIOs; and • Greater awareness of and behavior change toward the private sector and blended finance approaches. IMPLEMENTATION DEVELOPMENT HYPOTHESIS The development hypothesis for INVEST is that if the mechanism’s partner network of firms provides technical assistance to support the deployment of private capital and blended finance, then this will lead to or facilitate buy-in client development outcomes. INVEST is intended to support USAID’s work to foster the ability of countries and communities to more effectively harness domestic resources, private sector capital, and market-based solutions. Through learning capture, INVEST proposes the emergence of “a clearly-defined set of lessons and best practices in blended finance for application by USAID as part of the Agency’s effort to transform development through private sector engagement. Unlocking private investment through blended finance, a key focus in USAID’s 2018 Private Sector Engagement (PSE) Policy…is a cornerstone of this broader transformation.”6 EVALUATION PURPOSE, AUDIENCES, AND INTENDED USES This evaluation was conducted at an opportune time. USAID and the international community are looking for ways to leverage private sector capital and opportunities to promote sustainable development outcomes. USAID and its implementing partners from international development and nongovernmental organizations have been exploring different approaches to blended finance and to 4 As of September 2019 (see INVEST Y2 Annual Report, slide 5). 5 As of December 11, 2019. Annex B provides a list of INVEST buy-ins that DAI provided as of that date. 6 INVEST Monitoring, Evaluation, and Learning Plan, January 31, 2019, page 1. INVEST Learning Goals INVEST’s learning strategy seeks to develop a knowledge base on what approaches to blended finance work best across different circumstances and to fill gaps in the evidence base surrounding private capital mobilization for development. Specifically, INVEST seeks to answer three core learning questions: 1. What is USAID’s role in mobilizing private capital? 2. How can USAID best access novel private sector expertise to implement blended finance solutions? 3. How can USAID better connect positive financial impact with sustainable development results? MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 3 engaging non-traditional development actors. However, there have been few evaluations of these attempts, particularly in the context of either blended finance or procurement facilitation. This is the first evaluation of the large contracts (e.g., INVEST and CATALYZE7) implemented as part of the new USAID PSE hub. PURPOSE The purpose of this evaluation is to summarize what has worked well and what could have been improved over INVEST’s first two years and provide recommendations for the remainder of the mechanism’s performance period. AUDIENCES E3/PCM and DAI are the primary audiences for the evaluation results. USAID missions and the private sector firms involved with the INVEST network are key secondary audiences, as well as the broader donor community – particularly those working on blended finance, knowledge management, and procurement facilitation. Other stakeholders likely to have an interest in the evaluation findings and conclusions include USAID/Washington operating units and targeted beneficiaries of INVEST buy-ins. INTENDED USES USAID will use the evaluation findings, conclusions, and recommendations as learning opportunities to understand what the INVEST mechanism has achieved, how it is perceived and valued, what opportunities for collaboration were available and utilized, and whether it is advisable for USAID to continue with such a model. DAI and other INVEST stakeholders will have an opportunity to learn about and utilize evaluation results regarding the mechanism’s strengths and areas for improvement. EVALUATION DESIGN EVALUATION QUESTIONS The evaluation was designed to respond to the following evaluation questions (EQs), which are identical to those in USAID’s approved evaluation SOW (Annex A): 1. How has INVEST operationalized the mobilization of private capital approach through its network of participating firms? a. How has INVEST facilitated the procurement process for Mission and Operating Unit buy-ins to the activity? b. How has the network of participating firms under INVEST expanded and changed over time? 2. How has INVEST captured and validated lessons learned? a. How has INVEST acted upon captured learning and research? b. How have Missions, Operating Units and network member firms benefited from INVEST learning and research? 3. How has INVEST communicated to USAID and other stakeholders about its mobilization of private capital approach? a. How have other donors, host country counterparts, targeted beneficiaries and others learned about INVEST’s mobilization of private capital approach? 7 CATALYZE, which USAID awarded to Palladium in November 2019, takes a similar approach to INVEST and also utilizes the partner network. USAID/E3/PCM designed both INVEST and CATALYZE as part of the newly initiated Private Sector Engagement Hub. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 4 b. How have Missions and Operating Units modified their programmatic/strategic efforts to integrated mobilization of private capital models, particularly the level of buy-ins, the number of follow-on buy-ins and future expressions of interest? DATA COLLECTION APPROACH This section briefly describes the multiple methods the team employed to triangulate information and fill gaps based on the approved evaluation design proposal. To collect evidence to develop the findings and conclusions to answer these EQs, the evaluation team employed a mainly qualitative approach. Table 1 summarizes the data collection methods the team used to answer each EQ and Annex C provides a getting to answers matrix with more details on the team’s approach for each EQ. The primary data collection method was 64 in-depth semi-structured key informant interviews (KIIs) with INVEST stakeholders, including USAID buy-in clients from missions and other operating units and firms that have worked with INVEST. The evaluation team also conducted an online survey and a desk review of all available INVEST documentation, as well as secondary sources, across each EQ to fill gaps in the primary data and provide comparative references to verify or contrast other data sources. TABLE 1: EVALUATION QUESTIONS AND ASSOCIATED METHODS Key informant interviews: the evaluation team created a separate interview template for each stakeholder group (e.g., INVEST staff, partner firm respondents). Annex D provides an anonymized respondent list and Annex E provides the KII guides. The team conducted all KIIs remotely through Microsoft Teams or Google Meet and obtained informed consent from all interviewees. By gathering information, observations, and insights from individuals with a range of perspectives and experiences, the team sought to identify common themes as well as areas of divergent views. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 5 Quantitative analysis of network data: the evaluation team requested as much available data from the INVEST team as possible to understand how the partner network has changed over time. As noted in the limitations discussion below, INVEST does not capture time series data on the network, but the evaluation team used descriptive statistics of available data to help contextualize the KII data. Annex F lists all INVEST data and documents the team reviewed. In-depth case studies: to better understand specific INVEST buy-in activities, the team conducted case studies of buy-in activities for the Southern Africa Regional and Vietnam Missions. The Southern Africa Regional Mission case study covers three buy-ins related to Prosper Africa, a presidential initiative that seeks to promote trade and investment between the U.S. and countries in Africa. The Vietnam Mission case study addresses energy- and health-related INVEST activities. Both sets of buy-in activities were relatively new for INVEST, with work largely starting in late 2019 and early 2020 or postponed due to COVID-19. The case studies therefore provide less insight into INVEST’s approach to operationalize the mobilization of private capital than anticipated during the evaluation design phase. As explained in the case studies in Annex I, the evaluation team chose to frame the case studies within USAID’s PSE Policy to highlight how INVEST’s work fits within the broader goal of improving private sector engagement to achieve development outcomes. Online survey: the evaluation team created a closed-ended, 16-question online survey using the SoGoSurvey platform that it sent to 603 INVEST partner firm staff. The team received 21 responses from 20 different partner firms, which includes 17 percent of the firms that worked on buy-ins during INVEST’s first two years. The goal of the survey was to broaden the firm-respondent sample to better understand whether firms were aware of and have used INVEST learning materials. The survey was not meant to be representative, but instead provides additional data to confirm or counter themes that emerged from the KIIs. Annex G provides the survey instrument and Annex H provides supplemental survey results. Process mapping: for EQ3, the evaluation team created a formal process map to better understand how INVEST communicated its mobilization of private capital approach. The evaluation team developed a visual map of INVEST’s approach, as reported to the team through KIIs and provided through documentary evidence. This method provides a visual reference for where challenges arise in the communication process, as well as areas that were reportedly strong. SAMPLE The evaluation team initially selected KII respondents purposively. For each EQ, the team sought to interview INVEST, partner network, and USAID stakeholders. The target selection of buy-in clients sought to provide a full picture of the INVEST process from USAID mission and operating unit perspectives. The team’s selection of USAID buy-in clients attempted to stratify by conflict/post-conflict and non-conflict countries, regional and bilateral missions, and non-mission (i.e., Washington-based operating unit) respondents. Because the evaluation team was able to interview respondents from 28 of 318 buy-ins reported by INVEST, the team believes that the resulting sample offers a full and appropriate 8 Annex B lists the 32 buy-ins that INVEST provided to the evaluation team in a document dated December 19, 2019. However, other INVEST documentation showed that work did not actually start for one of the listed buy-ins until March 2020 -- well into the third year of the mechanism. Additionally, the buy-in SOW was not approved until March 12, 2020. This evaluation report presents the number of buy-ins occurring prior to September 30, 2019 as 32 when based on an INVEST source that reports 32 buy-ins, but as 31 when based on detailed data such as INVEST-prepared spreadsheets that included only 31 entries. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 6 representation of the population of USAID mission and operating unit respondents, based on the contact information listing that INVEST provided. The team purposively selected network firms that have completed a buy-in as well as those with an ongoing buy-in and those that are part of the network but have not participated in any buy-in activities. The team first reviewed all selected buy-in clients based on being the primary RFP awardee in the partner network. Where primary awardees were duplicated or exhausted, the team selected secondary awardees using convenience sampling. The team ordered firms based on when their associated buy-in started, first seeking out respondents from firms implementing older buy-ins and then from firms implementing more recent buy-ins. Figure 1 shows the general affiliations of the final KII sample. FIGURE 1: INTERVIEWS BY RESPONDENT AFFILIATION (n=64) The evaluation team conducted 64 KIIs with 60 unique respondents. In one case, multiple respondents joined an interview, and the team counted each respondent as part of its sample. In many cases, respondents cut across stakeholder groups. For example, some USAID/E3/PCM staff are also considered to be buy-in clients. Table 2 further breaks down the sample and illustrates how respondents may have filled multiple roles. The team interviewed five respondents more than once either due to time constraints or because the team wanted to target questions that addressed different respondent groups to which the respondent belonged. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 7 TABLE 2: TARGETED AND ACTUAL RESPONDENTS INTERVIEWED Respondent Group Target Group Actual Group INVEST buy-in clients 20 29 INVEST/DAI staff 3 14 Partner firm with an ongoing buy-in 6 7 Partner firm with no buy-in 6 7 Partner firm with a completed buy-in 5 4 USAID/E3/PCM staff 3 4 Other USAID staff 3 4 Subject matter experts 2 1 Beneficiary organizations 2 0 Host-country government 3 0 Donor organizations 3 0 USAID staff that did not pursue a buy-in 3 0 DAI technical specialist 1 0 As shown in Table 2, there were several respondent groups that the evaluation team proposed to interview in the evaluation design proposal but then determined to be no longer relevant as data provided a clearer picture of INVEST, or they were unavailable or inaccessible. For example, the team was able to obtain insight on the perspectives of certain donor groups by reviewing the report of the Design Challenge Workshop and interviewed at least three workshop participants as part of other respondent groups. Other organizations such as the World Bank seemed less relevant than anticipated at the evaluation design stage, primarily because the results of interviews and review of INVEST documents did not disclose interactions or unresolved issues that would have benefitted from such interviews. Additionally, INVEST provided heavily redacted documents in support of its financial transactions, which prevented the team from being able to make selections while preserving the respondent’s anonymity. Interviews with USAID and INVEST staff did not flag specific individuals from host-country governments who would likely offer good opportunities for interviews, and COVID-19 was cited as a reason that less interaction with host-country governments had occurred than might have otherwise been anticipated. In another case, the team attempted but was unable to schedule an interview with a DAI technical specialist not affiliated with INVEST. ANALYSIS APPROACH THEMATIC/CONTENT ANALYSIS The team examined qualitative data from KIIs and the online survey to identify patterns, themes, and trends relevant to each EQ to better understand context and meaning. When the team found divergence in responses, it explored possible reasons for divergence in fact, opinion, and perception using other respondent group interviews, and in some cases follow-up interviews. The team compared preliminary conclusions drawn from the testimonial evidence with information developed as part of the analytical review of reports and supporting documentation, as well as other data collected. Thematic analysis utilizes an approach to qualitative data that facilitates the identification of common traits and occurrences, a characteristic that can facilitate the search for causes and effective recommendations. One particular value of thematic analysis for narrative material, as in the case of an analysis of a set of KIIs, is the ability to identify unexpected or unanticipated aspects of the material being studied. That is, if certain members of a sample of respondents report a similar phenomenon, MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 8 careful review of the narrative descriptions offered by each can sometimes identify common themes that suggest causes or foster better understanding of the phenomenon under review. The team utilized MaxQDA qualitative software to analyze transcribed data from KIIs via content analysis techniques, coding text according to key themes of interest across the interviewees. Using coded interview transcripts, the team calculated summary statistics to analyze the distribution, number, and average responses by theme and respondents. DESCRIPTIVE STATISTICS Descriptive statistics provided simple summaries of the analyses the team conducted. This included the count, mean, median, and general distribution of key measures, such as firm entry into the network. The team sought to visualize results whenever possible. Quantitative data sources for descriptive analysis included the online survey conducted to respond to EQ2, data that INVEST shared, and information pulled from INVEST reporting. ANALYTICAL REVIEW OF INVEST REPORTS AND SUPPORTING DOCUMENTATION To enhance its assessment of how INVEST has managed its partner network; collected and validated lessons learned; and communicated its captured learning and research, the team also reviewed relevant documents such the INVEST monitoring, evaluation, and learning (MEL) plan, annual and quarterly reports, published “close-ups,” and recorded webinars. The document analysis ranged from simple comparisons to more complex assessments of the relationships between various data elements, such as comparing how the partner network is described in INVEST reporting and in KIIs, and how this relates to the actual partner network database. The team also used the document review to highlight variation or gaps in responses or findings from other data sources. TRIANGULATION The evaluation team made use of the various data sources through a triangulation process to enhance the credibility of the analysis. Triangulation synthesizes multiple perspectives and leads to a fuller understanding of the issues being studied.9 Data from the various lines of inquiry, including interviews, written documents, analytical procedures, and other sources (e.g., the online survey) were considered both separately and together to develop findings and conclusions. Annex J provides an example of how the various lines of inquiry were considered as part of the mixed-methods approach the evaluation utilized. GENDER ASPECTS OF EVALUATION QUESTIONS USAID evaluation policy guidance calls upon Agency staff and evaluation teams to examine EQs and processes from a gender perspective. Gender aspects must be considered and incorporated during all phases of an evaluation. The nature of the INVEST mechanism differs from direct beneficiary interventions in that gender outcomes may seem less direct. However, the team attempted to track the gender breakdown of the sample, where possible. Around one-third of USAID interviewees (n=10) were female, while 8 of the 18 partner firm respondents were female, as were all DAI respondents. 9 Salkind, N. J. (2010). Encyclopedia of research design (Vols. 1-0). Thousand Oaks, CA: SAGE Publications, Inc. doi: 10.4135/9781412961288 MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 9 In line with USAID’s Gender Equality and Female Empowerment Policy and Automated Directives System (ADS) 201, the evaluation team attempted to consider gender-specific and differential effects of the INVEST mechanism, where relevant. STRENGTHS AND LIMITATIONS OF THE EVALUATION DESIGN STRENGTHS The evaluation benefitted from several factors facilitated the efficient, effective, and accurate collection and analysis of data. • Coverage. The 64 KIIs covered a broad range of INVEST buy-ins and represent stakeholders from 90 percent of the buy-ins over INVEST’s first two years. For 14 of the 31 buy-ins listed in the INVEST Year 2 Annual Report, the team conducted KIIs with one or more USAID buy-in clients, the INVEST/DAI activity manager to whom the buy-in was assigned, and at least one representative of a network partner or other firm contracted to work on the buy-in. For these 14 buy-ins, the team obtained perspectives from all three main stakeholder groups. The KIIs provided perspectives from two viewpoints (e.g., USAID employee and INVEST/DAI activity manager) for another nine buy-ins. For five buy-ins, the team was only able to conduct KIIs with USAID staff. The team was unable to conduct a KII with relevant stakeholders for only 3 of 31 buy-ins. • Proximity and availability of respondents. Many of INVEST’s organizational and individual stakeholders are based in the Washington, DC area, including staff from USAID/E3/PCM, DAI, and several network partners. Although scheduling and accessibility still presented challenges, the evaluation team was able to interview almost all key informants it selected through the sampling process. • Mixed methods. The evaluation design combined qualitative interview data, quantitative survey data, quantitative activity data, formal process mapping, and thorough document review to respond to the EQs. In coordination, the range of types of evidence fills gaps in understanding, provided a robust understanding of INVEST’s first two years, and presented detailed evidence on which to make actionable recommendations. • Anonymity. As discussed in the background section, INVEST has received a ceiling increase and become a prominent mechanism to support USAID’s implementation of the PSE Policy. The evaluation team granted each respondent anonymity to gather open, honest feedback. This led to conversations that were balanced, with few respondents sharing only positive or negative perspectives, and provided a nuanced view into a complex mechanism. LIMITATIONS Challenges emerged in the design and implementation of the evaluation that limited team’s ability to accurately, effectively, and efficiently collect and analyze data. The team made multiple efforts to mitigate these challenges and limitations, as noted below. • Response bias. Key informants and online survey respondents were the primary sources of information to answer the EQs. Although the evaluation team attempted to triangulate as much of this testimonial evidence as possible, these data are subject to personal biases, opinions, and recollection. The team identified, selected, and interviewed a broad range of stakeholders to address this challenge. However, not every statement could be weighed against a similar or counter example to address potential response bias or gauge the magnitude of such bias. The team omitted statements that, while compelling or convincing, could not be confirmed, contextualized, or countered through other data sources. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 10 • Information availability. To address some EQs such as the partner network’s growth over time, time series data or detailed tracking information is needed. However, INVEST has not collected information in a way that allows for this type of analysis. In other cases, INVEST has deemed data or documentation related to buy-ins as “proprietary,” with details redacted to protect the parties involved. There are also buy-ins, such as those selected for the case studies, that are relatively new and have not yet generated detailed data or documentation. The team referenced all available information to the greatest extent possible. KII data also helped fill in gaps from other data sources. • Partner firm access and response rates. The evaluation team sent the online survey to 603 email addresses for each partner network firm (some firms had multiple points of contact) and received 21 responses from people at 20 different firms. To increase the response rate, the team sent the survey out multiple times and posted the survey link in the INVEST LinkedIn group. The team did not design the survey to be representative, but the survey responses represent 17 percent of partner firms that have worked on buy-ins and include 9 firms that have bid on, but not won, work through INVEST. The survey responses should not be taken as anything more than the perspectives of those who responded rather than broader findings about network firm perspectives. The team kept the survey brief and closed ended to encourage completion, but ultimately could not force people to respond. Although none of the evaluation’s conclusions or recommendations rely solely on the online survey responses, the survey data does complement and allow for comparison with the KII data despite the low response rate. EVALUATION TEAM The evaluation team consisted of the following members: Rebecca Anne Batts, DPA, Team Leader. Dr. Batts is a retired federal inspector general and certified public accountant, with 30 years’ experience leading and conducting highly technical audits, evaluations, and investigations at three diverse offices of inspector general. As team leader, Dr. Batts led the implementation of the evaluation including technical quality assurance, conducting interviews, oversight of analysis and reporting, and delivering presentations on the evaluation results. Peter Stoute-King, Subject Matter Expert. Mr. Stoute-King advises entrepreneurs, corporations, foundations, and international financial institutions on blended finance, business development, fundraising, strategic partnerships, stakeholder outreach and engagement, strategy, environmental, social and governance integration, and responsible/impact investing. Mr. Stoute-King provided input on the evaluation design and data collection instruments, and assisted with the case studies. Jacob Patterson-Stein, Evaluation Specialist and Coordinator. Mr. Patterson-Stein holds a master’s degree in international development policy and has experience in survey design, quantitative and qualitative analysis, and designing and implementing impact and performance evaluations. Mr. Patterson-Stein has also conducted credit risk, loan default, and predictive modelling for financial institutions, the U.S. Small Business Administration, and the Department of Housing and Urban Development. Mr. Patterson-Stein coordinated the execution of the evaluation design, ensured efficient and timely reporting, and monitored the fidelity of the evaluation design. Home Office Support. Bethany Little and Danielle Burke provided logistical support for data collection to ensure all KIIs were coordinated, recorded, and transcribed. They supported the team lead with technical coordination, research assistance, and data analysis. Additional E3 Analytics and Evaluation Project home office staff supported the core evaluation team with quality assurance, administrative oversight, and logistical support. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 11 FINDINGS AND CONCLUSIONS EQ1: HOW HAS INVEST OPERATIONALIZED THE MOBILIZATION OF PRIVATE CAPITAL APPROACH THROUGH ITS NETWORK OF PARTICIPATING FIRMS? SUMMARY RESULTS FOR EQ1 The INVEST mechanism was intended to support and enhance USAID actions to mobilize private capital in support of development goals. Over its first two years, INVEST has operationalized the mobilization of private capital through its partner network by supporting timely, targeted procurement through subcontracts. The network of firms that participate in INVEST buy-ins has expanded steadily over time, although it is difficult to know exactly how based on the data INVEST collects on the network. The mobilization of private capital was not a major feature of INVEST’s first two years, although additional blended finance transactions are anticipated during subsequent phases of its operations. INVEST’s annual and quarterly reports described a range of achievements including how INVEST operationalized its approach by identifying and assembling necessary private sector expertise to assess market conditions, develop and structure financial solutions to development challenges, and utilize transaction advisory services as a way of increasing the availability of private capital to address sustainable solutions. INVEST’s overall research goal is to develop and test a model that enables USAID to quickly engage new partners and mobilize private capital. To this end, respondents highly valued INVEST’s effectiveness in working with USAID missions, bureaus, and independent offices (MBIOs) to quickly facilitate procurement, find relevant partners, and support buy-ins. USAID staff also praised INVEST’s flexible, tailored approach to address the challenges MBIOs faced and the strong project management skills of INVEST staff and cited these as reasons to seek additional buy-ins. Respondents valued how INVEST functioned as administrative and procedural support for USAID MBIO development initiatives, regardless of their connection to the mobilization of private capital. Network partners appreciated INVEST’s simplified procurement format and the speed of its streamlined procurement approach. USAID MBIOs also appreciated how INVEST helped them access the services of new and previously underutilized partners. However, network partners flagged the quality and speed of INVEST’s feedback to offerors in response to RFPs as an area for improvement. Despite the speed that was deemed an asset of the mechanism, many INVEST stakeholders also found the mechanism’s procurement processes confusing. INVEST OPERATIONS INVEST’s structure is complex and not always fully understood by the mechanism’s stakeholders. Created pursuant to a Broad Agency Agreement (BAA), USAID stakeholders view INVEST as a research and development project that can bring innovation to help the Agency address a development challenge that lacks an obvious solution. DAI functions as the prime facilitating partner for INVEST (formerly referred to as the “apex partner”), and performs its work via a contract with USAID. As part of INVEST’s initial design, DAI subcontracted with three core partners – Convergence, CrossBoundary, and Tideline – that work under separate contracts in privity with DAI. As part of this arrangement, DAI is also responsible for managing a network of participating firms that can do business with INVEST by responding to RFPs. A buy-in can have multiple RFPs, depending on the nature of the work.10 The partner network was established out of the initial group of co-creation firms, including DAI and the core partners. After co-creation, INVEST 10 This evaluation report uses both “RFP” and “buy-ins” to refer to INVEST partner firm activities, depending on how the work was subcontracted (e.g., solely through an RFP process, through task orders alone, or a combination of RFPs and task orders). MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 12 continued to promote network participation. Within established USAID contracting parameters, and working in concert with the MBIO buy-in client, when work becomes available, DAI in its role as INVEST’s prime facilitating partner chooses whether to (1) do the work itself, (2) subcontract the work to one of the three core partners as a task order under established contracts, or (3) issue an RFP that is generally made available to the entire partner network or any other firm that chooses to respond, including core partners (Figure 2). The three core partners and other network partners are eligible to respond to RFPs as are firms that are not members of the INVEST partner network. FIGURE 2: INVEST CONTRACT FLOW Stakeholders described INVEST’s approach to the mobilization of private capital in a number of ways. The interview data suggest that INVEST took different approaches to different buy￾ins as part of its mobilization of private capital, a tactic that DAI and USAID staff frequently referred to as tailored or “bespoke.” The various services that were requested from the mechanism generally sought to manage risk and capital for both investors and firms, which typically fit into broader development goals or challenges. These challenges included the presence of dysfunctional markets or markets where private investment does not take place but probably should because there are un- or under-exploited opportunities. Some of INVEST’s “bespoke” approach involved markets that were seen to need information, advice, legal services, or convening power to bring capital and projects together. In other cases, policy advice to the public sector and transaction advisory services were critical to the overall INVEST approach. The following four buy-ins illustrate some of the ways INVEST operationalized its approach to mobilizing private capital through the partner network. During its first two years, INVEST reported mobilizing private capital as part of two buy-ins, Power Africa Kenya and USAID/E3 Office of Gender Equality and Women’s Empowerment (GenDev). Two other buy-in examples illustrate INVEST’s operationalization of the mobilization of private capital: capacity-building efforts under the Practical InnoVative On-the-Job Training (PIVOT) activity and the landscape analyses completed for the Water and Energy for Food (WE4) Grand Challenge for Development. The case studies in Annex I provide a detailed view of how MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 13 two buy-ins, the Southern Africa Regional Mission’s Prosper Africa buy-in and USAID/Vietnam’s buy-in, functioned and fit within broader PSE Policy goals. Power Africa Kenya Based on interviews with multiple USAID, INVEST, and partner firm stakeholders with knowledge of the buy-in, the development objective of the Power Africa Kenya buy-in was to support off-grid energy companies. These firms tended to be small- to medium-size enterprises that extend credit to end users to deploy electricity solutions. For this buy-in, collaborative efforts took place along two parallel channels. First, because USAID/Southern Africa had previously collaborated successfully with an INVEST core partner and was familiar with that firm’s expertise and team members, in-country USAID respondents described a close relationship between the USAID and Power Africa teams and the other INVEST subcontractors for Power Africa. However, DAI facilitated and served as the prime interlocutor for USAID/E3/PCM, based on the viewpoint that it was more expedient to work through DAI and thus avoid having ‘too many hands.’ As a way of mobilizing capital, the Power Africa Kenya buy-in used a two-pronged structure that provided assistance to both the energy companies and to potential investors. If a successful transaction occurred between the energy company and investor as a result of the assistance, a performance success fee was paid to the core partner and to the subcontractor. For the energy companies, the INVEST core partner drew on new and existing relationships in Kenya to identify a pipeline of potential transactions. DAI and its core partner developed and implemented screening criteria to select the off-grid energy companies most likely to become viable with an infusion of private capital, an action that was in alignment with Power Africa’s overarching goal of enabling electricity access by adding new electricity connections. The INVEST core partner and another subcontractor then developed a tailored plan of action for each selected energy firm. Based on the energy firm’s identified needs, action steps included introductions to appropriate investors, fine tuning of the firm’s pitch decks, or enhancements to financial models. For potential investors, the INVEST core partner and a subcontractor provided assistance including supporting the investor in developing appropriate criteria for selecting an investment and helping the investor think through or visualize what a potential investment might “look like” within the off-grid energy sector. If the potential investor actually completed the transaction by investing in the energy company, the INVEST core partner and its subcontractor were eligible to earn a performance success fee equal to more than half of the contract value, once the transaction closed and capital had been successfully mobilized. Figure 3 summarizes this process. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 14 FIGURE 3: POWER AFRICA KENYA INVEST PROCESS GenDev Women’s World Banking The development objective for the GenDev buy-in was to improve women’s financial inclusion through a blended finance approach by leveraging concessional equity to attract commercial investors. According to INVEST,11 this buy-in represented USAID’s first use of a subcontract-based first-loss investment approach.12 INVEST’s approach, according to the INVEST Close Up: Catalyzing Private Investment for Women’s Economic Empowerment, was to tell potential investors of USAID’s commitment with the intention of providing a kind of “USAID seal of approval” that would make future investment more attractive.13 For the GenDev buy-in, INVEST operationalized the mobilization of private capital by leading the RFP processes and administering finance-related technical management to reduce USAID’s management burden. MBIO respondents reported they were not concerned that anything would be lost by permitting INVEST to lead the procurement and management process, particularly given DAI’s responsiveness to questions and concerns. USAID reported that Agency representatives still took part in some phone calls with subcontractors implementing the GenDev work and reviewed invoices as appropriate, despite not working as closely as might have occurred under other approaches. A DAI respondent noted that they worked with multiple Agency staff to ensure they could run a thorough due diligence process as part of procurement, explaining that for the GenDev buy-in: 11 USAID INVEST Quarterly Report, Year 2, Quarter 3, pg. 15. 12 In donor-funded finance, “first loss” capital is put forward when the donor (although it could also be a grant maker or impact investor) commits to taking on any initial losses as part of an investment as a way to mitigate risk for other investors. 13 https://www.usaid.gov/sites/default/files/documents/1865/Womens_Economic_Empowerment_CloseUp_final.pdf MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 15 We had scheduled conversations with general counsel inside USAID. We had the [general counsel, GC], all the lawyers on Women's World Banking, we had scheduled meetings to go over the structuring of it and the validation of that with [the Office of Acquisition and Assistance, OAA]. They will tell you we had many discussions with them. So we had GC, we had OAA, we had a number of the politicals involved in this. And then we had our own team doing the due diligence. We also used [firm name removed], [through] which we had a membership for USAID. PIVOT INVEST’s approach to supporting the mobilization of private capital also included capacity building. For the PIVOT buy-in, INVEST contributed content and technical mentors in blended finance and private capital mobilization for PIVOT change teams. As part of a more comprehensive six-mission initiative led by USAID’s Africa Bureau, PIVOT taught basic blended finance concepts and tools through webinars and in-person trainings, with the goal of enhancing mission-level readiness for private sector engagement. INVEST staff described this effort as working to change mission staff from resource managers to resource mobilizers. WE4 Grand Challenge for Development WE4’s planned initiatives include the establishment of regional innovation accelerators to identify potential new innovators and provide them with financial and non-financial support in form of brokering and technical assistance units. An initial step was the development of regional landscape analyses to identify and evaluate prospective enterprises as well as types of capital, appropriate terms, and financial products needed to grow operating capacity and attract growth capital. INVEST supported WE4 in this effort by collaborating on the development of various procurement documents, and an INVEST core partner was awarded a task order under the existing contract to complete one of the four landscape analyses. INVEST subsequently issued three RFPs for other regional analyses. The same core partner for the analysis issued under the existing contract was also the successful bidder on two of the three RFPs and a non-traditional network partner was awarded the final RFP. DAI held weekly meetings and acted as facilitator with the two selected firms. If the quality of deliverables was not “up to par” in the view of the USAID client, DAI put in extra work with the firm to ensure the final product met standards. A high-level summary and a more detailed document were prepared for each of the four regional landscapes analyses, including East Africa, West Africa, South and Southeast Asia (with a mini buy-in for Afghanistan-specific information) and the Middle East and North Africa. ADMINISTRATIVE AND PROCEDURAL SUPPORT USAID staff viewed INVEST as a valuable partner in supporting elements of the mobilization of private capital. Of the 30 USAID staff interviewed, at least 21 provided one or more examples of how INVEST had provided important administrative, logistical, or managerial support. MBIO staff were generally enthusiastic about the INVEST mechanism’s functioning as administrative and procedural support. In comparing the experience of working directly with contractors to the experience of working with the same contractors through the INVEST mechanism, a USAID office director explained his viewpoint that INVEST was expensive, but still a “value-add” because of the mission was so overworked and pressed for time. Thirteen MBIO buy-in clients and four partner firm respondents acknowledged INVEST’s expertise in communications support. A USAID staff member described INVEST overall as MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 16 “absolutely top notch” and further explained “they understand the private sector, they understand our audiences, they’re able to not just do communications for communication’s sake, but to understand that our communications need to serve this [development goal]…They bring this expertise around investment and the private sector to the communications work…I can’t speak more highly to the kind of support we’ve been given there.” A USAID/Washington staff member explained, “I really like their comm teams. Like how they approach their case studies. I think they’re one of our best partners in terms of providing insight into what is working…” USAID staff praised INVEST’s expertise in planning and executing major events and identified INVEST’s events as something that should be shared within USAID, built upon, and used in the future. The following text box describes the Prosper Africa Conference; INVEST reported that 67 percent of investors attending the event expressed “very high” enthusiasm for follow￾on meetings with companies they met at the conference. Four USAID staff commented on their appreciation for the thought leadership INVEST provided to MBIOs. One respondent explained how the DAI activity manager “has done an excellent job of disciplining both the thinking within the mission and the thinking within our implementing partners, where applicable.” Other important aspects included the DAI activity manager’s ability to make sure “that INVEST sticks to its core principles of demand driven investment support” and her ability to ensure that the buy-in “is not designed as another USAID project where we bring in $10 million and invest in a bunch of trainings or any other outputs.” He further explained that INVEST was designed to be innovative and that the DAI activity manager “has been wonderful about communicating this, explaining this, reminding people of this and, as I said, keeping us disciplined around it.” Another USAID employee described the INVEST staff with whom he worked as being curious and sufficiently knowledgeable to be a really useful thought partner. He contrasted INVEST’s newer Planning and Executing a Major Trade Event In February 2020, more than 900 participants from 33 nations attended a conference in Tunis, Tunisia that the U.S. Embassy and the American Chamber of Commerce of Tunisia organized. The conference was intended to promote and strengthen U.S. and African business connections, including investments and partnerships. INVEST provided event logistics and communications support. A stakeholder described how the event involved “Everyone coming together in order to identify and better understand the Prosper Africa initiative and to try to, in very practical terms, make those business-to-business linkages so that the U.S. government is needed less. There was a huge forum facilitating business-to-business events and a number of panels…about the vision for Prosper Africa and what we aim to achieve under Prosper Africa. And the president and other leaders of Tunisia talking about their vision for U.S. investments and U.S. partnership in Tunisia and more broadly the African marketplace overall.” Another stakeholder described INVEST’s contribution to this conference, noting that INVEST “… really blew me away…their core team…[was] exceptional, the communications expertise was on point. She fabulously coordinated with every, like 10 gazillion government entities that were all involved in clearance chains and processes around what needed to happen and how it could happen. The event was incredibly successful.” Attendees described the conference as very useful for its target audience. As one attendee noted about an informal roundtable held near the end of the conference, “we were given a direction essentially if we were to ask the private sector what they wanted, they wanted more events like this. And I think it's a great example of how comms can be done, not just as window dressing or kind of marketing, but as part of the groundwork of implementation itself.” MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 17 approach with that of “nonprofits and other mechanisms that we have where they build up a decade of research and history, so if you pay them to write a paper now, they’d write the same paper. [The nonprofits] wrote that paper 10 years ago…I didn’t want that. I wanted something different.” One MBIO stakeholder saw INVEST’s “creative and aggressive” team members as important to ensure that barriers would not “lock us down and make things impossible to proceed.” Eight of the 21 online survey respondents noted that being associated with INVEST provided “learning about blended finance and private capital mobilization.” MBIO respondents valued INVEST’s role as a thought leader for many reasons, including the importance of USAID having access to new ideas and the ability to explore those ideas and determine whether they are reasonable. MBIO and DAI respondents cited new challenges, such as COVID-19, as requiring new or innovative solutions. INVEST’s MEL Plan explained the role of thought leadership as, “Thought leadership and effective learning capture and dissemination on operational and technical topics strengthens the capacity of the INVEST partner network, USAID and other development actors to implement PSE development strategies.” Respondents were not unanimous in their support for INVEST’s thought leadership. One USAID staff member expressed a serious concern about the loss of technical leadership and expertise that occurred when activities and analyses traditionally exercised by USAID employees were delegated or otherwise devolve to contractors. According to this staff member, “the big weakness is that USAID loses its technical leadership a little bit. Because…INVEST does the work and they get the primary credit for everything, as well as they are the ones on the technical review committee…[although] they let USAID participate, and we have two USAID to three INVEST people.” This staff member saw INVEST’s thought leadership role as a workaround because “…the Agency is short staffed and doesn't have the technical capacity to deliver the magnitude of funding and work that needs to be delivered in a short time frame, especially in the private sector space where new partners are critical to our work…And so for the short term it's a good solution, but probably for the long term it's not ideal.” Stakeholders consistently felt INVEST staff had excellent project and time management skills, and six provided examples of how those skills had influenced their buy-in. A USAID staff member noted, “They reminded me what good project management could really be. There’s a lot of bad project management out there and they’re not that. If anything, it renewed my faith in that type of process, which I think is good.” When the evaluation team inquired about any procedures or practices that led to INVEST’s effective project management, responses were not detailed. While there was no one approach to project management that the evaluation team heard about, MBIO respondents often cited the overall effectiveness of INVEST’s project management approach in its operationalization of the mobilization of private capital. A typical MBIO interviewee explanation described INVEST’s practices as seamless or unobtrusive, “… in a way, it’s kind of invisible at a level. It’s organized, It happens. It’s very clear. It’s very concise. You don’t have to go behind the hood all that often to see what’s really going on.” Another MBIO staff member mentioned that his office joked that if a specific INVEST employee scheduled a 25-minute conference call, the call would last exactly 25 minutes. Summarizing their experience, this MBIO respondent noted, “I really think as project managers INVEST is very strong.” MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 18 EQ1A: HOW HAS INVEST FACILITATED THE PROCUREMENT PROCESS FOR MISSION AND OPERATING UNIT BUY-INS TO THE ACTIVITY? USAID MBIOs generally gave INVEST high marks for the speed and relative ease of its procurement processes, as well as for INVEST’s ability to contract with new, previously underutilized, and/or local firms. At least 12 of 23 MBIO respondents mentioned one or more examples of how INVEST had facilitated contracting activities. However, 40 percent of the USAID and partner firm stakeholders interviewed, 14 associated with 16 of the 32 buy-ins from INVEST’s first 2 years, found INVEST’s practices to be confusing. This indicates the need for INVEST to provide more information about the steps for a typical procurement. As noted above, MBIO respondents relayed positive experiences with INVEST’s procurement process. However, 60 percent of partner network member KIIs with prior USAID experience were dissatisfied with some aspect of INVEST’s procurement practices. Additionally, representatives of 7 of 17 firms expressed concerns about transparency and/or fairness in subcontractor selection as part of the INVEST RFP process. MBIOs frequently expressed appreciation for INVEST’s ability to bring in new partners and 10 USAID staff described the importance of working with new or underutilized partners. INVEST used what it referred to as a New Partner Facilitation Model to encourage and expand the participation of firms that had not been a prime contractor with USAID during the prior five-year period or that had received less than $25 million in direct or indirect awards during the prior five-year period.15 As part of this approach, DAI helped new and underutilized firms navigate the complexities of U.S. government contracting and attempted to reduce or eliminate barriers that may have prevented the firm from doing business with USAID. MBIO interviewees saw access to new partners as a key benefit of working with INVEST. A USAID investment advisor explained the importance of this access, stating “We were very clear that we didn’t want the same ideas from old partners. We were very much looking for new ideas…I think a lot of our 14 Ten of 18 partner firm respondents (53 percent) and 11 of 32 USAID respondents (34 percent) reported this. 15 INVEST writes, “New Partner Facilitation Models follow the umbrella approach to partnering in which the Facilitating Partner uses subawards to build a network of partners with specialized expertise.” See “Our Partner Network,” https://www.usaid.gov/invest/our-partner-network. Strong Project Management Skills to Respond to Changing Conditions INVEST worked closely with the USAID Digital Connectivity and Cybersecurity Partnership and an implementing partner to ensure a successful rollout of the Agency’s Last Mile Connectivity Initiative research report on barriers to investing in last-mile connectivity. As part of the effort to drive investment and innovation to extend internet access to the world’s four billion unconnected people, USAID had planned an in-person event to explore research findings from the report and discuss concrete ways to increase investment in last-mile connectivity. The event’s timing was important and the release was scheduled to occur shortly after issuance of the Agency’s digital strategy. Plans were in place and a room had been rented for a spring event. Then, COVID-19 made travel impossible and an alternative approach was needed to disseminate the important research findings and recommendations. INVEST worked to shift from an in-person event to an online environment, which allowed the implementing partner, Connectivity Capital, to focus on the substance of the meeting and USAID to focus on generating an online audience and assemble a quality respondent panel. USAID estimated that about five times as many people attended virtually as would have attended in person. Participants from around eight countries, including individuals from Africa, Southeast Asia, and Europe, were able to take part. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 19 partners have very strong feelings about USAID does this, this, and this, and we don’t do these other things. We didn’t want a partner that was hamstrung by those limitations and those expectations.” Another MBIO staff member compared working with INVEST to more traditional approaches typically utilized by USAID: “It’s the diversity of partners they bring, non-traditional partners and the ease of working with them, the rapid pace of which we can sign up new contracts. I’ve worked with new partners before at USAID, and getting through our OAA takes about seven months when we have to do all of the paperwork, The fact that we can do this with new partners in two months sometimes with INVEST is incredibly, incredibly good for the industry. It builds competition in the development industry. It diversifies who’s out there doing the work and it lowers the cost of the work to the taxpayer…this makes our job easier. You don’t have the same people winning every contract. You have lots of diversity because you have partners who may be specialists that can apply to these things.” The chief executive officer of a new implementing partner working with USAID for the first time described her firm’s experience: “We had not looked at doing work with USAID in the past, because we didn't really think that we fit the profile of a company that would be doing USAID work. We do very financially and technical-focused work, and we didn't really know that that was a priority of USAID…We have tried to identify other similar programs, and we haven't figured out yet how to get to be part of them. So, in that sense, INVEST was very, very easy to become part of the network and start to see the opportunities. Other programs we've heard about we've looked into, and it hasn't seemed that accessible. So comparatively speaking, they were an easy port of entry into the USAID world.” USAID interviewees noted the importance of the speed of INVEST’s procurement process. INVEST quarterly reports show that the mechanism issued 43 subcontracts in its first two years and an additional 36 subcontracts in the first three quarters of the third year. A USAID stakeholder described this accomplishment as, “I think that’s light speed in USAID time.” Although the INVEST procurement team is fairly lean and usually composed of only two or three individuals, USAID and INVEST respondents cited the use of a less bureaucratic evaluation process and deliverable-based contracts that are easily replicable as reasons for INVEST’s ability to move quickly despite the large number of deliverables, reported to be more than 350 across the entire mechanism. Another USAID staff member considered speed to be the primary benefit of INVEST mechanism, given the typically slower pace of alternative procurement channels. “I would say that’s the biggest drawback of partnering with USAID, it’s very hard to get to fruition on something when there’s 100 people reviewing it, and it often gets stuck in the bureaucracy. And good ideas do come out of the bureaucracy, but you have to spend triple the amount of time working on the process of it…So I think it’s a good example of working with a partner to get…new commercial actors that USAID has been trying to work with for many years but struggling with our procurement mechanism. They’ve been able to help us with that.” Firms submitting bids in response to INVEST RFPs singled out two aspects of the procurement process: format and feedback. • Format: Nearly all network partners commenting on INVEST’s simplified proposal process liked its streamlined format and appreciated the speed with which proposals were reviewed. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 20 • Feedback: Firms were less satisfied with the quality and speed of feedback they received from INVEST as part of a debriefing process for unsuccessful bidders. Firms described a number of advantages to INVEST’s 10-slide PowerPoint approach for proposals and generally applauded INVEST for creating a better method for responding to RFPs. For example, an established USAID contractor with subcontracts for multiple INVEST buy-ins described himself as a “huge fan” of INVEST’s process, noting the efficiencies his firm achieved due to the use of the standardized financial proposal template and the consistency with which the same questions were asked from one RFP to the next. Based on his experience with this new procurement format, he concluded, “if USAID takes anything away from this, [it should be] ‘Let's rethink what our proposal request process is.’” Another respondent appreciated the ease and efficiency of INVEST’s process, noting “what we liked is the format, which is very different to other donors.” He further noted, “I think they have two-week deadlines on deciding on stuff. That was actually something that worked quite well for us.” Another firm noted that using this new procurement format became easier with repetition, “especially now that we've done it once or twice, it is a much easier lift than doing something more intensive. So that's been great.” INVEST generally offered to provide formal feedback to unsuccessful bidders upon request, using a process similar to other government contracting approaches. DAI also offered feedback at other points in the procurement process, such as feedback on documents that had been provided as deliverables. Interviewees noted that INVEST’s feedback was not always received promptly and did not necessarily advise bidders of who had won or why. The experience of a newer INVEST partner was typical of the 18 respondents who mentioned feedback. This firm described concerns with the level of detail in debriefing documentation, explaining: “We have received letters that, from my perspective, give general feedback as to why we were not selected. So, an example might be you had less local experience than the winning bidder. Or the price was a consideration, which for us is very different than dealing with commercial clients, where, of course, they give you very direct feedback. Like, ‘Hey, you were way off. You didn't have the right people. You didn't have this project. We didn't like the way you did this or that.’” For this nontraditional government contractor, the lack of detail has led to a reduction in the number of proposals submitted because “we haven't been able to figure out why we are not being selected at least to provide additional information or clarification on the proposals.” An experienced USAID contractor compared the quality of INVEST debriefs with those typically received from USAID, describing INVEST’s debriefs as “very short and probably not up to the standard of what a USAID debrief was. It was like something that someone just had to pull together and just provide some generic comments.” Another contractor with extensive USAID experience described his firm’s reaction to losing a bid for which he felt his firm had been highly competitive: “We bid on it and we didn't win. That's fine, we're very accustomed to losing, it happens. But when we got the debrief, it sort of changed everything for us. The debrief was minimal, limited and what was ironic about the entire thing was we were implementing a [very similar project for USAID]…So it was a comedy of error…The debrief was so weak and it was probably written by somebody who'd never been to [country] never had done any financial [work of a very similar nature]…So it was just so out of left field, that we were just like, ‘This is it. This clearly will never work for a firm like [firm].’ So that was sort of the last straw for us.” MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 21 The above respondent shared a copy of an INVEST debrief with the evaluation team. Based on a review of the debriefing materials as provided by the firm, the debriefing document provided by INVEST appeared to be general, with few actionable suggestions for future bids. Many key informants found INVEST’s procurement processes confusing, including USAID staff. Ten USAID interviewees associated with 11 buy-ins, including one respondent who was a key contact for two buy-ins, expressed concern, or a lack of clarity, about how INVEST procurements operate. A private enterprise officer who worked with both completed and in-process buy-ins described their mission’s experience with INVEST procurement: “It wasn’t too clear, it seemed to be a very complex structure. I understand more of it now, but we had to basically learn as we went…it hasn’t been the smoothest process when it comes to clarity, but it seems to be a promising one…it seems like what we would like to do too. But the steps we need to take were not always straightforward.” Some mission staff would have preferred a more predictable path and more advance notice of what was yet to be accomplished; one noted, “The expression we used was ‘the gift that keeps on giving’ because it’s after every phone call that we found out about a next step…What else is going to pop up or what other step I will be told that I have to do, because I don’t have a clear line of sight on exactly how everything has to go.” Despite this confusion about the procurement process, the mission noted that once the buy-in happened, it was “very happy” with the final report and recommendations. Respondents cited the incremental nature of the information INVEST provided as one reason why the procurement process was confusing or unclear. Instead of providing a generalized roadmap of the tasks to be completed and the overall activity goal, some MBIO staff felt INVEST was sometimes parceling information out step by step. INVEST has committed to developing a process chart that lays out clear procurement steps. This action will likely be helpful to both USAID staff and partner network firms. EQ1B: HOW HAS THE NETWORK OF PARTICIPATING FIRMS UNDER INVEST EXPANDED AND CHANGED OVER TIME? The network of firms that participated in INVEST buy-ins has expanded steadily since the mechanism’s early incarnation as a prime facilitating partner (DAI) supported by 3 core partners and a 25-member network of partner firms, organizations, and consultants. As shown in Figure 4, the INVEST partner network has steadily grown over the mechanism’s first two years.16 The partner network includes a variety of organizations. According to data from an INVEST partner firm survey that DAI conducts for new partner firms, as of March 2020 72.6 percent of firms have 50 or fewer employees (n = 135). One hundred and twenty-nine firms responded to DAI’s survey about whether they were registered in the U.S. as a small business, with 48 percent (n=62) reporting they are small businesses. The partner network includes universities, nonprofit organizations, and firms in partner countries. Around 72 percent (n=106) of the firms that responded to INVEST’s partner survey and reported on their USAID experience stated their organization had USAID experience outside of INVEST.17 16 Graph is based on INVEST quarterly and annual reports. 17 This figure is difficult to verify since not every firm that took INVEST’s partner survey responded to this question. Additionally, the INVEST partner survey asked this question as an open text field resulting in non-standardized replies, some of which were ambiguous or did not directly respond to the question. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 22 FIGURE 4: NUMBER OF INVEST PARTNER FIRMS FROM 2018 TO 2020 Reviewing available INVEST data also highlights which firms have most frequently partnered as subcontractors to DAI through INVEST. As shown in Figure 5, a core partner, CrossBoundary, won the most work in INVEST’s first 2 years, with about 26 percent (n=11) of the 43 subcontracts issued.18 The small business designation was included in INVEST’s data, but is not necessarily applicable for non-U.S.- based firms that have not registered with the Small Business Administration. For example, Convergence, which was awarded three subcontracts in INVEST’s first two years, is based in Canada but has fewer than 50 employees. The evaluation team has maintained INVEST’s own designation for the purposes of this analysis. FIGURE 5: TOTAL SUBCONTRACT VALUE BY FIRM AND INVEST SMALL BUSINESS DESIGNATION 18 Figure 5 only shares the figures that were made available to the evaluation team and may not reflect all funding awarded through INVEST. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 23 Numbers and graphs tell part of the story but do not fully explain how INVEST, its core partners, and the partner network have changed over time. USAID/E3/PCM developed a concept paper that outlines the initial ideas of INVEST and documents its earliest days. The concept paper describes how DAI was to serve as the “apex partner” of INVEST’s initial core team and provide technical and operational oversight to the mechanism. DAI would be supported by core partners Convergence, CrossBoundary, and Tideline, which would provide integrated support in their areas of technical expertise and serve as INVEST’s initial strategic “voice.” Twenty-five additional partner firms, organizations, and consultants – the entities that had participated in the co-creation event that conceived INVEST – comprised the founding members of the initial partner network. However, based on data provided to the evaluation team, 3 of the 25 original non-core partner firms obtained INVEST contracts during the period the evaluation team reviewed. Over time, as INVEST began to implement buy￾ins, additional organizations expressed interest in joining the partner network. Many of the new firms were interested in opportunities to contract with USAID via INVEST, while others joined as a way to keep abreast of USAID priorities and activities. Network partners have played various roles on buy-ins, including providing investment support and a range of business development, transaction support, and knowledge management activities. Data from the interviews and online survey suggest new partners frequently joined the network based on referrals from USAID, INVEST, or industry colleagues. Others became aware of the network after bidding on an INVEST RFP or from market research INVEST conducted to identify potential new and underutilized firms to fill needs for local companies or specialized and niche skills. FIGURE 6: HOW SURVEYED FIRMS JOINED THE PARTNER NETWORK (N=21) Figure 6 shows how the evaluation’s online survey respondents reported becoming a partner network member. The evaluation team did not find INVEST data that captured when or how firms joined the network to triangulate this information.19 As noted above, DAI conducts a partner firm survey for new partner firms that join the network, but does not ask how or when they joined. A DAI employee explained the process of adding new partners to the network as follows: “When a firm joins the network, they fill out a survey. So, the survey collects some basic information about the firm, primarily the firm name, their country of registration, and it's all self-reported, the technical capacity that they have, and expertise that they have, and the geographic experience that they have…we follow all of the normal USAID, and U.S. government requirements around due diligence and vetting. We check their SAM registration, and we make sure that they're not on any U.S. government excluded lists. We collect information on their financial viability, the insurance that they carry…For just membership in 19 Written communication from DAI at the start of the evaluation noted that, “exact dates on when partners joined were not collected by INVEST.” MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 24 the partner network…we take…an open tent approach, like where we're trying to bring these firms in. But before we were to do any work with them, put them under contract, have any agreements with them, certainly, before we would distribute any money against deliverable completion, we make sure that we followed all of the requirements around vetting, and due diligence to make sure that they are a responsible offer.” Some USAID/Washington staff viewed parallels between the CATALYZE initiative and INVEST, and believe CATALYZE may affect the future of INVEST and the partner network. USAID awarded CATALYZE to Palladium in late 2019. CATALYZE has a similar goal as INVEST of mobilizing private capital using the partner network.20 Another USAID employee noted the broader trend toward “mega-contracts” with “a huge pool of money for everyone to fight after, which creates an interesting dynamic…It’s hard to talk about INVEST without putting it into context side-by￾side with CATALYZE…there are options for the Agency to choose from between INVEST and CATALYZE to do procurement related to private sector engagement…whereas before INVEST was the only contract mechanism to buy into to procure services related to private sector engagement…it’s good to have both basically be doing the same thing because it provides options. If a mission doesn’t like working with DAI, then they can potentially pivot to working with Palladium under CATALYZE, if they want, for their next procurement. So I guess…that’s helpful.” At least two member firms currently working with INVEST reported being confused by the inception of CATALYZE. One respondent from a firm with multiple ongoing INVEST subcontracts described being “pitched” by CATALYZE and explained: “I don’t know [first] what is the ability of an INVEST project…to lead to a transaction…and secondly, there’s a question about how this project would be procured. Would they be through INVEST because your original study was done through INVEST, but it ended in CATALYZE? Or would it even be something…just sourced through a different procurement portal and we’re not on it.” EQ 2: HOW HAS INVEST CAPTURED AND VALIDATED LESSONS LEARNED? SUMMARY RESULTS FOR EQ2 INVEST has just begun to capture and validate the broad lessons to be learned as it executes its action research project. Some MBIOs and network partner firms viewed participation in INVEST as an 20 See https://thepalladiumgroup.com/news/Mobilizing-2-Billion-in-Private-Capital-USAID-Awards-CATALYZE-Contract-to￾Palladium. Adopting the Partner Network Concept for a Specialized Niche A USAID staff member who had worked successfully with INVEST noted the value of having a network of experts that could be tapped into as needed and is reportedly borrowing the concept for another project through which he will try to create a network of regulatory experts. He hoped to gain access to different voices and explained how he valued the ability to reach new partners, stating “Okay, this is a different perspective, and it intentionally has a different perspective." He plans to curate the list beforehand to facilitate a quick turnaround on projects. In describing the planned niche network, he stated that curation takes time and does not happen on its own. “Curating and cultivating experts, keeping them engaged, even if there isn’t a project that relates to them, is a cost, and you have to recognize that that’s a cost…it’s going to result in a higher core cost and things like that, and you just have to live with it, because that’s the value that you end up getting.” MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 25 experiment in finding new ways to conduct development activities. Learning activities in INVEST’s first two years generally focused on building interest and buy-ins from missions. Because many of its efforts are at an early stage, INVEST has been limited in its ability to generate and disseminate learning about what works in blended finance and why. The limited generation of formal learning with respect to past buy-ins presents a risk that lessons will be lost over time. While having DAI as the primary facilitating partner can create collective knowledge across buy-ins, absent formal, documented learning processes there is little to mitigate staff changes and fading memories. It may be difficult or impossible to validate research after the fact if careful plans have not been made to collect and analyze information and materials during the research process. INVEST risks losing much of the benefit of its efforts if intentional and careful plans are not made to collect and validate its action research as it unfolds. INVEST reports that, going forward, the mechanism plans to set two “deep dive topics” per quarter, for 2020, 2021, and 2022. INVEST believes that learning will be more effective in frequent, well-targeted, and small doses rather than through bulky learning products. For that reason, the mechanism will target at least one hands-on learning event with USAID for each topic and plans to ensure a USAID owner and messenger for each, both for knowledge transfer and to better speak to the point of view of USAID staff. According to INVEST, this direct approach will help INVEST understand and respond to skeptics, and provide a safe space to talk frankly about these approaches, what worked and what did not, and challenges faced. INVEST plans to support each topic with easy-to-digest communications materials and the learning agenda will be viewed as an important input into the editorial calendar for communications. Key informants including staff from USAID and network partner firms have not made wide use of the learning materials INVEST has shared to date. This is in part because of limited awareness of how to access to learning products, especially technical materials, and stakeholders sometimes describing available learning products as not particularly relevant to their needs or not technically useful. USAID staff reported they would value having greater access to INVEST’s various learning documents, exemplars, templates, and studies. One of the three firms that reported in interviews using learning materials associated with INVEST was a non-traditional partner firm that was new to working with USAID. According to a representative of that firm, INVEST “provided a lot of case studies, helpful case studies about other projects and through the, just the INVEST website, especially when we were familiarizing ourselves with the opportunities and the contract we were getting ready to execute.” Another firm with a long history of working with USAID and participation in multiple INVEST buy-ins, explained that his firm was more a creator than a consumer of content and described a webinar that had been “tightly coordinated by, and organized by, the INVEST Team.” A third firm that was also experienced in working with USAID and participated in multiple INVEST buy-ins was also a creator of content and stated, “at least in the early days, it was very informative to be able to share what we learned from the project and for everybody to hear what work was underway.” Around 80 percent of respondents to the partner firm online survey (n=17) were aware of INVEST publications, while 57 percent (n=12) had actually read any of INVEST’s documentation, reporting, or online material. Of the 12 online survey respondents who reported reading INVEST materials, 3 had never bid on an INVEST RFP, 3 were from firms that had subcontracts under INVEST, and the rest were from firms that had unsuccessfully bid on buy-in RFPs. CAPTURING AND VALIDATING LESSONS LEARNED INVEST has just begun the process of capturing and validating the broad lessons to be learned as it executes its action research project. As described in DAI’s proposal, INVEST’s research goal is the generation and aggregation of field-tested evidence on what works in blended finance and why, to create a foundation of knowledge upon which USAID can build new partnerships, programs, and approaches. One non-buy-in USAID respondent who was involved in the early design of MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 26 the mechanism explained DAI’s role in capturing learning as being able to connect the experience of one buy-in to inform the process of future buy-ins: “So the team that was working...We would work on Haiti. Then we would take what we learned in terms of developing a structure for the investment platform. When we did the assessment in the DRC, we would take all of those lessons learned and almost feed it to the mission and even the sub.” However, when asked how this was formalized, the respondent was uncertain about how well lessons were transferred. Learning activities in INVEST’s first two years generally focused on building interest and buy-ins from missions. As summarized in Annex F, INVEST generated a large amount of documentation, including blog posts, reports, and presentation materials. A leader within INVEST/DAI described the primary indicator of learning for the mechanism’s first years as the number and dollar value of buy-ins secured as well as USAID’s 2019 approval of a ceiling increase for the mechanism. At least one USAID stakeholder held a similar view about the lessons to be learned from MBIO demand for services: “Whereas I think for USAID in general, and one of the things that I think INVEST is proving, is that if you learn, and if you have this iterative model where you continually learn and continually adapt, you are much more relevant to the market, you are much more relevant to your stakeholders. And I think what is happening with INVEST, if I'm not dreaming, I think they have extended the project, or the buy-in to the project, the ceiling for the buy-in, three times now. Just showing that demand for the service that INVEST puts out there is massive. And I think the counterfactual might be that if they weren't this lean and if they weren't learning and adapting as they're going around, and reacting to what they were learning and to what the market is looking for, then maybe they wouldn't have been that successful. And that's maybe a good lesson that we could share.” Not all INVEST stakeholders viewed the ceiling increase as clear evidence of learning. One MBIO respondent who described INVEST as “a very good mechanism” said, “So to me, in terms of really seeing how successful the program was, there does need to be a deeper drill down of the actual projects that were bought into. And then how many of those were really successful and how many of them were successful because of INVEST versus, I think some of those initiatives would have happened anyway with or without that, the INVEST mechanism.” Perhaps different from a “deeper drill down,” each INVEST buy-in does have a MEL plan. However, one respondent noted that not all buy-in clients prioritize monitoring and evaluation, which affects the budget allocated to any MEL activities. Looking forward to INVEST’s next two-and-a-half years, the documentation suggests that INVEST is poised to begin a focus on learning capture, with a greater emphasis on the technical models, evidence, data, outcomes, and new partner profiles emerging from the buy-ins secured in the mechanism’s first two years. Based on interviews with INVEST stakeholders, especially MBIO staff, there is a strong desire to be able to easily access the various learning documents, exemplars, templates, and studies developed under INVEST buy-ins. For example, a USAID staff member who had worked closely with INVEST expressed: [INVEST is] doing hundreds of things a year now, and I only get to see 10 or 15. There's no place I can go to kind of get a sense of where all their different activities are. Maybe there is, and I don't know where it is. But who is doing what where, who they've already worked with. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 27 One of the things I've learned through this is sometimes, I'm going to do something that some other mission's already done something on…But what I don't have is…a way to go look at those things. If USAID had already paid for all this stuff, as a USAID employee, if I sign a nondisclosure agreement, I should be able to go see that stuff, because it'll help me not replicate the wheel. I don't know if that's in the works. I don't know if it already exists, but I've not seen it, and I think it should be there. INVEST has reported developing a library of buy-in SOWs and RFPs or statements of objectives (SOOs) that could be shared to help missions refine their thinking and requests for blended finance support, but this did not occur during the mechanism’s first two years. While INVEST launched a website in October 2019 with three RFP examples (www.invest-procurement.com), no MBIO interviewees said they had visited this website and it does not provide a full library of materials. INVEST has recently committed to expanding the site to include all live procurements and prior awards, an important step that will enhance the utility of the procurement website for the INVEST network partners.21 INVEST’s Learning and Dissemination Strategy describes a variety of channels such as “blogs, brown bags, and quarterly reports”22 intended for both USAID and external audiences to promote future learning. INVEST has developed written products such as close-ups, reports, research, thought leadership articles, and blog posts. Its hands-on activities have included closeout briefings, brown bags, conferences, events, webinars, and trainings. However, in response to questions from the evaluation team about INVEST learning materials that may have been useful, interviewees typically either stated they were unaware of such materials or that they had not made use of them. Of the 50 MBIO staff and partner firm representatives the team interviewed, 24 reported they made little or no use of INVEST learning materials. As one respondent explained, “I wasn’t necessarily looking for knowledge tools on how to mobilize investments.” The learning tool that respondents mentioned most frequently was INVEST’s periodic emails and newsletter, which 11 of 50 interviewees cited. A MBIO interviewee noted, “I do look at their weekly emails or whenever they put out that INVEST Monthly or something like that. I find that to be very valuable, very useful.” Nine of the 50 non-DAI respondents mentioned using the webinars and other learning events. A partner firm interviewee explained, “It’s more about getting a few ideas and see how that can fit into a broader sense of which I’m working on…And some of them have been useful from that perspective.” Two of the three firms INVEST identified as U.S. small businesses that were included in the sample reported participation in one or more INVEST webinars. Responding to questions about access to learning, 7 of 50 stakeholders viewed INVEST’s website as a source of learning, but INVEST’s web presence was not well known to 86 percent of the key informants. As one MBIO stakeholder noted, “Does INVEST have a website? I actually don’t know what you mean. I’m sorry.” Three respondents mentioned INVEST’s blog as a source of information and none reported making use of INVEST’s LinkedIn page. In comparison, as noted earlier, about 57 percent of the partner firm online survey respondents (n=12) reported reading INVEST publications, three of whom have never bid on an RFP. EQ2A: HOW HAS INVEST ACTED UPON CAPTURED LEARNING AND RESEARCH? DAI interviewees articulated aspirational learning targets for the future that included capturing technical models, evidence, data, outcomes, and new partner profiles that come out of buy-ins during INVEST’s first years. The perspective from INVEST/DAI leadership is that 21 This is based on correspondence shared with the evaluation team after the draft evaluation report was provided to INVEST. 22 Internal INVEST slide deck on learning, “Learning_PowerPoint_7_21_20.pptx.” MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 28 these targets will take on more importance “near the end of the project when more evidence has emerged and pandemic travel restrictions have passed.” Because learning activities during INVEST’s first two years focused on building interest and buy-ins from USAID, there was a relatively limited amount of learning and research captured and acted upon. Although INVEST’s annual reports detailed various “procurement innovations” and lessons, few key informants reported that they had benefitted from or acted on what had been captured. Respondents generally did not see as beneficial the examples of captured learning that INVEST described.23 An example of a lesson learned about the Afghanistan Trade and Investment buy-in that an annual report spotlighted was “not all sectors are equally ready for investment; some face significant operational and policy challenges.” A USAID staff member referred to these types of lessons learned as “more self￾promotions about what INVEST does or has done. I haven’t seen any technical work out of INVEST, really…I haven’t seen anything that has been all that useful, technically, in the market.” One partner network firm described a lesson learned and explained how that lesson was communicated to others: “We learned that it’s challenging to be a neutral advisor on transaction advisory work…typically, you were advocating for a client. And in one of the scenarios…, we were expected to be a neutral advisor, which meant we couldn’t really advocate for either side…I think looking as a lesson learned collectively, the team thought that it didn’t really serve either side as much as perhaps it could have, because each side then had to have their own lawyers, their own technical teams…So there was a lot of duplication of effort.” In response to questions about whether and how these lessons may have been shared, the above interviewee explained that the lesson learned had been included in the firm’s final report and had been discussed at INVEST partner network events, including one in-person event and one virtual event. Interviews with MBIO staff and DAI activity managers as well as document searches through USAID’s website (both the DEC and usaid.gov/invest) and other INVEST-related platforms suggest that some INVEST lessons learned may not have been shared widely. For example, a partner network firm described the use of a progressive risk mitigation model in which risks are proactively identified and de￾risked at each stage of a deal. As part of adapting and mitigating risks that came up in this buy-in, the partner firm developed a variety of documents to de-risk elements of the transaction. As one partner firm respondent described, “They’re documents, of course, but really they’re tools. So every time you come up against a risk and you develop a tool to mitigate it, whether it’s a legal document or a training document, or some other mechanism, you developed something new that helps counter a risk that somebody else may face next time.” Responding to questions about whether INVEST has passed on these tools or whether others would have access to the tools, that respondent said: “I don’t know the answer to the question. We give the tools to the INVEST team as well as the local partners and clients. So, in our project, [local government] received all of those documents as did the investors. Any my understanding is INVEST did send them to USAID, because there was a deliverable process involved with that. So, my understanding is it does go to USAID.” 23 INVEST’s annual reports described “procurement innovations” such as ensuring “that there is explicit language in the RFP detailing any specific country or sector experience that will be required” and testing “teleconference interviews as an efficient way to distinguish between top-rated proposals.” MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 29 Based on the KIIs and a review of INVEST’s communication and learning distribution channels, the evaluation team did not identify anything that showed that any of these tools were passed on to other missions or made more widely available for use. EQ2B: HOW HAVE MISSIONS, OPERATING UNITS AND NETWORK MEMBER FIRMS BENEFITED FROM INVEST LEARNING AND RESEARCH? MBIOs and network partner firms viewed participation in INVEST as an experiment in finding new ways to conduct development activities. For example, an MBIO that utilized INVEST did so partly because of an interest in addressing a research question differently than had been done in the past. In other words, USAID was interested in garnering a deeper understanding of an issue that had already been researched several times. Prior USAID research had identified business models that appeared extremely promising and had all the characteristics often associated with the potential for success. As one USAID staff member explained, “I didn’t feel like just having another development research project that would go to the same people that had been researching this topic for the past decade. To me [INVEST] conveyed enough information to say ’Okay, this is a different perspective and it intentionally has a different perspective.’ I think that was very valuable.” This MBIO respondent and two others saw INVEST as a mechanism that offered the opportunity to address this problem experimentally and with “fresh eyes.” When good business models failed to produce expected results and USAID was interested in an alternative perspective, MBIOs concluded that INVEST would provide “a new voice” and credited INVEST staff with the idea of conducting an examination of failures. In two instances, MBIOs reported that working with INVEST had led to changes in their own operations.24 For example, after a successful experience with an important buy-in, the Liberia Mission set up an office of private enterprise and brought in a foreign service officer to support that area of work. According to a USAID staff member who described the impact of INVEST activity, this action occurred because of the Mission’s work with INVEST. EQ3: HOW HAS INVEST COMMUNICATED TO USAID AND OTHER STAKEHOLDERS ABOUT ITS MOBILIZATION OF PRIVATE CAPITAL APPROACH? SUMMARY RESULTS FOR EQ3 INVEST had not effectively communicated critical information about aspects of its mobilization of private capital approach to key stakeholders. Processes from the INVEST BAA that made it possible to quickly award task orders to core partners have not been communicated, which has led to partner network firms questioning the fairness and transparency of procurement processes. This lack of transparency and limited communication has also fostered concerns related to how awards are selected generally and what oversight mechanisms exist. USAID staff valued one-on-one communications over media-based (e.g., internet, email, social media) efforts. Other donors, host-country partners, and targeted beneficiaries have generally not been the focus of INVEST’s communication in its first two years. INVEST’s events, workshops, and documentation have been useful for broadening the reach of its communication. 24 Although DAI did not share monitoring and evaluation data with the evaluation team during the data collection and analysis process, after the submission of the draft report USAID/E3/PCM forwarded a summary of monitoring and evaluation data for Y2Q4, which reports on INVEST indicator 3.3, “Number of USAID programming or policy adaptations informed by INVEST.” INVEST reported two programming or policy adaptations, which appear to align with the evaluation team’s independent finding reported here. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 30 INVEST has helped show some MBIO staff what is possible through a creative approach to procurement. Although at least four missions have multiple INVEST activity buy-ins, most USAID stakeholders felt their options were limited for integrating the mobilization of private capital into their strategic efforts due to bureaucratic challenges in the Agency. Challenges that multiple USAID stakeholders mentioned included a perception that people at USAID do not understand investment mobilization, the need to make individuals within USAID understand what can be done with private capital and how development can be facilitated, and a lack of understanding of how the private sector works, including the proper language to talk with the private sector, particularly in developing countries and closed, frontier markets. COMMUNICATION ABOUT THE MOBILIZATION OF PRIVATE CAPITAL APPROACH INVEST’s most effective channel of communication about its mobilization of private capital approach occurred as part of direct interactions between DAI and USAID staff. In response to questions about the utility of INVEST’s learning and research materials, stakeholders including USAID staff emphasized the value of one-on-one communications over media-based efforts (e.g., internet, email, social media, other published documents). A USAID communications professional explained her perspective on INVEST as a learning resource and described how she had made use the mechanism to develop a monitoring and evaluation plan: “I’m not necessarily reading analysis in my …role, I’m more seeking the advice, the support and the expertise of their communications professionals on their team to solve problems as they arise.” Similarly, another USAID staff member acknowledged that he had engaged with INVEST documentation and the website “a little bit,” but explained that “Most of my engagement is through the people I work with.” While 21 MBIO respondents noted the benefits of working closely with DAI, two USAID interviewees specifically noted that working in co-located spaces outside of USAID’s offices provided a forum for improved collaboration and communication in developing and understanding INVEST’s approach. While co-location is not necessarily scalable for a global mechanism built around subcontracting, the emphatic mention of it highlights how USAID valued the close, team-like communication with INVEST. An individual who was active in INVEST’s development emphasized the importance of people for any communication efforts: “not just the right talent but the ability to really work together in a cohesive team. We had such a…really strong working relationship and I think that makes all the difference, in terms of something functioning well. Ultimately, it comes down to people…the dynamics of the team…a flatter structure[…] That working in the same space and having constant communication. Really seeing each other as a team, rather than a funder and a fundee and a fundee under that, is important.” MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 31 Communication gaps around INVEST’s mobilization of private capital approach led to confusion and concerns among USAID and partner firm respondents. INVEST did not effectively communicate critical information about its procurement processes to key stakeholders. Most of the issues that interviewees described did not appear to be complaints from unsuccessful bidders or unhappy USAID buy-in clients. Instead, these interviewee observations were typically thoughtfully presented and many were accompanied by comments that demonstrated a commitment to INVEST and to USAID and its development goals. According to interviewees, the main challenges that arose due to what INVEST has communicated involve: • Communication about INVEST procurement processes • Transparency and communication • Intellectual property • Oversight These themes, which are further discussed below, are not mutually exclusive. In many cases, how a respondent understood one area (e.g., procurement) affected how they viewed another area (e.g., intellectual property). Regarding working through INVEST, one MBIO respondent reflected: “It wasn't too clear, it seemed to be a very complex structure. I understand more of it now, but we had to basically learn as we went…DAI was the main POC, and [firm] after they had been the one selected […] DC's contracting officers and the mission's contracting officers…seem to speak each other’s language and understand what was needed. So I'm not too concerned that the next step is not fully clear with me,…They'll just let us know what they need from us….With INVEST it's been different because you add the fact that it's based out of DC and it somehow has to work with our contracting office […] But my problem was you don't know what you don't know. So if we didn't have the right question, we might not have gotten information that could have been important or relevant for us.” COMMUNICATION ABOUT INVEST PROCUREMENT PROCESSES Based on KIIs with 17 firms, including 12 with successful buy-in bids, the evaluation team concludes that INVEST procurement practices are not well understood, particularly the distinction between RFPs and task order subcontracts. At least seven respondents specifically noted concerns about fairness and transparency in INVEST’s procurement practices. The evaluation team did not conduct a procurement Technical Vocabularies – When Words Mean Different Things to Different People Respondents often cited the challenge of conveying precise terms to groups with different professional backgrounds as a complicating factor in INVEST’s communication about its mobilization of private capital approach specifically, and blended finance generally. George Bernard Shaw’s reference to America and England as “two countries separated by a common language” was used to explain the relationship between vocabularies for development finance and commercial finance. One stakeholder explained the hazards of different types of parties making investments with different expectations. “The financial profit motivated investors in these partnerships or these investment funds…speak about leverage, speak about returns from their perspective, which is [from the] Wall Street Journal MBA perspective. But then those same topics, just those specific terms of leverage and returns, mean something completely different when you're talking to the donor community or USAID, for example. Those two terms…reflect different motivations…for forming these partnerships or these collaborations. Being able to understand…what type of leverage or what type of returns drive a corporation or drive a financial investor is really important for developing a partnership that works.” MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 32 review; instead, these and other comments from interviewees represent their personal perspectives and speak to what and how INVEST communicated about its approach. INVEST documentation notes that, “INVEST is also using [indefinite quantity contract] mechanisms that allow task-order specific flexibility for phasing in or out support as required by the buy-in client’s needs,” but does not detail when task orders are used compared to RFPs.25 Absent a full understanding of INVEST’s mobilization of private capital approach, respondents – particularly partner network firms – developed their own theories about how the process worked. As one interviewee from a partner network firm noted, “Very few firms who have the capability in blended finance trust the DAI is running a fair procurement process for USAID.” Based on the results of 64 KIIs with respondents from 23 MBIOs, as well as with staff from DAI and 17 firms, the evaluation team developed the process map in Figure 7 to show that communications actually occurred throughout the INVEST mechanism even though the process was not always clear to stakeholders. The process map highlights the lead point of action or contact through the buy-in process as explained to the evaluation team. In some cases, respondents had contradictory experiences or explanations about how the communication process occurred, but the process map provides an overall model of the most frequently described outreach and actions. What emerged from interviews is that partner network firms rarely have a view into any steps that do not directly occur in the “Partners” column or those that directly connect INVEST to partner firms. The process map is not intended to definitively describe exactly how INVEST operationalized its procurement processes. Instead, the graphic shows how the individuals interviewed perceived the process to operate. 25 See “INVEST: FAQs,” p. 3. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 33 FIGURE 7: MAP OF INVEST'S PROCUREMENT PROCESS, BASED ON INTERVIEW DATA MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 34 TRANSPARENCY AND COMMUNICATION At least 14 interviewees, including 5 MBIO clients and 9 partner firms of which 4 were successful proposal winners, viewed transparency as an important area to improve in the mechanism. Interviewees primarily saw issues with transparency in communications as a by-product of DAI’s position as a private sector firm directly involved in a USAID procurement process. Figure 8 shows an example of where a lack of communication across stakeholders creates confusion. INVEST’s general RFP process, as perceived by network firms who are not core partners, is that RFPs are shared and then firms respond. According to data reported to the evaluation team, core partners accounted for around 37 percent of the 43 subcontracts issued in INVEST’s first two years. FIGURE 8: GENERAL RFP AND RESPONSE PROCESS (as described) FIGURE 9: DECISION PROCESS FOR BUY-IN ACTIVITIES Despite the fact that INVEST states that it seeks to adhere to all Federal Acquisition Requirements for subcontracting,26 around a third of KII respondents (five MBIO clients and nine partner firms) felt mechanism transparency was an issue. For example, a representative from a firm that had won multiple buy-in subcontracts and had a long history of working successfully with USAID explained, “There was just no transparency.” Based on the information that had been made available to him, he felt that a disproportionate amount of work was being implemented by DAI and its core partners. Noting the small number of original network partners (excluding the three core partners) that had been awarded contracts, he continued, “you start to say, ‘Something's not right here.’” 27 The respondent 26 For example, this is noted on the investment-procurement.com website. However, as noted elsewhere in the report, only two respondents were aware of this site’s existence. 27 The Initial Network Organization consisted of DAI, the three core partners (Convergence, CrossBoundary, and Tideline) and 25 other organizations. Based on information provided by INVEST, 3 of the 25 non-core original network partners were awarded contracts with INVEST during the first 2 years of mechanism’s operation. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 35 expressed his belief that his firm’s status as a competitor to DAI was at the root of the issue. “But the bigger thing is, why would DAI ever give [his firm] work?28 … And that's the problem when you outsource the procurement process to a private sector firm, there's no transparency there.” Interviews with DAI staff suggest that the process shown in Figure 9 is more complex, although only 2 of the 64 interviews mentioned additional complexities. This process involves determining if a given request fits the technical capabilities and the timeframe required. As one respondent involved in INVEST’s early design process explained, “the intent was to be able to capture and rely on these [core] partners very quickly, in case we did get buy-ins that required a really rapid response.” Yet the process shown in Figure 9 was not reported by any of the non-core partner firm interviewees. Indeed, three USAID respondents noted that lack of understanding around the role of core partners to quickly respond to buy-in requests could have the potential to create confusion or mistrust. DAI noted in response to the draft evaluation report that “RFPs procurements always go to the full partner network as the preferred option.” However, this explanation does not include the various task orders issued as part other contractual arrangements. DAI, in reviewing the process that was explained to the evaluation team and displayed in Figures 7 and 9, noted that its actual process differs from what is depicted. This discrepancy highlights the disconnect between partner firm and MBIO understanding and INVEST’s practices as DAI sees them. INVEST’s procurement website also includes a section on its procurement process; however, this summary does not mention task orders. The topic of transparency related to communication also raised issues about future engagement with INVEST. A partner firm executive explained how concerns about transparency might preclude his firm from working with INVEST in the future, despite a long history of working with USAID: “this is just the natural outcome of when you give an entire mechanism to one firm, you outsource the procurement process to one firm to hold, and then that firm decides … what they make public.” Speaking more specifically about how this relates to communication of the process, the respondent explained: “I take DAI at their word that…the evaluation processes are fair. I think they're doing their best. I have no reason to suggest that anything untoward is happening…We have no clue who sits on these evaluation committees. We have no idea who votes, how the votes get tallied. We wouldn't know that about technical evaluation committee with USAID, but this isn't a USAID technical evaluation committee. Yes, they have told us that these things are in place and they're best practices. But I don't think that's sufficient for us to move forward with INVEST in any meaningful way.” Seven of the 17 partner firms interviewed, including 2 firms that worked on multiple buy-ins, questioned whether INVEST procurement processes resulted in outcomes that were consistently fair and transparent to contractors and partner network members. These concerns were expressed across the entire spectrum of INVEST partners, including established firms that had been awarded multiple buy-in contracts and had long history of working with USAID as well as newer firms that had not won an INVEST award. In five instances, questions of fairness and transparency were causing the firms to reconsider whether they wished to continue working with INVEST and, in a few cases, with USAID. As noted above, the evaluation team did not seek to assess the issue of fairness. Instead, it relates to how the RFP, technical review, and award process was communicated to INVEST partner network firms and USAID stakeholders. 28 DAI does not decide unilaterally which firm will be awarded a contract. Winning bids are chosen through a process that involves a Technical Evaluation Committee (TEC). Membership on the TEC is more-or-less evenly split across DAI and USAID, but DAI has at least one additional TEC member given that it holds fiduciary responsibility for the network. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 36 A long-time USAID contractor that had been awarded subcontracts under multiple buy-ins described the transparency of INVEST’s procurement practices as “horrific.” He further explained that, “I just really feel like I don't even know what's being bid on, I don't know who's winning these things and why.” As a result of these concerns, the individual interviewed explained that his firm’s interest in participating in future buy-ins had “definitely waned.” Of the seven partner firm interviewees who raised concerns about the fairness of the procurement process, four asserted a belief that these issues of favoritism or bias were the reason for INVEST’s lack of transparency with regard to procurement practices. One partner firm respondent explained his wish for additional transparency about the selection process, stating “They have their own preferences for their own reasons, they don't disclose that, but that would be, I think, an interesting shift or move if they actually made that transparent.” INTELLECTUAL PROPERTY INVEST has not communicated with the partner network about how intellectual property would be protected or shared. Because INVEST was created through a BAA process, many of its component initiatives involve co-creation events with multiple stakeholders. As designed, the INVEST model results in extensive sharing of firms’ intellectual property, primarily during the procurement process and as part of learning capture events. DAI noted in response to the draft evaluation report that the intellectual property shared through INVEST’s procurement process is stored securely, with access restricted to its operations team. INVEST has also noted that staff who review proposals sign non￾disclosure agreements (NDAs). In one interview, a DAI respondent noted they were aware of concerns with this issue, saying, “I had somebody tell me, a month ago, ‘I'm sure you're sharing, you share all the proposals that come in with DAI for new business development.’ And I'm like, ‘Are you kidding me? Absolutely not. Those are under strict NDA. They go into a folder.’ Not even everybody on the team sees them. There's so many misconceptions.” Indeed, interviewees from 8 of 17 firms reported INVEST’s treatment of intellectual property as a concern, including three firms that have worked on multiple buy-ins. When asked whether INVEST shared its NDA or file security approach with partner network firms, the DAI respondent said, “We don't put that out there, we're not leading with, we're not sending that around to every partner.” As shown in the interview guides in Annex E, the evaluation team did not seek out information related to INVEST’s handling of intellectual property. The eight respondents who noted it was a concern brought it up in reflecting on their experiences with INVEST (either through working on buy-ins or responding to RFPs). For example, a representative of a firm implementing multiple buy-ins explained: “in the implementation and in the contract that we had signed, essentially DAI gets 100 percent of the intellectual property that we produce. That's actually, I think one of the key things that is quite or can be quite problematic…Obviously USAID gets [deliverables], but that's fine because they are the end client. DAI gets it, but [they are] also a competitor in the market.” Two interviewees related stories to the evaluation team of how they believed their intellectual property had been misused. For example, a representative of a partner firm that was participating in multiple aspects and buy-ins of INVEST operations described a situation where various documents were used by INVEST without attribution or other information to identify the source. Another partner firm respondent noted, “we're giving away intellectual property to DAI, and we don't know how good of a firewall there is between DAI INVEST. And we have some recent incidents where they've kind of been releasing information to DAI.” MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 37 INVEST has committed to including issues of intellectual property and the safeguarding of partner proposals as part of the nearly finalized partner governance policy, an important step toward addressing this issue. OVERSIGHT Transaction advisors, partner firms, and others who participated in INVEST activities did not describe a clear understanding about the ethical and professional responsibilities associated with INVEST activities. Little or no information had been disseminated about bureaucratic oversight mechanisms in place or norms and standards around investment activities. Based on interviews with 7 respondents (4 USAID and 3 partner firms) associated with 15 buy-ins and two DAI staff, as well as comparisons of descriptions of workflows, roles and responsibilities, and other documentation, partner firms and USAID employees were unsure about or unaware of what mechanisms were in place to address any duties that INVEST and its contractors might have toward the best interests of firms, investors, and others with whom they might be involved as part of complex and bespoke multi-party transactions. One firm that participated in multiple INVEST buy-ins understood the potential for future problems and noted the possibility of “…conflicts, there would be questions of independence of competition…it’s a space we do keep an eye on. Especially as I have seen a number of the large USAID contracting firms are coming in to have their own direct investment vehicles.” This firm currently has a policy that restricts the firm and its subsidiaries from investing in a third party while another unit of the firm provides advisory services, except with written authorization of the relevant government agency or prime contractor. Following the end of an advisory engagement, a cooling-off period (typically six months) is observed before the firm provides investment or direct or indirect financing to a client. Other good practices in the firm’s published policy include steps to prevent unequal access to information, such as an information firewall. When asked about the potentially challenging situation in which an INVEST subcontractor might provide concurrent investment and transaction/strategic advisory services, a USAID staff member stated, “it’s been my impression that [firms] have been able to draw very bold lines, that they don’t walk across as they serve in each of these roles.” However, the USAID interviewee did not offer further details about how INVEST might handle this situation or where responsibility and the “very bold lines” were located. The USAID interviewee agreed that the question deserves a deeper analysis. An academic interviewee who was also a managing partner of a firm with extensive USAID experience reflected on the ethical challenges associated with various development contexts: “I actually believe you need people to look at these problems. They're very complex. There are large societal issues…you need multi-stakeholders. People from the policy side, people from the finance side, people from the advocacy side and people from the monitoring and evaluation side to come together to solve large problems.” He further explained his viewpoint about USAID investment projects that raise concerns and noted his view that many do not follow the Chartered Financial Analyst Code of Ethics and Standards of Professional Conduct, which provides guidance on how to deal with various finance-related ethical issues. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 38 Interviews with INVEST staff suggested INVEST is poised to address potential concerns occurring through the intersection of various investment and advisory roles, though not necessarily through a formal process or mechanism. As one INVEST respondent noted: We're pretty careful about how we make an award to make sure that these days if you're looking at working with a fund, there can be a mix of different capital coming in. We want to be very responsible and we're pretty careful about that…We look at that…I think that the idea of conflict of interest…it may be that we’ve been thinking about it a little too narrowly.” EQ3A: HOW HAVE OTHER DONORS, HOST COUNTRY COUNTERPARTS, TARGETED BENEFICIARIES AND OTHERS LEARNED ABOUT INVEST’S MOBILIZATION OF PRIVATE CAPITAL APPROACH? Respondents provided a few examples to demonstrate how INVEST’s mobilization of private capital approach was shared with donors, host-country counterparts, targeted beneficiaries, and others. As an example of INVEST communications with groups external to USAID and the partner network, the INVEST Design Challenge Workshop (also called the Donor Practitioner’s Forum) was an event co-hosted by USAID, Global Affairs Canada, the Shell Foundation, and the Ford Foundation. With a reported purpose of surfacing shared challenges, identifying solutions, and building a network that can support and learn from its members, the 2-day event included 30 representatives from a diverse group of leading donor institutions (including aid agencies, development finance institutions, private foundations, and private donors). As described in the summary of meeting takeaways and next steps, the group expressed optimism about “a growing political consensus on the importance of harnessing investment as a complement to traditional grant-making, along with the passion of future generations for impact.” A partner network firm interviewee described his understanding of communications between INVEST and the donor community, placing emphasis on the value of donor agencies developing their own communication materials and stating, “I think from a communication standpoint, again, the few resources that I have seen I think have been fairly straightforward and informative. And again, I think a lot of those materials have been made publicly available…Because these aren't challenges that are unique to USAID and I think the broader community again, really deserves to benefit.” In describing their interactions with the INVEST model, INVEST stakeholders only touched peripherally on the issue of how others may have learned about INVEST’s mobilization approach. Interaction with host-country counterparts was not a feature of most interviews with INVEST stakeholders, although one MBIO staff member explained how he was driving PSE Policy by closely working with his counterparts in the country where he was located and with the world. In a similar way, while little was said in the interviews specific to INVEST’s beneficiaries, INVEST’s published work on strategic corporate engagement describes the need for USAID to “ask the right questions to ascertain how a partnership or investment will impact that market and its beneficiaries.” EQ3B: HOW HAVE MISSIONS AND OPERATING UNITS MODIFIED THEIR PROGRAMMATIC/STRATEGIC EFFORTS TO INTEGRATED MOBILIZATION OF PRIVATE CAPITAL MODELS, PARTICULARLY THE LEVEL OF BUY-INS, THE NUMBER OF FOLLOW-ON BUY-INS AND FUTURE EXPRESSIONS OF INTEREST? Missions and operating units have largely not modified their programmatic or strategic efforts to integrate mobilization of private capital models. However, at least four MBIOs have enacted follow-on buy-ins and INVEST has highlighted new contracting approaches MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 39 for some USAID staff. Based on KIIs with 60 INVEST stakeholders, the modifications to MBIO strategic efforts that have been identified to date were based on INVEST’s organizational structure. In one instance, an MBIO decided to adopt the partner network model and create a specialized niche network of regulatory experts, with a goal of having ready access to “different voices.” As another example, the Liberian Mission has reportedly set up an office of private enterprise and brought in a foreign service officer to support private enterprise work, which was attributed to the Mission’s work with INVEST. There were no evident changes to programmatic or strategic efforts toward the integrated mobilization of private capital models. However, it is not necessarily within INVEST’s power, or even a given MBIO stakeholder’s ability, to modify programmatic efforts. As one MBIO respondent explained, “I wouldn't say that we're necessarily at the point where we're ready to go ahead and start looking at [public-private partnerships] yet, but at least it opens up our mind to two other ways of perhaps approaching service improvements through perhaps private engagement.” A MBIO interviewee also mentioned technical capacity limitations as a constraint to integrating the mobilization of private capital into future efforts. Another MBIO respondent put the onus on USAID for how quickly the integration of private capital into strategic efforts could occur: You know, there's a lot of things that I would like to do, that I would like to grab [from INVEST’s approach]. But USAID is, as you might very well known being contractors to USAID, it's a very large, slow-moving bureaucratic machine. And unfortunately when it comes to being very efficient, very quick, very nimble in how we do things, in how we procure, it's very hard to change. Tracking such changes was not a major part of INVEST’s approach in its first two years. INVEST initially developed a Monitoring, Evaluation and Learning Plan, including performance indicator reference sheets, that could be used to accumulate evidence of modifications to MBIO practices. However, INVEST has not formally captured information about many of the performance indicators included as part of this Plan, including its performance metric 3.3 “Number of USAID programming or policy adaptations informed by INVEST.”29 According to an INVEST stakeholder, USAID has not defined the frequency with which the indicator data should be provided nor was any MEL data shared with the evaluation team. 30 Additionally, the performance indicators of significant interest to USAID are number of transactions, value of capital mobilized, and the leverage ratio as reported by the INVEST stakeholder. 29 As noted in EQ2, after submission of the draft evaluation report, INVEST provided indicator data that shows two policy adaptations reported for indicator 3.3, which aligns to the two cases the evaluation team identified. 30 As one INVEST respondent noted, “We have a number of performance indicators that were identified by USAID at the beginning of the project that we track and report on. Although I should flag that PCM, the client, has not defined the frequency to which you have to submit the indicator data. That being said, they're not reported on formally like in a table, for example, rather than the statistics are incorporated into our quarterly reports…it was never a formal deliverable. It's not in our contract for example. But the three main performance metrics of priority for us to report on and collect to PCM is regarding our transactions. So those are, I think it's like 1.1.1.3. So it's number of transactions, value of capital mobilized and what the leverage ratio is.” MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 40 RECOMMENDATIONS Based on these findings and conclusions, the evaluation team recommends the following. RECOMMENDATIONS FOR INVEST • Seek feedback from unsuccessful bidders, especially new and underutilized partners, to understand how they perceive the debriefing materials. Enhance the quality and effectiveness of debriefings as warranted. • Develop and implement a methodology to capture and validate learning materials in real time, ensuring retention of important documents and memorialization of key decisions, including rationales. • Capture and act on lessons learned, even if they are not success stories, by creating a formal, documented process that can be tracked showing the challenge/solution, actions taken/not taken, etc. • Learn how stakeholders access and utilize learning and communication materials to better target MBIOs and partner network firms. • Develop and distribute a policy or set of standards/values/norms around intellectual property. Convene a webinar or other interactive session about the issue of working collaboratively with commercial competitors, and carefully consider the impact of relationships between firms within the partner network. RECOMMENDATIONS FOR USAID AND INVEST • Make all INVEST deliverables available for use in planning and future research efforts, unless there is a compelling and articulable reason not to do so. • Develop and publish a clear explanation of how work is distributed via the INVEST mechanism, including a high-level explanation of the contract process and why some work may not be competed via the RFP process. Make a listing of all issued RFPs available for network partners’ easy reference. • Determine where the responsibility lies for protecting all parties to investment transactions from inappropriate or conflicting duties or roles. Locate the “bold line” regarding INVEST practices and ensure that activity designs reflect good practices in management control. RECOMMENDATIONS FOR USAID/E3/PCM • Conduct an assessment of the technical quality and utility of INVEST’s technical products, considering factors such as the quality, depth, and precision of content and the utility of such products to technically sophisticated users in USAID and the development or financial/investment communities. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 41 ANNEX A: EVALUATION STATEMENT OF WORK MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM Introduction This is a Statement of Work (SOW) for a mid-term performance evaluation of the USAID-funded INVEST mechanism, implemented by Development Alternatives, Inc (DAI) under Contract No. AID￾OAA-C-17-00090. The mechanism runs from September 2017 through September 2022, with a total estimated cost of $95 million. The award is administered by the Office of Private Capital and Microenterprise (PCM) in USAID’s Bureau for Economic Growth, Education, and Environment (E3). The current Contracting Officer’s Representative (COR) is Cameron Khosrowshahi; the Alternate COR (A/COR) is Thomas Rojas. Evaluation Purpose The purpose of the INVEST mid-term performance evaluation is twofold: (1) to assess the effectiveness of the INVEST Project in expanding USAID efforts to mobilize private investment and engage with the private sector in jointly finding innovative solutions for development challenges, and (2) provide recommendations for the remainder of the performance period. Use of Evaluation Findings, Conclusions, and Recommendations USAID will use performance evaluation findings, conclusions, and recommendations as learning opportunities for understanding what the INVEST mechanism has achieved, how it is perceived and valued, what opportunities for collaboration were available and utilized, and whether it is advisable for USAID to continue with such a model. Other stakeholders with an interest in the evaluation findings and conclusions include USAID’s overseas missions; USAID/W operating units; INVEST private sector network member firms; and targeted beneficiaries. DAI and other stakeholders will have an opportunity to learn about INVEST’s strengths and areas for improvement. Background Information Blended finance is the strategic use of philanthropic or public resources to catalyze private capital flows in pursuit of development results and financial returns. Conceived and initially funded by USAID’s Office of Private Capital and Microenterprise (PCM), the Blended Finance Program represents an opportunity for USAID to engage new partners, test and scale up approaches, improve programming, and mobilize capital to achieve development objectives. Designed with a primary focus on researching, developing, and disseminating best fit blended finance solutions, the Blended Finance Program will build and flexibly deploy a network of experts from an array of organizations to frame and test customized financing solutions that address investment constraints, leverage additional capital, and generate development impacts to create value for USAID missions and operating units. The Blended Finance Program will enable USAID to quickly access relevant expertise from a broad and diverse network of partners, provide tailored, co-created, customer-centric solutions, and drive cross-agency learning and advancement, operationalized through a Blended Finance Innovation Center. Some of the technical challenges to USAID’s use of blended finance overlap with difficulties facing other development actors in the space. For example, at the project investment level, investors frequently lament a lack of investment-ready projects, barriers to capital supply, a lack of access to affordable and appropriate risk mitigation instruments, and high firm-level transaction costs. At the ecosystem or sector level, challenges include a lack of skills among and imperfect information from financial MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 42 intermediaries and advisors, immature market conditions, and lack of connective tissue among actors, all of which inhibit development of a blended finance network and, thus, impede the scaling of blended finance instruments. Further, Agency insights from current blended finance initiatives are scattered among missions, offices, and bureaus. As a result, champions of blended finance and their experiences and learned best practices are isolated, and overall awareness and confidence in the use of blended finance for achieving development objectives is low. The overarching goal of the Blended Finance Program is to develop and test a model that enables USAID to quickly engage new partners and mobilize private capital, improving the ways in which the Agency achieves its development objectives. In September 2017, USAID issued the INVEST prime contract to Development Alternatives, Inc. (DAI) as a result of a co-creation process under the Catalyzing Private Investment Broad Agency Announcement (BAA). INVEST is a groundbreaking USAID award that jumpstarted the Agency’s expanded efforts to mobilize private investment and engage with the private sector to jointly find innovative solutions to development challenges. To ensure robust competition, the co-creation period was held over five months, in which 58 applicants submitted expression of interests and 27 were selected for co-creation opportunities as partners of DAI under the BAA. The INVEST mechanism is structured so that the prime contractor (i.e., DAI) does not compete with the small firms for business. DAI provides knowledge management, monitoring and evaluation, and strategic communications for the overall INVEST program while facilitating a “clearing house” function through which information is shared and best practices are realized from the buy-ins and subcontracts. Those subcontracts are competed via separate requests for proposals through the INVEST network of providers to implement a projected eighty percent of the technical programming. The network and subcontractors are a variety of organizations: small and large businesses, universities, non-profit organizations and local counterparts. Many of the firms currently in the provider network are those that have never done any business with USAID. Over 18 months, the provider network has expanded significantly, from 27 to more than 170 firms, who respond to the subcontract opportunities, form teaming arrangements, and share expertise and learning; thereby, resulting in a highly efficient integration of new, specialized and relevant network of providers complemented by an open call for proposals. The INVEST network of specialized investment advisors and investors co-research, develop, test, and iterate blended finance approaches that lower transaction costs, address information barriers, mitigate risk/reward imbalances, provide relevant expertise, facilitate partnerships, and enable channels for sharing lessons learned—thereby increasing the flow of capital into investment opportunities that drive measurable and sustainable social, economic, and environmental impact. Thoughtful and adaptive monitoring and evaluation (M&E) and knowledge management ensures that lessons learned are captured and that successful blended finance approaches are validated and become solutions available to the Agency and other stakeholders for deployment elsewhere. Supported activities include: 1) accelerated procurement tools to help USAID align to private sector timelines; 2) flexible instruments to periodically expand partner specialized expertise; and 3) collaborative design for tailored support, rapid iteration and learning. Scope of Work The Contractor will assess the effectiveness of the INVEST Project in expanding USAID efforts to mobilize private investment and engaging with the private sector to jointly find innovative solutions to development challenges. In particular, the Contractor will answer the following questions (numbers do not reflect the priority): MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 43 1. How has INVEST operationalized the mobilization of private capital approach through its network of participating firms? a. How has INVEST facilitated the procurement process for Mission and Operating Unit buy-ins to the activity? b. How has the network of participating firms under INVEST expanded and changed over time? 2. How has INVEST captured and validated lessons learned? a. How has INVEST acted upon captured learning and research? b. How have Missions, Operating Units and network member firms benefited from INVEST learning and research? 3. How has INVEST communicated to USAID and other stakeholders about its mobilization of private capital approach? a. How have other donors, host country counterparts, targeted beneficiaries and others learned about INVEST’s mobilization of private capital approach? b. How have Missions and Operating Units modified their programmatic/strategic efforts to integrated mobilization of private capital models, particularly the level of buy-ins, the number of follow-on buy-ins and future expressions of interest? The Contractor will ensure that the evaluation of above mentioned activity is consistent with USAID ADS (Chapters 201 and 578) and USAID’s Evaluation Policy (2011) requirements and recommendations. In answering evaluation questions, the Evaluation Team (ET) should highlight gender specific approaches promoted by INVEST and practiced by its partners and related outcomes, as appropriate. For the evaluation purposes, “relevance” is a measure of the ability of a particular program intervention being pertinent to program objectives; and “effectiveness” is a measure of the ability of a particular program intervention to produce a planned effect or result that can be qualitatively measured. Evaluation Design & Methodology TBD based on our questions and purpose... Evaluation Team Qualifications and Composition ET Leader: The Contractor must designate one ET member to serve as the ET Leader. The ET Leader must have sufficient experience in designing and/or conducting performance evaluations of international development projects and good knowledge of USAID Evaluation Policy and evaluation reporting requirements. Excellent communication, both verbal and written, skills and experience in managing performance evaluations of similar size USAID activities are desirable. Experience in designing and conducting performance evaluations of similar size USAID activities is desirable. Evaluation Specialist: The Contractor must assign at least one Evaluation Specialist with strong understanding of data collection and analysis methodologies, network analysis, and substantial experience in designing and conducting evaluations of international development activities. Evaluation Specialist(s) must have good knowledge of USAID Evaluation Policy and evaluation reporting requirements. Experience in designing and conducting performance evaluations of USAID private investment and/or private sector engagement-related activities is desirable. Note: One individual may act as both the ET Leader and an Evaluation Specialist or International Governance Consultant if all qualifications requirements are met. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 44 USAID asks that gender be considered in the formation of the ET. One or more team members should have experience in engendered evaluation methods and knowledge of gender issues in the public governance sector. The ET should also include one or more members with local cultural expertise, including an awareness of gender norms, how gender interacts with other identity elements, and which sub-groups of women may be at risk for exclusion from the project or evaluation. The ET Leader, Evaluation Specialist(s), and any additional technical experts will be key personnel under this <>. Proposed personnel are expected to be the people on the job. Any substitutes to the proposed team must be vetted and approved by the Evaluation COR before they begin work. Evaluation Management E3/PCM will appoint an Evaluation COR to provide technical guidance and administrative oversight of the INVEST evaluation, to review the Evaluation Work Plan, and to review and accept the draft and final Evaluation Report(s) (ER). E3/PCM will also appoint an Alternate COR (A/COR). E3/PCM may delegate one or more USAID staff members to work full-time with the ET and/or participate in the data collection. The Evaluation COR will inform the Contractor about any full-time/part-time USAID delegates no later than three working days after the submission of a draft Evaluation Work Plan (EWP). All costs associated with the participation of full-time/part-time USAID delegates in the evaluation will be covered by E3/PCM. To facilitate evaluation planning, the COR will make available to the Contractor the following INVEST documents within one working day of the award effective date (as warranted, the Contractor will receive additional project-related documentation): the Program Description, Annual Work Plans, one Performance Monitoring and Evaluation Plan, Quarterly Reports and Annual Reports, as well as the lists of INVEST partners. To keep E3/PCM informed about the status of the INVEST evaluation, the Contractor will submit an electronic version of a draft INVEST EWP to the Evaluation COR within 15 working days following the award and at least 10 working days prior to the proposed ET’s departure for the field data collection. The submitted EWP should be fully consistent with the Scope of Work requirements and Contractor’s proposal (if the latter is fully or partially incorporated into the PO). The EWP should highlight all evaluation milestones and include: (1) a preliminary list of interviewees, (2) a preliminary list of survey participants (when survey is planned), (3) a preliminary schedule of the ET interviews/meetings, surveys, and focus group discussions (FGDs) (when planned), (4) all draft evaluation questionnaire(s), survey(s), FGD guides, etc., which the ET may use for evaluation, (5) locations and dates for piloting draft evaluation questionnaires and survey(s), (6) the proposed evaluation methodology including selection criteria for comparison groups (if applicable), and (7) an ER outline (if it will be different from the attached template (Attachment 1)). The Contractor will update the submitted EWP (first of all, the lists of interviewees, the lists of survey participants, the schedule of interviews/meetings/surveys/FGDs, etc.) and submit the updated version to the COR on a weekly basis. The Contractor may prepare EWP as a Google-based document to facilitate USAID staff access. The ET will conduct weekly briefings for the Evaluation COR/Activity Managers, and other relevant USAID in order to keep them informed of the progress of the INVEST evaluation and any issues that may arise/have arisen. The ET shall be prepared to conduct a briefing for the Evaluation COR, Activity Managers, and other relevant USAID within two working days after their arrival for the field data collection. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 45 The ET will invite the Evaluation COR and other relevant USAID to participate in all meetings, group discussions, site visits and other activities planned in conjunction with the INVEST evaluation as soon as those events are included in the EWP. The ET shall be prepared to have USAID staff and other activity stakeholders invited by the Evaluation COR to any meetings, site visits, or other activity planned in conjunction with the evaluation as observers. The ET will discuss any evaluation barriers/constraints and significant deviations from the original/updated EWP with the Evaluation COR and seek USAID’s guidance on those matters. All modifications to the required elements of the Evaluation SOW, whether evaluation questions, evaluation team composition, or timeline, need to be agreed upon in writing by the COR. Any revisions should be updated in the SOW that is included as an annex to the ER. Logistical Support The Contractor will be responsible for all logistical support of the evaluation activities, including translation/interpretation as needed, transportation, accommodation, meeting and site visit arrangements, office space, equipment, supplies, insurance and other contingency planning. The Contractor must not expect any substantial involvement of the USAID staff in either planning or conducting the evaluation (except for full-time/part-time USAID delegates discussed above). Upon request, E3/PCM will provide the Contractor with introductory letters to facilitate meeting arrangements. USAID requests that any forthcoming American and local holidays be considered in scheduling evaluation meetings, group discussions, surveys, and site visits in the United States and abroad. Deliverables To document the mid-term performance evaluation of INVEST, the Contractor will submit a clear, informative, and credible ER (up to 30 pages, excluding annexes and references) that reflects all relevant ET findings, conclusions, and recommendations made in conjunction with the performance evaluation of INVEST. The ER must describe in detail the INVEST evaluation design and the methods used to collect and process information requested in the Evaluation Purpose, Scope of Work, and Evaluation Design & Methodology sections. It must disclose any limitations to the evaluation and, particularly, those associated with the evaluation methodology (selection bias, recall bias, unobservable differences between groups, etc.). The ER Executive Summary Section should be three-five pages long and reflect the purpose of the evaluation, evaluation methodology and its limitations, key evaluation findings, conclusions, and recommendations. A preliminary version of the ER Executive Summary Section must be provided in a PowerPoint format to share with the PSE Team during the evaluation debriefing. The ER must be in line with relevant USAID ADS (Chapters 201, 205, 320, and 578 as well as relevant mandatory references such as the 201mah and 201maa ) and USAID Evaluation Policy requirements and recommendations. In particular, the ER should represent thoughtful and well-organized efforts that include sufficient local and global contextual information so the external validity and relevance of each activity evaluation can be assessed. Evaluation findings should be based on facts, evidence, and data. The findings should be specific, concise and supported by reliable quantitative and qualitative evidence [i.e. there should not be words like “some”, “many”, “most” in the report and frequency of responses and absolute number of interviewed respondents should be given, e.g. five out of 11 experts agreed that …; 30 percent of survey respondents reported that …]. Evaluation conclusions should be supported by a specific set of findings. Evaluation recommendations should be clear, specific, practical, action-oriented, and supported by a specific set of findings, conclusions, estimates of implementation costs, and suggested responsibility for the action. The Contractor shall ensure that conclusions and recommendations are based on data that are accurate, objective, and reliable. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 46 In the annexes, the ER should include the Evaluation SOW; description of the ET and its member qualifications; the final version of the EWP; the conflict of interest (COI) statements, either attesting to a lack of COI or describing existing COI, signed by all members of the ET; the tools used for conducting the evaluation such as questionnaires, checklists, and discussion guides; in-depth analyses of specific issues; properly identified sources of information; and statement(s) of differences regarding significant unresolved difference (if any) of opinion reported by either ET members or E3/PCM or the implementer(s) of INVEST. The ER will be written in English and submitted in electronic form readable in MS Word 2010 based on MS Word Times New Roman 12 or other legible font of similar size. Any data used to prepare those reports (except for the data protected by any formal agreements between the Contractor and interviewees and survey/focus group participants) will be presented in the MS Office compatible format suitable for re-analysis and submitted either by email or on a CD or a flash drive to the COR. The data should be fully documented and well organized for use by those not fully familiar with the evaluated activities or the evaluations. All quantitative data collected by the ET must be provided in machine-readable, non-proprietary formats at www.usaid.gov/data as required by USAID’s Open Data policy, at www.usaid.gov/data (see ADS 579). The data should be organized and fully documented for use by those not fully familiar with the project or the evaluation. USAID will retain ownership of the survey and all datasets developed, copies of which are provided to the COR. The ET will present their major evaluation findings and preliminary conclusions in either MS PowerPoint or Google Slides format at a briefing for E3/PCM and INVEST stakeholders. Draft ER will be due 15 working days after a corresponding briefing for E3/PCM. The draft ER must include all relevant ET findings and conclusions made in conjunction with the evaluation of INVEST, as well as preliminary ET recommendations. The draft ER shall be prepared in line with general requirements (clarity, credibility, length, font size, etc.) set for the final ER. It may include the feedback received from E3/PCM and INVEST stakeholders at the briefing(s). E3/PCM will have 15 working days to review the draft ER and provide comments to the Contractor. E3/PCM will decide whether INVEST stakeholders will be invited to comment on a draft. The final ER will be due ten working days following the receipt of E3/PCM’s comments on a draft ER. The Contractor will use either a cover memorandum or similar format to explain how comments provided by E3/PCM and INVEST stakeholders (when solicited) were addressed in the final ER if the final ER differs substantially from the draft one. Both E3/PCM and the Contractor will have a right to initiate an extension of the ER review or preparation/completion time for up to ten working days at no additional cost. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 47 Attachment 1: Evaluation Report Outline Template EXECUTIVE SUMMARY Evaluation purpose and questions Background and context Evaluation methods and limitations Evaluation findings Evaluation conclusions Recommendations for future programming Lessons learned (if applicable) 1.0 EVALUATION PURPOSE & QUESTIONS 2.0 EVALUATION TEAM MEMBERS 3.0 PROJECT BACKGROUND AND CONTEXT 4.0 EVALUATION METHODS AND LIMITATIONS 5.0 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 5.1 RELEVANCE OF INVEST ACTIVITIES 5.1.1 Findings 5.1.2 Conclusions 5.1.3 Recommendations 5.2 EFFECTIVENESS OF INVEST ACTIVITIES 5.2.1 Findings 5.2.2 Conclusions 5.2.3 Recommendations 5.3 EVALUATION QUESTION 1 5.3.1 Findings 5.3.2 Conclusions 5.3.3 Recommendations 5.4 EVALUATION QUESTION 2 5.4.1 Findings 5.4.2 Conclusions 5.4.3 Recommendations 5.5 EVALUATION QUESTION 3 5.5.1 Findings 5.5.2 Conclusions 5.5.3 Recommendations MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 48 5.6 EVALUATION QUESTION 4 5.6.1 Findings 5.6.2 Conclusions 5.6.3 Recommendations 5.7 EVALUATION QUESTION 5 5.7.1 Findings 5.7.2 Conclusions 5.7.3 Recommendations 6.0 LESSONS LEARNED ANNEXES Annex A: Evaluation Statement of Work Annex B: Description of the Evaluation Team and Member Qualifications Annex C: Conflict of Interest Statements Annex D: Final Evaluation Work Plan Annex E: List of Documents Reviewed Annex F: Lists of Key Informants, Focus Group Discussants (if applicable), and Survey Respondents (if applicable) Annex G: Data Collection Tools Annex H: Focus Group Summaries (if applicable) Annex I: Survey Results (if applicable) Annex K: Table of Evaluation Findings, Conclusions, and Recommendations Annex L: MS PowerPoint (or Google Slides) based Presentation of Evaluation Design, Findings, Conclusions, and Recommendations MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 49 ANNEX B: INVEST BUY-INS FY17-19 • INVEST Core Funding • Power Africa Kenya • Afghanistan Trade/Investment • PIVOT Training • Strategic Corp Eng. BFS • WE4F Landscape • ASHA DRC • GenDevWEE • FinGap Legacy • GlobalHealth Roadmap • Haiti PPP • Haiti INVEST • India 3.0 Innovative • Moldova Investment • Prosper Africa – Southern Africa Regional Mission • Prosper Africa- East Africa Regional Mission • SCE Tunisia • Power Africa Uganda • Vietnam INVEST • Prosper Africa - Ex.Sec • Last Mile Connectivity • South Africa CDCS • WomensGlobal-WGDP • Innovative Finance • Serbia Media Fund • Pension Fund Support • Tibetan Banking Support • Liberia Telecom • ASHA Strategic Framework • Haiti cholera • ITAN Asia Bureau • USAID AFR SD/EGEA MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 50 ANNEX C: GETTING TO ANSWERS MATRIX Evaluation Question Data Sources Sample Size Data Collection Instruments Analysis Methods 1. How has INVEST operationalized the mobilization of private capital approach through its network of participating firms? • KIIs • INVEST documentation • Quantitative analysis of network data • In-depth case studies • 64 KIIs • Analysis of all available partner network data • Respondent -specific interview guide • Thematic/content analysis • Descriptive quantitative analysis 2. How has INVEST captured and validated lessons learned? • KIIs • INVEST documentation • Online survey of INVEST partner firms • 64 KIIs • Online survey of partner firms • Respondent -specific interview guide • Online survey • Thematic/content analysis • Descriptive quantitative analysis 3. How has INVEST communicated to USAID and other stakeholders about its mobilization of private capital approach? • KIIs • INVEST documentation • Process mapping • 64 KIIs • Respondent -specific interview guide • Thematic/content analysis • Process analysis MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 51 ANNEX D: ANONYMIZED RESPONDENT LIST The table below provides an anonymized list of the respondents the evaluation team interviewed. Stakeholder Group Associated Buy-in Status Month of Interview USAID Complete 04-2020 USAID Complete 04-2020 USAID Ongoing 04-2020 USAID Complete 04-2020 USAID Complete 04-2020 USAID Ongoing 04-2020 USAID Ongoing 04-2020 USAID Ongoing 04-2020 USAID Ongoing 04-2020 USAID Ongoing 04-2020 USAID Ongoing 04-2020 USAID Ongoing 05-2020 USAID Ongoing 05-2020 USAID Ongoing 05-2020 USAID Ongoing 05-2020 USAID Ongoing 05-2020 USAID Ongoing 05-2020 USAID Ongoing 05-2020 USAID Ongoing 05-2020 USAID Ongoing 05-2020 USAID Ongoing 05-2020 USAID Ongoing 05-2020 USAID Ongoing 07-2020 Partner Firm Completed 06-2020 Partner Firm Ongoing 06-2020 Partner Firm Ongoing 06-2020 Partner Firm Ongoing 06-2020 Partner Firm Completed 06-2929 Partner Firm Ongoing 06-2020 Partner Firm NA 06-2020 Partner Firm Ongoing 06-2020 Partner Firm Completed 06-2020 Partner Firm NA 06-2020 Partner Firm NA 06-2020 Partner Firm NA 06-2020 Partner Firm NA 06-2020 Partner Firm NA 06-2020 Partner Firm NA 06-2020 Partner Firm Ongoing 06-2020 Partner Firm Completed 06-2020 Partner Firm Ongoing 08-2020 USAID NA 05-2020 USAID Ongoing 07-2020 USAID Ongoing 07-2020 MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 52 Stakeholder Group Associated Buy-in Status Month of Interview USAID Ongoing 07-2020 USAID NA 07-2020 USAID Ongoing 07-2020 Partner Firm NA 07-2020 USAID NA 08-2020 USAID Ongoing 08-2020 USAID Ongoing 05-2020 DAI NA 06-2020 DAI NA 06-2020 DAI NA 07-2020 DAI NA 07-2020 DAI NA 07-2020 DAI NA 07-2020 DAI NA 07-2020 DAI NA 07-2020 DAI NA 07-2020 DAI NA 07-2020 DAI NA 07-2020 DAI NA 07-2020 DAI NA 07-2020 DAI NA 05-2020 MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 53 ANNEX E: KEY INFORMANT INTERVIEW GUIDES USAID/E3/PCM INTERVIEW GUIDE Name of Interviewer: Date of Interview: Location of Interview: Start Time of Interview: End Time of Interview: Organization name: Respondent’s organizational title: CONSENT 1. Introduction Hello, my name is __________________ and I work for MSI (Management Systems International). 2. Purpose MSI has been contracted by USAID to conduct an independent, third-party evaluation of the INVEST activity. The goal of the evaluation is to assess INVEST’s work mobilizing private capital, capturing and implementing lessons learned, and communicating with USAID staff, partners, and other interested parties. We are conducting interviews to better understand INVEST’s activities over the past two years, your specific experience, and perspective. Your help in this endeavor is greatly appreciated. 3. Interview Process Today I will ask you some questions about your experiences and views. The interview should take us about 45 minutes to an hour 4. CONSENT Before we start the interview, I would like to ask for your formal consent, including the following: - You have the right to not participate, this is completely voluntary - You have the right to stop at any point - If there is anything you do not understand, please ask me to clarify - You have the right to decline to answer any question - The information you provide is only for our evaluation and your responses will be anonymous. Neither you nor your organization will be named directly in our report or in any information we share with USAID, INVEST, or other stakeholders. We will not use your name or specific title should we choose to quote something you say. CONSENT CONSENT TO RECORD Do we have your consent for an interview? ☐ YES ☐ NO We would like to record this interview to ensure we accurately capture your responses. We will not share this recording with anyone outside of the evaluation team. We will not share this recording with USAID, INVEST, or anyone else related to your firm or this activity. Do we have your consent to RECORD? ☐ YES ☐ NO MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 54 General questions about respondent 1. Can you please provide your name and title? 2. How long have you been with this organization? EQ1 Questions 3. Have you heard of the INVEST mechanism? a. What do you think are the INVEST mechanism objectives? What is your understanding of the INVEST mechanism? b. How has INVEST mobilized private capital? (If respondent is not familiar enough to answer, ask whether the respondent has had any interaction with representatives of the INVEST mechanism and what that interaction is.) c. What is your background in terms of blended finance? What do you think the main barriers are to blended finance for USAID? i. Has INVEST addressed any of these barriers? ii. How did this occur? iii. What was the outcome? 4. Have you heard of the INVEST partner network? What is your understanding of the INVEST partner network? a. What else do you know about this network? What are some of its positive features? Any negative features? b. Do you feel that the partners have helped facilitated private capital to achieve development outcomes? Which partner(s) did the facilitation and what, specifically, did the partner do? What was the outcome of this effort? i. Do you think these partners have worked in ways that differ from traditional USAID partners? If so, how? c. When did you become aware of the partner network? Have you noticed any changes (either positive or negative) since you first became aware of the network? 5. Thinking about your own experience, what aspects of the INVEST’s private capital mobilization have worked well in your opinion? What aspects have proved challenging? 6. Have you been involved in procurements relating to the INVEST mechanism? If so, what was your experience of the procurement process through INVEST? a. Did this process differ from other procurements? If so, how? b. What observations do you have about INVEST’s and the procurement process? c. What are the strengths of the INVEST procurement process compared to other procurement processes? What weaknesses have you observed? 1. How has INVEST operationalized the mobilization of private capital approach through its network of participating firms? a. How has INVEST facilitated the procurement process for Mission and Operating Unit buy￾ins to the activity? b. How has the network of participating firms under INVEST expanded and changed over time? MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 55 EQ2 Questions 7. How much engagement with INVEST documents or the website have you had? a. Which specific learning or research documents or resources have you used? How did you use them? Were they helpful? If so, how did the information help you? Did the information lead to any behavior change on your part? On the part of anyone else? 8. Did you find the learning documents/research to be accurate/relevant/useful to your own efforts? a. What, if anything, could have made the materials more accurate/relevant/useful? 9. Do you think INVEST’s learning and research differs from other activities’ monitoring, evaluation, and learning? If so, how? If not, why not? a. Has INVEST’s published documentation changed your perspective on private capital and blended finance? If so, how? 10. Have you participated in any INVEST meetings, debriefs, or reflection sessions? a. What was your impression of these sessions? If you provided input to one or more of the session, was your input appropriate reflected into INVEST documentation? 11. Have you perceived any difference in how operating units or missions handle procurement since working with INVEST? a. If so, how? If not, why not? 12. Have you perceived any difference in how operating units or missions handle/think about private capital since working with INVEST? a. If so, how? If not, why not? 13. What do you see as the primary benefit of working with INVEST? a. What INVEST services have you utilized or benefited from? EQ3 Questions 14. How did you first hear about INVEST? a. Can you describe how INVEST was explained to you? i. Who did you talk to? ii. How did you get this information? b. How many people at INVEST have you talked to or met with? c. Are you aware of other offices, or even other donors, hearing about INVEST? Do you know how they have learned about it? 15. How would you describe the type and nature of the communication you have seen from INVEST? 16. We would like to get an idea of INVEST’s communication processes overtime. Can you walk me through the communication process, whether it was in person or online, who you talked to and when? 17. Has your office modified its approach to private capital since INVEST started? a. If so, how? If not, why not? 2. How has INVEST captured and validated lessons learned? a. How has INVEST acted upon captured learning and research? b. How have Missions, Operating Units and network member firms benefited from INVEST learning and research? 3. How has INVEST communicated to USAID and other stakeholders about its mobilization of private capital approach? a. How have other donors, host country counterparts, targeted beneficiaries and others learned about INVEST’s mobilization of private capital approach? b. How have Missions and Operating Units modified their programmatic/strategic efforts to integrated mobilization of private capital models, particularly the level of buy-ins, the number of follow-on buy-ins and future expressions of interest? MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 56 18. How have the types of activities your office supports changed since INVEST started? a. If they haven’t changed, why not? b. If they have changed, can you explain a bit more about INVEST’s role in informing these changes, whether directly or indirectly? 19. What do you see as the biggest challenges in understanding and communicating issues around private capital mobilization? 20. We have talked a lot about the details of INVEST’s outreach, but stepping back, what do you think of the overall INVEST communication strategy? a. What could be improved? What worked well? 21. Yes/No: Have you accessed the INVEST website? What did you find most helpful about the website? 22. Yes/No: Have you read INVEST communication materials, such as blog posts? What stands out in your mind about the blog posts? Are you a member of the LinkedIn group where the blog posts are located? Have you contributed a blog post about your experience with INVEST? What else do we need to know to fully understand your interactions with the INVEST mechanism? What should we have asked, but didn’t? Thank you for speaking with us today. If there are any documents or other materials that you feel will help us understand your responses about the INVEST mechanism, we would appreciate receiving them via email. Additionally, if you think of additional information that you feel would help us to better understand your perspective on the INVEST mechanism, or if you realize, upon reflection, that any of your answers requires additional context or explanation, please feel free to contact us at _________. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 57 NETWORK PARTNER INTERVIEW GUIDE Name of Interviewer: Date of Interview: Location of Interview: Start Time of Interview: End Time of Interview: Organization name: Respondent’s organizational title: CONSENT ● Introduction Hello, my name is __________________ and I work for MSI (Management Systems International). ● Purpose MSI has been contracted by USAID to conduct an independent, third-party evaluation of the INVEST activity. The goal of the evaluation is to assess INVEST’s work mobilizing private capital, capturing and implementing lessons learned, and communicating with USAID staff, partners, and other interested parties. We are conducting interviews to better understand INVEST’s activities over the past two years, your specific experience, and perspective. Your help in this endeavor is greatly appreciated. ● Interview Process Today I will ask you some questions about your experiences and views. The interview should take us about 45 minutes to an hour ● CONSENT Before we start the interview, I would like to ask for your formal consent, including the following: - You have the right to not participate, this is completely voluntary - You have the right to stop at any point - If there is anything you do not understand, please ask me to clarify - You have the right to decline to answer any question - The information you provide is only for our evaluation and your responses will be anonymous. Neither you nor your organization will be named directly in our report or in any information we share with USAID, INVEST, or other stakeholders. We will not use your name or specific title should we choose to quote something you say. CONSENT CONSENT TO RECORD Do we have your consent for an interview? ☐ YES ☐ NO We would like to record this interview to ensure we accurately capture your responses. We will not share this recording with anyone outside of the evaluation team. We will not share this recording with USAID, INVEST, or anyone else related to your firm or this activity. Do we have your consent to RECORD? ☐ YES ☐ NO MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 58 General questions about respondent 1. Can you please provide your name and title? 2. How long have you been with this organization? EQ1 Questions 3. What was your understanding of INVEST and the partner network when you joined? a. How did you join the network? b. Has this understanding changed since joining? c. Are you familiar with other firms in the network? i. Have you told peers at other firms about INVEST? 4. Can you explain the buy-in that you worked on through the INVEST activity? What was the goal of the buy-in? a. Had you or someone within your organization attempted to work with USAID prior to working with INVEST? If so, what happened? If not, why not? i. Clarify: was this your first time working with USAID? With USG? b. Thanks, can you walk me through the process of how this buy-in happened? c. What was INVEST’s role in this process? d. Overall, what was your impression of this process? i. What were the challenges to working with INVEST? 5. To what extent was private capital leveraged as part of this buy-in? a. What was INVEST’s role in facilitating private capital? b. What was role of private capital in the activity? 6. Can you describe the relationship between you, INVEST, and the USAID mission or operating unit who worked on this activity? a. How does this compare to other work your firm typically does? 7. What was your experience of the procurement process through INVEST? a. What is your understanding of the review process? b. What did you think about INVEST’s support or facilitation during the procurement process? c. Are their strengths to this process compared to other projects you have worked on? Are there weaknesses? d. What would you have done differently as part of the buy-in? 8. Are you working with USAID through INVEST or other mechanisms on other projects since working with INVEST? EQ2 Questions 9. How much engagement with INVEST documents or the website did you have before your project? a. What about during and after? 1. How has INVEST operationalized the mobilization of private capital approach through its network of participating firms? a. How has INVEST facilitated the procurement process for Mission and Operating Unit buy￾ins to the activity? b. How has the network of participating firms under INVEST expanded and changed over time? 2. How has INVEST captured and validated lessons learned? a. How has INVEST acted upon captured learning and research? b. How have Missions, Operating Units and network member firms benefited from INVEST learning and research? MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 59 b. Can you describe what learning or research documents or resources you used? How did you use them? Why were they helpful? 10. Can you describe the engagement from INVEST after procurement? After implementation? a. Was there any internal debrief or reflection within your team about the process? b. Was there any external debrief or reflection with USAID? With INVEST? i. If so, what was your impression of INVEST’s follow-up after you responded to a potential buy-in opportunity? 11. Do you have plans to work with USAID again? a. If so, do you feel that the information that INVEST shared was helpful for understanding this client? How so? What specifically was helpful? b. If not, why not? 12. What resources have you and your team used to inform navigating the USAID process? a. Has this changed since working with INVEST? 13. What do you see as the benefit of working with INVEST? a. What INVEST services or resources unrelated to the buy-in have you utilized? i. How did you use these resources? How did they affect your firm’s processes or activities? EQ3 Questions 14. How did you hear about INVEST? a. Can you describe how INVEST was explained to you? i. Who did you talk to? ii. How did you get this information? b. How many people at INVEST did you talk to or meet with? 15. Can you walk me through the communication process, who you talked to and when, before, during, and after your engagement with INVEST? a. Can you describe the communication you received from INVEST after joining the partner network? b. Was this communication helpful for understanding how to work with USAID? c. Was this communication helpful for understanding how your firm could help achieve development outcomes? 16. How would you describe INVEST and USAID’s understanding of private capital? a. How prominent was capital mobilization in the communication and outreach? If it was not prominent, what were the main themes of the communication you received? 17. What do you see as the biggest challenges in understanding and communicating issues around private capital mobilization? 18. We have talked a lot about the detailed of INVEST’s outreach, but stepping back, what do you think of the overall INVEST communication strategy? a. What could be improved? What worked well? 19. Yes/No: Have you accessed the INVEST website? 20. Yes/No: Have you read INVEST communication materials, such as blog posts? 3. How has INVEST communicated to USAID and other stakeholders about its mobilization of private capital approach? a. How have other donors, host country counterparts, targeted beneficiaries and others learned about INVEST’s mobilization of private capital approach? b. How have Missions and Operating Units modified their programmatic/strategic efforts to integrated mobilization of private capital models, particularly the level of buy-ins, the number of follow-on buy-ins and future expressions of interest? MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 60 INVEST STAFF INTERVIEW GUIDE Name of Interviewer: Date of Interview: Location of Interview: Start Time of Interview: End Time of Interview: Organization name: Respondent’s organizational title: CONSENT ● Introduction Hello, my name is __________________ and I work for MSI (Management Systems International). ● Purpose MSI has been contracted by USAID to conduct an independent, third-party evaluation of the INVEST activity. The goal of the evaluation is to assess INVEST’s work mobilizing private capital, capturing and implementing lessons learned, and communicating with USAID staff, partners, and other interested parties. We are conducting interviews to better understand INVEST’s activities over the past two years, your specific experience, and perspective. Your help in this endeavor is greatly appreciated. ● Interview Process Today I will ask you some questions about your experiences and views. The interview should take us about 45 minutes to an hour ● CONSENT Before we start the interview, I would like to ask for your formal consent, including the following: - You have the right to not participate, this is completely voluntary - You have the right to stop at any point - If there is anything you do not understand, please ask me to clarify - You have the right to decline to answer any question - We will not use your name or specific title should we choose to quote something you say. It is helpful for us to be able to describe how DAI officials view the areas of focus for our evaluation. However, if there is anything that you wish to tell us that you would like to have kept anonymous, please tell us and we will honor your request. CONSENT CONSENT TO RECORD Do we have your consent for an interview? ☐ YES ☐ NO We would like to record this interview to ensure we accurately capture your responses. We will not share this recording with anyone outside of the evaluation team. We will not share this recording with USAID, INVEST, or anyone else related to your firm or this activity. Do we have your consent to RECORD? ☐ YES ☐ NO MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 61 General questions about respondent 1. Can you please provide your name and title? 2. How long have you been with INVEST? EQ1 Questions 3. I want to start a high-level and then discuss more details of the INVEST mechanism. a. What do you think are the INVEST mechanism objectives? b. Can you walk me through how INVEST mobilizes private capital? c. Based on your own observations, what proportion of the partner network firms are providing private capital as part of INVEST? i. Of those that are providing private capital, what proportion have worked previously with USAID? ii. Which network partners, if any, are currently providing private capital AND have not previously worked with USAID? iii. What was it about INVEST that made the difference – that resulted in the decision of the partner to provide private capital as part of INVEST? 1. Where might this information have been captured/documented in INVEST records? 4. Can you walk me through INVEST’s interactions with missions and operating units during the buy-in procurement process? a. What are some of the positive features of this process? Any negative features? 5. How long have you been involved with the partner network? a. Have you noticed any changes (either positive or negative) since you became involved? b. How has the partner network changed over INVEST’s first two years? 6. Can you describe the relationship between INVEST, USAID, and the INVEST network partners? 7. Have the buy-ins INVEST has facilitated changed over the activity’s first two years? a. If so, what do you think has driven this change? b. Are there any changes that need to occur/should occur and why? 8. What challenges have you faced in mobilizing private capital through the partner network? a. For buy-ins that did not involve private capital mobilization, what challenges have you faced? 9. How do the missions and operating units INVEST works with view the role of private capital? 10. Overall, how do you feel INVEST has balanced its three priorities of market assessment, structuring funds, and facilitating transaction services? a. Do you think INVEST could have mobilized more private capital over its first two years? b. How much of INVEST’s role is assisting with compliance and navigating USAID’s procurement system? 11. What role, if any, does INVEST have in managing or curating the partner network? a. What specific activities are involved in this process? b. How do you manage or vet out-of-network partners? 12. With regard to the partner network and private capital mobilization, in what areas do you feel INVEST has achieved its goals? a. In what areas do you feel there were challenges in the first two years? b. What are the most significant risks to accomplishment of INVEST mechanism goals? 1. How has INVEST operationalized the mobilization of private capital approach through its network of participating firms? a. How has INVEST facilitated the procurement process for Mission and Operating Unit buy￾ins to the activity? b. How has the network of participating firms under INVEST expanded and changed over time? MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 62 EQ2 Questions 13. Following up on that last question, I would like to know more about how INVEST captured lessons learned. Can you explain the internal process for debriefing or review of buy-ins and partner network management? a. How does this vary depending on the context? b. What formats has this taken? Who was involved? 14. What changes have occurred over the past two years as a result of this “lessons learned” process? 15. How much engagement have you had with USAID on capturing lessons learned? What about with firms? a. What format has this taken? b. How did this go overall? 16. Do you feel that the lessons learned have been absorbed, communicated and acted upon within INVEST? a. What about among USAID missions and operating units? b. What about among partner firms? 17. To what extent are INVEST’s learning and research products supply or demand driven? a. Do you know what learning or research documents or resources have been used and by whom? Which, specifically, have received the most feedback? i. How were each used? How do you know? ii. What was the nature of the feedback? iii. Did the information lead to any behavior change on your part? On the part of network partners? On the part of USAID? iv. Was the captured learning and/or research accurate/relevant/useful to your own efforts? To the efforts of network partners? To USAID? 18. What resources have you and your team used to inform process management? a. Has this changed over the past two years? If so, how? 19. What challenges do you feel INVEST has encountered in capturing and validating lessons learned? 20. What opportunities have there been over the past two years for capturing and validating lessons learned? 21. What quality control processes or peer review processes, if any, are in place to ensure the integrity of published INVEST data? EQ3 Questions 2. How has INVEST captured and validated lessons learned? a. How has INVEST acted upon captured learning and research? b. How have Missions, Operating Units and network member firms benefited from INVEST learning and research? 3. How has INVEST communicated to USAID and other stakeholders about its mobilization of private capital approach? a. How have other donors, host country counterparts, targeted beneficiaries and others learned about INVEST’s mobilization of private capital approach? b. How have Missions and Operating Units modified their programmatic/strategic efforts to integrated mobilization of private capital models, particularly the level of buy-ins, the number of follow-on buy-ins and future expressions of interest? MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 63 22. Please walk me through the communication process for outreach regarding the INVEST mechanism, such as who makes initial contact, who is involved, how outreach takes place, and the timeframe? 23. Please walk me through the communication process with partner network firms? a. How much engagement does INVEST have with its partner network after firms have joined? 24. What are the biggest challenges in communicating and explaining the role of INVEST? a. What do you see as the biggest challenges in understanding and communicating issues around private capital mobilization? 25. What type of follow-up does INVEST conduct after engagement with missions and operating units? With partner firms? 26. Have you seen changes in how missions and operating units integrate private capital? a. If not, why not? b. If so, what aspects of the INVEST mechanism seem to have contributed to these changes? c. What aspects of broader USAID policy have guided these changes? 27. How much interest have other USAID implementing partners or other donors had in INVEST? 28. We have talked a lot about the detailed of INVEST’s outreach, but stepping back, what is the overall purpose of the INVEST communication strategy? a. What do you think of the overall INVEST communication strategy and why? b. What could be improved? What worked well? MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 64 BUY-IN CLIENT INTERVIEW GUIDE Name of Interviewer: Date of Interview: Location of Interview: Start Time of Interview: End Time of Interview: Organization name: Respondent’s organizational title: CONSENT ● Introduction Hello, my name is __________________ and I work for MSI (Management Systems International). ● Purpose MSI has been contracted by USAID to conduct an independent, third￾party evaluation of the INVEST activity. The goal of the evaluation is to assess INVEST’s work mobilizing private capital, capturing and implementing lessons learned, and communicating with USAID staff, partners, and other interested parties. We are conducting interviews to better understand INVEST’s activities over the past two years, your specific experience, and perspective. Your help in this endeavor is greatly appreciated. ● Interview Process Today I will ask you some questions about your experiences and views. The interview should take us about 45 minutes to an hour ● CONSENT Before we start the interview, I would like to ask for your formal consent, including the following: - You have the right to not participate, this is completely voluntary - You have the right to stop at any point - If there is anything you do not understand, please ask me to clarify - You have the right to decline to answer any question - The information you provide is only for our evaluation and your responses will be anonymous. Neither you nor your organization will be named directly in our report or in any information we share with USAID, INVEST, or other stakeholders. We will not use your name or specific title should we choose to quote something you say. CONSENT CONSENT TO RECORD Do we have your consent for an interview? ☐ YES ☐ NO We would like to record this interview to ensure we accurately capture your responses. We will not share this recording with anyone outside of the evaluation team. We will not share this recording with USAID, INVEST, or anyone else related to your firm or this activity. Do we have your consent to RECORD? ☐ YES ☐ NO MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 65 General questions about respondent 1. Can you please provide your name and title? 2. How long have you been with this organization? EQ1 Questions 3. Can you explain the buy-in that you worked on through the INVEST activity? What was the goal of the buy-in? a. Had you or someone within your organization attempted to fund the buy-in activity prior to working with INVEST? If so, what happened? If not, why not? b. Thanks, can you walk me through the process of how this buy-in happened? c. What was INVEST’s role in this process? d. Overall, what was your impression of this process? 4. To what extent was private capital leveraged as part of this buy-in? a. What was INVEST’s role in facilitating private capital? b. What was role of private capital in the activity? 5. Can you describe the relationship between you, INVEST, and the INVEST partner who worked on this activity? a. How does this compare to other activities your office has managed? 6. What was your experience of the procurement process through INVEST? a. How did this differ from other procurements with which you have been involved? b. What did you think about INVEST’s support or facilitation during the procurement process? c. Are their strengths to this process compared to other procurement processes? Are there weaknesses? d. What would you have done differently as part of the buy-in? 7. What is your understanding of the INVEST partner network? a. Have you participated on an INVEST TEC Panel? If so, can you describe the process? i. How were firms chosen? ii. What strengths and limitations did you feel were part of this process? EQ2 Questions 8. How much engagement with INVEST documents or the website did you have before your buy-in? a. What about during and after? b. Can you describe what learning or research documents or resources you used? How did you use them? Why were they helpful? 9. Can you describe the engagement from INVEST after procurement? After implementation? a. Was there any internal debrief or reflection within your team about the process? b. Was there any external debrief or reflection with other USAID offices? With INVEST? 10. Have you or your team handled procurement processes differently since working with INVEST? 1. How has INVEST operationalized the mobilization of private capital approach through its network of participating firms? a. How has INVEST facilitated the procurement process for Mission and Operating Unit buy￾ins to the activity? b. How has the network of participating firms under INVEST expanded and changed over time? 2. How has INVEST captured and validated lessons learned? a. How has INVEST acted upon captured learning and research? b. How have Missions, Operating Units and network member firms benefited from INVEST learning and research? MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 66 a. If so, how? If not, why not? 11. What resources have you and your team used to inform process management? What about activity design and procurement? a. Has this changed over the past two years? b. Has this changed since your engagement with INVEST? 12. What do you see as the benefit of working with INVEST? a. What INVEST services unrelated to the buy-in have you utilized or benefited from? EQ3 Questions 13. How did you hear about INVEST? a. Can you describe how INVEST was explained to you? i. Who did you talk to? ii. How did you get this information? b. How many people at INVEST did you talk to or meet with? c. How would you describe your understanding of INVEST prior to starting the buy-in process? 14. Can you walk me through the communication process, who you talked to and when, before, during, and after your engagement with INVEST? 15. Has your office or mission modified its approach to private capital since working with INVEST? a. If so, how? If not, why not? 16. How have the types of activities your office or missions supports changed since working with INVEST? a. If they haven’t changed, why not? b. If they have changed, can you explain a bit more about INVEST’s role in informing these changes, whether directly or indirectly? 17. Have you or your office/mission considered follow-on buy-ins through INVEST? a. If so, what is driving this decision? b. If not, why not? c. What pieces of INVEST do you feel you would like to adapt into your current processes? What pieces would you leave out? 18. What do you see as the biggest challenges in understanding and communicating issues around private capital mobilization? 19. We have talked a lot about the detailed of INVEST’s outreach, but stepping back, what do you think of the overall INVEST communication strategy? a. What could be improved? What worked well? 20. Yes/No: Have you accessed the INVEST website? 21. Yes/No: Have you read INVEST communication materials, such as the blog 3. How has INVEST communicated to USAID and other stakeholders about its mobilization of private capital approach? a. How have other donors, host country counterparts, targeted beneficiaries and others learned about INVEST’s mobilization of private capital approach? b. How have Missions and Operating Units modified their programmatic/strategic efforts to integrated mobilization of private capital models, particularly the level of buy-ins, the number of follow-on buy-ins and future expressions of interest? MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 67 ANNEX F: DOCUMENTS REVIEWED Annual and Quarterly Reports ● INVEST Annual Report – 2018 – DEC ● INVEST Annual Report_FY1_External ● INVEST Concept Note – External Version 10_31_17 ● INVEST Quarterly Report Y2Q3_July_30_2019External ● INVEST Quarterly Report_03_EXTERNAL ● INVEST Quarterly Report_External_FY2Q2_Apr2019 ● INVEST Quarterly Report_FY1Q1_External ● INVEST Quarterly Report_FY1Q1_Jan2018_EXTERNAL ● INVEST Quarterly Report_FY1Q2_External ● INVEST Quarterly Report_FY1Q3_External ● INVEST Quarterly Report_FY2Q1_External_Jan2019 ● INVEST Quarterly Report_FY2Q1_Jan2019 (External) ● INVEST Quarterly Report_FY2Q2_External ● INVEST Quarterly Report_FY2Q3_External ● INVEST Quarterly Report_FY3Q1_Jan2020_External ● INVEST Y2 Annual Report with Financials_Final ● INVEST Y2 Annual Report_External INVEST Buy-In SOWs USAID Africa Bureau ● INVEST Africa Bureau - Blended Finance Expertise Incubator - Final Draft USAID CCD (DCCP) ● INVEST Buy in SOW CDD DCCP FINAL-Clean USAID CDI ● WE4F Buy-In SOW INVEST FINAL_CLEAN USAID DCHA ASHA ● INVEST Final Buy in SOW DCHA ASHA_CLEAN USAID DRC ● DRC - Buy-In Scope of Work INVEST - FINAL Clean USAID Gen Dev ● WE3 Finance Statement of Work USAID Ghana ● Feasibility Analysis for FinGAP Legacy Vehicle SOW USAID Global Health ● INVEST FINAL Buy in SOW_CII_Global Health Clean USAID Global Health-Haiti ● INVEST Buy in SOW_CII_Haiti 6.8.18_INVEST FINAL Clean USAID Haiti IP ● Haiti INVEST Buy-in Scope of Work 09 14 2018 USAID Haiti PPP ● FINAL Haiti PPP #2 Buy-in SOW_INVEST – CLEAN ● Haiti CUP Transaction Support Buy-in SOW_INVEST_FINAL - CLEAN USAID India 3.0 ● Buy-in SOW for USAID India 3.0_FINAL Clean USAID Kenya-Power Africa MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 68 ● Power Africa Buy-In SOW FINAL Clean USAID Liberia ● INVEST Buy-in SOW Liberia FINAL_CLEAN USAID Moldova ● Invest Moldova Buy-In SOW_CLEAN FINAL USAID Prosper Africa ● INVEST SOW for Prosper Africa FINAL USAID Serbia ● Buy-In SOW Media Investment Fund - INVEST SERBIA - FINAL_CLEAN USAID South Africa ● FINAL INVEST SA CDCS Buy-in SOW USAID Strategic Corporate Engagement ● Initial INVEST Corporate Investment Buy-In SOW_FINAL - CLEAN USAID Uganda ● INVEST Uganda Buy-In SOW FINAL Clean PIVOT MADAGASCAR TDY REPORT_FINAL INVEST Buy-Ins ● Buy-In Procurement ● SOO-RFP INVEST-019 WE4F GCD Landscape Analysis East Africa_FINAL ● Statement of Objectives - RFP INVEST-004 ● Buy-In Reporting Documents ● Haiti INVEST Semi Annual Report 042919 Final ● INVEST Afghanistan May 2019 Monthly Report.FINAL.combined ● Buy-in SOW for USAID India 3.0_FINAL ● Haiti INVEST Semi Annual Report 042919 Final ● INVEST Afghanistan May 2019 Monthly Report.FINAL.combined ● INVEST Buy in SOW_CII_Haiti 6.8.18_INVESTeditsv2_vDave ● INVEST FINAL Buy in SOW_CII_Global Health Clean ● INVEST Vietnam Buy-in SOW FINAL_CLEAN ● INVEST_Buy In List_7_31_19 ● INVEST_Buy In List_12_11_19 ● Power Africa Buy-In SOW FINAL Clean ● SOO - RFP INVEST-003 USAID India 3.0 ● SOO - RFP INVEST-038 PAES Roadmap_Mod 2 ● SOO INVEST-002 Power Africa USAID Kenya ● SOO-RFP INVEST-019 WE4F GCD Landscape Analysis East Africa_FINAL ● Statement of Objectives - RFP INVEST-004 ● WE4F Buy-In SOW INVEST FINAL_CLEAN ● INVEST RFPs_SOOs ● USAID Afghanistan ● INVEST RFQ 002 – Event Firm, Dubai ● RFP INVEST-012 – Energy and Extractives Transaction Advisory Support_FINAL ● FRP INVEST-020-Mining and Extractives Central Asia Mining Event Mod 1 ● RFP INVEST-023 FINAL ● SOO RFP INVEST-023 FINAL ● Statement of Objectives (SOO) RFP INVEST-012_FINAL ● USAID Africa Bureau ● INVEST RFP_Introductory Webinar on Blended Finance for PIVOT Cohort Champions MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 69 ● SOO_PIVOT Blended Finance Webinar ● USAID CDI ● RFP INVEST-019 West Africa Landscape – Mod 1 ● RFP INVEST-022 Asia Landscape ● SOO – RFP INVEST-019 WE4f GCD Landscape Analysis East Africa_FINAL ● SOO – RFP INVEST-019 WE4f GCD Landscape Analysis West Africa_FINAL ● SOO – RFP INVEST-019 WE4f GCD Asia Landscape Analysis ● USAID DCCP ● RFP INVEST-028 Last Mile Connectivity ● SOO – RFP-028 Last Mile Connectivity ● USAID DRC ● RFP INVEST-005 USAID DRC FINAL ● SOO – RFP INVEST-005 DRC FINAL ● USAID Gen Dev ● FINAL SOW GenDev – CB ● RFP INVEST – 013 – Fisheries Traceability – FINAL ● RFP INVEST – 013 – WEFMFT_Updated ● RFP INVEST-015_WWB ● SOO INVEST-015_WWB ● SOO RFP INVEST – 013 – Fisheries Traceability – FINAL ● Statement of Objectives – RFP INVEST-013 WEFMFT_Updated ● USAID Ghana ● Feasibility Analysis for FinGAP Legacy Vehicle SOW ● USAID Global Health ● INVEST Global Health RFP001 FINAL ● Statement of Objectives for RFP INVEST 001 FINAL ● USAID Global Health-Haiti ● RFP INVESt-004 GH-CII Haiti ● Statement of Objectives – RFP INVEST-004 ● USAID Haiti IP ● RFP INVEST-014 Haiti Diaspora Engagement ● RFP INVEST-016 Haiti INVEST Platform Ag VC Finance ● RFP INVEST-017 Haiti TA ● SOO – RFP INVEST-014 Haiti INVEST Platform Diaspora Engagement ● SOO Haiti INVEST Transaction Advisory Support ● SOO-RFP INVEST-016 Haiti INVEST Platform Ag VC Finance ● USAID Haiti PPP ● RFP INVEST-006 USAID Haiti ● RFP INVEST-0021 USAID Haiti Final_Mod 01 ● Statement of Objectives – RFP INVEST-006 USAID Haiti ● Statement of Objectives – RFP INVEST-021 USAID Haiti_Mod 01 ● USAID India 3.0 ● Amendment 002 – SOO – RFP INVEST-003 USAID India 3.0 FINAL ● Amendment 003 – RFP INVEST-003 USAID India 3.0 FINAL ● USAID Kenya-Power Africa ● RFP INVEST-002 Power Africa USAID Kenya ● SOO INVEST-002 Power Africa USAID Kenya ● USAID Moldova ● RFP INVEST-033_Moldova Assessment ● SOO-RFP INVEST-033_Moldova Assessment MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 70 ● USAID Pensions ● RFP INVEST-027 – Institutional Investment ● SOO – RFP INVEST-027 – Institutional Investment ● USAID Prosper Africa ● RFP INVEST 025-Prosper Africa Pipeline_TA FINAL ● RFP INVEST-027 Institutional Investment ● SOO – INVEST Prosper Africa Track One FINAL ● SOO – RFP INVEST-027 Institutional Investment ● SOO Appendix RFP INVEST 025-Prosper Africa Pipeline_TA FINAL ● SOO RFP INVEST 025-Prosper Africa Pipeline_TA FINAL ● USAID PSE Training ● SOO – RFP INVEST-007 PSE Learning Strategy_FINAL ● USAID South Africa ● RFP INVEST – 024 – South Africa CDCS Assessment – FINAL ● SOO INVEST – 024 – South Africa CDCS Assessment - FINAL ● USAID Strategic Corporate Engagement ● INVEST RFP 004 – Animation video_updated ● RFP INVEST-007 PSE Learning Strategy ● RFP INVEST-008 Corporate Venturing ● RFP INVEST-009 Supply and Value Chain ● RFP INVEST-010 Innovative Use of CSR ● SOO – RFP INVEST-007 PSE Learning Strategy ● Statement of Objectives – RFP INVEST-008 ● Statement of Objectives – RFP INVEST-009 ● Statement of Objectives – RFP INVEST-010 ● USAID Uganda ● INVEST SOO – USAID Uganda SOGE Follow-On ● RFP INVEST USAID Uganda SOGE Follow-on ● USAID W-GDP ● RFP INVEST-032 W-GDP Co-creation Facilitation Partner ● SOO INVEST-032 W-GDP Co-creation Facilitation Partner INVEST Learning and Communication Products ● INVEST-procurement.com ● USAID.gov/invest ● USAID INVEST Design Challenge Workshop_Event Summary_February 2019 ● Corporate_Investment_Partnerships_Executive_Summary ● Innovative-Finance-for-Development-A-Guide-for-International-NGOs ● INVEST Partners Blog ● INVEST WebLinks Nov2019 ● INVEST__PartnerList ● INVEST_AboutOurPartnerNetwork ● INVEST_FAQs ● INVEST_Mechanism_CloseUp_final ● INVEST_OnePager_AboutUs ● INVEST_Snapshot_PowerAfrica ● INVEST_Snapshot_WWB ● INVEST_Spotlight_Gender ● INVEST_Spotlight_Haiti MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 71 ● Learning Strategy_Summary 11_19_2019 ● Power_Africa_CloseUp_final ● USAID INVEST Design Challenge Workshop_Event Summary_February 2019 ● Womens_Economic_Empowerment_CloseUp_final INVEST MEL Documents ● Annex B_PIRS 2019 o 1.1 Number of Transactions o 1.2 Capital Leveraged o 1.3 Leverage ratio o 1.4 number inst. receiving support o 1.5 number of investees receiving support o 2.1 Number of buy-ins o 2.2 Value of buy-ins o 2.3 Number firms in p. network o 2.4 Number of subcontracts issued o 2.5 perc. value of P4P contracts o 2.6 perc. value of contracts to US_smallbus o 2.7 Number of proposals received o 2.8 Procurement Duration o 2.9 Satisfaction of buy-in clients o 2.10 Satisfaction of institutions o 2.11 Investee satisfaction o 3.1 Number of procurement changes o 3.2 Innovative finance solutions o 3.3 USAID adaptations o 3.4 Number of USAID staff trained o 3.5 Number USAID MBIOs receiving TA ● Transaction Close-Out Documents o Power Africa Kenya ▪ Azuri ● Azuri - Crossboundary Engagement Letter - Executed (1) ● Azuri Verification Letter_Redacted ● Transaction Close Report_Azuri_Redacted ▪ Green Light Planet ● 180912_GLP_OCA_Engagement_Executed (2) ● Email Technical Approval ● GLP Verification Letter_Redacted ● Transaction Close Report_GLP_Redacted ▪ PowerGen ● Email acceptance by USAID Kenya ● Power Gen Letter of Engagement Executed (1) ● PowerGen Verification Letter_Redacted ● Transaction Close Report_PowerGen_Redacted o Women_s World Banking ▪ INVEST FFP Subcontract WWB - RFP 015_PE ▪ INVEST GenDev - Transaction Close Template - WWB 6.1.20_redacted ▪ REDACTED WWBCP II Confirmation of First Close Investors_redacted ● Final MEL Plan_Annex B 7_15_2019 MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 72 ● Guideline INVEST Zoho System_Revised Oct ● INVEST Pause and Reflect Session Guide_Internal ● INVEST Pipeline Data_Reporting_PBi ● INVEST Pipeline Data_Updated May 2020 ● INVEST Transaction Data_Reporting_PBi ● MELP_Annex C_Guidelines INVEST_Zoho System_FINAL ● Pause and Reflect Session Guide_Internal ● USAID INVEST - MEL Plan v 1 31 2019 FINAL ● USAID INVEST - MEL Plan_7_15_2019 ● USAID INVEST - MEL Plan_7_15_2019_FINAL INVEST Partner Network ● INVEST Partner Network Members 7.9.19 ● INVEST_PartnerNetwork List Feb3 2020 ● INVEST_PartnerNetwork List Feb3 2020 ● MSI_PN_request_-_May_19_2020 ● Partner Network Survey ● PSE Hub Partner Network Survey Data 3_23_2020 ● RE First meeting on INVEST evaluation with MSI INVEST Quarterly Reports ● INVEST Quarterly Report_FY1Q2_EXTERNAL ● INVEST Quarterly Report_FY2Q1_Jan2019 (External) Southern Africa Regional Documents ● CB Task Order and Mods o DAI Task Order - Prosper Africa Track 1 FE o DAI Task Order - Prosper Africa Track 1 Mod 1 PE o DAI Task Order - Prosper Africa Track 1_Mod 2 ● CB Word Plan and Monthly Reports o 1. Prosper Africa Monthly Report (August 2019) o 2. Prosper Africa Monthly Report (September 2019) o 3. Prosper Africa Monthly Report (October 2019)1 o 4. Prosper Africa Monthly Report (November 2019) o 5. Prosper Africa Monthly Report (December 2019) o 6. Prosper Africa Monthly Report (January 2020) o 7. Prosper Africa Monthly Report (February 2020) o 8. Prosper Africa Monthly Report (March 2020) o 9. Prosper Africa Monthly Report April 2020_CB Revised o 10. Monthly Report May 2020 o CrossBoundary Prosper Africa Detailed Workplan_09 August 2019 ● Pensions – MiDA o Deliverable 1a Delegation Trip 1 Concept note revised April 2020_Finalized o Deliverable 3a. AEMS Concept note Final_finalized o Deliverable 3a. AEMS Concept note Final_finalized.docx-comment o Deliverable 3b. INVESTOR OUTREACH - COMMUNICATIONS PLAN Final clean o DELIVERABLE 4.D. Complete Capacity Building Advisory Mission Finalpptx MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 73 o DELIVERABLE 4.D. Complete Capacity Building Advisory Mission Finalpptx.pptx-comment o DELIVERABLE 4.E. Batseta Complete Capacity Building Advisory Mission Final AB o DELIVERABLE 4.E. Batseta Complete Capacity Building Advisory Mission Final AB.pptx￾comment o DELIVERABLE 4A - KEPFIC Advisory and Consortium Building Plan Revised Final o Deliverable 4b. Batseta Advisory and Consortium Building Plan revised Final o Memo to File - MiDA TO 1 Mod 1 o MiDA Advisors TO 1_Mod 1_FE ● Prosper – Track 2 ● 1. SA Procurement TEC Composition ● 2. SA Procurement PN Distribution List ● 3. INVEST SA REGO Comms Pieces Vietnam Documents ● 1. Scoping Mission Agenda ● 1. Vietnam INVEST scoping mission report ● 2. MCEDF Subk SOW ● 3. Vietnam INVEST MELP_Draft ● 3. Vietnam INVEST MELP_Draft.docx-comment ● 4. PPP Law_final ● 5. RFP INVEST-042 - Vietnam MCEDF ● 6. SOO_RFP-042 Vietnam MCEDF ● 7. RFP INVEST-050 - Vietnam Healthcare Sustainability Assessment ● 8. SOO INVEST - Vietnam Healthcare Sustainability Assessment ● 9. Vietnam INVEST_NPT Final Report Phase 1 ● 10. Cat Lai Summary Report Single Files ● 2.9 Satisfaction of buy-in clients 8_20 ● Blended-Finance-Roadmap-508 ● Buy-In Procurement and SubKs 3_23_2020 ● Buy-In Procurement and SubKs_May 6 ● Buy-In_Client_Survey_8.20_Draft ● INVEST Buy-Ins, Clients 3_23_2020 ● INVEST Buy-Ins, POCs, Clients 5_5_20 ● INVEST Buy-Ins, POCs, Clients 5_5_20_1 ● INVEST Concept Note - External Version 10_31_17 ● INVEST Kenya Final Report - 4 May 2020 Final ● INVEST Org Chart FEB 2020 ● INVEST Pipeline Data_Updated May 2020 ● INVEST Section C ● INVEST Y2 Annual Report with Financials_Final ● INVEST Y2 Annual Report_Final ● INVEST_PartnerNetwork List Feb3 2020 ● Learning_PowerPoint_7_21_20 ● List of INVEST staff, emails_5.19.20 ● MSI Information Request_VN ● Partner Network Survey Data 2_21_2020 MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 74 ● Partner Network Survey ● PCM BAA Addendum 02 ● PSE Hub Partner Network Survey Data 3_23_2020 ● RE EXTERNALRE Data Updates ● RE EXTERNALRE EXTERNALINVEST PE buy-in firm details MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 75 ANNEX G: ONLINE SURVEY INSTRUMENT The United States Agency for International Development (USAID) is conducting a mid-term evaluation of the INVEST mechanism. The evaluation is being conducted under USAID's E3 Analytics and Evaluation Project implemented by Management Systems International (MSI). The goal of the evaluation is to learn about the experiences and perceptions of participants and those familiar with INVEST. This information will be used to assess the first two years of the INVEST implementation, as well as make forward-looking recommendations to USAID. We appreciate your time. If you have any questions about the survey or evaluation please contact Jacob Patterson-Stein, jstein@msi-inc.com 1. Before you start, we are required to ask for your consent to participate in our survey. Please note that: ● This survey is completely voluntary and you have the right to decline participation. ● You can stop at any point and exit the survey. ● You have the right to decline to answer any question for any reason. Your responses to this survey are confidential. Participation in this survey poses no known risks to your person or property. A unique ID will be given to each survey, and all self-identifiers will be removed from the data and analysis. No personal identifiers will be shared with any third party and will only be used for evaluation research purposes. Would you like to proceed? (Select one option) Yes No NOTE : IF ANSWER TO Q1 is Yes Go to Page No. 2 No Stop, you have finished the survey If Did Not Answer Then Go to Page No. 2 2. What is the name of your firm? MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 76 3. How did you join the INVEST partner network? (Select one option) Email to Invest/DAI team LinkedIn Through the bidding process Referral from USAID, INVEST or industry colleagues I am not part of the partner network Other (Please specify) __________ NOTE : Answer the below question only if answer to Q#3 is NOT (I am not part of the partner network) 4. When did you join the partner network? _ _ /_ _ /_ _ _ _ [mm/dd/yyyy] 5. How have you participated in a buy-in? (Select one option) Been awarded one or more contracts through INVEST Bid on one or more contracts, but was not selected As an investor through INVEST Never bid on USAID work via INVEST Other (Please specify) __________ NOTE : Answer the below question only if answer to Q#5 is Been awarded one or more contracts through INVEST OR Bid on one or more contracts, but was not selected 6. What was the INVEST RFP or Task Order Number? MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 77 7. Had your firm worked with USAID before joining the partner network or working with INVEST? (Select one option) Yes No 8. Other than through INVEST, have you sought work with USAID (e.g. responding to RFPs or RFIs) since joining the INVEST partner network? (Select one option) Yes No 9. Have you read any of the reports, documents, or blog posts published by DAI-INVEST? (Select one option) Yes No NOTE : Answer the below question only if answer to Q#9 is Yes 10. Were these materials helpful for better understanding USAID, blended finance, or other related activities? (Select one option) Yes No NOTE : Answer the below question only if answer to Q#9 is No 11. Have you seen or are you aware of any reports, documents, or blog posts published by INVEST? (Select one option) Yes No MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 78 12. How has being part of or associated with INVEST has actively benefited your firm? Tip: Select all that apply Access to opportunities to contract with USAID Learning about blended finance and private capital mobilization Opportunities to interact or network with other firms doing work similar to mine My firm has not actively benefited from my association with INVEST Other (Please specify) ______________ 13. Is your firm either of the following? Tip: "Small business" as defined by the US SBA guidelines. See https://www.sba.gov/size-standards/ for more information. "Woman owned" is defined by at least 51 percent of ownership by one or more women. Yes No (a) Small business (b) Woman owned 14 . In what country is your firm headquartered? (Select one option) [drop-down list of all countries] 15. Is there anything else you would like to share about your experience with INVEST? 16. Check here if you are interested in having a discussion with the evaluation team about your experience with INVEST? NOTE : Answer the below question only if answer to Q#16 is - 17. Please enter your email address MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 79 ANNEX H: SUPPLEMENTAL ONLINE SURVEY FINDINGS The following findings from the online survey supplement those presented in the main evaluation report. The survey had 21 respondents, 14 of whom represented small businesses (7 of which were woman￾owned, with another respondent representing a woman-owned business that was not reported as small). The survey did not require respondents to answer every question. As shown in Figure 10, nine of the survey respondents joined the partner network between September 2019 and April 2020, with one respondent saying they joined the network in January 2018. FIGURE 10: DATE RESPONDENTS JOINED THE PARTNER NETWORK Four of the more recent partners that responded to the survey had never bid on an INVEST buy-in. The distribution of when partners joined varied among those that had bid on INVEST work is shown in Figure 11. Notably, the four respondents who joined the network at the end of 2019 and in 2020 had not bid on any INVEST work. FIGURE 11: DATE RESPONDENTS JOINED AND ENGAGEMENT WITH INVEST MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 80 Twelve of the 21 respondents had worked with USAID prior to joining the partner network. Fourteen of the 21 survey respondents reported seeking out new, non-INVEST USAID work since joining the network; of these, 4 had not previously worked with USAID. The main purpose of the online survey was to understand engagement with INVEST communication and learning materials. To this end, the survey asked whether respondents were aware of INVEST publications and if they had read INVEST publications. As shown in Figure 12, about 70 percent (n=12) of those who were aware of INVEST publications had actually read them. One of the four respondents who said they were unaware of INVEST publications also reported successfully winning work through INVEST, while two said they had never responded to INVEST RFPs, and one said they had unsuccessfully bid. FIGURE 12: SURVEY RESPONDENTS AWARENESS AND ENGAGEMENT WITH INVEST PUBLICATIONS Eleven of the 12 respondents who said they had read INVEST publications reported they were helpful. Most survey respondents saw INVEST as a way to access USAID opportunities, yet almost 40 percent (n=8) of respondents felt their firm had not actively benefitted from INVEST (Figure 13). MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 81 FIGURE 13: SURVEY RESPONDENTS' PERSPECTIVES ON THE BENEFITS OF INVEST Five respondents noted that they hoped to work with other firms in the partner network. Interview data with DAI and partner network firms suggest there was little interaction between network firms, with the exception of workshops and the co-creation process. A review of available subcontract data suggests there has been some collaboration within the network, but this has been limited to 11 firms. As shown in Figure 14, most buy-ins have not involved multiple network partners but some firms have worked together on subcontracts. In two cases, this collaboration consisted of two firms working on the same buy-in (as shown by the two dots connected to each other on the right and left sides of the network plot in Figure 14). The data available to the evaluation team do not make clear whether these joint subcontracts were part of a deliberate collaboration or simply the way that work was awarded. FIGURE 14: NETWORK MAP OF PARTNER FIRMS SUBCONTRACTING ON THE SAME BUY-IN MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 82 ANNEX I: CASE STUDIES MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 83 INVEST BUY-IN CASE STUDY: USAID/SOUTHERN AFRICA – SUPPORT FOR PROSPER AFRICA INTRODUCTION This case study highlights key details of USAID’s INVEST mechanism’s work with the Southern Africa Regional (SAR) Mission. The case study complements the report for the mid-term performance evaluation of the INVEST mechanism, which USAID’s Office of Private Capital and Microenterprise in the Bureau for Economic Growth, Education, and Environment (E3/PCM) commissioned and USAID’s E3 Analytics and Evaluation Project31 designed and implemented. The case study also describes how INVEST’s support of buy-in activities relates to USAID’s broader Private Sector Engagement (PSE) Policy. The mid-term evaluation report provides more background on the INVEST mechanism and the evaluation. The SAR Mission and E3/PCM began discussions of a buy-in activity to support Prosper Africa in early 2019. Prosper Africa is a multiagency U.S. presidential initiative to foster U.S.-Africa business connections, trade, and investment.32 Since those initial discussions, two additional SAR Mission buy-ins related to Prosper Africa have started. This case study draws from interviews the evaluation team conducted as part of the overall mid-term evaluation and from the team’s review of dozens of related documents and communication materials. BUY-IN ACTIVITIES This case study details three INVEST buy-in activities: 1. Track 1: A task order DAI subcontracted to an INVEST core partner, CrossBoundary, that started in July 2019. Under this task order, CrossBoundary provided transaction advisory services to an initial set of six firms that were part of CrossBoundary’s potential deal pipeline. Between August 2019 and April 2020, CrossBoundary provided advisory services to six “investment-ready” firms.33 These firms cover diverse sectors from poultry in Uganda to software as a service in Nigeria and Kenya. 2. Track 2: Two task orders that started in November 2019 for Lion’s Head Global Partners and Open Capital Advisors to develop a pipeline of transaction opportunities in southern and eastern Africa. The goal of pipeline development is to identify opportunities above $5 million, although the request for proposals notes that smaller investment deals could also be supported. A key criterion for inclusion is that at least 20 percent of the firm or investor equity must be American held. 3. Pensions/Institutional Investors: A subcontracted activity to MiDA Advisors that started in December 2019 to promote investment in Africa to institutional investors, such as pension funds, through relationship building, roadshows, and delegation trips, as well as to provide research 31 Management Systems International (MSI), a Tetra Tech Company, implements the E3 Analytics and Evaluation Project in partnership with Palladium and NORC at the University of Chicago. 32 See https://www.trade.gov/prosper-africa-about-us for background on the initiative. 33 See CrossBoundary Prosper Africa Monthly Report April 2020. What is INVEST? The INVEST mechanism was awarded to DAI in 2017 to support private investment in USAID partner countries. DAI helps facilitate and manage a network of partner firms that can respond to investment￾related proposals and task orders. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 84 and pipeline facilitation services. Documentation review and interviews with INVEST stakeholders suggest that this buy-in activity seeks to demystify and “de-risk” investment in Africa. As one USAID buy-in client respondent described, “the pension work is really designed to get underneath that [avoidance of investment in Africa] and really understand why that is and is it just an information asymmetry? Is it a relationship issue? Is there actual risk out there that we can mitigate? Or is it something where we could solve for it just by increasing the exposure of U.S. asset allocators to the continent?” Each of these buy-in activities fits into Prosper Africa’s goal of strengthening investment connections between the U.S. and Africa. Track 1 addresses specific deals that CrossBoundary identified before its work with INVEST, and Track 2 and the Pension/Institutional Investor activities relate to identification, information sharing, and assessment. However, these INVEST activities are relatively new, with work starting toward the end of INVEST’s second year (i.e., November and December 2019). Therefore, this case study does not examine development outcomes as a result of these activities or even the completion of capital allocation—it is simply too soon. Credit: Jacob Patterson-Stein, MSI In addition, each buy-in activity underwent a contract modification, with Track 1 and the Pensions/Institutional Investor activities extending their timelines to account for travel restrictions imposed by the COVID-19 pandemic response. To summarize INVEST’s SAR Mission support, the evaluation team has framed the case study within the key questions of the PSE Policy. Announced at the end of 2018, this policy is a “mandate” for the Agency to “work hand-in-hand with the private sector.”34 The PSE Policy contains five questions for USAID and its partners to consider when approaching development: 1. Can the private sector solve this problem by itself? 2. Could there be a market-based approach to addressing this challenge? 3. What are the roles and interests of the private sector in addressing this challenge? 4. Are there factors constraining the private sector from involvement and investment? 5. Is there a role for USAID to help alleviate or eliminate these constraints? Within each of these PSE Policy areas, a picture emerges of how INVEST has operationalized private capital mobilization through its partner network.35 CAN THE PRIVATE SECTOR SOLVE THIS PROBLEM BY ITSELF? Foreign direct investment inflows to Sub-Saharan Africa are down from a peak of $45.5 billion (at current U.S. rates) in 2012 to $2.8 billion (at current U.S. rates) in 2019, according to the International Monetary Fund.36 Between 2013 and 2017, U.S. foreign direct investment in Africa overall fell by $11 34 See https://www.usaid.gov/work-usaid/private-sector-engagement 35 This directly relates to the first evaluation question of the mid-term evaluation. 36 See International Monetary Fund, Balance of Payments database MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 85 billion, according to the United Nations’ World Investment Report. 37 In this context, the challenge of connecting U.S. investors with firms working and investing in Africa may not be “solved” by the private sector itself. The Track 1 INVEST activity started with two transactions in August 2019, aiming to provide advisory services on six transactions. The first transaction exemplifies how INVEST’s support can fill a gap left by the private sector. The first firm, Agilis, is a U.S.-owned agricultural holding company seeking to expand through debt and equity capital. Although Agilis received investment from John Deere and an impact investor, it was facing challenges. As one USAID respondent explained, “They were having a challenge raising capital, they were facing a lot of issues around their balance sheet, their capital structure, and they were hitting some roadblocks on their capital raise.” In addition, the firm was “targeting a close for the equity in three months” (CrossBoundary Prosper Africa Detailed Work Plan), suggesting that assistance was timely. CrossBoundary’s support with market and regulatory assessments was reportedly able to support the firm’s planning and capital raise despite dealing with capital constraints and a limited window to obtain advisory services, all in support of the firm’s non-market food security goals. The Track 2 INVEST activity, the development of an investment pipeline, attempts to identify investment opportunities that align with USAID priorities. Using a pipeline tool, Track 2 partners identify the size of potential investment deals, support needed, existence of a gender component, the U.S. nexus (i.e., 20 percent U.S. equity), United Nations Sustainable Development Goal support, and other priorities that private-sector investors may not consider. Roadshows, investor summits, and other forms of convening and networking are typical private-sector solutions to filling information gaps; the U.S. Securities and Exchange Commission formally addresses road shows in the Securities Act Rule 433. Many institutional investors have researchers who work to identify opportunities, but the private sector on its own may not move beyond its own biases and perceived and actual risks to begin the relationship building needed for investment. Through INVEST, MiDA has planned a “delegation trip” to Nairobi, Kenya, for asset owners, pension fund managers, and industry partners to learn about “African market fundamentals” (Delegation Trip Concept Note), as well as a suite of other outreach efforts to fill this gap. Many planned activities under the Pension/Institutional Investor work have been rescheduled due to COVID-19-related restrictions, but the gap this work aims to fill remains. One respondent interviewed for the INVEST mid-term evaluation was skeptical that such efforts are needed, but USAID staff, as well as one private-sector firm, noted that facilitating events were a potential value add and a first step toward providing a forum. COULD THERE BE A MARKET-BASED APPROACH TO ADDRESSING THIS CHALLENGE? For each of these activities, there could be a market-based approach to addressing the challenge of connecting investment-ready firms with transaction services (Track 1), identifying a pipeline (Track 2), or connecting institutional investors with new opportunities in Africa (Pensions/Institutional Investors). The firms implementing these activities under INVEST have implemented similar projects outside a donor context. In Track 1, for example, CrossBoundary had already selected firms in its pipeline, but INVEST supports the timely provision of services to help the pipeline firms. However, observed changes in foreign direct investment, as well as general perceptions that INVEST stakeholders have acknowledged they are pushing against,38 suggest that the market may not have 37 See p. 34. 38 For example, see https://nextbillion.net/past-outdated-narrative-investing-in-africa. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 86 provided the necessary support within the timeframe and under the terms that would support the firms’ goals and Prosper Africa’s priorities. The Track 1 activity’s support of Field Intelligence, an American-run, Nigerian-based, software-as-a￾service startup provides a more nuanced example. Field Intelligence had already received grant funding from the Gates Foundation, with “Blue Haven Initiative, FINCA Ventures, Johnson & Johnson impact fund…conducting active due diligence processes” when the INVEST buy-in started (CrossBoundary Prosper Africa Detailed Work Plan). CrossBoundary provided financial model support, including cash flow analysis, valuation tools, and operational projections, as well as a detailed market assessment. With the support of CrossBoundary’s services, Field Intelligence completed a $3.6 million Series A fundraising round in February 2020. It is difficult to know what the private sector counterfactual would have been. Field Intelligence theoretically could have hired a firm to provide advisory services—Field Intelligence opened an additional Berlin office in 2018 and expanded to Kenya shortly thereafter39—but the support INVEST provided helped an already growing firm translate its operational trajectory into business valuation that helped support its Series A round. WHAT ARE THE ROLES AND INTERESTS OF THE PRIVATE SECTOR IN ADDRESSING THIS CHALLENGE? The challenge that Prosper Africa’s three INVEST buy-ins seek to address all relate to connecting firms and investors on the continent to firms and investors in the U.S. For the Track 1 activity, there is a clear private sector role and incentive to find opportunities for a debt transaction, in the case of Agilis, or to find an equity transaction in a market that other investors may not have tapped into yet. The private sector in the INVEST context can help firms grow, fund research, and change their operations (e.g., shift to vertically integrated production). The Track 1 activity provides an example of the private sector’s role. CrossBoundary provided due diligence and support to Tomato Jos, a woman-owned, Nigeria-based firm that makes tomato paste. Firms such as Monsanto, Heinz, and DuPont, had already established research partnerships with Tomato Jos, but additional private capital was needed for the firm to scale its operations. Tomato Jos closed $4.4 million in equity investment from two U.S. investors and one local investor (CrossBoundary Monthly Report May 2020). Equity investment that leads to improved processing and distribution systems will help Tomato Jos and its investors increase their revenues, assuming sales increase as a result, while also showing that investment in Africa can be mutually beneficial. ARE THERE FACTORS CONSTRAINING THE PRIVATE SECTOR FROM INVOLVEMENT AND INVESTMENT? INVEST’s Prosper Africa work addresses or has highlighted three broad constraints: transaction and other costs, governance constraints, and coordination and network constraints. Transaction and other costs. CrossBoundary identified local value-added taxes as increasing the cost of investment, including that “in Nigeria, a U.S. company may actually optimize their tax burden by limiting the number of local suppliers they include in their value chain” (Monthly Report May 2020). CrossBoundary notes in its monthly reporting that local policies may push private firms to vertically integrate to reduce their tax liability. Governance constraints. As part of the Track 1 activity, CrossBoundary identified multiple lessons learned from its experience providing advisory services, which include constraints to investment. 39 See https://field.inc/about/. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 87 CrossBoundary noted that “local content policies” that aim to promote local sourcing or employment often lack regulatory structures, which can “leave foreign investors at risk of being noncompliant” (Monthly Report May 2020). One USAID respondent described the Track 1 beneficiary firm as facing a similar situation: “The commercial ag farm [Agilis] was looking to vertically integrate, so they would produce feed for the chickens from their farm, and move into the chicken space. They were unfamiliar with the industry…the viability, trade, exporting, license, all of those things.” CrossBoundary identified issues around governance and transparency as potential constraints to investment as well. The partner firm noted that a lack of transparency around government procurement leaves private U.S. investors and firms at a competitive disadvantage to firms that do not need to comply with the U.S. Foreign Corrupt Practices Act. Coordination and network constraints. Although work under the Pensions/Institutional Investors activity has been limited, this buy-in as well as the Track 1 and Track 2 activities aim to address the challenges private-sector actors in Africa face in connecting with USAID to obtain de-risking or transaction services support or with investors interested in but uninformed about investing in Africa. A Track 2 respondent described the work as “to source businesses and find ones that would be a good fit and that may be ones that the USAID would want to support, that entails essentially us going to the market, tapping our networks to identify those businesses.” This respondent explained that under Track 2, the buy-in partners developed a conference tracker to coordinate who was going to what events to find potential pipeline opportunities and connect with people who might otherwise not find support. Under Track 1, CrossBoundary provided support to the Savannah Fund, a venture capital fund, to help with “introductions to potential U.S. investors and support in executing due diligence on [a] new fund” (CrossBoundary Detailed Work Plan). The entire suite of planned Pensions/Institutional Investors activities, which are slated to commence in late 2020 or 2021, focus on finding ways—events, outreach, media—to connect investors to potential opportunities that they do not know about or have not considered. IS THERE A ROLE FOR USAID TO HELP ALLEVIATE OR ELIMINATE THESE CONSTRAINTS? Each of the activities highlights USAID’s role—through INVEST—in addressing the aforementioned constraints. Much of the work completed under Track 1, developed and ongoing as part of Track 2, and planned for the Pensions/Institutional Investors buy-in involves providing a strong foundation for investment. As part of Track 1, INVEST’s support provided a process through which investment-ready firms could move closer to obtaining investment. CrossBoundary provided market and regulatory assessments and due diligence, valuation metrics, financial models, and outreach as part of its advisory services through INVEST. Track 2 provides a pipeline to support in the future, and the Pensions/Institutional Investors buy-in helps build the connections required to move from considering investment in Africa to committing to it. One case study interviewee summarized USAID’s role in addressing the constraints facing firms: You do a number of things [to address constraints]. Identifying transactions and doing the diligence on those transactions, given that those costs are quite prohibitive for many companies thinking about doing business in Africa because they'd much rather and more easily expand their investments in Asia, for example, rather than trying to break new ground in Africa. We have a policy-enabling environment. You have de-risking deals, and then thirdly would be aligning the tools that the U.S. government does have to make them more accessible and to make them more easily understood. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 88 KEY TAKEAWAYS The INVEST SAR buy-in activities each work to support Prosper Africa’s overarching goal of increasing U.S.-Africa trade and investment, and each fits into the PSE Policy’s initial questions: 1. Can the private sector solve this problem by itself? The private sector may not simply step in to address the problems facing some firms in Track 1. In cases where firms have a non-market-driven, broader development impact goal and face regulatory, capital, and time constraints, private sector support may be slow and costly. Although the private sector may, on its own, identify opportunities and incentivize the connection of investors to opportunities generally, investors have not always sought out opportunities in Africa on their own. 2. Could there be a market-based approach to addressing this challenge? In some cases, a non-donor approach may have addressed the challenges the INVEST buy-ins sought to support but perhaps not within the desired timeline. In other cases, a market-based approach would not likely have addressed the challenges the buy-ins seek to address. 3. What are the roles and interests of the private sector in addressing this challenge? The private sector can support debt and equity capital and provide technical support within the context of the SAR Mission buy-ins. 4. Are there factors constraining the private sector from involvement and investment? Transaction costs, governance constraints, and coordination challenges – whether real or perceived – all constrain private-sector investment in Africa. The buy-ins this case study summarized attempt to mitigate or address these constraints. 5. Is there a role for USAID to help alleviate or eliminate these constraints? The Prosper Africa INVEST buy-ins highlight the role USAID can play in laying a foundation for investment in Africa. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 89 INVEST BUY-IN CASE STUDY: USAID/VIETNAM INTRODUCTION This case study highlights key details of USAID’s INVEST mechanism’s work with USAID/Vietnam (known as “INVEST Vietnam”). The case study complements the report for the mid-term performance evaluation of the INVEST mechanism, which USAID’s Office of Private Capital and Microenterprise in the Bureau for Economic Growth, Education, and Environment (E3/PCM) commissioned and USAID’s E3 Analytics and Evaluation Project40 designed and implemented. The case study also describes how INVEST’s support of buy-in activities relates to USAID’s broader Private Sector Engagement (PSE) Policy. The mid-term evaluation report provides more background on the INVEST mechanism and the evaluation. USAID/Vietnam launched its INVEST buy-in in August 2019 to explore and facilitate private investments in the energy, healthcare, education, and infrastructure sectors. The goal of this activity was to improve governance systems and catalyze investments in the targeted sectors by supporting market-based and public-private partnership solutions to address development priorities. This case study draws from interviews conducted as part of overall mid-term evaluation as well as a review of related documents and communication materials. BUY-IN ACTIVITIES This case study discusses four INVEST Vietnam activities: 1. Transmission Bond with Vietnam Electricity/National Power Transmission Corporation (NPT): A standalone bond issuance with a domestic tranche denominated in Vietnamese dong and an international tranche denominated in U.S. dollars would help NPT raise needed capital from a market that has not yet been tapped. The proposed bond would be a standalone NPT bond issue, without a sovereign guarantee.41 A pre-feasibility assessment report issued in April 2020 described the “genuine interest on the part of both the Government of Vietnam (GVN) and NPT to proceed” and included a recommendation that USAID/Vietnam continue support of the NPT bond effort. 2. Renewable Energy Catalytic Capital: Vietnam INVEST plans to offset some of the regulatory risks and transaction costs that private-sector actors who wish to invest in the energy sector face. The proposed approach involves commissioning discrete activities such as pre-feasibility and feasibility studies, legal analyses/risk assessments, and environmental analyses. Developers will be asked to submit promising projects or concepts they would like to pursue and projects will be selected based on feasibility and impact criteria developed in coordination with the USAID/Vietnam Urban Energy Security project, another DAI-implemented initiative.42 One interviewee explained how the Urban Energy Security project, considered together with the clean energy facility described 40 Management Systems International (MSI), a Tetra Tech Company, implements the E3 Analytics and Evaluation Project in partnership with Palladium and NORC at the University of Chicago. 41 A sovereign guarantee, given by a government, provides assurance that all obligations will be satisfied if the primary obligor (in this instance, NPT) goes into default. Because NPT is sovereign-owned, but not sovereign, a standalone NPT bond would receive a rating slightly lower than a similar bond with a sovereign guarantee. Methods of mitigating this lower rating could be considered including seeking a Multilateral Investment Guarantee Agency guarantee. More information about sovereign debt and this type of guarantee can be found at https://www.nationalstandardfinance.com/news/sovereign-guarantees/. 42 See https://www.dai.com/our-work/projects/vietnam-urban-energy-security What is INVEST? The INVEST mechanism was awarded to DAI in 2017 to support private investment in USAID partner countries. DAI helps facilitate and manage a network of partner firms that can respond to investment￾related proposals and task orders. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 90 below, formed “a set of ideas on a risk spectrum that would leave the mission in a good place from a risk-management perspective.” 3. Municipal Clean Energy Development Facility (MCEDF): The MCEDF will build the capacity of the Danang municipal government to engage the private sector in clean energy projects under the GVN’s new public-private partnership (PPP) law. If municipal governments receive the legal and technical assistance needed to establish, structure, and implement PPPs through a transparent and rigorous process, the theory of change for this initiative holds that the governments can then drive innovative, clean, and renewable energy solutions to meet demand and attract investors. Credit: USAID/Vietnam Described as offering a medium risk, this project concept was described by an interviewee as appropriate for a pilot in Danang “because they were quite well progressed in clean and renewable energy, forward-looking administration, and there was also some very interesting synergy to be had with the Vietnam Urban Energy Security team that felt like we could implement with some decent degree of success.” 4. Market-Based Healthcare Sustainability Activity: USAID began supporting vital HIV/AIDS programs in the mid-1990s and since then has worked with community-based organizations (CBOs) and social enterprises (SEs) based in urban centers to deliver services in partnership with GVN via the U.S. President’s Emergency Plan for AIDS Relief. INVEST’s effort is planned to identify and pilot scalable fee-for-service models and expanded service offerings to assist CBOs and SEs to become financially sustainable and deliver more diversified quality care. Phase I of this initiative will involve assessment of 60 to 65 CBOS and SEs, recommendation of parameters for an innovative pilot fund to implement new fee structures or services, and a shortlist of 15 to 20 entities ready to diversify and expand services. To summarize INVEST Vietnam, the evaluation team framed this case study within the PSE Policy’s key questions. USAID announced the PSE Policy at the end of 2018, and it is a “mandate” for the Agency to “work hand-in-hand with the private sector.”43 The PSE Policy contains five questions for USAID and its partners to consider when approaching development. These questions offer a helpful way to frame the earliest stages of INVEST Vietnam: 1. Can the private sector solve this problem by itself? 2. Could there be a market-based approach to addressing this challenge? 3. What are the roles and interests of the private sector in addressing this challenge? 4. Are there factors constraining the private sector from involvement and investment? 5. Is there a role for USAID to help alleviate or eliminate these constraints? 43 See https://www.usaid.gov/work-usaid/private-sector-engagement MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 91 Within each PSE Policy area, a picture of how INVEST has operationalized the mobilization of private capital through its partner network emerges.44 However, the advent of the COVID-19 pandemic in early March 2020 resulted in the deferral or delay of much planned effort. All in-person meetings related to the buy-ins were cancelled as GVN’s response to the pandemic took priority. Nevertheless, using the five PSE Policy threshold questions to review INVEST Vietnam provides an appropriate critical framework to assess the buy-in despite it still being relatively new and much of the planned work having been postponed. CAN THE PRIVATE SECTOR SOLVE THIS PROBLEM BY ITSELF? According to the World Bank, Vietnam’s shift from a centrally planned to a market economy has raised the nation’s status from being one of the world’s poorest to that of a lower-middle income country. With gross domestic product per capita increasing 2.7 times between 2002 and 2018, poverty rates have declined from 70 percent to below 6 percent. As of 2016, for 99 percent of the population the main source of lighting is electricity and electric consumption tripled over the past 10 years. This rapid growth and industrialization is reported to have had serious negative effects on the environment.45 A USAID respondent noted that the Vietnam Mission has been leading the Agency in thinking around private sector engagement for years and explained the market in Vietnam, saying, “there's a lot of foreign investment here. There's a lot of domestic investment here. It's a buzzing economy and if you're not doing private sector engagement here, then something's wrong because private sector's everywhere. So I would say the Mission was already more oriented than most around the private sector partnerships.” However, dysfunctional markets where capital supply and project demand are disconnected continued to exist. In some cases, those markets simply needed information, advice, legal services, and convening power to address what would otherwise be a market failure. In November 2019, INVEST led a scoping mission in which DAI’s strategic investment advisor and a bond consultant met with more than 40 informants to discuss, among other things, transaction facilitation opportunities and challenges for USAID in Vietnam’s energy and health sectors. Identified issues included how, as a result of a system of feed-in tariffs, the private sector had invested extensively in solar energy, leaving an array of untapped clean and renewable energy sources. European and U.S. energy players were not comfortable with the risks allocated to them in the power purchasing agreements that had developed as the template within the market and had therefore mostly stayed on the sidelines. With regard to healthcare, it was noted that a great deal of private capital had already been invested into the sector and more was anticipated. COULD THERE BE A MARKET-BASED APPROACH TO ADDRESSING THIS CHALLENGE? Based on the reported results of the initial scoping mission, as well as interviews with multiple INVEST stakeholders, a market-based approach can reasonably be contemplated for each of the four challenges addressed in this case study. The proposed NPT bond issuance can be viewed as a classic market-based approach to addressing the need for investment to expand and modernize the energy transmission network and smart-grid capability. If successful, the bond issuance would build and strengthen the NPT’s capacity to access financing in national and international capital markets. This approach could also support GVN’s aim of achieving a bond market operated in accordance with international best practices. If the bond does 44 This directly relates to the first evaluation question of the mid-term evaluation report. 45 See https://www.worldbank.org/en/country/vietnam/overview. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 92 operate within best practices, it could introduce the possibility of a bond market to finance further Vietnamese infrastructure development. The planned use of catalytic capital for renewable energy companies is an example of using up￾front funding to attract additional private capital that could not otherwise be attracted without concessional support in the form of feasibility studies and legal and environmental analyses. This market￾based approach is intended to help remove some uncertainty about whether projects would be eligible under the Ministry of Industry and Trade’s upcoming National Power Development Master Plan 2021- 2030 (Power Development Master Plan 8). Ensuring project eligibility under Master Plan 8, reducing up￾front transaction costs, and helping to determine the economic and technical viability of projects will help mobilize private capital to meet Vietnam’s increasing demand for renewable energy. Since PPPs allow the public and private sectors to share risks and rewards in the delivery of services and infrastructure, the MCDEF is anticipated to include significant market-based elements, depending on the manner in which the pilot project is designed. Although a feasibility assessment has yet to prepared and, therefore, details of the pilot project have not been defined, mobilization of additional private sources of funding and financing is a possible outcome, as are opportunities to develop objectives that align private sector incentives with those of the Danang municipal government or other contracting authority. As suggested by its name, the Market-Based Healthcare Sustainability Activity is an effort to engage the private sector to achieve greater scale and sustainability of outcomes. Historically, GVN has directly delivered services, operated facilities, trained medical professionals, and conducted medical research and development. Healthcare has been viewed as a state function. However, GVN has begun to allow investors to open private hospitals and clinics, which are seen as offering more and improved services for those who are able to pay. INVEST’s approach involves de-risking a pilot project of new services while concurrently delivering technical assistance to support implementation. WHAT ARE THE ROLES AND INTERESTS OF THE PRIVATE SECTOR IN ADDRESSING THIS CHALLENGE? Facilitating private investment in Vietnam’s energy and healthcare sectors is a vital element of INVEST Vietnam. The roles and interests of the private sector are being considered as part of INVEST’s planning and execution of the four activities addressed in this case study. For the NPT bond issuance, the most obvious role for the private sector is the provision of financial resources via national and international financial markets. However, there are important additional roles that can likely best be carried out by the private sector. There are several options for organizations that could take the lead in bond issuance, including consultancy firms, international accounting firms, capital markets advisory firms, merchant banks, and investment banks. INVEST’s prefeasibility study included an illustrative list of private-sector firms with the necessary qualifications. The firms that get involved would ideally have a long track record of advising on bond issues. Other private-sector actors will include one or more underwriters and, if multi-year International Financial Reporting Standards-compliant financial statements are not already available, international accounting and auditing firms. Other entities that could be important to the bond issuance include banks, securities brokerages, guarantors, capital markets authorities, exchanges, law firms, rating agencies, and rating advisory service providers. Government policies that do more to encourage private-sector engagement may occur as a by-product of the catalytic capital for renewable energy activity. GVN may be motivated to encourage and facilitate private-sector investment in renewable energy production based on the impact of INVEST Vietnam’s more positive and transparent collaboration with private sector renewable energy actors. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 93 The feasibility study for the MCEDF has yet to be prepared, so the specific way in which the private sector will be engaged in that activity has not been determined. Nevertheless, it is anticipated that the proposed PPP will include a role for private interests and investors. As noted in the project’s statement of objectives, there is active private-sector participation in Vietnam’s infrastructure sector. In the first half of 2018, investments totaled more than $40 billion in 164 projects and renewable energy comprised 64 percent of the total investment in electricity generation.46 Private-sector roles and interests in the Market-Based Healthcare Sustainability Activity are likely to be extensive. The sustainability of the CBOs and SEs that deliver care to HIV/AIDS patients is essential and many do not have the funding or financial flexibility to test new models or fee structures. The proposed Technical Assistance Fund to strengthen the capacity of these organizations may offer business plan development, coaching, training, or other technical support to enhance organizational capacity. ARE THERE FACTORS CONSTRAINING THE PRIVATE SECTOR FROM INVOLVEMENT AND INVESTMENT? Interviewees viewed identifying challenges and constraints as an important aspect of INVEST’s thought leadership. Describing INVEST’s responsibility for the INVEST Vietnam buy-in, a USAID respondent explained how DAI has excelled at helping USAID “use our expertise to ground truth these transaction ideas, think about potential designs, and think about entry points and roles for the INVEST program into the transaction that we were considering.” Another respondent with significant involvement in INVEST Vietnam explained an important aspect of INVEST’s involvement as, “we help [USAID] not just implement and procure, but think through what might some of these approaches be, what might be some of the challenges, how might they be mitigated, what are alternatives and so on.” The inefficient legal framework governing PPPs (including the proposed MCEDF) has historically been cited as an impediment to certain types of private-sector engagement in Vietnam. The requirements and conditions applicable to PPPs were scattered across a large number of legislative instruments, making it difficult for investors to navigate the separate decrees, circulars, and decisions addressing PPPs as well as existing laws dealing with public and private investment generally. A draft PPP law was developed and included changes to impose discipline, transparency, and accountability on regulators and market participants. Further, the proposed law established eligible project types and minimum investment size, stipulated investor equity and revenue sharing norms, eliminated sole-source procurement, and delegated thresholds and authorities for different project types. On June 18, 2020, the GVN National Assembly passed the new law on PPP investment.47 IS THERE A ROLE FOR USAID TO HELP ALLEVIATE OR ELIMINATE THESE CONSTRAINTS? As INVEST conducts and completes the various feasibility studies to identify next steps for the four buy￾in activities, USAID’s role in addressing identified constraints will become clearer. However, in one respect, USAID has already helped enhance Vietnam’s ability to work with the private sector. USAID Vietnam has been providing ongoing support to the National Assembly via the a separate contract mechanism, LEAP III, including an interactive question-and-answer platform as well as regional consultative workshops.48 This action, which aided in the passage of the new PPP legislation, was seen as demonstrating the government’s legal commitment to domestic and foreign investors, and is expected 46 See https://www.vietnam-briefing.com/news/vietnam-among-top-five-private-participation-infrastructure-projects.html/. 47 See https://vietnamnewswire.com/national-assembly-passes-law-on-ppp-investment/. 48 See http://www.integrallc.com/leap-3/ and https://www.usaid.gov/vietnam/program-updates/oct-2019-usaid-supports-vietnam￾first-public-private-partnership-ppp-law. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 94 to encourage investors to join GVN in PPP projects. However related challenges, including the lack of capacity among state agencies to prepare, procure and manage PPP projects are still to be addressed. Going forward, INVEST has contracted for a firm to develop a feasibility study of the MCEDF and pilot project recommendations for the Danang municipal government. Key tasks that address the challenge presented by the new PPP law include identifying constraints and risks at the national, municipal, and investor levels that would impact municipal-level investment in renewable and clean energy projects; meeting with relevant national and municipal government entities, project agencies, and development partners including private-sector companies to note anticipated staff changes and processes under the new PPP law and document their perspectives; and performing an analysis to identify any municipal-level policy changes that may be required to align with the new law. KEY TAKEWAYS INVEST Vietnam was developed to explore and facilitate private investments in the energy, healthcare, education, and infrastructure sectors. This case study highlighted how the buy-in’s first four activities fit into USAID’s PSE Policy threshold questions: 1. Can the private sector solve this problem by itself? It is unclear whether the private sector could address INVEST Vietnam’s development goals on is its own. Given the progress made to date and recent rapid growth, it is possible that increased investment would occur, even without USAID intervention. However, the continued existence of dysfunctional markets indicates the likelihood that additional support in the form of information, advice, legal services, and convening power to bring capital and projects together will shorten Vietnam’s journey toward self-reliance. 2. Could there be a market-based approach to addressing this challenge? Market-based approaches can make a significant addition to the impact of each of the four current INVEST Vietnam activities. INVEST is planning to use a range of market-based approaches to support Vietnam in building the requisite skills, resources, knowledge, local institutions, and incentives that enable local systems and markets to enhance its ability to become self-sustaining. 3. What are the roles and interests of the private sector in addressing this challenge? While it is anticipated that the private sector will be an important factor in each of INVEST Vietnam activity, specific private sector roles have only been described for the NPT bond issuance. As additional feasibility analyses are conducted, more information about roles and interests will likely be developed. 4. Are there factors constraining the private sector from involvement and investment? The inefficient legal framework governing PPPs (including the proposed MCEDF) has historically been cited as an impediment to certain types of private sector engagement in Vietnam. 5. Is there a role for USAID to help alleviate or eliminate these constraints? USAID/Vietnam made an important contribution by supporting passage of a recent law that is expected to encourage investors to join GVN in PPP projects. INVEST Vietnam’s planned feasibility study will further explore the additional efforts needed to effectively implement the new legislation. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 95 ANNEX J: EXAMPLE OF DATA TRIANGULATION The evaluation team used multiple data sources and multiple data analysis approaches to enhance the credibility of the evaluation’s findings and conclusions. The following example demonstrates how the team combined the various line of inquiry to support reported results. The evaluation report included the observation that “Seven of 50 stakeholders viewed INVEST’s website as a source of learning, but INVEST’s web presence was not well known.” Evidence in support of this observation included information from four lines of inquiry. First, KIIs with 50 individuals who were USAID buy-in clients, other USAID staff, or representatives of firms with INVEST buy-in contracts or members of a partner network firm were the primary data source for this observation. Five of 32 USAID respondents reported making some use of the website, generally in response to the question “How much engagement with … the website have you had?” Additionally, 2 of 18 respondents from firms reported engagement with the website. Thus, 14 percent of the respondents (7 of 50) mentioned engaging with the website. Another seven respondents (two USAID staff and five partner network firm staff) reported only limited use of the website, but provided no details about how the website was used or what material had been accessed. A second line of inquiry involved a review of USAID’s INVEST website and the USAID INVEST Learning and Dissemination Strategy. The INVEST website did not include a listing of issued RFPs or details of open procurement opportunities, two items that had been of interest to firms interviewed. The Learning and Dissemination Strategy did not describe how INVEST would determine whether either website was reaching the desired audience or how useful the materials were to USAID and to partner firms. Nothing on the site or in the materials provided offered insight about the degree of stakeholder engagement with the website. The online survey of network partner firms served as a third line of inquiry. While the survey’s response rate was too limited to allow for inferences about the overall population, the small number of responses received were generally consistent with the KII results. That is, only 11 of 21 online survey respondents reported reading any of INVEST’s reports, documents, or blog posts. If only about half of the respondents who answered an online survey had read any learning materials, it would not be surprising that only 14 percent of the individuals interviewed had made use of the website. That is, the results of the KIIs tended to reinforce the online survey results and support the observation about the website as a source of learning and that INVEST’s web presence was not well known. A final line of inquiry involved the INVEST procurement forecast website. DAI did not provide information about this website through any of the communication and learning materials shared with the evaluation team. However, the team became aware of the procurement forecast website through the KII process and located and reviewed the website to determine whether it addressed issues of concern raised by partner network firms. Two partner network firms made positive mention of the new website. It is unclear when this website went online, but no respondents mentioned it prior to May 27. A long￾time USAID contractor participating in multiple INVEST buy-ins noted in a June 8 interview, “They fairly recently started doing a forecast, which has been good. I’m very happy that they’ve done it. Once they did it, I realized how much I missed it prior.” The informant’s reference to “fairly recently” suggests that the website may have only recently been put into operation. Once the evaluation team became aware of this website, four DAI employees provided information about it. That information, as well as comments from the two partner network firm representatives and the contents of the website itself, were used to inform the report’s findings, conclusions, and recommendations. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 96 The evaluation team compared and assessed each of the four lines of inquiry to triangulate the report’s observation about the INVEST website as a source of learning. The most valuable insight came from interviews with USAID and partner firm respondents, but additional information offered by DAI as well as other data gathered through the online survey and review of the websites and the Learning and Dissemination Strategy added to the credibility of the team’s finding. MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 97 ANNEX K: CROSS-WALK OF FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS BY EVALUATION QUESTION Evaluation Question Findings Conclusions Recommendations EQ1: How has INVEST operationalized the mobilization of private capital approach through its network of participating firms? • 21 of 30 MBIO respondents provided at least one example of how INVEST had provided important administrative, logistical, or managerial support • 13 MBIO buy-in clients and 4 partner firm respondents acknowledged INVEST’s expertise in communications support • 4 USAID staff commented on their appreciation for the thought leadership INVEST provided to MBIOs • 12 of 23 MBIO respondents mentioned at least one example of how INVEST facilitated contracting activities • Over its first two years, INVEST operationalized the mobilization of private capital through its partner network by supporting timely, targeted procurement through subcontracts • Respondents highly valued INVEST’s effectiveness in working with MBIOs to quickly facilitate procurement, find relevant partners, and support buy-ins • Seek feedback from unsuccessful bidders, especially new and underutilized partners, to understand how they perceive the debriefing materials. Enhance the quality and effectiveness of debriefings as warranted. • 40 percent of USAID and partner firm interviewees found INVEST’s practices to be confusing and expressed a desire for more information about the expected process and steps for a typical procurement • 60 percent of partner network members interviewed with prior USAID experience were dissatisfied with INVEST’s procurement practices and representatives of 7 of 17 firms expressed concerns about transparency and/or fairness in subcontractor selection as part of the INVEST RFP process • INVEST’s debriefing materials were not viewed as helpful for understanding why a firm won or lost a procurement • 10 USAID interviewees expressed concern, or a lack of clarity, about how INVEST procurements operate • INVEST’s structure is complex and not always fully understood by the mechanism’s stakeholders • The partner network grew from 25 initial firms to more than 200 in INVEST’s first two years • Based on data provided to the evaluation team, 3 of the 25 original non-core partner firms obtained INVEST contracts during the period the evaluation reviewed • 4 of the 22 firms that participated in buy-ins during INVEST’s first two years were designated small businesses • The network of firms that participated in INVEST buy-ins has expanded steadily since the mechanism’s early incarnation as a prime facilitating partner (DAI) supported by 3 core partners and a 25-member network of partner firms, organizations, and consultants EQ2: How has INVEST captured and validated lessons learned? • INVEST has developed written products such as close-ups, reports, research, thought leadership articles, and blog posts • 24 of 50 interviewees reported they made little or no use of INVEST learning materials • 3 of 50 interviewees mentioned INVEST’s blog as a source of information and none reported use of INVEST’s LinkedIn group • INVEST has just begun to capture and validate the broad lessons to be learned as it executes its action research project • Key informants including staff from USAID and network partner firms have not made wide use of the learning materials INVEST has shared to date • Develop and implement a methodology to capture and validate learning materials in real time, ensuring retention of important documents and MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 98 Evaluation Question Findings Conclusions Recommendations • 12 of 21 online survey respondents reported having read any INVEST documentation or online material • Documentation suggests that INVEST is poised to begin a focus on learning capture, with greater emphasis on the technical models, evidence, data, outcomes, and new partner profiles emerging from buy-ins in the mechanism’s first two years • No MBIO respondents reported visiting INVEST’s procurement website • Interviews with MBIO staff and DAI activity managers as well as document searches through USAID websites (both the DEC and usaid.gov/invest) and other INVEST-related platforms suggest that some INVEST lessons learned may not have been shared widely • There is a strong desire to be able to easily access the various learning documents, exemplars, templates, and studies developed under INVEST buy-ins memorialization of key decisions, including rationales • Capture and act on lessons learned, even if they are not success stories, by creating a formal, documented process that can be tracked showing the challenge/solution, actions taken/not taken, etc. • Learn how stakeholders access and utilize learning and communication materials to better target MBIOs and partner network firms • Make all INVEST deliverables available for use in planning and future research efforts, unless there is a compelling and articulable reason not to do so • Three MBIO respondents viewed INVEST a way to view development problems with “fresh eyes” • Two MBIO respondents reported that working with INVEST had influenced their future • MBIOs and network partner firms viewed participation in INVEST as an experiment in finding new ways to conduct development activities EQ3: How has INVEST communicated to USAID and other stakeholders about its mobilization of private capital approach? • 4 USAID respondents noted the value of working directly with the INVEST team • 5 MBIO respondents and 9 partner firms viewed transparency in procurement processes as an area for improvement • 7 of the 17 partner firms, including 2 firms that worked on multiple buy-ins, questioned whether INVEST procurement processes resulted in outcomes that were consistently fair and transparent to contractors and partner network members • 8 of 17 firms reported INVEST’s treatment of intellectual property as a concern • 2 interviewees related stories of how they believed their intellectual property had been misused • 7 respondents from DAI, buy-in firms, and USAID were uncertain about what mechanisms INVEST has in place to address ethical or professional issues that could arise • INVEST’s most effective channel of communication about its mobilization of private capital approach occurred as part of direct interactions between DAI and USAID staff • INVEST has not effectively communicated critical information about aspects of its mobilization of private capital approach to key stakeholders • INVEST has not communicated with the partner network about how intellectual property would be protected or shared • Transaction advisors, partner firms, and others who participate in INVEST activities did not describe a clear understanding about the ethical and professional responsibilities associated with INVEST activities • Develop and distribute a policy or set of standards/values/ norms around intellectual property. Convene a webinar or other interactive session about the issue of working collaboratively with commercial competitors, and carefully consider the impact of relationships between firms within the partner network. • Develop and publish a clear explanation of how work is distributed via the INVEST mechanism, including a high￾level explanation of the procurement and contracting process and why some work may not be competed via the • One partner firm respondent noted that publicly available documents that align with broader donor community goals are an example of INVEST-donor communications. • There are few examples to demonstrate how INVEST’s mobilization of private capital approach was shared with donors, host-country counterparts, targeted beneficiaries, and others MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 99 Evaluation Question Findings Conclusions Recommendations • Two programmatic modifications were described during the 29 MBIO interviews: o 1 MBIO decided to adopt the partner network model and create a specialized niche network of regulatory experts o The Liberia Mission has reportedly set up an office of private enterprise and brought in a foreign service officer to support private enterprise work • 2 USAID respondents noted it was not necessarily possible to adapt new policies shortly after working with INVEST • MBIOs have largely not modified their programmatic or strategic efforts to integrate mobilization of private capital models RFP process. Make a listing of all issued RFPs available for network partners’ easy reference. • Determine where the responsibility lies for protecting all parties to investment transactions from inappropriate or conflicting duties or roles. Locate the “bold line” regarding INVEST practices and ensure that activity designs reflect good practices in management control. • Conduct an assessment of the technical quality and utility of INVEST’s technical products, considering factors such as the quality, depth, and precision of content and the utility of such products to technically sophisticated users in USAID and the development or financial/investment communities MID-TERM PERFORMANCE EVALUATION OF THE INVEST MECHANISM 100 ANNEX L: STATEMENT OF DIFFERENCES INVEST Midterm Evaluation Statement of Differences INVEST Midterm Evaluation Statement of Differences, October 28, 2020 PAGE 1 INVEST’s Procurement Process A rigorous, fair, compliant procurement process is INVEST’s highest priority. INVEST takes seriously the evaluation findings that members of the USAID Finance and Investment Network (“partner network” or “network”) would benefit from more visibility into our rigorous procurement processes. INVEST was designed to make it easier for the private sector to work with USAID by shouldering much of the administrative burden of government contracting, which can be complex and labor-intensive, especially for private-sector partners that are not traditional implementers. INVEST acknowledges that enhanced communication, without overburdening partners with information, will increase trust in the fairness and rigor of our procurement processes. The description of the INVEST procurement process used as the basis for evaluation findings and recommendations is based on the opinions of key informants rather than a review of the actual process. As noted, “The evaluation team did not conduct a procurement review; instead, these and other comments from interviewees represent their personal perspectives and speak to what and how INVEST communicated about its approach (p. 46).” For this reason, INVEST’s Statement of Differences is focused on providing a detailed description of the INVEST procurement process, the mechanism’s use of core partners, and our outreach and engagement with the USAID Finance and Investment Network. INVEST staff draft each technical procurement in the form of a Statement of Objectives (SOO), written in coordination with USAID Missions, Bureaus, and Independent Offices (MBIOs) that have buy-ins with INVEST. These SOOs are included with every solicitation released by INVEST as an accompaniment to the Request for Proposals (RFP) document. RFPs issued by INVEST detail the requirements to submit a compliant proposal, including the release date, due dates for questions and answers (Q&A), proposal submission deadline, estimated budget amount, and expected subcontract type. Most importantly, RFPs include detailed instructions for the formatting and structure of technical and cost proposals as well as the evaluation criteria by which proposals will be scored to make an award determination. The evaluation criteria are developed by INVEST and reviewed by the buy-in client prior to release to ensure that INVEST and the MBIO are in agreement that the criteria are clear and will result in the most responsive proposal receiving an award. RFPs also include a link to the contractual standard terms and conditions offerors will be subject to should their proposal be successful. 1 Throughout this document, a  symbol indicates that the mentioned resource or data point references a time period beyond the evaluation period. INVEST releases RFPs to the entire partner network and maintains a record of all recipients of INVEST solicitations. INVEST also supplements the standard RFP distribution list with additional relevant vendors identified through desk research to ensure INVEST is engaging the right partners in a particular country or technical area to meet the needs of each procurement. Every procurement includes a window of time for offerors to ask questions about the solicitation. This step is vital to ensure a level playing field by providing all offerors, in particular new and underutilized partners, with access to the same information. This allows INVEST to address everything from typos to substantive questions concerning the technical approach and cost parameters. For particularly complex or unique assignments, INVEST will hold bidders’ teleconferences in addition to a written Q&A; INVEST held seven bidders’ teleconferences in its first three years.1 The teleconferences and written INVEST Midterm Evaluation Statement of Differences, October 28, 2020 PAGE 2 Q&A period are designed to ensure that offerors’ questions are fully answered. All Q&As as well as transcripts from bidders’ teleconferences are released to the entire vendor list for each proposal without attribution to ensure that all bidders have access to the same information. After proposals are submitted, INVEST convenes a Technical Evaluation Committee (TEC) whose members are responsible for evaluating each proposal according to the evaluation criteria set forth in the RFP. TEC members identify the strengths and weaknesses of each proposal and determine which proposal is best suited to meet the objectives of the procurement and should be recommended to implement the Scope of Work. TECs are typically comprised of INVEST (DAI) technical staff, buy-in MBIO USAID representatives, and stakeholders with expertise relevant to the country and/or sector on which the RFP focuses. INVEST (DAI) always maintains a voting majority on TECs, given the contractual liability that falls to DAI as the prime implementer of INVEST. DAI takes full responsibility for all TEC outcomes and the compliance and transparency of the process. Participation by USAID MBIO staff and other stakeholders is critical to fruitful conversations during the evaluation process given their country or sector-based knowledge, their assistance in developing RFPs and SOOs, and the advisory role they will play during implementation of the award. INVEST adheres to all U.S. Government procurement regulations as set forth in the Federal Acquisition Regulation (FAR) and documents all aspects of the procurement process, proposal evaluation, award decision, and subcontract negotiations, in compliance with approved DAI processes and the FAR. Every voting member of the TEC receives an Excel scoresheet that mirrors the evaluation criteria included in the RFP. TEC members review every proposal independently, sending their completed scoresheets to a designated member of INVEST’s procurement team who compiles them before convening and facilitating a TEC panel discussion. The facilitator shares the average scores to help frame the TEC panel’s conversation and ensures that the discussion is limited to the evaluation criteria in the RFP and the information provided in submitted proposals. In most cases, the TEC makes a determination to advance one or more firms to competitive range, shortlisting the highest-rated proposals. Shortlisted firms are typically invited to a teleconference interview with the TEC, which the facilitator leads, inviting members of the TEC to ask follow-up questions as needed. If a large number of firms are shortlisted, written questions may be used instead, and the facilitator compiles and shares all responses with the TEC. After conducting interviews or receiving responses, the facilitator reconvenes voting members of the TEC to discuss the responses and make an award recommendation. When the TEC panel decides who to recommend for award, a summary of the process is sent to the INVEST Chief of Party for concurrence in the award decision. Following concurrence, the INVEST procurement team contacts the successful offeror to inform them of the decision and schedule an orientation call. INVEST uses these orientation calls to provide additional context on the assignment, walk the firm through required contractual inputs, outline the timeline and process to sign the subcontract, and answer any questions the organization has regarding DAI’s subcontract template. This template captures all essential flow-down clauses from DAI’s prime subcontract with USAID as well as all mandatory FAR language. Once all inputs are received, INVEST sends a consent request to the USAID Contracting Officer (CO) that has primary responsibility for INVEST. This step is only required if the subcontract is labor-based or its value exceeds the simplified acquisition threshold of $250,000. At the end of Year 3 (September 30, 2020), 43 of 44 CO subcontracting requests had been approved. (The only subcontract request that was not approved was ultimately cancelled due to changing circumstance in Afghanistan that made the work infeasible). Following the signing of each subcontract, INVEST offers debriefs to unsuccessful offerors who wish to learn which elements of their proposals were successful and which could be strengthened, to improve their chances in INVEST Midterm Evaluation Statement of Differences, October 28, 2020 PAGE 3 future procurements. By September 30, 2020, INVEST had sent 160 debrief letters to the 96 firms that requested them following notification of their unsuccessful offer.Since the start of the project, all debrief letters include the name of the firm or firms that were awarded for the sake of transparency. In response to recent partner feedback, INVEST has committed to making debriefs more substantive and timely, as well as including a note on the composition of the TEC and the ranking of the firm’s proposal in relation to all other proposals received. Key Statistics TEC Statistics for Years 1 and 2 (September 28, 2017 - September 30, 2019) o 28 TECs convened o 18 TECs convened with USAID/MBIO staff participating as voting members o 49 non-INVEST staff who participated in any capacity on TECs o 213 Non-Disclosure/Conflict of Interest certifications completed by all members of TECs (DAI, USAID/MBIO, Stakeholders) o 11 USAID Contracting Officer (CO) Consent Requests approved out of 12 submitted TEC Statistics for Years 1, 2, and 3+ (September 28, 2017-September 30, 2020) o 74 TECs convened o 56 TECs convened with USAID/MBIO staff participating as voting members o 137 non-INVEST staff who participated in any capacity on TECs o 604 Non-Disclosure/Conflict of Interest certifications completed by all members of TECs (DAI, USAID/MBIO, Stakeholders) o 43 CO Consent Requests approved out of 44 submitted o All 56 TECs that included USAID/MBIO representatives reached their award decisions unanimously INVEST Core Partner Utilization Policy INVEST was co-created through a BAA process and in the concept note and eventual proposal submitted during the co-creation process, DAI included three “core” or “institutional” partners. INVEST has a formal policy on the utilization of these core partners, vetted by DAI’s Home Office Contracts & Grants Department, that sets forth when and how core partners can be activated in support of INVEST buy-in activities. Activation means that INVEST releases an RFP only to a core partner rather than releasing and openly competing among the full partner network. The overall approach and the specific mechanics of the procurement process are the same for core partners and partner network members as outlined above. As instructed by USAID’s Office of Acquisitions and Assistance at the beginning of the INVEST contract, the preferred option is to issue INVEST RFPs to the full partner network by default. As of September 30, 2020, 85 percent of competitive INVEST RFPs had been released to the full partner network and an additional 23 RFPs were in open procurement, evaluation, or subcontracting that are not captured in the statistics, but which were released to the full partner network. INVEST typically releases RFPs to a core partner when: (1) the core partner’s defined scope as presented in the initial INVEST proposal is directly aligned with the INVEST requirement; and (2) USAID and INVEST have a very short timeframe to implement an activity, making it impossible to follow the regular procurement process, which takes 8-10 weeks on average from RFP release to award. Core partners may also be used in rare situations in which USAID identifies a procurement as politically sensitive and inappropriate for public release to the Partner INVEST Midterm Evaluation Statement of Differences, October 28, 2020 PAGE 4 Regardless of whether INVEST subcontracts to a core partner under the core partner utilization policy or releases a solicitation to the full Partner Network, INVEST releases an RFP and requires a full technical and cost proposal from each offeror. The same applies to any IQC (Indefinite Quantity Contract) subcontracts INVEST has with firms; IQCs are umbrella subcontracts that are also awarded competitively through procurements released to the full partner network. These umbrella subcontracts are valuable tools to manage risk in uncertain environments since work can be implemented in shorter timeframes with a more flexible, phased approach that allows for iterative co-design where needed. Every INVEST RFP notes the subcontract type (e.g. Firm Fixed Price or Time and Materials), whether the subcontract will be a single award or an IQC subcontract, and whether there will be one or multiple holders of the IQC. The only difference between IQCs and typical RFPs is that Requests for Task Order Proposals (RFTOPs, released under IQCs in lieu of RFPs) are not released to the entire Partner Network; they are only released to the holders of the umbrella subcontract. Even if the IQC has only a single holder, INVEST still releases a RFTOP to the firm and requires a technical and cost proposal to be submitted and evaluated prior to award. Key Statistics Subcontracting Statistics for Years 1 and 2 (September 28, 2017 - September 30, 2019) o INVEST subcontracted $9,869,972 o $7.53 million was awarded to firms through open competition o $0.98 million was awarded to core partners through open competition o $2.34 million was awarded to core partners through core partner activation o 21 unique firms were awarded Subcontracting Statistics for Years 1, 2, and 3+ (September 28, 2017-September 30, 2020) o INVEST subcontracted $38,944,584 o $34.43 million was awarded to firms through open competition o $3.65 million was awarded to core partners through open competition o $4.52 million was awarded to core partners through partner activation o 49 unique firms were awarded In the first three years of INVEST, 165 unique firms responded to technical procurements, an indication of the mechanism’s positive engagement with the partner network.+ Network. As members of the network, core partners are also invited to bid on all RFPs released to the Partner Network and have been awarded work through openly competed subcontracts for which they submitted the strongest proposal. In its first three years, INVEST activated core partners 19 times, in accordance with the policy. While core partners can be useful in terms of supporting thought leadership and rapid activation for new buy-ins, core partners are not privy to the inner workings of INVEST, do not share office space with INVEST, and do not attend planning meetings or participate in buy-in strategy development. Subcontracts awarded to core partners through activation represent only 12 percent of the total value of subcontracts awarded by INVEST. INVEST Midterm Evaluation Statement of Differences, October 28, 2020 PAGE 5 Partner Network Outreach and Engagement INVEST is committed to facilitating communication among the members of the network and with USAID to help build lasting relationships. In addition to engaging the network through the procurement process, INVEST holds biannual network webinars, sends communications to the network on behalf of other PSE mechanisms (such as CATALYZE), circulates news and updates related to USG policy, and releases calls for participation in conferences or events. INVEST also features partner activities through INVEST communication channels to amplify their voices and perspectives. The network receives updates, thought pieces, and partner spotlights through emails and platforms like the INVEST Partner Network newsletter and LinkedIn group. INVEST also has web-based resources publicly available to all partners. In response to feedback from network members in Years 1 and 2 of the program, and to promote transparency in the procurement process, INVEST published a Procurement Forecast website2 in October 2019. The website focuses exclusively on INVEST procurements and provides tools and information to help firms respond effectively to solicitations, including a forecast of upcoming RFPs to give firms advance notice of what will be released soon. The forecast includes a brief description of the scope of work and sector, geographic scope, estimated budget amount, and expected release month. The site also includes helpful resources like sample RFPs, a description of the INVEST Procurement Process, and resources with information on the evaluation process and advice for firms on presenting a compelling proposal in response to INVEST procurements.3 This website is publicly available, and INVEST highlights the website in its bi-annual partner network webinars, monthly newsletters that are sent to the partner network, and RFP release emails. The Procurement Forecast website is a key tool in ensuring transparency in INVEST procurements and distributing information to the partner network and other organizations interested in INVEST solicitations. The website received 6,745 pageviews between its launch in October 2019 and September 2020 and showed a considerable uptick in views between May 2020 and September 2020, increasing by 21.6 percent over the previous three months. In the same time period, the number of users rose 2.8 percent and session duration rose 44 percent. The trust and confidence of partners is paramount to the success of the INVEST mechanism and to securing high-quality proposals from a set of the most qualified firms. INVEST complies with its Data Protection Policy for the USAID Finance and Investment Network, which covers the collection and storage of partner data, use and sharing of partner data, and safeguarding of technical contributions from partners. Access to submitted proposals for the purpose of technical evaluation is restricted to those who sign conflict of interest (COI) certifications and affirm adherence to non-disclosure of information requirements specific to each procurement. The larger INVEST team and DAI corporate staff do not have access to proposals submitted in response to INVEST procurements. At no point in the life of INVEST will any partner procurement information be made available beyond those few reviewers covered by the COI certification and non-disclosure stipulations and the operations team responsible for subcontracting. All proposals are submitted to the INVEST_Procurement@dai.com email box which only members of the INVEST Procurement team can access. All submitted proposal materials are saved in a restricted access electronic database that only the same INVEST Procurement team can access. The Procurement team then prepares a non-disclosure affirmation and COI certification which names all firms that submitted a response. The non￾disclosure language affirms that reviewers do not share proprietary proposal materials, and the COI certification requires reviewers to disclose any conflicts of interest with submitting organizations. Every TEC member must certify that they will not share the materials with any party, including a prohibition of sharing proposals with the larger INVEST team members or DAI staff not participating in the TEC. Only after the form is received do TEC participants receive access to the proposals through an electronic drop box. 2 https://invest-procurement.com 3 https://invest-procurement.com/2019/09/27/tips-for-stronger-proposals.html INVEST Midterm Evaluation Statement of Differences, October 28, 2020 PAGE 6 Conclusion In the final years of the project, INVEST will work in close collaboration with USAID to increase INVEST’s impact, share lessons learned, and maximize value to USAID and the blended finance community. INVEST will improve the Procurement Forecast website by uploading two key policies. The first is the Core Partner Utilization Policy, which covers when and how core partners are activated and their role on the project, as well as their ability to submit proposals in response to openly competed RFPs. The second is the Data Protection Policy for the USAID Finance and Investment Network, covering intellectual property and how proposals and information received from partners are handled. The website will also include a list of all live procurements and documentation, a list of all awardees made under each INVEST procurement, and a process chart describing INVEST’s procurement procedures. All award information is already available here through mandated Federal Funding Accountability and Transparency Act (FFATA) reporting. However, by making this information more easily accessible for partners, INVEST will ensure greater transparency and confidence in the fair and compliant procurement processes INVEST follows. INVEST will ensure partners are aware of these resources – highlighting them in channels such as the Network newsletter, LinkedIn Group, and webinars – so that all understand the government regulations INVEST follows and the processes that protect members’ intellectual property. INVEST will also intensify its learning activities and enhance two-way communication with the partner network on INVEST procurement processes and lessons learned across the portfolio. While INVEST published more than 85 learning products in its first three years – including case studies, articles, and reports – INVEST will dedicate even more resources to harvest evidence and learning in the final years of the project as activities mature and lessons emerge. INVEST recently released an RFP for a robust learning platform and evidence and learning roadmap for the second half of the project. INVEST reviews its MEL plan at the start of each fiscal year, and this year it will do so with an eye to intensifying the rigor of data collection at the activity and project levels. With a continued focus on a strong partner network, streamlined and rigorous procurement, and useful research and learning, INVEST will continue to support USAID's efforts to mobilize private capital effectively for better, more sustainable development results. Kristi R. Ragan DAI Global, LLC INVEST Chief of Party October 28, 2020 INVEST and USAID develop solicitation and scoresheet with evaluation criteria. Offerors submit proposals. (Network membership is not required to submit a proposal.) INVEST releases solicitation to: A Core Partner For rapid activation according to Core Partner Usage Policy. Used Rarely. INVEST and USAID receive and answer questions from offerors and send answers to all potential bidders. Full Network Plus Vendors Identified via Research For procurements requiring specialized capabilities. Used frequently. USAID INVEST Procurement Process Map Stage 1: Solicitation Facilitator coordinates with offerors to answer written clarification questions or have a phone interview with the TEC panel. TEC reconvenes to review short-list responses and discuss tradeoffs. INVEST procurement facilitator identifies Technical Evaluation Committee (TEC) of INVEST and USAID staff. INVEST holds the majority on all TECs. Select Successful Offeror(s) TEC agrees on the best firm or consortium, making a unanimous best value determination Cancel or Reissue the RFP (rare) The TEC panel decides to: All TEC members certify they have no conflicts of interest and will adhere to non-disclosure requirements. TEC members receive technical proposals and score proposals individually according to RFP evaluation criteria Facilitator compiles all scores. Facilitator convenes a TEC panel discussion on technical proposals, limiting discussion to pre-established selection criteria from the RFP. Identify a Shortlist (most common outcome) USAID INVEST Procurement Process Map Stage 2: Evaluation USAID INVEST Procurement Process Map Stage 3: Subcontracting SOW and contracting paperwork finalized. Subcontract is fully executed. INVEST notifies unsuccessful offerors and provides evaluation debrief letters upon request. Technical kickoff call INVEST sends request for consent to subcontract to USAID Contracting Officer. Facilitator sends award decision to Chief of Party for concurrence. Implementation begins. Is subcontract over $250,000 or labor-based? Facilitator notifies successful offeror and sends required contracting paperwork, including compliance documentation, draft SOW, and draft subcontract template. INVEST holds orientation call on subcontracting process and compliance requirements No Yes