BELARUS I3 PERFORMANCE EVALUATION December 2020 This publication is made possible by the support of the American People through the United States Agency for International Development (USAID) and was prepared by Integra Government Services International LLC for the Learning, Evaluation, and Analysis Project (LEAP III) Activity. 2 LEAP III FINAL EVALUATION REPORT BELARUS INCREASING PRIVATE SECTOR COMPETITIVENESS THROUGH IMPROVING THE ENABLING ENVIRONMENT, IMPROVING MANAGEMENT AND OPERATIONAL CAPACITY, AND INCREASING ACCESS TO FINANCE (I3) PERFORMANCE EVALUATION Contract Title: LEAP III: Learning, Evaluation, and Analysis Project Contract Number: GS-10F-083CA / 7200AA18M0004 Activity Number: 1009.1028 Submitted: December 29, 2020 Contractor: Integra Government Services International LLC 1156 15th Street NW, Suite 800 Washington, DC 20005 USAID Office: USAID/Belarus Prepared by: Dr. Lyubov Palyvoda, Mick Mullay, and Ganyapak (Pin) Thanesnant DISCLAIMER This report is made possible by the support of the American people through USAID. The authors’ views expressed in this publication do not necessarily reflect the views of USAID or the United States Government. TABLE OF CONTENTS ACRONYM LIST 4 ACKNOWLEDGEMENTS 5 EXECUTIVE SUMMARY 7 I. INTRODUCTION 10 2. METHODOLOGY 14 3. EVALUATION FINDINGS 21 4. CONCLUSION 52 REFERENCES 57 ANNEX 1: STATEMENT OF WORK 59 ANNEX II: EVALUATION DESIGN MATRIX 73 ANNEX III: DATA COLLECTION TOOLS 75 ANNEX IV: LIST OF INTERVIEWS 99 ANNEX V: ONLINE SURVEY RESPONSE DETAILS 101 4 ACRONYM LIST AID-VENTURE Facilitating Access to Venture Funding in Belarus AOR Agreement Officer Representatives BBAN Belarus Business Angels Association BEROC Belarus Economic Research and Outreach Center BIF Belarus Innovation Fund CEE Center of Excellence in Entrepreneurship CLA Collaborating, Learning, and Adapting COP Chief of Party COR Contracting Officer Representative COVID-19 Coronavirus Disease 2019 CSISS Creating Sustainable Infrastructure to Support Start-Ups and Small Business Development in the Regions of Belarus DCOP Deputy Chief of Party DO Development Objective DRIVES Delivering Regional Instruction Vital to Entrepreneurial Success EAEU Eurasian Economic Union EBRD European Bank for Reconstruction and Development EQ Evaluation Question ET Evaluation Team EU European Union EWP Evaluation Work Plan FGD Focus Group Discussion GBSN Global Business School Network GDP Gross Domestic Product GESI Gender Equity and Vulnerable Groups/Social Inclusion GOB Government of Belarus I3 Increasing Private sector Competitiveness through Improving the Enabling Environment, Improving Management and Operational Capacity, and Increasing Access to Finance Project IFC International Finance Corporation IR Intermedia Result 5 IT Information Technology KII Key Informant Interviews LEAP III Learning, Evaluation, and Analysis III Project LED Local Economic Development MEL Monitoring, Evaluation, and Learning NGOs Nongovernmental Organizations NUPAS Non-U.S. Organization Pre-Award Survey PSE Private Sector Engagement R&D Research and Development SIBS Society of Innovative Business Support SME Small and Medium Enterprise SOE State-owned enterprise SOW Scope of Work TA Technical Assistance TOT Train the Trainer USAID United States Agency for International Development USD United States Dollar USG United States Government UNDP United Nations Development Programme VC Venture Capital ACKNOWLEDGEMENTS The authors would like to acknowledge and thank the input and comments from numerous respondents and beneficiaries of the I3 project during the evaluation, particularly those who shared their time and experiences during remote data collection and during the preparation of this paper. The authors would like to thank I3 AID￾VENTURE, DRIVES, and CSISS Activity Managers for providing information and support on this evaluation. The authors would also like to thank the following key informants for their important contributions during the evaluation, namely: Valentina Sidorkina and Nataliya Khalanskaya (CSISS), Dmitryi Matveev and Kiril Golub (AID￾VENTURE), Anna Perunova and Maryna Naydovych (DRIVES). The team would like to acknowledge USAID/Belarus for their time and efforts in providing sufficient project documents for review during this evaluation. The team would also like to thank Ms. Svetlana Zinkevich, the local Belarus consultant, for which remote data collection would not have gone so smoothly. Lastly, the team would like to acknowledge Ms. Liesl Kim, LEAP III Associate, for editorial and formatting support of this report. 7 EXECUTIVE SUMMARY INTRODUCTION The United States Agency for International Development (USAID)/Belarus requested the Learning, Evaluation, and Analysis Project III (LEAP III) team1 to conduct a performance evaluation of the Increasing Private Sector Competitiveness through Improving the Enabling Environment, Improving Management and Operational Capacity, and Increasing Access to Finance (I3) project, implemented by several local Belarusian and American implementing partners. The evaluation looks at the performance of three of the project’s ongoing activities intended to contribute to private sector development in Belarus: Facilitating Access to Venture Funding in Belarus (AID-VENTURE), Delivering Regional Instruction Vital to Entrepreneurial Success — Belarus (DRIVES), and Creating Sustainable Infrastructure to Support Start-Ups and Small Business Development in the Regions of Belarus (CSISS). Ultimately, the evaluation will outline actionable recommendations to USAID/Belarus about any necessary adjustments to the implementation of the selected activities and future programming needs and approaches. In particular, the evaluation will answer the following five evaluation questions (EQs): 1. Have the I3 project activities contributed to increasing the competitiveness of small and medium enterprises (SMEs) in Belarus? 2. Have the current modes of implementation of AID-VENTURE, DRIVES, and CSISS activities been efficient in achieving respective project goals? What works well and what doesn’t? What unintended effects have resulted to date from the project approaches, tools, and activities? 3. How organizationally efficient are the AID-VENTURE, DRIVES, and CSISS implementing partners? 4. How successful have implementing partners been in their private sector engagement (PSE) efforts? What PSE approaches can be scaled up? 5. Are there new, emerging entrepreneurship development needs to be addressed in Belarus? METHODOLOGY The Evaluation Team (ET) used a mixed-methods approach, combining qualitative, in-depth remote interviews2 with key stakeholders, project partners, and informants paired with online surveys and group interviews to answer the EQs focused on specific interventions, such as entrepreneurship promotion, private investment expansion, business training, and building sustainability of activities and organizations. All proposed semi￾structured interviews and focus group discussions, as well as the mini surveys, are organized around the EQs and supported with detailed questionnaires. Each questionnaire (in Annex III) was developed for each specific group of interviewees and includes both common questions and questions unique to the group to obtain a full range of opinions regarding specific projects but also to ensure that data is comparable across all respondent groups. 1 Implemented by Integra Government Services International LLC (‘Integra’) – www.integrallc.com. 2 The interviews were conducted remotely, due to the COVID-19 pandemic. I3 ACTIVITY FINDINGS Based on the evaluation, the three project activities evaluated were properly designed, relevant, and delivered assistance aligned with beneficiaries’ needs. Current modes of implementation used by AID-VENTURE, DRIVES, and CSISS have been efficient in achieving project goals and creating a supportive ecosystem for aspiring entrepreneurs, start-ups, SMEs, and local investors. Each of the I3 activities successfully contributed to strengthening various aspects of the economic ecosystem needed for SMEs and start-ups to launch, grow, increase employment, and become competitive. More detailed findings are outlined below and discussed throughout this report. Source: Authors’ representation ACTIVITY NAME KEY FINDINGS AID-VENTURE ● Contributed to the improvement of legislative and regulatory frameworks conducive to angel investment and venture funding. ● Launched productive policy dialogue with national authorities by increasing their attention and interest in venture funding. ● Assisted in building the local community of private venture investors and strengthening their capacity—resulting in improved relationships between local start-ups and investors. ● Created communities of local investors by establishing and institutionalizing the Belarus Business Angel Association (BBAN). ● Provided access for local investors to quality training, international experts and expertise, networking opportunities, study trips, and partnership opportunities with foreign investors. ● Advanced start-ups’ capacity to attract and work with local and foreign investors through matching events, such as forums, venture days, and promotion of Global Entrepreneurship Week. Facilitated cooperation between start-ups and investors, provided mentorship, and guided investments. ƒ² .p>]Sox >_I >GGKmmSFS]Sox oa FpmS_Kmm KIpG>oSa_ „² _GlK>mKI >ooK_oSa_¸mpiialo oa mo>loÄpim­ S_G]pIS_Q xap_Q FpmS_KmmKm¸K_olKilK_KplmRSim …² pmS_Kmm K_>F]S_Q K_uSla_^K_o >_I ]KQSm]>oSa_¸lKQp]>oSa_ †² _GlK>mKI >ooK_oSa_¸mpiialo oa va^K_Ä K_olKilK_Kplm 9 ACTIVITY NAME KEY FINDINGS DRIVES ● Provided systematic, needs-based, access to practical knowledge and skills necessary to start and grow a business, including at the regional level. ● Five regional Centers of Excellence in Entrepreneurship (CEEs) were established to provide access to entrepreneurship support services and business education. ● Raised recognition of the value of business education in regions outside of Minsk, while increasing access to high quality instructors and business experts simultaneously. ● Improved understanding of the local context and regional differences important for marketing and delivering of business courses. CSISS ● Improved overall public image of, attitudes toward, and awareness of the benefits of being an entrepreneur. ● Increased entrepreneurial activity in regions by transferring expertise of Society of Innovative Business Support (SIBS) in start-up and SME support to local partners. ● Supported new businesses by validating ideas and assisting in developing business projects. ● Built entrepreneurship and leadership skills through informal education of high school students. ● Created a network of local organizers of start-up support activities. ● Supported complementary local initiatives and women’s entrepreneur clubs in regions. ● Made well-developed and practical business trainings and materials (i.e., how-to guidelines to conduct start up school and expert days in Russian) accessible to a broader audience. The biggest unexpected challenge for the three USAID-supported activities was the COVID-19 pandemic in spring of 2020. However, evaluation participants also noted that all three activities migrated and adapted their work to online channels when the COVID-19 pandemic began. The evaluation confirms that technology not only helped prepare implementers for impact of COVID-19 but also allowed efficient training outreach to the regions and was efficiently leveraged for other events as well. 10 I. INTRODUCTION 1.1 EVALUATION PURPOSE The purpose of this evaluation is to assess the relevance, effectiveness, and efficiency of AID-VENTURE, DRIVES, and CSISS activities focused on entrepreneurship development and entrepreneurship funding in Belarus. In particular, the evaluation will answer the following EQs: 1. Have the I3 project activities contributed to increasing the competitiveness of small and medium enterprises (SMEs) in Belarus? 2. Have the current modes of implementation of AID-VENTURE, DRIVES and CSISS activities been efficient in achieving respective project goals? What works well and what doesn’t? What unintended effects have resulted to date from the project approaches, tools, and activities? 3. How organizationally efficient are the AID-VENTURE, DRIVES, and CSISS implementing partners? 4. How successful have implementing partners been in their private sector engagement (PSE) efforts? What PSE approaches can be scaled up? 5. Are there new, emerging entrepreneurship development needs in Belarus to be addressed? In particular, the evaluation was to (or intended to): 1) assess the relevance, effectiveness, and efficiency of the aforementioned three selected activities3; 2) analyze the status of the activities in relation to the set objectives and the activities’ successes and weaknesses; 3) assess the implementing organization’s performance, including the identification of any external factors that might have impacted activity performance; 4) provide recommendations on adjustments needed to the implementation of the selected activities; and 5) inform the United States Agency for International Development (USAID)/Belarus on future programming needs and approaches, in particular, on the design of the follow-on private sector development project in 2021. Based on the findings, the ET has outlined key recommendations in the Conclusion section of this report. These recommendations are drawn from the key findings of each EQ, looking at opportunities for change and possible adjustments that could not only enhance the current activities but also for USAID/Belarus to consider in planning for future activities. The recommendations are presented looking at potential areas of intervention based on the need and country situation, classified as either high, moderate, or low priority. 3 For evaluation purposes, “relevance” is a measure of the pertinence of a particular activity to project objectives, “effectiveness” is a measure of the ability of a particular activity to produce a planned effect or result that can be measured, and “efficiency” is a measure of the ability of an activity to use the minimum amount of project resources to achieve the desired results. 11 1.2 PROJECT BACKGROUND PRIVATE SECTOR IN BELARUS In recent years, the Government of Belarus (GOB) has been increasing its efforts to promote private sector development, strengthen the position of SMEs, and sustain growth, employment, and resilience. Belarus has pursued a gradual transition path, characterized by limited structural reforms and a modest expansion of the private sector. Currently positioned 49th globally in the World Bank’s Doing Business rankings, the business climate in Belarus has been improving steadily over the past five years. The report reflects positive changes in the indicators such as Starting a Business (reduced registration time); Dealing with Construction Permit (reduced permit registration time); Getting Electricity (reduced connection cost); Paying Taxes (reduced time to pay taxes); Resolving Insolvency (increased recovery coefficient rate); and Trade Across Borders (reduced cost of border and customs controls in export operations and the time of border and customs controls in import operations). For the past several years, Belarus has had the best conditions for trade across borders in the Eurasian Economic Union (EAEU), ranking 24th. Of the various topics that the Doing Business report covers, Belarus is quite advanced in property registration (ranked 14th globally), although it lags behind in paying taxes and getting credit, where it is ranked 99th and 104th in 2020 among the 190 countries, a significant drop in rankings over the last two years. Compared to the previous report from 2019, however, Belarus was down in seven out of ten positions in the rankings, remained the same on two, and was up on one indicator. It should be noted that Belarus' downward movement in the Doing Business 2020 rankings, however, was largely due to the dynamics of other countries, which were more active in improving their business environment across 10 indicators in 2019.4 SMEs in Belarus account for around one-fifth of gross domestic product (GDP), below the European Union (EU) average, with small privatized companies struggling to flourish. The share of SMEs in the country’s GDP is 24.6 percent. The largest proportion of SMEs is located in the capital city of Minsk and the greater Minsk region, jointly accounting for more than one-half of all SMEs in the country. Large, primarily state-owned enterprises remain the principal employers and economic mainstays for oblast and district-level towns. Government policies in Belarus have recently focused on upgrading state-owned enterprises (SOEs), the foundation of the Belarusian economy, with the help of subsidies and preferential loans through state-owned banks. SMEs’ contribution to added value and employment in the business sector remains limited, as SOEs continue to account for the majority of the nation’s income and employment. The economy of Belarus is export-oriented—approximately 70 percent of its GDP is exported, mainly petrochemicals, potash fertilizers, and agricultural and other heavy machinery, with high dependency on the Russian market for exports. In contrast, new private sector companies in high-tech industries have been burgeoning in Belarus, mainly in information technology (IT), over the past decade. Despite its modest share of the Belarus GDP (3.6 percent, equivalent to $2 billion USD), the industry is rapidly expanding and becoming a role model for the economy. Belarus is known as having the largest IT cluster in Central and Eastern Europe due to Presidential Decrees on establishment of the Hi-Tech Park and the development of the digital economy, along with substantially simplified 4 Economy Ministry comments on Belarus' position on Doing Business 2020 - https://eng.belta.by/economics/view/economy-ministry-comments-on￾belarus-position-on-doing-business-2020-125320-2019/#:~:text=Belarus%20ranked%2049th%20out%20of,the%20ease%20of%20doing%20business. 12 regulations for doing business. It includes provisions for minimizing state interference in business operations, presuming the good faith of enterprises, and banning the introduction of new taxes until 2020. USAID/Belarus has aimed to capitalize on these areas for private sector growth in particular, supporting entrepreneurship development in technology sectors and helping link Belarusian start-ups to international investors and markets. USAID was the first donor in the country that provided targeted and strategic support to entrepreneurship in Belarus, and it remains a key donor in this field. USAID’s support for entrepreneurship is further discussed as part of the I3 project intermediate results (IRs)—2.2 and 2.3. I3 PROJECT USAID’s history of economic programming in Belarus dates to the mid-1990s and early 2000s, when assistance was provided to support small-scale privatization, the development of business associations, and the introduction of international accounting standards. Then, in the early 2000s, USAID helped develop the first business school in Belarus. For the next decade, until 2010, USAID did not provide any notable private sector development support in Belarus, with the exception of business advocacy. In 2010, USAID resumed and significantly increased private sector support activities, as it became a distinct U.S. Government (USG) foreign assistance priority. In addition, over the past years, the GOB has gradually changed its position vis-à-vis the private sector and now expresses readiness to support its development in public policy. The combination of these two critical shifts has created a window of opportunity for USAID to engage substantively in private sector development in Belarus. In 2015, USAID/Belarus designed the five-year I3 project with the purpose of increasing private sector competitiveness and thus contributing to private sector growth in Belarus. On the project level, the targeted I3 activity results are: 1) expanded access to venture funding, contributions to the growth of start-ups, and expansion of the role of the private sector in the economy; 2) increased access to and quality of business education throughout Belarus; and 3) the expanded and strengthened network of local organizers of start-up development activities and system of informal entrepreneurship education and business career orientation for school children. During the I3 project design process, the following three Intermediary Results (IRs) were determined to be critical to achieving the project purpose and USAID’s Development Objective (DO) of increased private sector competitiveness: IR 2.1 – Improved Enabling Environment IR 2.2 – Improved Management and Operational Capacity IR 2.3 – Increased Access to Finance USAID/Belarus recently discontinued targeted activities supporting IR 2.1 – Improved Enabling Environment, as it achieved noticeable progress in this area as shown by several laws and policies adopted and improved international ratings, and left this objective as cross-cutting. The portfolio now focuses on the two remaining IRs: IR 2.2 – Improved Management and Operational Capacity and IR 2.3 – Increased Access to Finance. The main activities under this evaluation are as follows: DRIVES, AID-VENTURE, and CSISS. DRIVES and CSISS contribute to IR 2.2. AID-VENTURE is grouped under IR 2.3 but also contributes to IRs 2.2 and 2.1. 13 ACTIVITIES TARGETED BY THE PERFORMANCE EVALUATION Table 1 below outlines brief descriptions of each of the activities’ objectives and intended results. TABLE 1: ACTIVITY OBJECTIVES ACTIVITY NAME ACTIVITY OBJECTIVES INTENDED RESULTS AID-VENTURE ● Contribute to the creation of the legal and regulatory framework conducive to venture funding. ● Expand the availability of venture capital to help build a local community of private venture investors and integrate Belarus into the international venture funding ecosystem. ● Build capacity for venture fund creation. ● Strengthen connections between demand and supply—bring investors closer to start-ups. 1. Legal and regulatory framework conducive to venture funding created and strong community of local venture investors established. 2. Capacity of the involved organizations strengthened, enabling them to continue the venture funding support activities beyond the life of USAID support. 3. Venture investments in Belarus, local, regional, and global, expanded. 4. One or more venture funds established in Belarus. DRIVES ● Build the capacity of institutions in Belarus to provide relevant and current entrepreneurship courses; ● Increase educational opportunities that are geographically and financially accessible to Belarusian entrepreneurs, especially outside of Minsk, to increase entrepreneurial activity and the number of new businesses in Belarus. ● Provide opportunities for young entrepreneurs to network, share business plans, and be mentored by fellow entrepreneurs. 1. IPM Business School faculty demonstrates improved capacity and knowledge of technologies and current business subjects; 2. Businesses throughout Belarus have increased access to modern, high-quality business education resources; and 3. Support networks that encourage participation in entrepreneurial activities are available in the regions of Belarus. CSISS ● Create a network of local organizers of start-up development activities. ● Build a sustainable system of promotion, cultivation, training and expertise for start-ups and small businesses. 1. The infrastructure for start-up and small business support represented by the network of Start-Up Schools and Expert Days is strengthened and expanded. 2. Public image and awareness of entrepreneurship improves. 3. Entrepreneurship activity in the regions of Belarus grows. 4. School children improve understanding of entrepreneurship and leadership skills. 5. Popularity of entrepreneurship as a career option grows. 14 Source: USAID Activity Program Documents 2. METHODOLOGY 2.1 EVALUATION QUESTIONS This evaluation assesses the relevance, effectiveness and efficiency of AID-VENTURE, DRIVES, and CSISS interventions focused on entrepreneurship development and entrepreneurship funding in Belarus. To accomplish this task, the ET has developed an evaluation approach to address a set of five EQs outlined in the Evaluation scope of work (SOW) (Annex 1). Responses to these five questions (Table 2) have been informed by thorough data collection protocols that expand on, and are supported by, more detailed lines of inquiry. TABLE 2: EVALUATION QUESTIONS EQ1: Have the I3 project activities contributed to increasing the competitiveness of SMEs in Belarus? EQ2: Have the current modes of implementation of the AID-VENTURE, DRIVES, and CSISS activities been efficient in achieving respective project goals? What works well and what doesn’t? What unintended effects have resulted to date from the project approaches, tools, and activities? EQ3: How organizationally efficient are the AID-VENTURE, DRIVES, and CSISS implementing partners?5 EQ4: How successful have implementing partners been in their PSE efforts? What PSE approaches can be scaled up? EQ5: Are there new, emerging entrepreneurship development needs in Belarus to be addressed? Source: Evaluation Scope of Work The evaluation covered three activities from their start-up until the evaluation was conducted in July 2020. These timelines are shown below: AID-VENTURE: October 2016 – July 2020 DRIVES: September 2014 – July 2020 CSISS: October 2017 – July 2020 The evaluation also considered geographic distribution of projects’ activities, examining the capital city of Minsk and the other oblasts that contain project participants. In answering evaluation questions, the ET will highlight gender-specific and social inclusiveness approaches promoted by AID-VENTURE, DRIVES, and CSISS, where relevant. 5 The evaluation team should analyze management of the activities: implementing partners’ efficiency in planning interventions; reporting to USAID; communicating with stakeholders; taking mitigation measures promptly when implementation issues arise; collaborating, learning, and adapting; taking leadership positions in the respective sectors, etc. 15 2.2 EVALUATION METHODS The ET consisted of the following core team members: Dr. Lyuba Palyvoda, Team Lead; Mr. Mick Mullay, Private Sector Engagement Expert; Ms. Svetlana Zinkevich, Local Facilitator/Coordinator; and Ms. Ganyapak Thanesnant, Project Manager and Evaluation Specialist. The ET used a mixed-methods approach, combining qualitative key informant interviews (KIIs), focus group discussions (FGDs), an online survey, event observation, and a desk review. This approach reflects USAID’s parallel combinations approach, in which two or more different methods are used to collect and analyze information that is then synthesized to answer individual evaluation questions. 2.3 OVERVIEW OF REMOTE DATA COLLECTION Due to the Coronavirus Disease 2019 (COVID-19) pandemic, the ET agreed with USAID/Belarus that all data collection would be done remotely.6 The ET conducted remote data collection from July 13, 2020 to August 26, 2020. Telephone and internet-based options were used to carry out all interviews. Most interviews were conducted through online platforms, such as Zoom and Google Hangouts, but the Telegram channel7 was used as well. The ET was assisted by a local consultant in Minsk to help set up and follow up on interviews, especially in rural areas. Key informants were selected from the following groups of stakeholders, counterparts, and beneficiaries:8 A. USAID/Belarus staff, including the Agreement Officer Representatives (AORs). B. AID-VENTURE, DRIVES, and CSISS and any other relevant project staff, as well as project beneficiaries. C. Donors and international partners (i.e., the U.S. Embassy, the International Finance Corporation (IFC), European Bank for Reconstruction and Development (EBRD), United Nations Development Programme (UNDP), and others). D. State entities, including the Belarus Innovation Fund (BIF) and the Bank of Development of the Republic of Belarus. E. Private businesses, including Belarus Business Angel Association (BBAN), Zubr Capital Private Equity Fund, private companies supporting start-up competitions in the framework of DRIVES activity, and the like. F. Think Tanks, including the Belarus Economic Research and Outreach Center (BEROC) and the IPM Research Center. In addition to the KIIs, the ET sent out an online survey to all I3 beneficiaries from July 20, 2020 to August 17, 2020. 6 Due to the situation in Belarus during the time of the evaluation, the team faced some difficulties in receiving feedback from stakeholders and internet connectivity issues. Thus, the period of remote data collection was extended. More details on the difficulties of remote data collection are presented in the Limitations Section below. 7 During the first days after the country's presidential election, when the entire country went nearly completely offline, the Polish-based Telegram messenger service was one of the only sources of information available for use. 8 A final list of interviewed stakeholders may be found in Annex IV., which has been redacted for the public version of this document 16 SECONDARY DATA The ET conducted extensive desk review of key project and external documents, including secondary data and background documents (i.e., relevant academic, periodical publications, and other donors’ reports; project surveys and monitoring and implementation plans; and quarterly and annual reports, etc.). These documents provided a deep dive into the development context, challenges and priorities; relevant national laws, policies, and regulations; as well as insight into the business environment and entrepreneurship development in Belarus. The purpose of the review was to provide an introduction to the key activities and to understand private sector competitiveness to situate the evaluation. The team worked with USAID/Belarus in advance to retrieve project documents, including the initial task order, annual (and quarterly, where applicable) performance reports, business development survey data, activity work plans, and others. QUALITATIVE DATA COLLECTION A total of 55 KIIs were carried out with I3 implementing organizations and their partners, direct beneficiaries at national and regional levels, including local partners of implementing organizations (local organizers, local networks of entrepreneurs, BBAN, etc.); participants in projects activities (individuals, local investors, start-up school/junior participants, SMEs and entrepreneurs in the regions, young entrepreneurs and women entrepreneurs, winners of local initiatives and business plan competitions, etc.); faculties, teachers, instructors, business experts trained and/or engaged to educate/mentor/consult SMEs and start-ups; state entities (national and local); donors and international partners; and economic think-tanks. The ET prioritized the main groups of stakeholders and beneficiaries most representative of the public, civic, and private sectors across all regions of Belarus. Details of interviewees’ distribution by gender and oblast are provided below. Figure 1: Key Informant Interviews by Gender and Oblast Source: Authors’ representation 17 ONLINE SURVEY To understand the current state of I3 activity, the ET developed an extensive survey instrument. The survey explored the current state of private enterprise development in Belarus, changes occurring in the last five years, factors influencing and hindering SMEs competitiveness, effectiveness and efficiency of the I3 implementation modes, implementing partners’ organizational effectiveness, the status of PSE in I3 activities, and emerging entrepreneurship development needs. The ET used the existing I3 activities’ databases supplied by each of I3 project managers to establish the sampling frame. In total, online survey invitations were sent to all 9,180 beneficiaries, but only 50 responses were received. A low response rate was unfortunately expected and was discussed in advance of remote data collection, due to the timing of the evaluation (i.e., during the summer months) and to the ongoing political climate, which resulted in arrests of candidates and protestors. Consequently, people were especially wary about responding to an online survey collecting information about business activities, even though it was sent by a USAID contractor. More on this is discussed in the Limitations section of the report. With a response rate of .005 percent, however, the ET believes it is necessary to more closely examine the breakdown of respondents to assess the usefulness of survey responses. Although the summer holiday would be likely to affect all groups fairly equally, it is possible that a low response rate due to political unrest could affect some categories of respondents more than others, therefore introducing selection bias. The distribution of survey invitations sent and received by different I3 activities is presented in Table 3 below, broken down by intervention type. Annex V further details the breakdown of online survey respondents. Note that the numbers of respondents (below) by category is greater than invitations sent, because some respondents self-identified as participating in more than one of the project interventions. Here, N=132, even though there were only 50 responses received. This still makes it difficult to determine the representativeness of those who responded based on category of intervention, but in geographic and gender terms, responses do reflect the demographics of the overall population of beneficiaries for all three activities. Unfortunately, it was not possible to collect more detailed information about survey respondents to compare samples, because that would have further reduced the likelihood of receiving responses in general. TABLE 3: SURVEY RESPONSE RATE BY INTERVENTION INTERVENTION TYPE # OF SURVEY INVITATIONS SENT # OF SURVEY RESPONSES RECEIVED RESPONSE RATE Training of trainers 45 5 11.1% Training 8,305 23 0.3% Education program 203 35 17.2% Local initiatives support9 9 16 177.8% Networking events 523 27 5.2% Consultations 80 20 25% 9 These initiatives are supported by the CSISS activity – there were 10 initiatives in total. 18 TABLE 3: SURVEY RESPONSE RATE BY INTERVENTION INTERVENTION TYPE # OF SURVEY INVITATIONS SENT # OF SURVEY RESPONSES RECEIVED RESPONSE RATE Acceleration program 15 6 40% Total 9,180 132 1.4% Source: Authors’ representation DIRECT OBSERVATION Based on discussions with USAID/Belarus, the ET also took part in the partners’ online events planned for the period of this evaluation to assess the quality of services or training provided, event organization, beneficiaries’ skills, and the like. Specifically, the ET observed the following during the time of the evaluation: ● A start-up competition organized by IPM, the flagship business school in Belarus, within the DRIVES activity on June 19, 2020. During this online event, selected start-ups had time to present their ideas and the selection committee, composed of IPM faculty, entrepreneurs, and business consultants, had an opportunity to clarify details and get additional information from the presenters. ● A presentation of the TechMinsk Accelerator Summer Batch by the TechMinsk team and consultants (Dima Sarle, Will Cardwell) on June 23, 2020. The event served as a way to provide background information to potential applicants and answer questions on the TechMinsk Accelerator Summer Batch. The event was also looking for Summer Batch teams who will be going through an intensive two-week online program planned for July 21-31, 2020. Selected start-ups will receive up to $50,000 of investment and acceleration, including through 1:1 mentoring sessions with top-notch experts. ● A presentation of the results of a recent Belarus start-up survey on the impact of COVID-19. This event discussed the influence on income of firms and employment, what measures have been taken by companies and what measures entrepreneurs expect from the government. The presentation was then concluded by listing steps for businesses and government. ● Selected AID-VENTURE activity events recordings available at YouTube (i.e., B-Venture, Business Angels Meetups & Camp, Business Angels Academy, Imaguru Start-Up Sprint, Imaguru Start-Up Lab, VC Evolution, Venture Day Minsk 2020: https://www.youtube.com/user/ImaguruHub/videos). 2.4 LIMITATIONS The following inherent challenges and risks were considered and mitigated as much as possible to obtain reliable data collection throughout the evaluation. ● COVID-19 pandemic: When LEAP III was first approached to conduct this evaluation back in early 2019, COVID-19 was non-existent, and fieldwork was expected to occur as planned. However, due to the year-long delay, the ET conducted the entire evaluation remotely starting in May 2020. Although the ET adapted its methodology accordingly, it was no doubt a disadvantage that the ET members could not be in-country to speak with stakeholders and beneficiaries in person and experience activities on the 19 ground. Regardless, the ET attempted to mitigate all issues by planning in advance and working with a local team member in Belarus to help coordinate in-country logistics and interviews. All plans were communicated and agreed on with USAID/Belarus in advance of the evaluation to ensure that all parties were satisfied with the methodology. In addition, due to this evaluation shifting to being strictly remote, this is one of the contributing factors to why survey response was extremely low. The ET discussed this early in the evaluation, as USAID/Belarus feared that most beneficiaries would be in rural regions, and thus, access to the internet would be difficult. The team sent numerous follow-ups and asked the local consultant to contact beneficiaries by mobile phone throughout these months. Had there been no pandemic, the ET would have wanted to hire a team to provide handwritten surveys to aid in the evaluation. The analysis of the survey responses received did prove to be consistent among all project activities.10 ● Effects of presidential election timing. The Belarusian presidential election, held on August 9, 2020, restricted availability of respondents during remote data collection, especially after Election Day. Holding interviews related to future entrepreneurs’ development issues after the election proved difficult, as heightened sensitivity and uncertainty caused respondents to be unsure of the country’s future. Many respondents chose not to speak about this to the ET. The ET conducted the majority of interviews before August 9, 2020. The availability of respondents after the election was limited, as some were arrested, and internet connection was often cut off entirely. Communication with USAID/Belarus was also cut off. Upon return of the internet, the ET immediately discussed the difficulties with USAID/Belarus and agreed on an extended remote data collection period of additional days to ensure that all interviews were conducted as planned after internet service was restored. It is extremely important to note that, after the elections occurred, participant attitudes toward the interviews changed drastically. The ET noticed that participants’ views of the future differed greatly before the election occurred compared to how they felt in the aftermath. Many of the businesses had high hopes for what the future held for their businesses before the elections. After them, however, many were unwilling to voice their opinions openly and/or refrained from commenting on their long-term plans at all. There were also instances whereby interviews were canceled, because stakeholders had no interest in participating and/or had urgent matters to tend to elsewhere. The ET believed that had the evaluation been conducted a year before the elections, respondents would not have thought or felt strongly about how the elections would affect their work. This year, the outcomes changed their outlook and caused them to question their future plans. The ET addressed this issue as much as possible during KIIs throughout the discussion, but inevitably, conducting an evaluation during a period of significant unrest introduces greater uncertainty into the findings, particularly when it comes to recommendations for future work. Nonetheless, the ET believes that the recommendations provided in this evaluation report still hold true and remain relevant to current events that occurred. ● Effect of timing of the evaluation. The timing of the evaluation fell during the summer/holiday period, when many stakeholders were out of reach. The team sent out a number of follow-up emails to no avail. The ET mitigated these challenges by careful planning, good logistical support, and division of labor among the team members. Where necessary, the team also worked with USAID/Belarus to ask for additional follow-up, but due to the current country situation, much was out of their control. 10 It is also important to note that had the evaluation been conducted a year prior, the results of this evaluation would have no mention of the consequences that the COVID-19 pandemic would create moving forward (i.e. uncertainty in day-to-day activities, adaptation to strictly virtual work, and questions about the future). 20 ● Large scope (geographically and wide variety of stakeholder/beneficiaries and project activities). Encompassing policy development, educational services and training/consulting, SMEs/start￾ups and local business organizers, local investors, and school students, this evaluation was quite a large undertaking to be done entirely remotely. To mitigate the risk of an unbalanced approach, the ET spent a significant amount of time conducting desk reviews and preparatory interviews with applied purposive sampling of sites, respondents, and direct observations to ensure exposure to a wide scope and variety of activity geographies, beneficiaries/stakeholders, and activities. ● Recall bias. A challenge of qualitative data is that responses rely on the interviewee’s recollection or perspectives. Many of the activities within the I3 started six years ago (DRIVES). Some respondents found it difficult to accurately recall efforts related to particular activities or changes over time. The ET mitigated this by incorporating best practices for qualitative data collection when recall is required, such as framing questions to anchor to memorable points in time to ease recall, or asking questions that rely less on recall of specific activities and more on the current perceived implications of those activities. Data was also triangulated with other respondent categories and sources, helping to verify where responses may have been biased due to recall limitations. Moreover, the team made efforts to speak with individuals who are part of groups one-on-one to ensure that sources of information were not affected in any way by their peers. ● Selection bias. The ET depended on USAID/Belarus implementing partners to identify and communicate with specific key stakeholders. There was thus some risk of selection bias due to the potential for selecting a large proportion of interviewees with only positive opinions of the program. Regardless, when the ET reviewed all project documents, the team made decisions on whom to contact at their own discretion as well. It is also possible that beneficiaries selected for KIIs were more likely to fill out the online survey, which presents overlap between those two groups of respondents. This could reduce the overall value addition of the survey as an additional data source. Due to the anonymous nature of the survey, however, it was not possible to look for direct matches using names, but the demographics of gender and location are similar across the samples, indicating that some caution should be applied in interpreting the results, due to the smaller sample size. 21 3. EVALUATION FINDINGS 3.1 KEY FINDINGS FOR EQ1 Have the I3 project activities contributed to increasing the competitiveness of SMEs in Belarus? Overall, analysis of various reports and documents11 showed that the Belarusian economy remains largely state￾controlled and its economic model is heavily oriented toward maintaining full employment. The government has shown increasing commitment to the development of the private sector. The flourishing information technology (IT) sector is a good example of a quickly emerging industry in Belarus, in which private businesses are thriving due to strong human capital available in the country, coupled with the strategic choice to introduce a highly preferential tax and regulatory regime for the sector.12 Despite SMEs’ 99.5 percent share13 of the total business population, the economic role of SMEs in Belarus remains limited, with a 24.6 percent share of the country GDP.14 The GOB has been increasing its efforts to promote private sector development; strengthen the position of SMEs15 ; and sustain growth, employment, and resilience. Recent introductions include the adoption of the SME Development Strategy 2030, along with the elaboration of provisions for the establishment of an SME agency through Presidential Decree No. 7 (2017) “On the Development of Entrepreneurship,” which complements 2010 Presidential Decree No. 4, “On the Development of Entrepreneurship and Stimulation of Business Activity in Belarus.” This substantially simplifies regulations for doing business in Belarus, bans the introduction of new taxes until 2020, and minimizes state interference in business operations with the assumption of good faith of enterprises.16 The latter improves government-business relations, which means that firms can now nominally conduct business operations more independently, without being subject to frequent and unjustified scrutiny from state authorities. This evaluation has shown that, despite all changes and improvements introduced by the GOB (either new legislation and regulations or less resource-consuming business administration procedures), awareness of these changes and improvements them is generally low among SMEs, and there has been a lack of awareness-raising activities conducted by the government in this regard. This is true both for entrepreneurs in Minsk and in the regions. For example, only 10 to 12 percent of businesses of different sizes knew about the adoption of the 2017 Decree No. 7 “On the Development of Entrepreneurship.”17 In addition, Belarusian SMEs mostly operate on a very small scale in non-innovative, low-productivity industries, which explains their limited contribution to value addition. When asked about the most important factors influencing the SME competitiveness in Belarus, KII and survey participants mentioned the following issues: 1) supportive legal and regulatory environment (i.e., business enabling 11 Please see References, page 57. 12IMF (2019), Belarus Country Report No. 19/9, International Monetary Fund, Washington, D.C., https://www.imf.org/en/Publications/CR/Issues/2019/01/18/Republic-of-Belarus-2018- Article-IV-Consultation-Press-Release-Staff-Report-and-Statement￾by-46526. 13 Individual enterprises represented 68.1 percent of all business entities; microenterprises 27.5 percent; small enterprises 3.3 percent; medium enterprises 0.6 percent; SME POLICY INDEX: EASTERN PARTNER COUNTRIES 2020 © EBRD, ETF, EU, OECD 2020, page 380. 14 Belstat (2019), Belarus in Figures, National Statistical Committee, Minsk. 15 SME definition has remained unchanged since 2010: SMEs are exclusively defined by the number of their employees and are divided into individual entrepreneurs, micro (up to 15 employees), small (16 to 100 employees) and medium enterprises (101 to 250 employees). 16 SME POLICY INDEX: EASTERN PARTNER COUNTRIES 2020 © EBRD, ETF, EU, OECD 2020, page 379. 17 Business environment in Belarus, December, 2017, IFC, EU, SIDA. 22 environment), 2) quality and accessible business education, 3) increased attention to and support for start-ups, including young and women entrepreneurs, and 4) positive government attitude/support.18 Respondents argued that despite important steps to improve the legal and regulatory environment for private enterprises and, especially, SMEs, further improvement of the legal and regulatory environment is still needed. First, it is important to establish an even playing field (rules and procedures) for SOEs and SMEs as well as for other sectors, including IT. Key informants acknowledge the progress achieved in approving legal framework and simplifying regulations for doing business and recognize that now is the time to “fine-tune” the business environment to unify their voices. Key informants believe that having a designated single body (such as a one-stop shop) to provide support to SMEs; coordinating the activity of related institutions (e.g., ministries and agencies, educational institutions, and civil society business associations) toward enterprise development; strengthening data collection regarding enterprise needs; and conducting evidence-based planning, monitoring, and evaluation will help strengthen the business enabling environment and increase attention to private enterprises. Recognizing the high educational level of the Belarusian population, respondents see the need for improvement of key competencies related to entrepreneurship and the establishment of more targeted and systematic support to students with entrepreneurial aspirations. To support women's entrepreneurship in Belarus, it is important to strengthen women’s business knowledge and skills, pay attention to networks and online platforms, and expand opportunities for women-owned businesses. Key informants identified the following obstacles that inhibit the development of a competitive, developed, and vibrant private sector: high tax rates, difficulty in accessing finance, lack of advanced legislative and policy reforms, an inadequately educated workforce in entrepreneurship-related areas, lack of regulations such as business licensing and permits, and customs and trade rules. Confirming the barriers spelled out by interviewees, survey respondents also cited an unsupportive business environment and government attitudes. These conclusions are echoed by World Bank research:19 “The SOEs’ preferential regime in access to financial resources, raw materials, lower energy prices, and softer attitude of regulatory authorities is a constraint on the development of a vibrant private sector.” This evaluation shows that USAID/Belarus I3 activities contributed to Development Objective (DO), increased private sector competitiveness and specifically improving start-ups’, SMEs’, and local investors’ management and operational capacity (IR2.2 through DRIVES, CSISS, and partially, AID-VENTURE) and increasing access to finance (IR2.3 through AID-VENTURE). All evaluation participants recognized USAID’s targeted, continuous support to private entrepreneurship. They mentioned the Mission’s advocacy efforts in improving legislation and regulatory condition for entrepreneurs and investors; capacity building support to start-ups, SMEs, and business support organizations; promotion of the business and investor enabling environment; support for outreach and access to business education for start-ups, SMEs, and others to the regions; increased attention and support to young entrepreneurs and women entrepreneurs; and shifts in attention to business development from capital to regional level. Survey participants noticed the following most important changes that occurred in SME development over the past 5 years in Belarus: 18 KIIs/Online survey, July-August 2020. 19 World Bank (2018), Enterprise Surveys: Belarus 2018 Country Profile, World Bank, Washington, D.C., http://www.enterprisesurveys.org/en/data/exploreeconomies/2018/belarus#firm- characteristics 23 Figure 2: Most Important Changes in SME Development - Attributed to I3 Source: Authors’ representation Measures of SME competitiveness address firms' capabilities and improved performance, such as increased sales or profits, cost reduction, or accessing new markets. To achieve this, the local ecosystem must ensure that start-ups and SMEs have access to key resources such as training and information, finance, and other resources and markets. Although the ET did not examine the profits or sales of participating firms,20 specific contributions of each of the I3 activities to increasing enterprise competitiveness in Belarus, discussed below, were mentioned during evaluation.21 Based on the evaluation findings, these USAID-supported activities contributed effectively to increasing the competitiveness of SMEs in Belarus. Specific findings by activity are outlined below. KEY FINDINGS FOR AID-VENTURE The purpose of the AID-VENTURE activity is “to expand access to venture funding, thus contribut[ing] to the growth of start-ups and expansion of the role of the private sector in the economy.”22 Interviews and survey results confirm that the activity made an important contribution to the improvement of legislative and regulatory frameworks conducive to angel investment and venture funding and assisted in building the local community of private venture investors and strengthening their capacity, building linkages between local start-ups and investors. Specifically, respondents mentioned the activity in correspondence with the following improvements in the business 20 These points were not included as part of the evaluation SOW. 21 KIIs, July – August 2020. 22 AID-VENTURE Program Description, page 17. ƒ² .p>]Sox >_I >GGKmmSFS]Sox oa FpmS_Kmm KIpG>oSa_ „² _GlK>mKI >ooK_oSa_¸mpiialo oa mo>loÄpim­ S_G]pIS_Q xap_Q FpmS_KmmKm¸K_olKilK_KplmRSim …² pmS_Kmm K_>F]S_Q K_uSla_^K_o >_I ]KQSm]>oSa_¸lKQp]>oSa_ †² _GlK>mKI >ooK_oSa_¸mpiialo oa va^K_Ä K_olKilK_Kplm 24 environment: ● Legislation and regulatory environment improvements23 were generally supportive of venture capital and funding. The activity team participated in working groups on regulatory improvements and launched productive policy dialogue with national authorities. This helped increase their attention to venture funding and the importance of developing a supportive regulatory and policy environment for start-ups, investors, business angels, and building a venture fund. The activity held regular meetings with policymakers to discuss results derived from investors' comments and recommendations and consulted to government officials on venture capital (VC) and information materials. AID-VENTURE involved 18 government institutions and agencies in its activity. ● The creation of a community of local investors and establishment and institutionalization of BBAN provided access for local investors. The activity provided a space and opportunity for quality training, international experts and expertise, networking opportunities, study trips, and partnership opportunities with foreign investors. As a result of AID-VENTURE efforts, a community of potential local investors and business angels numbered nearly 200 people, including those interested in working with VC funds. BBAN membership grew from zero to more than 90 since its establishment, and the association’s organizational capacity was strengthened through training, study tours, and mentoring by foreign experts. ● Recognized and well-organized networking events, including annual Global Entrepreneur Week in Belarus, Venture Day Minsk, Bootcamp for Business Angels, and Google Founders Day were key to the activities. These events included networking sessions, master classes, panel and small-group discussions, and matchmaking activities for start-ups and investors. Networking events attracted government participants. At the Global Entrepreneur Week Expo, BBAN Angel Band managers and founders were introduced to the Prime Minister of Belarus. ● There was focus on advanced start-ups’ capacity to attract and work with local and foreign investors. The activity not only conducted matching events and facilitated cooperation between start￾ups and investors but also improved the availability of funding necessary to launch a business, mentor new start-ups, and guide investments. Once investments were received, start-ups worked closely with activity experts and received consultation and training support. As of the time of evaluation, the TechMinsk Fund had invested up to $290,000 in six start-ups. ● A new program with comprehensive educational assistance helped new enterprises to build products. Start-ups throughout mentioned the Tech Minsk Accelerator as providing great support in capacity building and mentorship. Support for start-ups was provided through design marketing, sales plans, improvements in financial planning and strategies for entering local and foreign markets. ● There was significant support for women entrepreneurs, especially in male-dominated sectors. In areas such as IT, special networking events and discussions, such as the dedicated Women IT Week, were held to help build women’s confidence and ability to share success stories openly and safely. 23 The ET did not conduct an in-depth analysis of this as part of the evaluation as it was outside the SOW. 25 ● As an unexpected result, the TechMinsk Acceleration fund became operational as a prototype for testing. This fund currently supports local start-ups with investments.24 KEY FINDINGS FOR DRIVES DRIVES provided systematic, needs-based access to practical knowledge and skills necessary to start and grow a business, including at the regional level, with the overall purpose of “expand[ing] the private sector throughout Belarus.”25 Five regional Centers for Excellence in Entrepreneurship (CEEs) were established to provide access to entrepreneurship support services. All respondents from DRIVES-associated CEEs reported that the project activity significantly improved access to business education in regions. Specific DRIVES contributions in increasing enterprise competitiveness mentioned by KIIs include: ● Access to knowledge and skills necessary was increased to start and grow a business, including at the regional level. Five regional CEEs were established to provide access to entrepreneurship support services. ● Essential business training courses were created and delivered effectively at the regional level through a blended-learning format using new technologies and teleconferencing systems. The teleconferencing resources were also made available for use by other donors and technical assistance programs in an effort to maximize the regional impact of the technology investments. ● CEEs’ networking events increased the cooperation of business associations, training organizations, local agencies, and entrepreneurs from different regions. In addition, the events allowed leadership of the CEEs to share best practices and lessons learned during their efforts to make their centers effective and sustainable. This created a local business community of established SMEs, early-stage businesses, and start-ups. ● Greater access to high-quality instructors and business experts in regions increased the recognition of the value of business education. Owners and managers can now readily attend practical training courses online to acquire knowledge and skills needed to make their businesses more competitive. The technology and blended-learning format allow them to participate without traveling to Minsk, and recorded sessions allow them to listen to access lectures at their convenience. ● Business instructors were enabled to adapt business courses to the local context and deliver trainings through blended learning technology. Use of this type of learning continues to increase, meeting learner needs, especially during the pandemic. The growth of blended learning—through both individual and group activities—has been seen to promote collaboration and critical thinking. Teachers and students work together through online resources to personalize learning based on students’ needs. ● Quality of educational programs in the regions and the practical business knowledge and skills of their graduates were significantly improved. As a result of this activity, graduates’ ability to contribute to the success of existing businesses or launch new ventures was greatly enhanced. 24 AID-VENTURE Project Document, page 1. 25 DRIVES Program Description, page 19. 26 Specific DRIVES examples are included below—more success stories may be found on the CSISS page available26 on the USAID/Belarus webpage. 26 https://www.usaid.gov/belarus/news-information/news/startup-school-innovation-development-and-growth-startup-school-story 27 KEY FINDINGS FOR CSISS According to the CSISS Program Description, the activity’s key objective was “to lead to increased support for entrepreneurs and the establishment of new businesses, contributing to the local communities’ development and expansion of the private sector in the economy.”27 Interviews with project staff and management, local partners, and beneficiaries in regions confirmed that CSISS and its implementer, the Society of Innovative Business Support (SIBS), substantially contributed to: ● Growing entrepreneurial activity in regions by transferring SIBS expertise of start-up and SME support to local partners. Supporting new businesses through Startup Schools and its educational activities, validating their ideas during Expert and Investor Days, and assisting in developing business projects through networking events and information dissemination. Increased awareness of the risks and rewards of entrepreneurism improved attractiveness and acceptance at the regional level. A network of 38 local organizers of start-up support activities was created and established new business support infrastructure organizations. ● Entrepreneurship and leadership skills were built through informal education for high school students and teenagers (in Startup Schools and Startup Schools Junior). Ten complementary local initiatives were supported to provide business education. These initiatives helped to target teenagers, producing short films on entrepreneurship for students, designing games simulating business activity, and developing user-friendly educational courses on entrepreneurship for young children. This also provided support for women entrepreneurs’ networking, events, and clubs in regions. Piloting Startup Schools Junior allowed for the building of entrepreneurship and leadership capacity among motivated high school students through partnerships with formal (school) and informal educational institutions. Along with traditional education, students in Startup Schools Junior received basic economic and entrepreneurship knowledge and skills. The activity provided well-developed and practical business training materials accessible to a broader audience. CONCLUSIONS FOR EQ1 1. Overall, the evaluation found that the SME competitiveness improved for all three I3 activities. Each of the I3 activities successfully contributed to strengthening various aspects of the economic ecosystem needed for SMEs and start-ups to launch, grow, increase employment, and become competitive. Individual start-up businesses and SMEs attributed improvements in their operations and increased competitiveness to assistance received through participation in I3 activities. 2. The legal and policy framework of Belarus also improved. Activities, programs, and attitudes of the GOB in supporting SMEs in general and women in business specifically also improved. AID￾VENTURE activities united and empowered key stakeholders to advocate successfully for needed changes governing finance and venture capital. Efforts also contributed to government policies to support development of the IT sector. 27 CSISS Program Description, page 2. 28 3. The evaluation found that financing needed to launch and grow a company is more available and investors’ ability to advise and mentor start-ups was enhanced. Practical business training and mentorships of new startups improved the availability of funding needed to launch business and guide investments (AID-VENTURE). This also helped improve the image of private entrepreneurship in regions among youths (CSISS) and create better access to quality education throughout Belarus, which was necessary to create and operate competitive businesses. Significantly, students and aspiring entrepreneurs can now access needed training, knowledge, skills, and funding in the regions, which also improves the overall availability of human capital. 4. To increase SMEs’ value-addition contributions and competitiveness in the business sector, it is important to create level playing field conditions for all firms, regardless of size and ownership status. In this way, businesses can establish a culture of healthy competition in the economy and support the development of entrepreneurial human capital. 29 3.2 KEY FINDINGS FOR EQ2 Have the current modes of implementation of the AID-VENTURE, DRIVES, and CSISS activities been efficient in achieving respective project goals? What works well' and what does not? What unintended effects have resulted to date from the project approaches, tools, and activities? MODES OF EFFICIENCY OF I3 IMPLEMENTATION Activity reports, survey respondents, and KIIs provided strong evidence that USAID assistance advanced the I3 activity objectives of strengthening key target groups capacity to a significant extent. The evaluation confirmed that the I3 project efficiently used a broad spectrum of activities and new technology to enhance the capacity of investors, start-up entrepreneurs, and implementing partners. Survey respondents evaluated I3 activities’ relevance to their need on a five-point scale, where 1 = not relevant at all and 5 + totally relevant, as follows: Access to investments/venture funding (AID-VENTURE) N=928 4.3 Access to business education (DRIVES) N=12 4.6 Regional supportive SME infrastructure (CSISS) N=25 3.84 Survey respondents often said that the I3 activities addressed the following needs: 1) knowledge of how to move from a business idea to a ready-for-market product with lower expenses, and how to create local business development infrastructure for SMEs; 2) opportunities to share experience and involve more people in entrepreneurial activity in regions; 3) networking with international experts and businesspeople; 4) access to experts and speakers for start-up schools and finance (credits, investments, grants); 5) consultation and mentoring on business project design; 6) assistance in finding new customers; 7) learning new teaching technologies and practicing online educational program development; 8) direct communication with investors; and 9) establishment of local community of entrepreneurs, etc. “The project is very clear and accessible, an easy entrance for each person to the same start-up schools: Guys, come, we are glad to see you, we have something to help each of you who only dreams of your business, or is already developing it. I saw many examples of successful entrepreneurs from my city, although five years ago I did not believe in their existence, did not know that these people were ready to share their experience. Yes, and it seems to me that entrepreneurs did not know that they were ready to share their experience. A very significant example for me of my internship in the United States, when on the plane there, my colleagues, investors, and entrepreneurs almost all said that there was no point to speak in public, share experience, nobody needs it (in response to my invitation to speak at a start-up school). And on the way back, after just three weeks, each of them, EVERY ONE, wanted to share, perform, travel around Belarus -- and in the very first year after the internship, almost all of them came to my hometown, and some not just once, but at other start-up schools as well. And you can talk a lot about how many projects were born before my eyes, how many people who believed that the world of entrepreneurs exists in Belarus and in their native Vitebsk, not only in books and beautiful stories on the internet. I am infinitely grateful to the project for this!” -- Online Survey Respondent 28 N – number of online survey respondents associated with a specific I3 activity. 30 90 percent of survey respondents said that the USAID-supported activities they participated in offered an appropriate approach to support increased competitiveness of SMEs in Belarus. One respondent said that, “The project approach is correct already in that it opens the doors to another world for entrepreneurs—a world where risk is not a problem, but a means for development. This changes the worldview and gives more competitiveness to such entrepreneurs.”29 Survey respondents among I3 stakeholders and beneficiaries identified most useful types of support provided by USAID projects. Survey findings show that the most useful types of support include networking events (conferences, expert and investor days, start-up events, and the like), training, start-up schools (including junior schools), linking start-ups and investors, support to local initiatives, access to funding and investments, grant support, and training of trainers (see Table 4). TABLE 4: MOST/LEAST USEFUL TYPES OF SUPPORT PROVIDED BY USAID PROJECT (ON A 5-POINT SCALE, WHERE 1 IS NOT USEFUL AND 5 IS VERY USEFUL), N=50 TYPES OF SUPPORT USEFULNESS SCORE Improvement of legal and regulatory framework 3.3 Policy development 3.1 Training of trainers 3.6 Start-up schools, including junior schools 4.0 Training 4.2 Access to funding/investments 3.6 Local initiatives support 3.7 Networking events (conferences, expert/ and investor days, start-up events, etc.) 4.4 Partnership/bringing start-ups and investors together 3.9 Acceleration program 3.3 Grant support 3.6 Source: Authors’ representation The variety of approaches engaged representatives of key stakeholder groups while contributing to improvement of the SME ecosystem. Table 4 shows that beneficiaries (survey respondents) tended to identify as most useful the types of support that had a direct impact on them, while the majority of interviewees were more likely to identify less visible types of support, such as policy work on legislation and regulations and access to information and communication platforms. However, efficiency in terms of how funds were used for different types of support was not evaluated and requires further investigation. 29 Question 10, Online survey, July–August, 2020. 31 Respondents noted that all three activities migrated and adapted their work to online channels when the COVID￾19 pandemic began early in the spring of 2020. The evaluation confirms that technology not only helped prepare implementers for impact of COVID-19 but also allowed efficient training outreach to the regions and was efficiently leveraged for other events as well. KEY FINDINGS FOR AID-VENTURE AID-VENTURE used various types of activities to achieve its objectives. Interviewed activity stakeholders and beneficiaries confirmed that the activity acted as a ”catalyst” to ignite and fuel interest in venture funding, encourage local investors to cooperate, and build their capacity through training and building a network within Belarus and across borders. Respondents30 who participated in the project thought that activities under AID￾VENTURE were highly relevant to their needs (4.3 on a five-point scale), as they provided opportunities to create and expand the venture ecosystem in Belarus. The activities provided opportunities for networking with international experts and businesspeople and direct communication with investors, including entrepreneurs from regions, increasing access to venture funding, and the ability to demonstrate successes and failures. Nearly all respondents (21 out of 23) confidently said that AID-VENTURE offered an approach to support increasing competitiveness in Belarus. Survey respondents appreciated the comprehensive and well-designed trainings for business angels, local investors, and start-ups. Education activities were very important to build demand (start-ups) and supply-side (investors) capacities in effectively presenting start-ups’ innovative ideas and evaluating investors’ business projects in terms of product, marketing, selling, and potential to grow. Besides business-related issues, the majority of the training events focused on overcoming negative attitudes in society toward business, such as the fear of failure (and how to accept it), fear of being targeted by the government if business grows, and how to identify risk at an early stage. Moreover, training beneficiaries appreciated follow￾up advice and mentoring by experts from Belarus, Russia, Europe, and the United States. Networking events assisted in building a community of local experts, forming and establishing BBAN. As of the time of the evaluation, BBAN had nearly 90 members—well-trained angels and investors capable of identifying and counseling innovative start-ups, willing to (or already providing funding to) local start-ups, either individually or with other investors, advocating for improvements in legislation and regulatory frameworks needed for effective venture ecosystem, raising awareness of the needs of start-ups and local investors, and assisting in building partnerships between start-ups and local and foreign investors. BBAN served as a very important platform for communication, sharing information, and mentoring for its members and start-ups31. AID-VENTURE achieved a certain level of cooperation with the GOB by advocating for legislative change and for the Development Bank of Belarus to become a VC player. The evaluation determined that first steps in VC reforms were made. Respondents noted a lack of legislation and policies governing start-ups’ and SMEs’ second round of finance and uneven treatment of different sectors of the economy. IT is seen as a preferential area for state support. However, state financial support is not very accessible and requires excessive documentation to prove business legitimacy. At the same time, key informants mentioned that the time for simple decisions has passed and that it is now time for sophisticated tuning of specific issues within a complicated business support system.32 30 Online survey respondent, July-August 2020 31 KII#10, #11 32 KII#50 32 Access to finance was noted as a key issue for the existence of start-ups and SMEs in Belarus. Respondents ranked the improvement of the Belarusian environment in terms of access to finance and investment as marginally better (3.1 on a five-point scale).33 In KIIs, respondents cited types and sizes of financial and investment deals to show that 20 start-ups received several million dollars through the acceleration TechMinsk program, assisting in setting up several venture deals of $100,000 each, received $50,000 in investments, and prepared two agreements to implement pilot projects of $100,000 in total. Despite these anecdotal examples, a raised acceleration fund occurred sooner than the project expected. At the same time, beneficiaries of TechMinsk mentioned some challenges in receiving investments, including timing (for example, funds were provided toward the end of the investment agreement term rather than at the start) and funding from traditional sources, like banks, was very slow to be received. AID-VENTURE conducted very few activities focused on women entrepreneurs. However, where relevant, special sessions for women managers were very successful and show that women have solid experience in company management and are valid sources of knowledge, expertise, and communication with investors and clients. In interviews, AID-VENTURE partners and beneficiaries could not identify any specific challenges for women in business. Moreover, some respondents confirmed that women are more thorough, know foreign languages, and have better communication and marketing skills than men. In fact, these women were often considered business leaders who can bring a lot to the table and help companies develop faster. Respondents suggested changes they would make to the AID-VENTURE activity, including more comprehensive training programs oriented toward practical results and more microgrants and investments for start-ups in their early stages, with specific focus on regional participants. Moreover, respondents in the online survey stated, for example, that “it would be beneficial for Belarusian ecosystem to have more such activities that could provide even deeper influence.”34 AID-VENTURE faced some challenges, including a limited number of start-ups to work with; badly developed infrastructure for supporting start-ups/accelerators, especially in regions; lack of a second round of investments of $200,000 to $300,000; and minimal outreach and impact achieved in regions. A framework was created for future public sector engagement, but it is currently insufficient to make key AID-VENTURE activities sustainable. Despite steps made toward the sustainability of the BBAN activity (membership fees), strategic approaches are needed to diversify revenue streams to support activities and sustain network associations. KEY FINDINGS FOR DRIVES The modes of activity implementation delivered by DRIVES successfully enabled current and aspiring entrepreneurs and SME owners throughout the regions to acquire the knowledge and skills needed to launch start-ups or contribute to existing SMEs. Survey participants and interviewees highlighted that IPM faculty provided courses that were practical and addressed their specific needs. Courses such as “Business from Scratch” enabled students to chart pathways from idea to business planning to acquiring needed funds and launching a business. Additionally, faculty “Train 33 This answer is taken and analyzed from the survey question 11.3 in the Master Questionnaire: “Please rate the improvement of the Belarus environment in terms of access to financing and investment on a scale from 1 to 5, where 1 is no improvement and 5 is substantial improvement.” 34 Online survey respondent, July-August 2020. 33 the Trainer” (TOT) courses and study tours enhanced the capacity of IPM faculty members to adapt existing and create new courses tailored to participant feedback. The introduction of the new technology and blended-learning approaches, and the establishment of IPM regional centers allowed participants outside of Minsk to attend training courses without incurring travel expenses and excessive time requirements. To offset the drawbacks inherent in online learning, the DRIVES methodology included competitions, mentoring, and networking opportunities. This comprehensive approach significantly contributed to increasing the availability of quality business training in the regions and advancing SME competitiveness. Key achievements of DRIVES include the creation of practical SME and entrepreneur courses tailored to the Belarus environment. Introduction of blended learning and state-of-the-art technology (e.g., Cisco Webex). Training was user-friendly and included unlimited online sessions, integrated audio, content sharing, and the ability to record classes. The benefits included higher productivity and cost savings over traditional training. These strategies made the courses readily accessible throughout the country. IPM now has faculty trained in innovative course design and use in blended-learning methods delivered through “state of the art” online technology that allowed IPM to shift quickly to an online format.35 Having these new resources positioned IPM to respond quickly to challenges presented by the COVID-19 pandemic, making it possible to quickly shift most of their education and training programs to online formats. The network of established IPM Regional Centers allows access to new markets while contributing to the development of local ecosystems necessary for SME competitiveness. Start￾up business idea competitions and other networking events offset some negatives of online learning and facilitate the development of supportive professional relationships and mentoring needed for businesses to grow. DRIVES improved faculty members’ knowledge and skills, created practical business courses, and effectively delivered training through IPM’s newly established network of CEEs. Through the use of online technology and blended learning methods, current and aspiring entrepreneurs in the regions now have access to essential business knowledge and skills. The CEEs’ cooperation with other donor activities, including making the online technology available to them, adds to the prospects for sustainability of IPM’s network and their positive impact on the regional SME environment. However, many challenges need to be overcome for IPM regional strategy to be successful. Without USAID support, IPM training fees may become uncompetitive, as low-cost and free alternatives are introduced by other donor projects such as UNDP, along with other alternatives that are becoming available online. In addition to needing to generate enough revenues to cover fixed costs, most of the CEEs experienced high personnel turnover and difficulty in identifying local business trainers and mentors. IPM reportedly identified only one high￾potential local business trainer to leverage into the regional offices. The increasing number of online alternatives and IPM’s entry into the regional markets also has the potential to disrupt the advancement of university business education and other training providers. To overcome these challenges, IPM may consider organizing more events in cooperation with local universities, private businesses, government agencies, nongovernmental organizations (NGOs), and other donors. Survey respondents indicated a need for more follow-on guidance and mentoring, and additional regional events may help identify potential mentors and business advisors, as well as faculty resources. By expanding their pool of 35 State of the art refers to the highest level of technology that participants under the activities used including innovative websites, devices, and techniques. 34 qualified trainers and consultants, the CEEs could increase their revenues and diversify the business services they offer. Increasing the number of local events tailored to area business needs, expanding their personnel resources, and cultivating more local private sector support will be necessary, if IPM is to continue to sustain its presence in regional markets. KEY FINDINGS FOR CSISS To achieve CSISS’s objectives of 1) creating a network of local organizers of start-up development activities and 2) building a sustainable system of promotion, cultivation, training, and expertise for start-ups and small businesses, the activity developed a clear three-layer system of support to start-ups and SMEs in the regions and employed a variety of activities to improve the public image and popularity of entrepreneurship in regions. Respondents in the CSISS project think it is highly relevant to their needs36 (3.84 on a five-point scale), as it provided opportunities to learn to start and grow start-ups, take part in various events in the regions, promote and improve business ideas and present them to experts and investors, gain new ideas and communicate with local entrepreneurs, become part of the local business community, create and advance the infrastructure for start-ups and SMEs, and receive support for local initiatives. Most survey respondents37 (35 out of 39) said that CSISS offers the right approach to support increasing competitiveness in Belarus. To date, the project has built a network of 38 Start-Up Schools (however, only 17 were active at the time of the evaluation)38 and 11 Startup Schools Junior. Interviewees noted that local universities, and civil society organizations preferred free-of-charge rather than fee-based school models. In interviews, CSISS’s regional partners said that they appreciated the well-designed and regularly updated SME Grow-Up Toolkit, summarizing local experience in a step-by-step approach to organizing start-ups and SMEs. In addition, SIBS builds and strengthens local partner capacity and competency through TOT, mentoring, and assistance in organizing Start-Up Schools, Expert and Investor Days, building local partnerships, and supporting local initiatives. Respondents to the online survey confirmed that start-up business training was tailored to the needs of start-ups and young entrepreneurs and commented that “the project directly helps to get local support in regions that is very important for those who want to start working in the field of small and medium-sized businesses at [the] local level.”39 “We are testing different channels, but there is still a lot of work to do. Therefore, the awareness is not as high as we would like. The attitude has definitely improved among those who are involved in initiatives and communities. This is just a significant breakthrough. I myself am a perfect example! Five years ago, I believed that ‘Vitebsk is the most depressed region for entrepreneurship,’ and now I am at the head of the start-up community movement. I believe in people and see their incredible growth, I love entrepreneurship, and I am proud of my city.” -- Online survey respondent Survey respondents confirmed that entrepreneurship activity in other regions of Belarus has significantly increased.40 Key informants noted a large number of start-up schools, educational and 36 Online survey respondent, July-August 2020. 37 Ibid 38 KIIs#16, 17, 18 39 Online survey respondent, July-August 2020. 40 Question “To what extent entrepreneurship activity in the regions of Belarus has grown on a scale from 1 to 5, where 1 is no improvement and 5 - substantial improvement?”, Online survey, N=25, average 3.2 points on a five-point scale. 35 networking events, meetings of local entrepreneurs, and joint activity with regional governments. Implementation of local initiatives by CSISS grantees was also mentioned as a contribution to the improved image of entrepreneurship image. Moreover, the number of graduates of start-up schools, experts, and permanent followers of the start-up schools has grown exponentially, and these people publicize private entrepreneurship and contribute to more positive public attitudes toward business and beliefs that it is not so “dangerous.”41 More young people in Belarus—specifically, participants in CSISS start-up school events—now see their future in business and entrepreneurship. Respondents42 stated that they would like to open their own successful businesses (e.g., the Lavka project in Plotsk). IT companies that have opened offices in regional centers and large cities of Belarus have grown their business infrastructure. Moreover, business education has become prestigious and popular, and the creation of local business clubs and communities has catalyzed enterprise growth. For example, a closed group for the development of entrepreneurs has been working in Mogilev for five years. During this time, the businesses of approximately 70 percent of participants have increased five to ten times and continue to grow. This year, a group launched a start-up “Crowd Investing Platform of 41 Investors,” which received 16 applications for investment in two months. Participants in the CSISS activities began to feel not only part of local business communities, but members of the community of entrepreneurs throughout Belarus as they can communicate directly with entrepreneurs in other regions and with local colleagues. In the words of one survey respondent, “Their eyes light up, they begin to believe in their ideas, take on their implementation, assemble teams, and act together. This is the biggest project achievement to me.”43 In addition, the evaluation found that CSISS effectively worked with women entrepreneurs by introducing innovative events for women’s groups (e.g., business breakfasts in Molodechno44 or initiatives for young mothers in Bobruysk45) and supporting the social entrepreneurship school in Gomel to help women, including those from disadvantaged groups, develop business skills, find employment, and engage in entrepreneurial activity. Despite these achievements, respondents made many suggestions on changes they would make to the CSISS activity. They include: ● Start-up schools: More comprehensive training programs; discussion about management of start-up projects; more interactive distance learning activities to expand access to business education in regions and rural areas; formation of a pool of regional experts for high-quality organization and implementation of similar projects; teaching start-up methodology, scaling ideas, and rapid testing of hypotheses so that young entrepreneurs move quickly; increasing hands-on experience. ● Local initiatives (small sub-grants): Continuation and development of projects; more support for project implementation from local authorities to make local initiatives sustainable; streamlining the reporting process (both to the Ministry of Economy and USAID)46, which comprised 90 percent of the work on the project. 41 KIIs#20, 21, 22, 23 42 Online survey respondent, July-August 2020. 43 Online survey respondent, July-August 2020. 44 KII#25 45 KII#20 46 KII#20, 25 36 ● Communication/outreach: Improved promotion of project events and timely provision of information on upcoming events, which is often not received until the day before the event or even after the event took place. ● Local community of entrepreneurs: New formats for meetings and work with entrepreneurs, both active and beginners; more applied long-term project management under supervision; creation of places for entrepreneurs in the regions for networking and offline communication. ● Other: Increased attention to the development of youth and women’s entrepreneurship; accelerated development of the start-up ecosystem by creating a laboratory for start-ups to interact with specialists and solve problems rapidly. ● General: The start-up schools should use metrics that track the development of projects, such as the number of students, the number of registered businesses after training, investments received, the number of operating businesses two years after opening, and the like; fund local teams of start-up schools in the regions, because people burn out and leave. Despite successful partnerships and leveraged resources with universities, state technoparks and incubators, and technical assistance (TA) for training and networking events, the CSISS activity faced challenges in: 1) keeping its trained local trainers and partners who work free of charge, because their motivation to support start-ups and SMEs does not last; 2) finding regional businesses and investors to participate and present their experience during networking events such as expert and investor days; and 3) addressing limited resources and capacity to advance students on the pathway to becoming entrepreneurs. SIBS has made extensive efforts to consider different ways to sustain project activities. But in the current economic situation, fee-based training is not sellable, especially when other donors, like UNDP with state support, propose free courses. At the same time, start-up schools do not aim at achieving results in terms of the number of new businesses established and successfully operating that might have stipulated activity sustainability according to a survey respondent. CONCLUSIONS FOR EQ2 1. The current modes of implementation used by AID-VENTURE, DRIVES, and CSISS have been efficient in achieving project goals and creating a supportive ecosystem for aspiring entrepreneurs, start-ups, SMEs, and local investors. They were demonstrably effective and largely targeted the appropriate stakeholder groups to achieve the stated project goals. However, the extent of their efficient use of funds requires further investigation. Moreover, the limited number of start-ups and existing SMEs in local communities adds to the challenges and barriers to successful development of an entrepreneurial ecosystem and vice versa, despite reaching and training many aspiring and current entrepreneurs in the regions. 2. I3 project activities were properly designed and relevant, and delivered assistance aligned with beneficiaries’ needs. I3 project activities increased the availability of start-up and early-stage financing and helped improve the public image, attitudes toward, and awareness of entrepreneurship. However, additional assistance is needed to improve access to VC and traditional financing, and much work remains in overcoming the related risks and fear of being an entrepreneur. 37 3. The activities used the correct approaches to reach the targeted groups and achieve project goals efficiently. Activities targeting women entrepreneurs and disadvantaged groups47 through a social entrepreneurship approach and local initiative support proved effective, successfully contributing to their success in business. 4. DRIVES and CSISS modes of delivery successfully increased the availability of business education in the regions, but without additional support (including financial support) and TA, their sustainability is questionable. The modes of implementation, combined with new technology, online activities, and innovative actions of the I3 project teams overcame challenges presented by COVID-19. 5. I3 activities successfully facilitated networking among entrepreneurs, investors and, to a certain extent, representatives from the GOB. CSISS did this by creating local communities of entrepreneurs. DRIVES achieved this by attracting a broad spectrum of business representatives to its courses from different regions. DRIVES also contributed to organizing Kastryčnicki Economic Forum (KEF) events in the regions which proved to be successful. Finally, AID-VENTURE facilitated networking by conducting annual Global Entrepreneurs Weeks and other events in Belarus. 47 Includes groups such as youth, single mother families, etc. 38 3.3 KEY FINDINGS FOR EQ3 How organizationally efficient are the AID-VENTURE, DRIVES, and CSISS implementing partners?48 I3 stakeholders and activity staff and partners interviewed recognized USAID’s strategy of working closely with and through local implementing partners to ensure local ownership while enhancing the capacity of the partners to continue providing services in the long term. Moreover, USAID/Belarus was seen as highly effective and efficient in filling a critical development “niche” while working to increase the capacity of implementing partners. Analysis of project documents confirmed that each of the three I3 activities envisioned activities aimed at organizational capacity building. It is important to mention that the three implementing institutions were at different levels of their organizational development. For example, whereas IPM was experienced and well￾established, SIBS was a new civil society organization with young leaders and a dedicated team with very little organizational capacity. At the time of award, the BelBiz Group went through the USAID Non-U.S. Organization Pre-Award Survey (NUPAS) evaluation with plans to strengthen its organization capacity. KEY FINDINGS FOR AID-VENTURE AID-VENTURE’s partner, Association BelBiz, is a proven, well-known NGO leader that continues to improve and creatively adapt to address local needs. The stakeholders and beneficiaries interviewed see AID￾VENTURE as a leader in VC in Belarus, not only promoting this new topic among government officials but also creating a community of local investors by establishing BBAN and working with TechMinsk Acceleration Fund to raise investment for local start-ups. The activity succeeded in strengthening BBAN’s organizational capacity to serve as an advanced platform for consolidating and advocating for Belarus’s angel investor community. At the same time, opinions are divided as to whether BBAN can sustain its activity without USAID support. Some respondents claimed that the association is already sustainable, as it unites prominent local investors that pay membership fees; others, including the project, doubt BBAN’s sustainability, as members are not actively involved in network activity, and association members include few or no investors. Analysis of the activity’s work plans show that AID-VENTURE comprehensively lists planned activities under each objective and cross-cutting issue and reflects results achieved in the previous years, as well as justifications for what was not implemented, and lessons learned. The timeframes for planned activities are clear and presented in a user-friendly way. However, although capacity building is mentioned in the activity report, in terms of NUPAS’ recommendations and its assessment of the financials, no plan is available for analysis. The report is structured around project objectives and includes discussions of problems encountered during the reporting period or results achieved. Analysis of the activity reports shows that the information presented in the documents is not well organized, and there is no clear summary of achievement either of activities or expected results against the annual work plan. 48 The evaluation team was tasked to analyze the management of the activities focused on implementing partners’ efficiency in planning interventions; reporting to USAID; communicating with stakeholders; taking mitigation measures promptly when implementation issues arise; collaborating, learning, and adapting; taking leadership positions in the respective sectors, etc. 39 Moreover, the length and format (small size with big intervals accompanied by big pictures) of the reports is not user-friendly or informative. Stakeholder interviews49 revealed that the activity’s communication and outreach efforts could be improved by delivering of information on project events (often provided too close to events) in a more timely fashion. Some stakeholders50 suggested that it would be beneficial for information to reach a wider audience, including other regions. The activity’s collaborating, learning, and adapting (CLA) approach is described as part of the Monitoring, Evaluation, and Learning (MEL) plan. As USAID introduced the CLA concept quite recently, the AID-VENTURE CLA plan will benefit from better approaches to internal collaboration involving all project partners. AID-VENTURE plans pause-and-reflect sessions to better understand the changing context, validate the Theory of Change, identify risks and opportunities, assess the relevance of monitoring data to decision-making, and align monitoring, evaluation, and learning (MEL) efforts across the activity. Planned and implemented CLA activities and expected results and results used for decision-making, along with the adapted project implementation plan, are to be presented in the Annual Plan and Annual Report (either in the monitoring and evaluation matrix file or in separate chapters of the report). The biggest hurdle that the activity faced was COVID-19, but the project effectively and promptly moved all its activity online with almost no delays. Noticing the strengthened capacity of the AID-VENTURE partners, respondents think that Association BelBiz needs to increase transparency, improve communication with partners and beneficiary participants, and focus on time management skills. KEY FINDINGS FOR DRIVES DRIVES’ partner IPM is experienced, highly professional, and a recognized leader in business education and research. DRIVES’ organizational capacity building and plans were focused on three areas and three project partners: 1) internal communications for PYXERA; 2) external communications strategy development for Global Business School Network (GBSN) and IPM; and 3) relationship/partnership building for PYXERA, GBSN, and IPM. On further analysis of its project documents, IPM’s institutional capacity plan is clear and well thought out. Analysis of the activity’s work plans shows that it is comprehensive and clearly lays out annual activities and cross-cutting issues, with well-articulated expected results under each objective. Timeframes for planned activities for each objective are specified, along with names of activities and dates of delivery for each. Expected results are quantified and deadlines for their achievement identified. The activity’s quarterly and annual reports are well-structured around the project objectives and include a Performance Management Plan indicator table and opportunities and challenges sections. Report presentation is user-friendly and presents information in written and visual form with balanced quantitative (in graphs and tables) and qualitative (citations, photos) formats. Moreover, information in the reports is presented against implementation plan parameters. Interviews with the activity staff, partners, and stakeholders revealed that the external communication and outreach efforts of the project could be improved in terms of audience and reach. Despite a very 49 KII#6, 13 50 Ibid 40 informative website, access to information is limited: Only subscribers or IPM graduates can receive news or information regarding future courses. Although information is made available through Facebook, stakeholder feedback suggests that minor adjustments on the website and other promotional efforts are needed to raise awareness about planned activities. As with AID-VENTURE, the biggest obstacle that the activity faced was COVID-19. However, as the majority of project activities were already in online form, it was easy to move networking events and business idea competitions online. IPM’s CISCO telepresence equipment and distance learning platform allowed the project to move all its activity online, effectively and promptly, with almost no delays. Although the pandemic presented a clear challenge, IPM Business School was in a better position than many businesses, schools, and organizations to adapt to the need for a completely virtual operation. IPM faculty moved quickly to adapt course materials. Just one week into the height of the pandemic, all courses at IPM Business School had been adapted or presented via CISCO teleconference equipment, allowing students and customers to connect from home and preventing disruption to learning. Regardless, respondents to this evaluation stated that IPM’s infrastructure should be strengthened to invest in professional development for the faculty and staff of newly established regional CEEs. DRIVES’ CLA approach is described as part of the MEL plan. It is short but well written overall. It can, however benefit, from replacing terms such as “coordination” with “collaboration”51 and dedicating some attention to internal collaboration among partner institutions. This is important with newly established regional CEEs. Document review and interviews revealed that three out of five regional managers had been changed throughout the years. Interviews indicate that these regional managers felt they lacked attention and appropriate ongoing training from IPM headquarters. Interviews show that regional coordinators received training when CEEs were established, but guidance was provided only during the project annual planning process. Analysis of DRIVES documents show that the CLA tool is used for learning and project activity adaptation, which will continue for the rest of the project cycle. DRIVES did a good job of developing new activities based on research results and drawing on IPM’s extensive experience. Private businesses and entrepreneurs show great demand for modern business knowledge and tools to be delivered to them through customized business education products. The introduction of distance learning courses, along with face-to-face consultative support and mentoring through CEEs, perfectly fit the existing gaps in business education in Belarus. IPM has long experience cooperating with USAID to implement initiatives of different types and introducing commercially and financially viable products.52 Respondents noted that, along with the introduction of new innovative online approaches to business learning, it is important to improve the governance of IPM by including CEE representatives on its board. The monitoring and evaluation system of IPM business education has improved, and now focuses on quantitative data (number of businesses established, new workplaces created, income/profit raise) and qualitative results (case studies, best practices of IPM graduates) which are constantly being identified, described, and presented in the courses. 51When coordination is the integration, unification, synchronization of the efforts of the people/organization to provide unity of action, collaboration is a working practice whereby individuals/organizations work together to find novel means to achieve defined and common outcomes. Coordination is usually managed through formal systems and procedures, while collaboration emphasizes adaptiveness in management action through teams and the creative resolution of interpersonal differences and organizational constraints. 52 DRIVES Program Description, page 31. 41 During the evaluation period, IPM strengthened its organizational capacity and expanded its activity to the regions. However, IPM must significantly improve its governance53 to align its institutional structure to the regional level; introduce incentives and operation procedures to its regional network; design a monitoring and evaluation approach that includes course evaluation (at least level 2-3 of the Kirkpatrick model for evaluation of educational activity) and written case studies and success stories for Belarusian companies; and train more faculties and trainers from regions, given that only one has been trained to date. Lastly, interviewees noted that IPM needs to do more cross-selling in regions and make greater efforts to identify and leverage government and private sector resources in the regions. KEY FINDINGS FOR CSISS CSISS is implemented by a young, dynamic, fast-growing, and effective team that is engaged and eager to learn. Its capacity building efforts are robust and mostly aimed at the SIBS team to enable it to provide quality support to its local partners and constantly develop and improve the SME Grow-Up Toolkit based on the activity’s results. The CSISS implementation plan reflects seven milestones and the MEL Plan, where information needed for activity realization is clearly presented with identified deadlines and expected results. CSISS reports are presented for each milestone and built around activities conducted during the reporting period. Although the information is more descriptive than analytical, it is accompanied by graphs, tables, and easy-to-read visuals that clearly helps present results against planned activities. CSISS was the only activity that reported on SIBS’s capacity building activities and plans for the next milestone, even though no written capacity building plan was identified. CSISS’s CLA approach is thorough, well designed, and covers all important aspects. During interviews, key SIBS managers54 noted that the organization might benefit from structured and guided capacity building activities, because as a young institution, it lacks depth of knowledge and understanding of organizational building blocks (e.g., governance and management structure, mission and strategic management, management practices and systems, financial and human resource management, internal control systems, resource mobilization, strategic outreach and partnerships, service delivery, and project performance management) and a strategic approach to development of the organization. SIBS leadership admitted that the organization has not undergone any organizational/institutional evaluation like NUPAS. Although NUPAS is not required, SIBS leadership noted that an independent and guided evaluation can provide SIBS with the needed understanding of what organizational components the organization should have and what components are missing (if any); what capacities are less or well developed; and, in the end, help the organization develop an informed and evidence-based capacity building plan to guide its development. Additionally, KIIs mentioned that capacity building activities are taking place on their own initiative and that a guided strategic approach to organizational development is needed. SIBS created a monitoring system and procedures to get feedback and adapt activity interventions to the customers’ needs. Stakeholder and beneficiary interviews show that CSISS constantly sought new ways to engage with and motivate local partners, taking into account that partner institutions are not paid to organize start-up schools55 or other project activities at the regional level. Local partners' engagement with the project was built on the opportunity to acquire new knowledge and skills through TOT methodologies for entrepreneurship training. Respondents56 confirmed that all new approaches and innovation were built on the analysis of available and most reasonable approaches. SIBS took incremental steps and made detailed evaluations of what did and did not work and took those analyses into consideration for their future planning. Fee-based approaches to selling 53 KII#35, 40, 41, 45 54 KIIs #16, 17, 18, 19 55 Because access to schoolchildren who studied in junior start-up schools should be granted by parents, evaluation is limited only to understanding of start-up school-related activities and results. 56 KIIs#16, 17, 18, 19 42 and franchise education services were used, among others, but revealed that potential customers were unable to afford even modest costs for getting new knowledge and skills. However, the dedication of the SIBS team and its enthusiasm and desire to achieve results outweighed its lack of experience and capacity. Throughout the evaluation, stakeholders and the SIBS team recognized the need to improve organizational capacity, strategic communication, organizational and management structure, human resources management, and overall public relations to reach a wider audience base. As mentioned throughout this report, the biggest unexpected challenge for all three USAID-supported activities was the COVID-19 pandemic. However, it did not severely affect any of the I3 activities, which all quickly transferred any in-person activities to an online forum. Their quick adaptability helped ensure that there were no long-term damages. It is to note that USAID staff members were very supportive during this time as well, providing a mode of communication through project managers to any proposals, requests, and conservations expressed by the I3 activity team members. CONCLUSIONS FOR EQ3 1. USAID’s strategy to work closely with and through local implementing partners to ensure “local ownership” was recognized by all stakeholders and beneficiaries. USAID/Belarus is widely recognized as highly effective and efficient in filling a critical development “niche” while working to increase the capacity of implementing partners. 2. Organizational capacity building and strengthening were considered to be the most important traits needed for improvement to each of the three I3 implementing partners, despite the relatively high level of capacity at the time of contract award. The quality of capacity development plans and the corresponding activities of I3 partners was difficult to evaluate, as only the DRIVES work plan contained details; the other two partners used only the brief description provided in the project documents. Moreover, reports of capacity building activities were presented only by CSISS in its milestones reports. 3. The depth, quality, and presentation of quarterly and annual reports varies from activity to activity. Some KIIs57 noted that language was a barrier at times, as these reports needed to be written in English. The quality of implementing plans for all I3 activities was good, although each used a different format and presentation. No template was provided at the time of award, so each activity created its own form of reporting. 4. Each of the I3 implementing partners demonstrated some practices of CLA. Partners solicited participant feedback and used the information to better align training and other project activities to local needs. Sharing and exchanging information with private sector partners and other stakeholders was also done regularly, although results would be improved if this had taken place more systematically (e.g., newsletters and success stories or position papers regularly shared with GOB representatives or step-by-step case studies created to share experiences from idea to funding). I3 implementing partners were also able to quickly adapt to COVID-19 challenges and either use established online resources or quickly migrate to an online forum. 57 KIIs# 20, 21, 22, 25 43 5. The I3 implementing partners each have the capacity to sustain their position as viable organizations. Each I3 activity provided key services needed by the business community, but each could benefit from additional training and support in reporting, planning, and PSE. IPM would benefit from management training focused on the development of regional offices or perhaps introduction of a franchising strategy as a way to improve the sustainability of the regional office network. Although the current I3 implementing partners are strong, USAID may benefit from broadening the number of new implementing partners, especially in the regions. 44 3.4 KEY FINDINGS FOR EQ4 How successful have implementing partners been in their PSE efforts? What PSE approaches can be scaled up? USAID/Belarus is actively working with a cross-section of citizens and key stakeholder groups to accelerate the country’s transition to a democratic-based market economy, while improving the growth and competitiveness of private business. To achieve this, especially in recognition of limited budgets, implementing partners need to actively engage and collaborate with the private sector to leverage resources and align assistance efforts to achieve greater scale, sustainability, and outcomes. Ideally, for PSE to be successful, the private sector should be engaged in developing the initial project concept and play an active and cooperative role throughout implementing partners’ design and implementation of project activities. Although not required of the I3 implementing partners, each project did consult private sector representatives in designing I3 activities and applied PSE approaches in varying degrees, contributing to the achievement of project goals while improving prospects for future support and active collaboration with the private sector. KEY FINDINGS FOR AID-VENTURE The AID-VENTURE team used a holistic approach in its engagement with the private sector. The activity was well-positioned to serve as an essential catalyst to consolidate multiple private investors while delivering essential training that enhanced their financial analysis and start-up mentoring skills. AID-VENTURE responded well to feedback from investors and aspiring entrepreneurs to tailor workshops and other training activities. According to interviews, participants in angel investor international study tours and local training events became the core group founding the AngelsBAND,58 which played an important role in influencing positive changes in policy and legislation governing this segment of the financial sector. The private sector was engaged in organizing and implementing networking events and start-up competitions. Although perhaps not at the same magnitude as initial events, angel investors do indicate a willingness to provide funds necessary to continue some key events, according to an interview with a local investor. 59 AID-VENTURE accelerated local investors’ progress toward self-reliance by joining them to address problems that would not otherwise have been overcome. Without AID-VENTURE’s effective PSE approaches, it is doubtful that individual Belarusian angel investors would consolidate, increase their professional capacities, or make progress toward becoming a sustainable and effective platform uniting investment with start-ups, raising capital from local private investors into TechMinsk acceleration fund (approx. $1 million in commitments) and mentoring startups. KEY FINDINGS FOR DRIVES Managed by PYXERA Global and implemented by IPM Business School, the DRIVES program created a virtual learning platform that allowed aspiring entrepreneurs and SME owners outside of Minsk to access business courses tailored to their needs. Once again, a USAID-funded project acted as a catalyst by collaborating 58 KII#9 59 KII#12 45 with this private school to remove constraints preventing its access to regional markets. DRIVES provided funding needed to purchase state-of-the-art technology (at a negotiated discount), and IPM continues to fund maintenance and upgrades to the system, including creation of its internal TV network. IPM faculty also responded to feedback from business students to tailor the blended-learning courses, making them more practical. In addition, DRIVES actively engaged the private sector—specifically, businesses owned by, or employing, alumni of IPM programs—to present or judge events such as Best Start-Up Competitions, where start-ups were matched with mentors from IPM.60 Using PSE approaches, DRIVES successfully mobilized funding and expertise of the private sector while allowing the program to reach new markets through technology and establishment of a network of regional offices. KEY FINDINGS FOR CSISS The CSISS team used a broad spectrum of events to engage the private sector. Events such as Investor Day Belarus brought together business leaders, investors, aspiring entrepreneurs and other stakeholders, and regional roundtables conducted in 10 cities across Belarus took initial steps to raise awareness among leaders of business, education, and government about the importance of joining efforts to improve local entrepreneurial ecosystems. Although the targeted beneficiaries of CSISS activities were primarily students, aspiring entrepreneurs, and start-ups, the team successfully applied PSE approaches to create a foundation to build future private sector collaboration in the regions through the delivery of classes in Startup Schools by business people, sharing their knowledge and experience free of charge. CSISS includes about 100 business experts in its network. PSE APPROACHES FOR SCALING-UP USAID/Belarus successfully targeted key strategic areas in which limited technical assistance could achieve significant results by cultivating private sector partnerships. Each of the selected I3 implementing partners demonstrated that application of various PSE approaches contributed to the effectiveness and efficiency of project activities. However, to develop a deeper collaboration with the private sector the application of PSE approaches earlier in the project development process, inclusion of specific performance metrics into contractual agreements, and provision of additional PSE training and guidance are needed. A higher level of cooperation may be needed to ensure sustainability of project results and wider mobilization of private capital necessary to magnify their scale, particularly in Belarus’s smaller communities. ● Multiple private sector representatives should be engaged early in project concept development. It is not uncommon for USAID Missions to limit communications between the program team and potential private sector partners until approval of concept papers. This violates a basic tenet of PSE, which calls for open dialogue with prospective partners throughout the process. Early input from the private sector during project development and, later, into implementation partners’ work plans, can ensure better alignment of assistance and greater collaboration needed for sustainability. ● Inclusion of PSE performance requirements into partner agreements coupled with early trainings and continued guidance will advance the number and depth of private sector engagements. A contractual obligation to apply PSE approaches coupled with tailored training and 60 KII#45 46 guidance can be expected to result in greater collaboration and leveraging of private sector expertise and financial resources. ● Creation of project advisory councils, particularly in the regions, contributes to participation and support of the private sector and other key stakeholders. One way to improve the alignment and responsiveness of programming is to create structures where local stakeholders can give periodic feedback on their observations of both programming and context. Representatives of the private sector, education, and local government can share critical insights while also advocating for support and changes needed to improve their local entrepreneurial ecosystem, including identifying current constraints to business growth. Although I3 implementing partners reached out to the private sector for feedback and guidance, a wider use of more formal structures is recommended. ● Expanded use of local business leaders to lead trainings or provide guest lectures builds support and contributes to better alignment of assistance to local needs. Each of the I3 implementing partners applied this PSE approach to various degrees in multiple events, such as lectures, training courses, and judging competitions. However, the extent of their participation and effectiveness of their training could be enhanced by offering potential instructors opportunities to develop skills through TOT workshops and possible participation in study tours. This is especially true in the regions where businesses they represent can become candidates for greater PSE collaboration, including financial support, hosting student interns, and sponsoring research. ● Greater use of international and domestic study tours can develop critical knowledge, skills, and professional networks needed to improve SME performance. Study tours organized by AID-VENTURE and DRIVES proved to be both effective and popular with participants. Collaborating with the private sector to jointly design, implement, and co-fund study tours should be explored as an approach to identify value chain constraints and develop opportunities to mitigate the barriers. ● Start-up angel fund matching grants can help unlock additional private investment. AID￾VENTURE achieved great success in uniting angel investors while improving their capacity to support the growth of start-ups. The introduction of matching grants can reduce risks to investors as they work to build start-ups’ success. The additional funds for product development, combined with zero dilution of equity, can motivate angel investors’ willingness to invest in higher risk innovations, increasing the number of firms supported while increasing their likelihood for success. It is recognized that a careful review of governing legislation would be needed before considering this option. ● Cooperation with the private sector should be explored to provide financial support and technical assistance to the network of regional technoparks, incubators, and university research and development (R&D) centers: The technical park in Minsk enjoys great success, while regional efforts have struggled to become relevant. CSISS successfully cooperated with some state university structures. USAID may want to consider leveraging its convening power to bring together local representatives of the private sector, education, and local government to create a joint action plan aimed at enabling the technoparks and universities to support innovation effectively, contribute to SME performance, and facilitate growth of start-ups. 47 CONCLUSIONS FOR EQ4 1. All I3 implementing partners were relatively successful in applying various PSE approaches. The partners should continue to include PSE requirements in their contractual agreements, such as performance metrics, and provide PSE training early in implementation. This can result in greater collaboration and leveraging of private sector expertise and financial resources. 2. Introduction of new PSE approaches can facilitate sustainable improvements to regional ecosystems. This could include early introductions of private sector needs in project concept development to instill behavior changes overtime. The goal would be an intentional shift toward enterprise-driven development as a more sustainable way to empower people and communities towards a more collaborative and open business. 48 3.5 KEY FINDINGS FOR EQ5 Are there new, emerging entrepreneurship development needs in Belarus to be addressed? In the last five years, Belarus has been making steady improvements in the business climate for SMEs and entrepreneurship. The highly educated and inexpensive workforce, tax benefits from high-tech parks (HTP), and political stability have made the country a magnet for IT start-ups. This provides a rare bright spot in an economy still dominated by SOEs in heavy industry and collective farms. USAID/I3 assistance contributed to advancements in the IT SME and business sector. This assistance increased the availability and quality of business training, improved access to start-up financing, and enhanced the entrepreneurship ecosystem, particularly in regions outside of Minsk. Unfortunately, the recent ongoing crackdown on protesters demanding an end to President Lukashenko's 26-year reign threatens these achievements.61 This ongoing crisis means the landscape could be shifting in the business sector. During the time of the evaluation, the following emerging entrepreneurship development needs were assessed: BUSINESS EDUCATION AND SUPPORT INFRASTRUCTURE Absence of “One-Stop” Business start-up and SME Development Support. Multiple interviewees and survey participants highlighted the difficulty of starting a business in Belarus. In addition to political risks for foreign investors investing in Belarusian legal entities, the process is confusing and complex. Little, if any, guidance is available. As a result, many IT start-ups reportedly register their companies abroad. Establishing a network of "one-stop" support centers throughout Belarus, based on the U.S. Small Business Administration's model, may help USAID mitigate these challenges. Typical services of this model include guidance in completing required forms when starting a business. Additional services can include connecting entrepreneurs with funding and short courses on planning, launching, and growing a business. In the United States, most such centers are affiliated with universities or colleges. Ineffective Innovation and Entrepreneurial Support Facilities. Interviewees and survey respondents indicate that five of the six Science and Technology Parks established by universities in Belarus are poorly organized. Parks noted include the “Polytechnic” Technopark of the Belarusian National Technical University (Minsk), the Technopark of Vitebsk State Technological University, the Technopark of Polotsk State University, the “Polesye” Technopark of Polesye State University, the “Unitechprom BSU” Technopark of Belarusian State University, and the “Technolab” Technopark of Ya. Kupala State University (Grodno). Other than the Minsk facility, according to survey respondents, few offer quality business services needed by aspiring entrepreneurs or existing businesses. Technical assistance for the universities and their entrepreneurial support facilities tailored on a “case by case basis” holds great potential. Possible areas include promoting the development of innovative entrepreneurship for university teachers and students, improving the R&D capacity of personnel, and aligning the facility services and university business programs to support innovation and local business needs. 61 The effects of the election are discussed in the Limitations and Conclusions section. 49 TABLE 5: EXAMPLES OF ASSOCIATIONS OPERATING IN BELARUS62 ASSOCIATION NAME TYPE OF ORGANIZATION NUMBER OF ORGANIZATIONS WEBSITE Association of Business Education Mostly state providers of business education 18 www.facebook.com/abe.belarus/ Association of Management Development Mostly major private providers of business education 6 www.amdbelarus.wordpress.com Association of Companies Providing Consulting Services and Business Education Mostly minor private providers of business training 5 No website63 Source: Authors’ representation Application of best practices in incubators, research centers, associations, and business development programs within these institutions would advance the goals of the National Strategy for Sustainable Development of the Republic of Belarus—2030. The strategy envisages transition to an economy based on innovation and knowledge by advancing the concept of the entrepreneurial university. Noting that they are difficult to work with, most experts agree that Belarusian universities "can and should play a key role in the development of entrepreneurship, relying on their material and technical base, experience, and competence, as well as established international relations. They are difficult to work with, but hold good potential for helping a broad audience. If successful, the changes also should be sustainable.”64 ACCESS TO FINANCE Inadequate Financial Resources Available to Grow Business: Despite AID-VENTURE’s achievements in uniting investors and matching them to viable start-ups, our survey identified the inability to access funds needed to operate and expand a business as the most constraining obstacle to their success. Even if an entrepreneur has enough personal capital or is successful in receiving angel funds during start-up, the risk of running out of the financial resources necessary to survive remains significant. The inability or reluctance to access SME funds made available through the GOB complicates the issue, especially for entrepreneurs outside of Minsk or in rural communities. USAID should continue TA focused on angel funding and start-ups, while expanding the program to include 1) mobilization of greater amounts and types of seed funding, and 2) assistance in the development of the Belarusian VC market. Business angels need a well-functioning VC market to provide the follow-on financing that the businesses they support will require. At the same time, a well-developed angel market can create more investment opportunities and increase the deal flows for VC investors. Broadening the financing options available and accessible to SMEs after the start-up phase is a long-term challenge to improving the SMEs' capital structure and investment capacity and reducing their overreliance on the traditional lending channels. USAID/ Belarus should expand its assistance through more of the business lifecycle, including such tools as matching grants and leasing. 62 Belarus Digest, January 2017. 63 http://ta-aspect.by/belorusskie-biznes-shkoly-i-konsaltingovye-kompanii-obedinilis-v-associaciyu 64 Morozov, R. (2019). Why does Belarus need an entrepreneurial university 3.0? Retrieved from http://news.21.by/other￾news/2019/01/09/1697779.html 50 In the aftermath of the current democratic crisis, it can be expected that many private investors will have withdrawn from the market. At this point, the GOB may be open to technical assistance aimed at creating policies and programs sustaining these markets. At the very least, the government may be willing to address the financing gap for innovative or growth-oriented enterprises. A new challenge—and opportunity—may be to introduce approaches that leverage public resources or guarantees while developing appropriate risk-sharing mechanisms with private partners. MONOPOLY OF INDUSTRY BY STATE-OWNED ENTERPRISES Industries dominated by the state block development of private business: SOEs remain the largest employer in Belarus, not only providing critical public services to citizens but producing equipment and products for export and domestic consumption. The World Bank estimates SOEs contribute to 47 percent of GDP and 75 percent of industrial output. It is generally accepted that the majority of SOEs are plagued by mismanagement, corruption, poor governance, and weak regulations, and often produce losses instead of profits. Even before the current crisis, the GOB was exploring ways to restructure and improve SOE performance. One scenario of the post-crisis period includes mass restructuring, privatization, or closing of ineffective enterprises. This will result in layoffs for thousands. In some parts of the country, there may be no employment alternative to the state-run plant or a collective farm. USAID may be best positioned among the donor community to help improve regional service providers’ capacity to retrain the many downsized employees. Most will need to acquire new skills demanded by the market. Others will need to gain skills and support needed to start their own business.65 The process of SOE restructuring and privatization may also identify multiple production problems and value chain inefficiencies. This would present new opportunities for innovations and start-ups. CONCLUSIONS FOR EQ5 1. Continued assistance in the “niche” areas currently addressed by USAID is needed for start-up finance, business training, and facilitating access to professional networks and markets. Regardless of the outcome of the current crisis in Belarus, the economic environment in general and SMEs specifically will need additional assistance to improve the access to finance required for them to grow and become competitive. USAID should also assist in simplifying the regulatory environment for microenterprises and the self-employed. 2. Working with and through business associations is key. This may accelerate the development of a more competitive business training and consulting market in the regions of Belarus. Belarusian business associations are historically trusted by their members to provide needed advocacy, good networking opportunities, and training opportunities and other services needed for business development. Many have representative offices and member businesses based in the regions. 3. It would be beneficial to work in collaboration with the GOB. If, and when, the GOB recognizes the need to improve SOE management and operations, demonopolize sectors in which they are working, 65 KIIs #52-54 51 and promote privatization, USAID is well-positioned to assist new and existing SMEs and entrepreneurs wishing to take advantage of the created market opportunities. 4. The current size of the public sector and GOB-provided services is significant. Introduction of transparency and competition into the contracting process would potentially create numerous opportunities for SMEs. Initiating an e-government procurement program, accompanied by training focused on how SMEs can access public procurement contracts, would significantly contribute to the growth and success of the private sector. 52 4. CONCLUSION Based on the evaluation findings, each of the three project activities evaluated was properly designed, relevant, and delivered assistance aligned with beneficiaries’ needs. The three I3 activities successfully contributed to strengthening various aspects of the economic ecosystem needed for SMEs and start-ups to launch, grow, increase employment, and become competitive. The current modes of implementation used by AID-VENTURE, DRIVES, and CSISS have been efficient in achieving project goals and creating a supportive ecosystem for aspiring entrepreneurs, start-ups, SMEs, and local investors. The USAID/Belarus team was innovative and successful in identifying the right niche, using relatively small budgets aimed for maximum impact. In addition, the selected group of local partner organizations that contribute to the development of Belarus showed an impressive amount of dedication. The entrepreneurs participating in project activities recognize, appreciate, and value USAID assistance benefits. Belarusian women currently in business, or aspiring to become entrepreneurs, appear to be proactive, ambitious, well-educated, and recognized as valued team members and leaders. Throughout the evaluation process, the USAID/Mission staff were open, knowledgeable, and supportive. This was especially appreciated during the challenging environment of COVID-19 protocols, widespread protests, and the government crackdown. With the ongoing exodus of entrepreneurs and IT professionals, the future economic development environment and business needs of Belarus will be dramatically different. SHIFTS IN ENTREPRENEURSHIP PERCEPTION Stakeholders and beneficiaries acknowledged in interviews that the overall public image, attitudes, and awareness of entrepreneurship have improved since 2017. Respondents mentioned that, as more start-ups were created, their success became more visible to the public. Entrepreneurs have begun to use high-quality techniques to gain visibility; for example, new businesses have created websites that are more stylish and attractive. There is more information about start-ups in the local media (for example, the “About Business” section on the TUT.by website).66 Universities are actively involved in holding competitions for start-ups (for example, the Mogilev Invest Day 2019 competition, which was held at the Belarusian-Russian University). Entrepreneurs have begun to communicate more with each other to make common decisions. Successful examples of start-ups’ growth (with increased initiatives and events in the ecosystem) include the Business Awakening conference that has become popular and in demand in Belarus and Camps LipenPro, which have diverse accelerators, start-up contests, and hackathons. An increasing number of successful young entrepreneurs are setting the tone in society to show there are no boundaries to what can be achieved today. These efforts made certain contributions to changes in regulatory and legal acts aimed at supporting the development of entrepreneurship. A survey respondent said that the I3 project “creates a favorable environment for business in the regions, forms a positive image of entrepreneurship, in which people can believe in themselves, see successful examples from their own cities, when they know what others have done and convinced that I can succeed as well, when they can get support and expertise from famous entrepreneurs from all over Belarus, when they feel involved in the world of entrepreneurship. The 66 https://www.tut.by/ 53 project just created that community that supports and is full with knowledge that is necessary like air at the start of your business.” Although this holds true, some respondents still insisted that the public image of new business ventures is developing slowly as a significant part of entrepreneurs’ lack of knowledge about start-up schools, especially among those 40 to 55 years old. There is still a lot to do “to popularize and make the same start-up schools more accessible, because they are extremely difficult to promote, especially without a promotion budget, and still a large part of the population simply does not know about them.” PROJECT SUSTAINABILITY Each of the I3 implementing partners made progress toward sustainability in terms of organization and project activities.67 The AID-VENTURE BelBiz team's competence engaged successfully with the private sector to develop the Belarus Business Angels Network and the Business Angel Academy. Although threatened by the ongoing crisis, the Network and Academy have the skills and relationships needed to continue services in the long-term. The DRIVES IPM team plans to continue to deliver the entrepreneur courses they introduced. With new technology and a network of regional offices, they are well-positioned to continue to make the courses available throughout Belarus. IPM’s biggest challenge in achieving sustainability is learning how to manage the regional network effectively. The SIBS implementing CSISS has strong leadership and is well respected in the regions. Its sustainability challenges include the high turnover of local partners and administrators within partner institutions and the audience's willingness to pay fees. Moreover, SIBS has made extensive efforts to look at different ways to sustain project activities. But in the current economic situation, fee-based training is not sellable, especially when other donors, like UNDP with state support, propose free courses. In conclusion, it is important to mention that building the sustainability of each I3 activity will benefit from better evidence-based results based on the activity’s long-term outcomes and not exclusively on outputs. Outcome data can be used for selling services in the future (e.g., SIBS can say that a large percent of participants of Start￾Ups Schools established a significant number of new businesses and employed a lot of people). RECOMMENDATIONS FOR FUTURE PROGRAMMING The USAID/Belarus team proved to be innovative and successful in identifying the right “niche” to effectively use relatively small budgets for maximum impact. Investors and young entrepreneurs, start-ups and SMEs, local educational partners and private entrepreneurs participating in the project activities recognize, appreciate, and value the benefits of USAID assistance. Currently, the I3 project is the largest initiative in the field of private entrepreneurship development that has promoted the public importance of entrepreneurship, provided access to start-ups and SMEs for needed knowledge and business education in regions, and introduced the venture funding approach to local investors and firms. The I3 activities have substantial potential in: ● Policy development at the national level. 67 The evaluation SOW did not ask the team to focus on sustainability, however, in conducting the work, the theme did arise. After conversations with USAID/Belarus, the Mission asked the ET to provide a quick note on this and identify areas of sustainability where possible. This has been done throughout the sections, as relevant. 54 ● Institutional and individual capacity building of start-ups, SMEs, local investors, and women entrepreneurs, and strengthening business support infrastructure. ● Building and strengthening the community of local private venture investors, advancing start-ups capacity to work with investors, and raising funding for venture funds. These evaluation findings help to provide USAID/Belarus with possible adjustments that could enhance current activities as well as the design of the follow-on private sector development project. Based on the findings, the ET outlined key recommendations (see Table 6 below) drawn from the key findings of each EQ on the opportunities and key areas for USAID/Belarus to consider in planning for future activities. Based on the need and current country situation, the recommendations are classified as either high, medium, or low priority for USAID’s consideration. Table 6 outlines key recommendations for USAID/Belarus to consider: TABLE 6: LIST OF KEY RECOMMENDATIONS RECOMMENDATIONS AREAS OF POSSIBLE INTERVENTIONS PRIORITY Increase focus on regional-level programs ● Expand public outreach, education, awareness campaigns on the benefits and positive aspects of being entrepreneurs, produce newsletters and case studies highlighting success stories. ● Connect local education institutions and align their R&D services and short courses to business needs. ● Improve access to informal education by improving the capacity of business trainers, consultants, faculty of educational institutions, NGOs, and local civil society. ● Strengthen the capacity of business support organizations, and local experts and trainers. ● Strengthen coordination and collaboration with local public authorities through public councils, project advisory boards, joint events, agency cooperation, etc. ● Strengthen coordination and collaboration with the local private sector through joint support of regional technoparks and incubators, research centers, educational institutions, and financial assistance programs. High 55 TABLE 6: LIST OF KEY RECOMMENDATIONS RECOMMENDATIONS AREAS OF POSSIBLE INTERVENTIONS PRIORITY Increase the number of start-ups and SMEs to enhance competition, especially at the regional level ● Establish one-stop shops for start-ups and SMEs to provide information, administrative services, registration assistance, etc. ● Invest in infrastructure improvement (potentially in coordination with and/or based on established incubators, technoparks, or co-working spaces). ● Link programs between local and foreign sector associations, SMEs, and multinational companies. ● Simplify the policy and regulatory framework for new and existing businesses regardless of size, sector, and ownership status. ● Increase engagement of local successful entrepreneurs and leaders in training, consulting, mentoring, and joint initiative design and implementation. ● Improve metrics to measure start-ups’ and SMEs’ outcomes (number established, workplaces created, income, profit growth). High Increase Access to finance ● Provide grants or matching grants for local initiatives, social enterprises, and women’s initiatives. ● Continue improving the related legal and regulatory environment for VC. ● Work with local investors to increase their capacity and foster a second round of investments (through BBAN). ● Increase the accessibility of GOB-guaranteed loans. High Attract more attention to support to women entrepreneurs ● Broaden networking opportunities for female entrepreneurs. ● Initiate public image and awareness campaigns promoting women’s successes online and at events. ● Organize a community of practice in research. High Improve communication and outreach to entrepreneurs, especially youth, in regions ● Publish newsletter and/or establish social media. ● Systemize information and education campaigns. ● Engage foreign business representatives and leverage alumni of USAID activities and partners (such as IPM). ● Organize entrepreneur study tours within Belarus and neighboring countries to visit successful businesses, angel investors, and start-up support groups. Moderate Improve communication with state and regional authorities ● Increase cooperation with existing local economic development and employment agencies (or support their creation). ● Coordinate support to regional technoparks and incubators activity. Moderate 56 TABLE 6: LIST OF KEY RECOMMENDATIONS RECOMMENDATIONS AREAS OF POSSIBLE INTERVENTIONS PRIORITY Strengthen the organizational capacity of I3 local implementing partners ● General: unified activity report template; CLA/PSE training; template for local best practices, success stories, and case studies; pool of local implementing partners; project advisory councils. ● SIBS in strategic approach in development, governance, HR management (perhaps conducting a NUPAS evaluation to structure its understanding of organizational capacity elements). ● DRIVES in communication with potential beneficiaries, governance, M&E of educational courses; management of regional network. ● AID-VENTURE in communication with and outreach to clients with timely information and time management. Moderate CONCLUDING REMARKS The ET determined that the I3 project successfully improved the enabling environment, improved management capacity, and increased access to finance. Findings determined that USAID/Belarus identified key needs that other donors were not currently addressing and developed a “niche” where effective use of limited resources by the right implementing partners achieved ambitious goals. The evaluation process used to reach these conclusions was anything but typical. The COVID-19 pandemic and political unrest eliminated several normal evaluation circumstances. The team missed the opportunity to visit Belarus, so the entire evaluation was conducted remotely. In addition, with several connectivity issues and interruptions, challenges in communication were more present than ever. The arrest of numerous protesters, many whom the ET was scheduled to interview, magnified the challenges. The ET tried to address uncertainties as much as possible throughout the report, but inevitably, conducting an evaluation during a period of significant unrest introduces greater variability into the findings, particularly when it comes to recommendations for future work. With that said, the fact that the interviews and responses were met with the willingness and efforts of Belarusian citizens to find time to share their project experiences, views, and suggestions is a testament to their recognition of the value of the USAID project’s contributions to the future of their country. The importance of future USAID/Belarus programming is greater than ever before. Whatever the outcome of the current crisis, the recent achievements and progress made are now in danger. It is expected that USAID assistance will not only need to reclaim lost ground by continuing much of the current I3 programming, but will need to broaden and intensify efforts to overcome the new and great challenges of the post-revolution environment. 57 REFERENCES Aginskaya, H. and M. Akulava (2018), Женщины в бизнесе Беларуси: особенности, мотивация и барьеры (Women in Belarus’ Business: features, motivation and barriers), BEROC, Minsk, http://eng.beroc.by/research/policy_papers/pp56_en. AID-VENTURE Annual Work Plans, 2014-2020 AID-VENTURE, PMEP, 2015, 2018, 2019 AID-VENTURE, Program Description, 2014, AID-VENTURE, Quarterly and Annual Reports, 2014-2020 Belarus Economic Research and Outreach Center (BEROC) webpage, http://www.beroc.by/publications/ Belarus Presidential Decree No. 4, (2010), “On the Development of Entrepreneurship and Stimulation of Business Activity in Belarus” Belarus Presidential Decree No. 7, (2017), “On the Development of Entrepreneurship” Belstat (2019), Belarus in Figures, National Statistical Committee, Minsk, http://www.belstat.gov.by/en/ofitsialnaya-statistika/publications/statistical-publications-data￾books-bulletins/public_compilation/. BEROC. (2020). “SME Survey Results”, http://covideconomy.by/business. Cornell University, INSEAD, WIPO, (2019), Global Innovation index 201, https://www.wipo.int/edocs/pubdocs/en/wipo_pub_gii_2019.pdf CSISS, (2017), Program Description CSISS, (2020), SME GROW-up Toolkit CSISS, (2020), start-ups School digest, email from September 4, 2020 CSISS, Milestone 1-5 Reports, 2018-2020 CSISS, PMEP, 2017-2020 DRIVES, (2019), Impact Assessment Report DRIVES, Annual Work Plans, 2014-2020 DRIVES, PMEP, 2015, 2018, 2019 DRIVES, Program Description, 2014, 2017, 2019 DRIVES, Quarterly and Annual Reports, 2014-2020 EBRD, ETF, EU, OECD, (2020), SME POLICY INDEX: Eastern Partner Countries 2020 IFC, EU, SIDA. (2017). Business environment in Belarus, research IMACON Consulting Group, (2016), Mid-Term Evaluation Report of DRIVES IMF (2019), Belarus Country Report No. 19/9, International Monetary Fund, Washington, D.C., https://www.imf.org/en/Publications/CR/Issues/2019/01/18/Republic-of-Belarus-2018- Article-IV-Consultation-Press-Release-Staff-Report-and-Statement-by-46526. IPM Research Center webpage, http://www.research.by/ IPM Research Center, (2020), SME survey 2019 (full report), http://www.research.by/webroot/delivery/files/pdp2019r06.pdf National Bank of the Republic of Belarus, “Program for Social and Economic Development (PSED) 2016-2020” SME Development Strategy 2030 UNECE, (2020), Innovation for Sustainable Development Review of Belarus, https://www.unece.org/index.php?id=47157 USAID (2019), Project Appraisal Document (PAD) for I3 Project World Bank (2018), Enterprise Surveys: Belarus 2018 Country Profile, World Bank, Washington, DC, http://www.enterprisesurveys.org/en/data/exploreeconomies/2018/belarus#firm- characteristics 58 World Bank (2019), Doing Business 2020, World Bank, Washington, DC, http://www.doingbusiness.org/en/doingbusiness. World Bank (2019), World Development Indicators (database), https://databank.worldbank.org/source/world-development-indicators 59 ANNEX 1: STATEMENT OF WORK 60 61 62 63 64 65 66 67 68 69 70 71 72 73 ANNEX II: EVALUATION DESIGN MATRIX 74 75 ANNEX III: DATA COLLECTION TOOLS 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 ANNEX IV: LIST OF INTERVIEWS TABLE 7: LIST OF KEY INFORMANT INTERVIEWS Table Redacted in public version 100 TABLE 8: INTERVIEWEES REACHED STAKEHOLDER GROUP PLANNED REACHED INTERVIEWED DIFFERENCES USAID/Belarus 3 4 3 0 CSISS staff, partners and activity-related stakeholders and beneficiaries 63 15 14 -21 AID-VENTURE staff, partners and activity-related stakeholders and beneficiaries 17 14 DRIVES - staff, partners and activity-related stakeholders and beneficiaries 24 14 Donors 6 8 6 0 Think Tanks 2 5 2 0 National authorities 5 3 2 -3 TOTAL 79 75 55 -24 101 ANNEX V: ONLINE SURVEY RESPONSE DETAILS TABLE 9: ONLINE SURVEY RESPONDENTS BY LOCATION LOCATION/OBLAST NUMBER OF RESPONDENTS Minsk 26 Brest 5 Vitebsk 8 Gomel 3 Grodno 2 Mogilev 5 Outside of Belarus 1 TABLE 10: ONLINE SURVEY RESPONDENTS BY GENDER GENDER OF RESPONDENT NUMBER OF RESPONDENTS Male 30 Female 20 TABLE 11: ONLINE SURVEY RESPONDENTS BY I3 ACTIVITY I3 ACTIVITY NUMBER OF RESPONDENTS AID-VENTURE 9 DRIVES 12 CSISS 25 Other 4 102 TABLE 12: ONLINE SURVEY RESPONDENTS BY PROJECT INTERVENTION PROJECT INTERVENTION THAT RESPONDENT IS A PARTICIPANT OF NUMBER OF RESPONDENTS Improvement of legal/regulatory framework 2 Policy development 3 Training of trainers 5 Training 23 Education program 35 Access to funding/investments program/competition 8 Local initiatives support 16 Networking events (conferences, Expert/Investor days, start-ups events, etc.) 27 Consultation 20 Partnership/ bringing start-ups and investors - Acceleration program 6 Study visits 1 Grant 24 103 TABLE 13: ONLINE SURVEY RESPONDENTS BY CATEGORY RESPONDENT CATEGORY NUMBER OF RESPONDENTS Start-ups/Young entrepreneurs 13 Women entrepreneurs 9 Participants of Expert and Investors Days 7 Winner of local initiatives 6 Faculties, teachers, instructors, business experts 15 Business plan competition applicants 3 Venture/local investors 3 BBAN staff/ members 2 Training participants of business angels/ venture investor/ venture fund management 3 Participants of acceleration program 3 Participants of start-up events, etc. 14 Business support organization 11