CFG FINAL PERFORMANCE EVALUATION OF COOPERATION FOR GROWTH PROJECT Final Evaluation Report DISCLAIMER This work plan is made possible by the support of the American people through the United States Agency for International Development (USAID). The contents are the sole responsibility of the authors and do not necessarily reflect the views of USAID or the United States Government. This publication was produced at the request of the United States Agency for International Development. It was prepared independently by IMPAQ International, LLC under Contract No. AID-167-I-17-00004, Task Order No. 72016921F00002. Implemented by: IMPAQ International 1325 G Street, NW Suite 900 Washington, DC 20005 ABSTRACT: In 2018, USAID/Serbia launched the four-year, $8.8 million CFG Activity to improve Serbia’s private sector competitiveness through strengthening the business enabling environment and addressing regulatory constraints to private sector development, ultimately to improve conditions for broad-based, inclusive economic growth. Within this context, the CFG Activity is designed to engage SMEs and other stakeholders in open and effective dialogue to improve the business enabling environment, thereby increasing overall confidence in the private sector and building the foundation for increased private sector competitiveness and investment. Specifically, the CFG Activity aims to generate (1) an improved business environment and related administrative efficiency in selected areas and (2) improved SME access to finance. In 2021, IMPAQ International (IMPAQ) was awarded a contract from USAID/Serbia to conduct a final performance evaluation of the CFG Activity. IMPAQ conducted a rigorous utilization-focused evaluation that integrates quantitative and qualitative methods to enrich USAID’s understanding of the evaluation questions and provided USAID/Serbia with actionable recommendations to improve the efficiency and sustainability of its economic growth programming moving forward. CONTENTS ACRONYMS I EXECUTIVE SUMMARY III INTRODUCTION 1 ACTIVITY BACKGROUND 1 EVALUATION PURPOSE 1 EVALUATION METHODOLOGY 2 DATA COLLECTION 2 DATA ANALYSIS 3 KNOWN LIMITATIONS AND MITIGATION STRATEGIES 3 FINDINGS AND CONCLUSIONS 5 EVALUATION QUESTION 1: 5 EVALUATION QUESTION 2: 10 EVALUATION QUESTION 3: 19 EVALUATION QUESTION 4: 24 EVALUATION QUESTION 5: 28 EVALUATION QUESTION 6: 33 RECOMMENDATIONS 38 ANNEX 1: EVALUATION SCOPE OF WORK 41 ANNEX 2: EVALUATION METHODOLOGY 44 ANNEX 3: DATA COLLECTION TOOLS 55 ANNEX 4: LIST OF DOCUMENTS REVIEWED 81 ANNEX 5: ANALYSIS OF CFG KEY PERFORMANCE INDICATORS 82 ACRONYMS AFA All for All AIF Alternative Investment Funds AMELP Activity Monitoring, Evaluation, and Learning Plan B2B Business to Business CAGR Compound Annual Growth Rate CDCS Country Development Cooperation Strategy CEFTA Central European Free Trade Agreement CFG Cooperation for Growth CLA Collaborating, Learning, And Adapting COP Chief of Party COR Contract Officer Representative COVID Coronavirus DO Development Objective EBRD European Bank for Reconstruction and Development ECS eCommerce Association of Serbia EQ Evaluation Question EU European Union EUR Euro FGD Focus Group Discussion FIC Foreign Investors Council FX Law Law on Foreign Exchange Transaction FY Fiscal Year GDP Gross Domestic Product GEFE Gender Equality and Female Empowerment Policy GESI Gender Equality and Social Inclusion GoS Government of Serbia IBE Improving Business Environment IFC International Finance Corporation IR Intermediary Result IT Internet Technology KII Key Informant Interview LGBTQI Lesbian, Gay, Bisexual, Transgender, Queer, & Intersex MEL Monitoring, Evaluation, and Learning MoA Ministry of Agriculture MoE Ministry of Economy MoF Ministry of Finance MoTTT Ministry of Trade, Tourism and Telecommunications NALED National Alliance for Local Economic Development NAPA National Academy for Public Administration OECD Organization for Economic Co-Operation and Development PSE Private Sector Engagement SAM Serbian Association of Managers SME Small and Medium Enterprises i | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov USAID United States Agency for International Development USD United States Dollar USG United States Government VAT Value-Added Tax usaid.gov Final Performance Evaluation of CFG Activity, Final Evaluation Report | ii EXECUTIVE SUMMARY The United States Agency for International Development (USAID) engaged IMPAQ International LLC (IMPAQ) to conduct a final performance evaluation of the Cooperation for Growth (CFG) Activity in Serbia implemented by Cardno Emerging Markets. The $8.8 million, four-year CFG Activity started in January 2018 and extends through January 2022. The purpose of CFG is to engage Small and Medium Enterprises (SMEs) and other stakeholders in open and effective dialogue to improve each of the identified objectives, thereby increasing overall confidence and support within the private sector, and ultimately strengthening the foundation for increasing private sector competitiveness and investment, expanding Serbia’s overall economic growth. Specifically, the CFG Activity aims to generate (1) improved business environment and related administrative efficiency in selected areas and (2) improved SME access to finance. The evaluation team assessed the extent to which CFG achieved its intended results, identified key successes, challenges, and gaps, and documented good practices and lessons learned. The results of the evaluation will help inform USAID/Serbia, Cardno, and other stakeholders involved in promoting SME growth to design future actions in Serbia and elsewhere in the region. FINDINGS AND CONCLUSIONS EVALUATION QUESTION 1: Has the Activity demonstrated results that are consistent with its theory of change, and to what degree has it achieved its stated objectives? CFG’s interventions made significant contributions to enhancing the systems and legal frameworks and improving access to finance for SMEs through the diversification of available financial products to support SME growth in Serbia. Consistent with its theory of change, CFG helped to enhance the enabling environment for SMEs in Serbia and increased opportunities to obtain access to finance by supporting the systems that provide services to SMEs. CFG found effective ways to strengthen Serbia’s legal frameworks, increasing harmonization with EU standards while enhancing the availability of economic and business development services and financial products tailored to SMEs’ needs. While CFG did internally monitor the extent to which its supported initiatives led to financing for SMEs, including equity financing, digital platforms, and its collaboration with banks; these metrics were not included in CFG’s established performance indicators as defined in the Activity Monitoring, Evaluation, and Learning Plan (AMELP). This in turn has limited USAID’s understanding of CFG’s influence on relevant development outcomes related to strengthening SMEs growth. CFG adapted its implementation strategy, including its monitoring and evaluation framework, to reflect on the ground evidence and real-time learning in response to Serbia’s existing and emerging needs. Despite the limited or lack of available and relevant performance monitoring data, it is apparent that CFG conducted several important research activities and routinely monitored the overall operating environment. As a result, CFG quickly adapted its programming to align with the existing and emerging needs within Serbia’s SME ecosystem, reflecting the program’s ongoing learning. iii | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov EVALUATION QUESTION 2: To what extent did CFG improve the business enabling environment and related regulatory framework? CFG contributed to strengthening Serbia’s business enabling environment, most notably in the E-commerce ecosystem but there are still opportunities to increase access to finance for micro and SMEs. CFG implemented several activities that contributed to the significant improvement of the E-commerce ecosystem, notably the provision of (1) technical assistance to the Government of Serbia (GoS) for the adoption of the E-commerce Action Plan and Program Development, (2) the Strategy for Digital Skills Development and E-commerce regulations, (3) the development and distribution of the E-commerce Guide for Businesses, and (4) the promotion and dissemination of useful details on how to access and benefit from online sales. As a result of these efforts, representatives of CFG’s intended beneficiaries and key stakeholders all stated that the business enabling environment has improved over the last four years. The majority of informants noted that the greatest improvements were made in the E-commerce ecosystem due, in large part, to CFG’s significant contributions in pushing forward initiatives. CFG provided technical support to the GoS in the development of the regulatory framework such that the Serbian regulatory system now contains improved legislation for alternative investment funds and the systematically regulated tax treatment of digital assets. CFG supported legislative reforms related to Alternative Investment Funds and stimuli for owners of digital assets, as well as those to reduce the tax administration compliance burdens for SMEs by the codification of VAT rulebooks and implementation of e-invoicing system in Serbia. Notably, CFG provided technical assistance to the MoF and to the GoS in analyzing the tax treatment of digital assets and drafting proposals for amendments to tax laws. The Serbian regulatory system now contains a systematically regulated tax treatment of digital assets, while tax incentives additionally encourage owners of cryptocurrencies and other forms of digital assets to invest their income in the Serbian economy and thus achieve significant tax savings. In this way, an opportunity for digital assets to become an additional source of financing for companies in Serbia has been provided. Based on feedback from key informants and focus group discussion participants, the previously mentioned CFG activities were successful because the Activity involved all key stakeholders from the public and private sector in the process, establishing a positive dialogue between public and private sector actors. CFG effectively leveraged resources from other donor-funded programs, the private sector, and the GoS, building self-reliant support systems and enabling a high level of inclusiveness for SMEs. Cooperation has proved to be the key in generating sustainable and successful support to Serbian SMEs, as pointed out by almost all interviewed key informants. The interviewed key informants, all of whom were involved in private sector engagement (PSE) initiatives, stated that they will continue to implement the initiatives established with CFG support, thus ensuring their long-term sustainability of CFG’s programming to improve the business enabling environment and related regulatory framework after its completion. EVALUATION QUESTION 3: To what extent and in what ways did CFG improve SME access to finance through expanding the variety of financial products available? CFG’s support for the development and expansion of alternative financial and non-financial products for SMEs in Serbia has increased the ability of SMEs to apply for traditional lending opportunities as well as to expand its understanding of non-traditional financial usaid.gov Final Performance Evaluation of CFG Activity, Final Evaluation Report | iv sources to grow their businesses. The CFG Activity provided and supported several important and innovative improvements to expand the availability of financial and non-financial resources for SMEs in Serbia, including: (1) new sources of capital, (2) ongoing training and mentoring to improve business skills, and (3) expansion of E-commerce within Serbia to address the changing economic and social context. CFG recognized and provided the necessary resources to locally-led innovative ideas to address the constraints limiting SME growth. These important contributions by CFG have been needed and critical to assisting SMEs in Serbia to adapt and survive during this time of unprecedented external challenges. Without CFG’s support, many of these initiatives would not have been realized and thus accessible to SMEs as quickly as they were. CFG’s work to expand access to finance through innovative, online platforms has expanded the availability of new financial products for SMEs as well as contributed to the broader goal of strengthening E-commerce in Serbia. The two new financial platforms developed by CFG, Ventu and Finspot, each show promise for becoming important new sources of financing for SMEs in Serbia. The success of Ventu, the first Crowdfunding campaign and new non-banking source of financing in Serbia in more than ten years provides an important example for other businesses and potential investors in Serbia looking to raise funds via nontraditional sources such as the internet. Similarly, Finspot’s success in providing companies with a new way to immediately finance their customers’ invoices was another important example of how new tools can be leveraged to expand access to finance and allow SMEs to run their businesses. Both tools have the potential to become recognized and trusted financial tools for SMEs. CFG supported the development of non-financial resources for SMEs to become more competitive and to adapt to and meet the evolving requirements of the business environment, thus promoting widespread economic growth. While access to financial resources is essential, many SMEs still require non-financial support in the form of training, mentorship, and technical assistance, in order to grow their businesses. While outside of its mandate to diversify the types of financial products available to SMEs, CFG recognized the importance of improving SME’s awareness of the current and emerging opportunities within the financial market as well as improving their business skills to ensure that SMEs were well prepared to enter the marketplace. . EVALUATION QUESTION 4: How well did CFG adapt to the changing operational environment including incorporating adaptive and continuous learning in planning and delivering its assistance? CFG utilized data and beneficiary feedback to adjust program strategy and activities to better respond to SMEs ’ needs and the rapidly changing operating environment; however, CFG did not fully leverage existing opportunities to strengthen its indicators to better systematize data-driven decision making. CFG utilized several studies and reports to assess needs and inform programming. Similarly, in light of the COVID-19 pandemic, the Activity regularly sought feedback from stakeholders through online communications and surveys, adjusting activities based on such feedback including transitioning programs to e-learning courses, panels, and discussions. Although the Activity was able to incorporate data from reports and feedback from beneficiaries to adapt programming, as noted under EQ 1, the data collected as part of the Activity’s AMELP was limited, particularly with regard to monitoring and reporting progress towards increasing access to finance through its established performance indicators. v | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov Close collaboration and networking with other public and private actors oriented to SMEs’ support helped in making strategic decisions on and prompt responses to business community demands. Due to such an approach, the majority of key informants emphasized that CFG had respected their feedback and subsequently adapted its strategies to the needs and goals of their organizations. EVALUATION QUESTION 5: From the perspective of intended beneficiaries and other stakeholders, how relevant and effective has this Activity been in expanding SME growth? CFG programming was relevant, responsive, and effective in addressing the needs of Serbian SMEs; however, it is too early to determine if these initiatives will lead to sustainable SME development and growth. CFG used a targeted approach to address key challenges facing Serbian SMEs including limited access to traditional financial services, lack of resources to engage in E-commerce, administrative barriers associated with the inspection of goods and services, and an unfriendly regulatory environment for investment. Through the introduction of alternative funding sources and by working with the GoS to reform the tax and regulatory framework for investment and revamp the inspections system, CFG helped SMEs overcome financial barriers and compete in the regional marketplace, expand access to finance, and remove barriers to attracting outside investors. CFG addressed knowledge and capacity issues of SMEs, enhancing their ability to gain access to capital and grow their businesses. CFG helped address non-financial barriers through its cooperation and support for a range of training and information programs including those provided through Erste Bank, Pojacalo, Dobar za Banku, and the E-Commerce Academy, among others. CFG helped SMEs gain the skills and knowledge to (1) more effectively present their companies’ financial and operating information when applying for loans with banks, (2) manage their long-term business strategy and growth, and (3) contribute to the long-term expansion and growth of the Serbian economy. EVALUATION QUESTION 6: Based on the experience and knowledge from this Activity, what approaches are most likely to result in gender equity improvements in the area(s) of focus? The lack of a gender analysis in the inception phase and the absence of consistently monitored sex- and age- disaggregated data have led to a lack of awareness and missed opportunities. Opportunities have been missed to address and target competitiveness, women’s access to finance, and utilization of E-commerce, among others. Without accommodating the different needs of SMEs owned by men, women, youth, and minorities, as stated by key informants and demonstrated by CFG’s gender-blind approach, CFG operated under the assumption that all had equal access to the technical and financial assistance the Activity provided. When in fact previous and subsequent gender analyses show that heightened barriers for women exist in business development, access to assets, access to education and training, and access to finance. CFG identified some interesting findings during its implementation which can be used and replicated in the future or can serve as a basis for the design of future actions in the same field. CFG has identified and capitalized upon the finding that women entrepreneurs and SME owners have both a need for and an interest in taking part in capacity development initiatives – particularly regarding digital literacy and E-commerce. Additionally, CFG found that companies with an equal or majority share of women managers are more likely and more willing to use new Fintech solutions. usaid.gov Final Performance Evaluation of CFG Activity, Final Evaluation Report | vi RECOMMENDATIONS Based on the previous findings and conclusions, the evaluation team makes the following recommendations for USAID consideration to support future economic growth programming under its Development Objective 2, Equitable Prosperity Increased and related Intermediary Result 2.1 SME Economic Growth Accelerated.1 R1 – Continue to expand and diversify alternative access to finance for micro-business, start-ups, and SMEs broadly. It is important to introduce new non-bank financing tools such as leasing, angel capital, and microlending, as well as non-financial products and business development support services, to reduce the bank-centric structure of Serbia’s financial system. This will allow SMEs to grow and will further drive Serbia’s economic growth, building a stronger foundation for private sector investment and competitiveness. R2 – Building on CFG’s success, key areas of focus for future USAID economic growth activities could include: • Maintaining a strong focus on expanding and strengthening the recent progress in E-commerce, including continuing education in digital skills for all businesses, consumers, and citizens, including women, youth, rural residents, and other disadvantaged populations, to increase Serbia’s competitiveness in Europe and in the international marketplace. • Building on the success of CFG’s work with Erste Bank’s Social Bank Unit and its Step by Step program, USAID programs could support increased dialogue and support to other banks in Serbia to create similar units within their existing structure. This would expand lending to SMEs and provide much-needed mentoring in business and financial skills to help SMEs continue to grow. • Continuing cooperation with the MoTTT to harmonize regulatory reforms with EU standards in the areas of export promotion and the digital economy. Also important is establishing additional regulatory reform cooperation with other ministries to further enhance the existing regulatory framework for SMEs, such as the MoE, using CFG’s cooperation with the MoTTT as good model. • Drawing on CFG’s accomplishments in the food retail and hospitality industry, expand the available guidance for private sector actors to increase stakeholders’ accessibility and knowledge of the inspection process, utilizing online guides, trainings, and open days, among other methods, to address existing and emerging needs within the market. R3 – Future economic growth activities should target women, youth, and other marginalized groups. USAID and Implementing Partners could consider: • Promoting additional research to expand the existing evidence on women’s willingness to use Fintech Solutions. • Ensuring an increased usage of Fintech solutions since the evidence shows that women are more willing and more likely to use Fintech. • Capitalizing on the CFG finding that women entrepreneurs and SME owners have shown both a need for and an interest in capacity development initiatives, continue with similar trainings and workshops, particularly in the areas of digital literacy and E-commerce. 1 Country Development Cooperation Strategy (CDCS) - Serbia, 2020-2025 (usaid.gov) vii | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov • Implementing activities that specifically devise economic and entrepreneurship programs targeting women, youth, and other marginalized groups. By supporting and empowering women-owned SMEs and women entrepreneurs, future actions can simultaneously ensure an increased usage of Fintech solutions because the evidence shows that women are more willing and more likely to use Fintech. R4 – Expand implementing partner monitoring and evaluation reporting requirements and promote the continued application of collaborating, learning and adapting (CLA), drawing on CFG’s approach to adaptive management, throughout all stages of the activity cycle. USAID and Implementing Partners could consider: • Facilitating theory of change workshops during the design and launch of activities to ensure the inclusion of relevant indicators to inform all steps in the decision-making process and in future evaluation and assessment activities. For example, in activities designed to improve SMEs’ access to finance, consider requiring indicators such as the number and/or types of SMEs that participated and/or benefited from an intervention as well as the level of financing generated for SMEs. • Requiring routine reporting of performance indicators as part of ongoing activity and periodic reporting, including data tables with relevant disaggregations. • Creating CLA plans and/or Learning Agendas during the design and launch of activities to address existing knowledge gaps and promote adaptive management. • Leveraging CFG’s approach, continue to conduct relevant research and in-depth analysis to address existing knowledge gaps and better understand the dynamic SME landscape to enhance ongoing and future Activity design and implementation strategies in efforts to improve the business-enabling environment and expand broad-based economic growth. • Defining a set of indicators at the Mission level to monitor changes in the SME landscape beyond the average activity period of performance to fully understand the impact of USAID’s investments to improve Serbia’s complex regulatory environment and stimulate behavioral change. R5 – To ensure local ownership and promote the continuation and sustainability of programming, adopt a systems-based approach, working in close partnership with a range of private and public sector actors. USAID and Implementing Partners could consider: • Leveraging the learning from CFG’s PSE mechanism to strategically and systematically collaborate with the right partners at the right time to leverage local expertise and knowledge, ultimately promoting local ownership and sustainability. • Working closely with key Serbian Ministries (MoF, MoTTT, MoA, Inspections) to expand opportunities for continuous learning on both the supply and demand sides of the Serbian economy. • Drawing on public and private sector networks to expand SMEs’ knowledge and awareness of the broader economic market or supply chain. This will increase SMEs’ compliance with standards, the implementation of new technologies, and preparation for accessing traditional financial products. R6 – Pay greater attention to outreach and communication. USAID and Implementing Partners could consider organizing activities that contribute to strengthening key stakeholder’s understanding of usaid.gov Final Performance Evaluation of CFG Activity, Final Evaluation Report | viii SMEs and the challenges that they face. Additionally, USAID and Implementing Partners could consider promoting broader dissemination of activities and learning across the SME landscape. R7 – USAID should require Implementing Partners to conduct a gender analysis during the design and launch of future activities. This will ensure that gender analyses are an integral component of the design, implementation, monitoring, and evaluation of future SME programming. ix | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov INTRODUCTION ACTIVITY BACKGROUND Small and medium enterprises (SMEs) account for the largest share of the country’s economic development and private sector employment, representing 99% of all enterprises2 and 65% of the labor force.3 In recent years, the Government of Serbia (GoS) has demonstrated its commitment to the development of SMEs, recognizing the growth and competitiveness of SMEs as a critical element for the development of the Serbian economy, increased employment, and improved citizens’ standards of living. As a result, the Serbian Government released the SME Development Strategy and Action Plan 2015– 2020, to strengthen the business enabling environment and SME market by improving “access to finance, human resource development, consolidation of sustainability and competitiveness of SMEs and entrepreneurs, access to new markets, and development of entrepreneurial spirit, as well as women, youth, and social entrepreneurship.”4 Following the launch of Serbia’s SME strategy in 2015, several government institutions, chambers of commerce, business associations, international organizations, and non-governmental organizations partnered to launch Serbia’s Decade of Entrepreneurship, to build on the country’s commitment to SME development by providing targeted financial and non-financial support to encourage and foster entrepreneurial activities across the country. Despite the GoS’ commitment to SME development and recent progress to improve lending conditions, relax monetary policy, and lower the country’s risk premium, SMEs remain burdened and access to finance continues to constrain SME development and growth. Although great improvements have been made in creating an enabling environment for SMEs in the last 20 years, the enabling environment does not adequately respond to SME needs, but rather favors the interests of larger businesses and state-owned enterprises.5 To support the GoS’ objectives and build upon the success of USAID’s prior policy reform activity—the USAID Business Enabling Project (BEP)—USAID/Serbia launched the four-year, $8.8 million Cooperation for Growth (CFG) Activity in 2018. Working alongside the GoS, the private sector, civil society organizations, media, and other stakeholders, the CFG Activity seeks to “improve the competitiveness of the Serbian private sector, through improvements in the business enabling environment that remove regulatory constraints to private sector growth,”6 thereby improving “conditions for broad-based, inclusive economic growth.”7 Within this context, the CFG Activity is designed to engage SMEs and other stakeholders in open and effective dialogue to improve each of the identified objectives, thereby increasing overall confidence and support within the private sector, and ultimately strengthening the foundation for increasing private sector competitiveness, promoting job creation, strengthening exports, and expanding Serbia’s overall economic growth. EVALUATION PURPOSE USAID/Serbia commissioned IMPAQ International, LLC (IMPAQ) to conduct a final performance evaluation of the CFG Activity in Serbia. The evaluation provides USAID/Serbia with a full, evidence￾2 Financing SMEs and Entrepreneurs 2020: An OECD Scoreboard. 3 Country Development Cooperation Strategy (CDCS) – Serbia, 2020-2025 (usaid.gov) 4 SME Development Strategy and Action Plan, 2015–2020. Republic of Serbia, Ministry of Economy, 2015. 5 Country Development Cooperation Strategy. USAID/Serbia, 2020. 6 USAID/Serbia, Cooperation for Growth Project (CFG), Activity Monitoring, Evaluation, and Learning Plan (AMELP), Revised Draft, February 21, 2020. 7 In alignment with USAID/Serbia’s Country Development Cooperation Strategy (CDCS) goal which focuses on improving the conditions for broad-based inclusive economic growth improved as stated in Development Objective (DO) 2. 1 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov based, and independent assessment of the Activity’s effectiveness, including an in-depth analysis of the extent to which CFG achieved its intended results. The evaluation also identifies key successes, unintended results, challenges, and gaps as well as contributing factors, and documents good practices and lessons learned. In conducting the final performance evaluation, the team focused on the following six evaluation questions (EQs) specified in the evaluation’s scope of work, which can be found in Annex 1: 1. Has the Activity demonstrated results that are consistent with its theory of change, and to what degree has it achieved its stated objectives? 2. To what extent did CFG improve the business enabling environment and related regulatory framework? 3. To what extent and in what ways did CFG improve SME access to finance through expanding the variety of financial products available? 4. How well did CFG adapt to the changing operational environment including incorporating adaptive and continuous learning in planning and delivering its assistance? 5. From the perspective of intended beneficiaries and other stakeholders, how relevant and effective has this Activity been in expanding SME growth? 6. Based on the experience and knowledge from this Activity, what approaches are most likely to result in gender equity improvements in the area(s) of focus? EVALUATION METHODOLOGY To conduct the final performance evaluation of the CFG Activity, the evaluation team used a utilization￾focused approach, incorporating complementary qualitative and quantitative methods of data collection and analysis, including online data collection tools tailored for different stakeholder groups. Before collecting data, the evaluation team conducted a thorough review of the documentation related to the EQs to gain an in-depth understanding of CFG activities and to assess the availability and quality of CFG’s monitoring and evaluation data to answer the EQs. DATA COLLECTION From 24 September through 28 October 2021, the evaluation team collected data, both qualitative and quantitative, on the perspectives and experiences of key stakeholders involved in CFG. The primary research methods included key informant interviews (KIIs) and focus group discussions (FGDs), for a combined total of 53 participants (25 women and 28 men). The KII and FGD participants were identified from a complete list of direct financial beneficiaries that received support from CFG as well as key stakeholders based on their involvement, knowledge, and awareness of the Activity and the broader SME landscape in Serbia. The KII and FGD sample was reviewed and validated by USAID and the CFG Activity staff prior to data collection. The research also included an online survey administered to a pool of 798 SME beneficiaries and end users. Due to COVID-19-related public health restrictions, the evaluation team conducted all data collection activities remotely by telephone and online conferencing technology. usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 2 ● Key Informant Interviews: The evaluation team conducted online interviews with 37 individuals (18 women and 19 men) including (1) USAID, (2) the CFG Activity, (3) the GoS, (4) financial institution and business development support providers, (5) business associations and civil society organizations, (6) direct financial beneficiaries, and (7) other donors. Each KII followed an interview protocol adjusted for different respondent types and was conducted in English or Serbian, depending on the key informant’s preference. ● Focus Group Discussions: The evaluation team conducted six online FGDs with a total of 16 participants (7 women and 9 men), each representing a direct beneficiary of CFG’s programming. Due to the sensitive nature of sharing SMEs’ access to financing activities, several FGD participants preferred to provide their feedback through individual in-depth interviews rather than in a group setting. ● Online Survey. The evaluation team disseminated an online survey to 798 SME beneficiaries and end users. A total of 55 individuals (representing 15 women-owned SMEs, 34 men-owned SMEs, 5 jointly owned SMEs, and 1 nondisclosed [10 of which were youth-owned]) completed the survey, for a response rate of 7 percent. Survey respondents included beneficiaries from Pojacalo (29), Finspot (7), eCommerce Webinars (6), Open Days (6), Ventu (3), eCommerce Academy (3), and #erstenzali (3).8 For the detailed methodology refer to Annex 2. For the data collection protocols refer to Annex 3. DATA ANALYSIS The evaluation team employed several data analysis methods and approaches to identify key findings from the data, draw conclusions, and make recommendations. The team triangulated the available performance monitoring data, including data gathered through the annual Survey of 1,000 Business and Overcoming Crisis Together Survey,9,10 with qualitative data collected through document review, KIIs, FGDs, and the online survey. Evaluation team members independently reviewed the evaluation notes and tallied occurrences of key themes, comparing data from the various sources described above, to determine whether findings were divergent or convergent. The extent to which multiple informational streams provided consistent findings informed the certitude and internal validity of the conclusions. The evaluation team conducted content analysis by reviewing and coding KII and FGD data to identify and highlight notable examples of CFG’s successes, or lack of success, that contributed to, or hindered, achievement of the Activity’s objectives. Throughout the analysis, the evaluation team disaggregated all KII and FGD data by sex and age and analyzed them for possible differences in respondents’ perspectives associated with gender and age. KNOWN LIMITATIONS AND MITIGATION STRATEGIES The team identified several challenges and limitations in preparing for and conducting the evaluation, including attribution and cognitive bias as well as challenges associated with limited performance monitoring data, remote data collection, facilitating FGDs, and administering an online survey. 8 Respondents were asked to select all CFG interventions that they participated in. 9 USAID/Serbia, Cooperation for Growth Project (CFG) Survey of 1000 Businesses. 2019. Cardno 10 USAID/Serbia, Cooperation for Growth Project (CFG) Executive Summary Survey of 1000 Businesses. 2020. Cardno 3 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov Limitation of attribution bias. Since 2011, Cardno has been implementing economic growth programs in Serbia, including the predecessor program, the USAID-funded BEP (2011–2017). In addition to CFG, Cardno is currently implementing the USAID-funded Competitive Economy Project (2017– 2021) in Serbia. Cardno’s extensive experience implementing economic growth projects in Serbia introduced attribution bias into the evaluation, because partners, beneficiaries, and stakeholders may not easily distinguish between activities conducted under CFG and other Cardno programs. The evaluation team mitigated the potential for attribution bias through targeted questions, probing, and frequent prompting, to remind participants to respond about their experiences since the CFG award. Cognitive bias: Key informants and FGD participants constituted a key source for answering all EQs, but interview data is prone to cognitive biases, including recall and social desirability, as well as respondent bias when implementers facilitate contact with beneficiaries. To mitigate this risk, the evaluation team began each KII and FGD with a protocol that reviewed CFG’s objectives. Additionally, the team explained the evaluation purpose and how the data would be used, while ensuring confidentiality. Limited performance monitoring data: Although CFG had an established monitoring and evaluation framework, only limited performance monitoring data was provided to the evaluation team, namely, aggregated figures for CFG’s ten performance indicators through the end of Year 3. This data was presented in the USAID CFG Activity Monitoring, Evaluation, and Learning Plan (AMELP) Evaluation Report Y3, dated February 4, 2021. Raw performance data was not provided. The evaluation team mitigated this constraint by using qualitative data from KIIs and FGDs and its own survey. Challenges with remote data collection: Given the ongoing COVID-19 pandemic, the evaluation team collected all data remotely. Potential challenges associated with remote data collection included the risk of technology failures, diminished communication quality, and increased challenges in arranging meetings with respondents. To mitigate these limitations, the evaluation team had a variety of remote communication tools (Skype, WhatsApp, Zoom, mobile conferences, and regular mobile phone communication) available. In point of fact, the evaluation team encountered very few issues with connectivity and therefore was able to mitigate this challenge. Limitations in facilitating FGDs: Due to CFG’s intervention strategy of implementing a number of activities through intermediaries, the evaluation team encountered several challenges in identifying SME beneficiaries and end users who were willing to discuss their involvement with CFG’s activities. Many of the SMEs benefiting from CFG’s interventions were not aware of USAID’s investment nor of CFG’s involvement. In efforts to mitigate this limitation, the evaluation team worked directly with intermediaries to identify and schedule focus groups with their participating SMEs. Although this effort was largely successful, the evaluation team was unable to conduct the planned number of FGDs. Limitations to administering an online survey: While electronic surveys can reach large numbers of respondents, the challenge is often low uptake and the inability to ask probing questions. The evaluation team worked with CFG and its partners at the beginning of the evaluation, leveraging CFG’s existing network, to inform all potential respondents of the purpose and importance of the evaluation and the anonymous nature of the survey. In an effort to increase uptake and response rates, the evaluation team also used an online automated survey platform, Survey Monkey, that tracks response rates and sends respondents reminder emails if they have not completed the survey. Despite these efforts, the team received only 55 responses from the target pool of 798 beneficiaries. usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 4 FINDINGS AND CONCLUSIONS EVALUATION QUESTION 1: Has the Activity demonstrated results that are consistent with its theory of change, and to what degree has it achieved its stated objectives? To frame its performance analysis in response to EQ1, the discussion begins by highlighting changes in CFG’s intervention strategy, relating these changes to both the Activity’s theory of change and the corresponding AMELP. This subsection then provides an overview of CFG’s accomplishments as related to its theory of change, followed by an assessment of the extent and ways in which the Activity’s interventions contributed to each of its two objectives. FINDINGS Following the second year of implementation, CFG revised its intervention strategy. As articulated in CFG’s Year 1 workplan, as well as its revised AMELP submitted in Year 2, CFG’s original strategy, along with its corresponding theory of change and logical framework, had three aims: (I) strengthening behavior-driven policies to improve the business environment and related administrative efficiencies, (2) improving access to financial resources to bring about SME growth, and (3) improving the attitudes of government officials and entrepreneurs to one another to strengthen public and private dialogue and cooperation. As further confirmed by key informants representing USAID/Serbia and CFG staff, the CFG plan was originally designed with a strong focus on addressing challenges and constraints within the regulatory environment. However, based on learning generated from the annual Survey of 1,000 Businesses, in which CFG incorporated principles to better understand the drivers of a company’s growth, CFG recognized that certain parts of the economy were growing faster than average, including the digital economy. The annual survey, for example, showed that digitally oriented companies – those with an online presence, web-based sales, digital marketing, and employees with relevant digital skills – demonstrated a higher average growth rate of revenue. According to the findings from the 2021 annual Survey of 1,000 Businesses, the average growth rate of revenue as measured by the Compound Annual Growth Rate (CAGR) in Serbia was reported at 3.83. Companies that were not digitally oriented were found to have a CAGR of 4.11. In comparison, companies that had a strategic approach to strengthening digital aspects reported a CAGR of 4.95; those with an online store, 4.86; those with online payments, 4.57; and those that had employees with digital skills, 4.27. To leverage this opportunity, CFG adapted its intervention strategy, shifting away from general regulatory improvement toward a more digitally oriented strategy that leveraged the identification of “champions” in the SME community and focused on the provision of better opportunities for their growth. To support this learning and change in strategy, as part of its Year 3 Work Plan CFG introduced activity streams targeting SMEs' inclusion in the digital economy, including developing an E-commerce ecosystem for SMEs' access to markets. CFG also introduced electronic invoicing (e-invoicing) as a tool to reduce cost, improve cost management, and enable external financing options for SMEs. This change in strategy was fully aligned with the priorities of the Serbian Government, which placed the promotion of innovation, digitalization, and digital transformation at the top of its policy agenda.11 Additionally, to achieve its intended, revised purpose, CFG updated its stated objectives, which focus on two interrelated objectives: (1) an improved business environment and related administrative efficiency in selected areas and (2) improved SME access to finance. 11 Program of the Government of the Republic of Serbia – Candidate for the Prime Minister Ana Brnabic, June 28, 2017. 5 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov To reflect the shift in strategy, with less of a focus on the regulatory environment and with greater emphasis on the integration of activities focused on the digital economy, CFG revised its theory of change and corresponding logical framework and produced an updated AMELP. As stated in the revised AMELP submitted in Year 3, CFG’s revised theory of change states: “[I]f supporting systems provide SMEs with services to improve the business environment and access to finance, SMEs will become more competitive and will grow, driving Serbia’s economic growth and building a durable foundation for increased private sector investment and competitiveness.”12 To support this revised theory of change, CFG’s logical framework was also revised to focus on (1) improving systems and legal frameworks to better harmonize with EU standards and enhance the availability of economic and business development services and (2) improving access to financial resources in efforts to diversify the available financial products for SMEs, thus collectively supporting SME growth. Refer to Exhibit 1 for a summary of the changes in CFG’s theory of change and logical framework over time. EXHIBIT 1. CHANGES IN CFG’S THEORY OF CHANGE AND LOGICAL FRAMEWORK ORIGINAL DESIGN ADAPTED DESIGN Source Revised AMELP January 25, 2019 Revised AMELP February 21, 2020 Theory of Change If public and private collaboration to improve the legal and regulatory framework for private sector growth is broadened and strengthened to cooperatively develop effective solutions for the transparent, predictable, and accountable implementation of reforms to reduce regulatory burdens and improve access to finance, then private sector confidence in the regulatory environment will be increased, building a durable foundation for increased private sector investment and competitiveness. If supporting systems provide SMEs with services to improve the business environment and access to finance, SMEs will become more competitive and will grow, driving Serbia’s economic growth and building a durable foundation for increased private sector investment and competitiveness. Purpose Cooperation within the Serbian business environment improved Improve the competitiveness of the Serbian private sector, through improvements in the business enabling environment that remove regulatory constraints to private sector growth. Sub-Purpose 1 Behaviour driven policy measures strengthened Improve systems and legal frameworks to support SMEs growth Sub-Purpose 2 Access to financial resources improved Access to financial resources improved Sub-Purpose 3 Attitudes of government officials and entrepreneurs towards each other improved N/A Notably, in alignment with the integration of the digital economy, CFG expanded its performance metrics to include a composite indicator, the E-commerce Ecosystem Strengthening Index. This index has four pillars: (1) institutional and knowledge ecosystem, (2) understanding and awareness, (3) private sector engagement (PSE) in E-commerce, and (4) E-commerce share in GDP. The index is based on data 12 USAID/Serbia, Cooperation for Growth Project (CFG), Activity Monitoring, Evaluation, and Learning Plan (AMELP), Revised Draft, February 21, 2020. usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 6 generated from the annual Survey of 1,000 Businesses, official data reported by the National Bank of Serbia, and CFG’s internal records. Despite these regular updates to the AMELP, there is a lack of available data and reporting against the Activity’s performance indicators, thus limiting USAID’s ability to fully understand CFG’s achievements. The most recent AMELP, submitted in Year 3, includes a series of ten performance indicators at the output (5 indicators) and outcome (5 indicator) levels, reflecting both custom and standard indicators. According to the corresponding performance indicator reference sheets, CFG indicators are to be reported either quarterly (2) or annually (8). However, only one indicator has been reported in CFG’s periodic progress reports: “person hours of USG funded technical assistance in business enabling environment provided to counterparts or stakeholders (EG5.1-1).” Reporting against this indicator did not start until the final quarter of Year 1 implementation. Aside from this indicator, the performance data presented in this evaluation report draws on the aggregated figures presented in the USAID CFG AMELP Evaluation Report Y3, dated February 4, 2021. The raw performance data was not provided to the evaluation team. CFG tackled regulatory and legal constraints in four key areas: equity investment, inspections, E-commerce, and VAT, and intervened in efforts to bring about better harmonization with EU standards. The majority of key informants across stakeholder groups noted CFG’s important contributions to improving Serbia’s legal and regulatory environment. As described in more detail under EQ2, among CFG’s various interventions, key informants most often noted the Activity’s contributions to the Law on Alternative Investment Funds (AIF), the Law on Digital Assets, the Law on Foreign Exchange Transaction (FX Law), and the Law on E￾invoicing; the inspection policy for hospitality and industry and the food inspection code of procedures; the establishment of the E-commerce Association, the first National E-commerce program and, subsequently, the adoption of the E-commerce Law; and the VAT Rulebook. While one informant noted that Serbia is “still below the EU average, this is changing,” others, including government representatives, noted that “some root issues have been tackled by the Project,” acknowledging that “regulations are beginning to align, legal conditions reached, and legislation designed to support foreign investment and SME growth.” Greatest importance of the CFG is their ability to recognize the legit needs of businesses and to logically suggest the actions. — Government of Serbia Representative CFG has largely met its objectives to improve the business environment and related administrative efficiency in selected areas.13 Based on its performance against its established performance indicators, as presented in the CFG’s AMELP Evaluation Report Y3, CFG is on track to meet and/or surpass five of the seven indicators (refer to Exhibit 2). 13 As part of the analysis, the evaluation team classified performance in three categories: on track to meet targets, on track to significantly surpass targets, and not on track to meet targets. In order for CFG to receive the status "significantly surpass targets," the project had to have already met or exceeded its established LOP at the end of Year 3. Indicators receiving the status of "not on track to meet targets" mean that CFG performance has fallen below its Year 3 targets and is unlikely to achieve its LOP targets in the remaining project months. 7 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov EXHIBIT 2. CFG PERFORMANCE AGAINST OBJECTIVE 1 PERFORMANCE INDICATORS INDICATOR BASELINE TARGETS PERFORMANCE STATUS 2018 2019 2020 LOP YEAR 3 Person-hours of USG-funded technical assistance in business enabling environment provided to counterparts or stakeholders (EG5.1-1) 0 (2018) 1,000 5,000 8,000 10,000 14,527 Surpass Number of commerce laws and regulations developed/changed in accordance with international standards as a result of USG assistance 0 (2018) 5 12 20 25 58 Surpass Number of inspections based on risk assessments (%) 65% (2018) 65% 68% 69% 70% 19% Not on track Number of firms involved in preventive activities of inspection 2,140 (2018) 2,140 3,643 4,000 4,500 2,974 Not on track E-commerce Ecosystem Strengthening Index 28% (2019) N/A 28 35 40 37 On track Decrease in firms in Serbia reporting tax procedures having a negative impact on their doing business (%) 24% (2019) N/A 24% 23% 21% 22% On track Number of self-reliant systems established through project assistance 0 (2018) 0 0 2 4 2 On track With respect to the two indicators not on track to meet the established targets – the number of inspections based on risk assessments and the number of firms involved in preventive inspection activities – the COVID-19 pandemic had a substantial impact on risk assessments and preventive inspection activities. Since the beginning of the pandemic, the number of inspections has accelerated, due to the imposed curfews and greater enforcement of health and safety measures. The majority of these inspections are a result of extraordinary supervisions as opposed to planned inspections, as a best practice. In terms of preventive inspection activities, CFG has observed a decrease in advisory and self￾inspections due to close-outs and social distancing. Prior to Year 3, CFG was on track to achieve the LOP targets for both of these indicators. CFG contributed to the diversification of available financial products in Serbia through the launch of Ventu and Finspot. Nearly all key informants noted CFG’s contributions with respect to improving access to finance for SMEs by the Activity’s work in launching two new methods of alternative financing: a crowdinvesting platform and a digital invoicing financing or e-factoring platform. • Ventu, a crowdinvesting platform, was launched with the support of CFG as a breakthrough digital financing solution to combat the banks’ monopoly on financing, the first in Serbia in more than a decade. It is extremely difficult for SMEs to navigate the process of accessing finance through the banks because they are less likely to have a sound business plan, collateral, usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 8 and the financial literacy to meet the requirements. Banks in Serbia are the primary source of financing, with more than 90% of the market share. Ventu targets SMEs with innovative business ideas and scalable growth. To date, CFG has supported two successful campaign and as indicated by CFG, one more campaign is imminent. • Finspot. CFG supported the development and implementation of Serbia’s first e-factoring platform, which is accessible and usable by all SMEs, regardless of their size. The Finspot platform allows companies to upload invoices and receive financing in 24 hours, increasing liquidity and simultaneously facilitating the rapid emergence of E-commerce due to the pandemic. An additional strategy in designing Finspot was to ensure it was accessible to SMEs that lacked financial literacy, while simultaneously providing trainings to address this lack of knowledge. Finspot quickly proved its functionality, disbursing EUR 45,000 among five companies in the first two weeks. Since then, Finspot has grown significantly and has set a new record in the last quarter of 131 commercial invoices over the course of one day. To date, Finspot has provided financing in the amount of EUR 4,000,000 million through 1100 transactions. CFG has largely met its objectives to improve the systems and legal framework to support SME growth. However, the performance indicators monitored by CFG under Objective 2 were not very useful to fully measure improved access to finance. Based on its performance against its established performance indicators, as presented in the CFG’s AMELP Evaluation Report Y3, CFG is on track to meet or surpass two of the three indicators (refer to Exhibit 3). Although CFG is not on track to meet the established target of growth in new SME loans, the Activity has made significant progress since baseline, which was reported at -7.3%, according to the official National Bank of Serbia data. According to CFG reporting, this indicator has also been impacted by the COVID-19 pandemic because the National Bank of Serbia and other financial institutions have reduced lending due to higher risks. EXHIBIT 3. CFG PERFORMANCE AGAINST OBJECTIVE 2 PERFORMANCE INDICATORS INDICATOR BASELINE TARGETS PERFORMANCE STATUS 2018 2019 2020 LOP YEAR 3 Percentage of financing coming from own funds 84% (2017) N/A 81% 80% 79% 79.4% On track Number of new SME financial products introduced 0 (2018) 0 1 2 3 2 On track Growth of new SME loans -7.3% (2018) - 7.3% 2.6% 4% 4% 3% Not on track While CFG’s indicators under Objective 2, in part, signal how access to financial resources has improved, the established outcome level indicators do not permit a full understanding of the Activity’s achievements. For example, although CFG did internally monitor the extent to which its supported initiatives led to financing for SMEs, including equity financing, digital platforms, and its collaboration with banks; CFG did not include any indicators to measure the number and/or types of SMEs that 9 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov participated and/or benefited, nor the level of financing generated for SMEs as a result of its programming within its AMELP. CONCLUSIONS CFG’s interventions made significant contributions to enhancing the systems and legal frameworks and improving access to finance for SMEs through the diversification of available financial products to support SME growth in Serbia. Consistent with its theory of change, the evaluation findings suggest that the CFG Activity has helped to enhance the enabling environment for SMEs in Serbia and increased opportunities to obtain access to finance by supporting the systems that provide services to SMEs. The evaluation findings indicate that due to COVID-19, among other environmental factors, CFG will not achieve all ten of its performance metrics, which were designed to measure progress against its stated objectives. However, the findings also suggest that CFG quickly pivoted its programming to adapt to the changing environment. As a result, CFG found effective ways to strengthen Serbia’s legal frameworks, increasing harmonization with EU standards while enhancing the availability of economic and business development services and financial products tailored to SMEs’ needs. Furthermore, based on the available literature and existing evidence, the findings suggest that these contributions are likely to support Serbia’s economic growth and the creation of a durable foundation for increased private sector investment and competitiveness. Yet, while CFG did internally monitor the extent to which its supported initiatives led to financing for SMEs, including equity financing, digital platforms, and its collaboration with banks; these metrics were not included in CFG’s established performance indicators as defined in the AMELP. This in turn has limited USAID’s understanding of CFG’s influence on relevant development outcomes related to strengthening SMEs growth. CFG adapted its implementation strategy, including its monitoring and evaluation framework, to reflect on-the-ground evidence and real-time learning in response to Serbia’s existing and emerging needs. Despite the limited available and relevant performance monitoring data, it is apparent that CFG conducted a number of important research activities and routinely monitored the overall operating environment. As a result, CFG quickly adapted its programming to align with existing and emerging needs within Serbia’s SME ecosystem, reflecting the program’s ongoing learning. Subsequently, as part of these efforts and in alignment with best practices, CFG routinely updated its monitoring and evaluation framework through its AMELPs to ensure that its theory of change and corresponding objectives reflected its implementation strategy. EVALUATION QUESTION 2: To what extent did CFG improve the business enabling environment and related regulatory framework? The purpose of EQ2 was to examine CFG’s programming to determine whether the program contributed to the strengthening of Serbia’s business enabling environment and related administrative efficiencies. Additionally, in response to EQ2, the evaluation team sought to understand which of CFG’s business enabling activities were most successful and why. The team also explored how and to what extent CFG leveraged resources from other donor-funded programs, the private sector, and the GoS to improve the business enabling environment and identify opportunities to promote sustainability of results. FINDINGS CFG contributed to the strengthening of the E-commerce ecosystem in Serbia. As stated in the CFG Quarterly Progress Report, CFG provided technical assistance to the GoS for the adoption of usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 10 the E-commerce Action Plan and Program Development for 2019- 2020, Strategy for Digital Skills Development 2020-2024, and three regulations related to strengthening E-commerce: the Law on Trade, the amendment on the Law on Electronic Communication, and refinement of the Law on Postal Services.14 Supporting implementation of the GoS-adopted E-commerce Action Plan, CFG, in cooperation with the Ministry of Trade, Tourism, and Telecommunication (MoTTT) and the E-commerce Association of Serbia (ECS), developed and published the E-commerce Guide for Businesses.15 The guide provides useful details on how to access and benefit from online sales. In partnership with MoTTT, the Serbian Chamber of Commerce, the Belgrade Chamber of Commerce, and the ECS, CFG held an online workshop to introduce the E-Commerce Guide for Businesses, which was attended by representatives of 128 Serbian businesses.16 The CFG team was also invited to hold similar webinars to present E-Commerce Guide for Businesses by the Serbian, Italian, Japanese, American, and German chambers of commerce, as well as by other business associations (Serbian Association of Managers [SAM], Fashion Cluster, National Alliance for Local Economic Development [NALED]). Before the joint CFG and MoTTT Action, the E￾commerce topic had a negative reputation in the media. They [CFG] managed to change this. – Government of Serbia Representative Exhibit 4: To what extent do you agree with the following statement: “I better understand the benefits and value of E-commerce today that I did three years ago.” (n=50) As a result, over half of the key informants representing CFG’s beneficiaries and key stakeholders indicated that CFG had contributed to the development of the E-commerce ecosystem in Serbia. Representatives from the GoS emphasized that the MoTTT had the full support of CFG in strengthening E-commerce in the Republic of Serbia. Additionally, 86% of SME online survey participants confirmed that they better understand the benefits and values of E￾commerce today than they did three years ago. (refer to Exhibit 4.) 54% 32% 14% 0% 0% 0% Strongly Agreed Neutral Disagreed Strongly No Agreed Disagreed Response CFG prepared recommendations on how to increase the use of inspection advisory visits and self-inspections by businesses as part of an innovative approach to improving inspection. The recommendations provided step-by-step instructions on how to conduct a self￾inspection. Additionally, CFG provided a set of document templates for practical use and listed the best examples of how similar self-regulatory policies work in developed countries.17 These activities were instrumental in laying the groundwork for the development of innovative inspection policies. Furthermore, CFG helped improve inspectors’ knowledge and ability to apply new technology and digital Government-to-Business services. During the first quarter of 2018, CFG organized meetings with representatives of four inspectorates – Market Surveillance, Agriculture, Labor, and Sanitary – who all 14 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 4, FY 2019. Cardno; USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 1, FY 2020. Cardno 15 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 1, FY 2021. Cardno 16 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 2, FY 2021. Cardno 17 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 1, FY 2020. Cardno 11 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov expressed interest and willingness to participate in CFG’s reforms.18 As stated by the CFG Activity team, CFG later focused on collaboration with the food and food safety sectors. From 2019 through the first quarter of 2020, CFG successfully completed a one-year support program to build the Inspectors’ Network of Serbia (INS). The support program consisted of the following activities: training, exchange of experience through study visits, an annual conference, and development of a functional and modern website, among others.19 Additionally, CFG implemented a set of eight open house events across the country in partnership with the Serbian Chamber of Commerce, which were co-financed by the European Union (EU)-funded Improving Business Environment (IBE) project. Around 500 companies participated.20 The activity was temporarily on hold during the pandemic but was successfully moved to an online facilitation format through the Chamber of Commerce. To date, three virtual open house events have occurred.21 In the area of inspection, there was a positive influence from the CFG Project. In the last four years, they worked on the improvement of communication of the inspections and business entities. They had to present new Law articles and they worked in the implementation of this. – Government of Serbia Representative CFG, together with the Organization for Security and Co-operation in Europe Mission, developed the first online course for inspectors and one of the first online courses for public service in general for the National Academy for Public Administration (NAPA).22 The material developed to facilitate this course covers all major aspects of the Law on Inspections Oversight. The course itself has been integrated into the Annual Programs of NAPA, spanning both 2020 and 2021 as well as the anticipated program for 2022,23 and is seen as a critical component to support the GoS as it plans to hire 1,200 new inspectors in the coming year.24 Key informants representing inspectorates indicated that the training they received thus far through the program has helped to introduce and prepare newly hired inspectors for job requirements and procedures. CFG developed a new inspection policy for the food retail and hospitality industries. As stated in CFG FY 2020 Quarter 4 Progress Report, CFG developed a new inspection policy for the food retail industry based on redefined risk criteria and self-inspection, and an alternative inspection policy for the hospitality industry. Additionally, CFG developed templates and checklists to support the implementation of the new policy and drafted practical guidelines for retail food businesses and for the hospitality industry. Aligned with their focus on the digital economy, CFG developed specifications for the e-registration of food-based operators on a digital/web-based platform. CFG’s training program for cold storage, developed in partnership with the Ministry of Agriculture (MoA), resulted in removing the restrictive EU policies on Serbian frozen fruit exporters.25 CFG drafted a Food Inspection Code of Procedures to streamline inspections in all four inspectorates dealing with food (agricultural, veterinary, phytosanitary, and sanitary).26 The 18 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 1, FY 2018. Cardno 19 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 1, FY 2020. Cardno 20 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 1, FY 2020. Cardno 21 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 4, FY 2020. Cardno 22 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 2, FY 2020. Cardno 23 https://www.napa.gov.rs/tekst/49/godisnji-programi-obuka-naju.php 24 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 2, FY 2020. Cardno 25 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 4, FY 2020. Cardno 26 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 1, FY 2021. Cardno usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 12 purpose of the document is to align the practice of the inspectorates from planning to execution, and to create more transparency and predictability. As mentioned in the CFG FY 2020 Quarter 4 Progress Report, to focus on tangible results for the Serbian economy, CFG partnered with International Finance Cooperation (IFC) and AmCham to facilitate cross-border trade in the Central European Free Trade Agreement (CEFTA) region and to introduce risk-based phytosanitary inspection checks. The risk-based checks are part of the international trade facilitation policy forged by the World Trade Organization and the World Customs Organization. Risk assessment, checklist, and advisory visits were ranked positively by 1,000 businesses, as measured by the 2021 annual Survey of 1,000 Businesses. The majority of participants in the annual Survey of Businesses for 2021 positively ranked the following aspects of the regulatory environment: (1) regulations and procedures for employees; (2) risk assessments, checklists, and advisory visits in inspections; (3) VAT refund procedures and fulfillment of tax obligations; and (4) inspection supervision (inspection controls). The most negatively ranked aspects were related to (1) taxes and contributions on salaries, (2) other mandatory charges and various taxes and fees; and 3) the grey economy, in sequential order. Of these areas, CFG’s interventions have focused on risk assessments, checklists, and advisory visits and on inspection supervision (inspection controls). Exhibit 5: Regulatory environment ranking, 2021 annual Survey of 1,000 Businesses (n=1,000) 24% 10% 9% 8% 8% 6% 6% 6% -7% -19% -20% -25% -36% -39% -40% -43% -49% Regulations and procedures for employees Risk assessments, checklists and advisory visits in inspections VAT refund procedures and fulfillment of tax obligations Tax registration and payment of tax liabilities Inspection supervision (inspection controls) VAT control Procedures concerning exports and imports Collection of receivables from business partners Court procedures and requirements Taxes and contributions for entrepreneurs/self-employment VAT rates Abuse the dominant position within the market and unfair… Property tax Income tax Grey economy Other mandatory charges, various taxes and fees Taxes and contributions on salaries CFG supported the emergence of the private equity market in Serbia through interventions on both the supply and demand sides. Early in Year 1 of implementation, CFG provided support to the Ministry of Finance (MoF) in drafting the AIF. CFG conducted comparative research into the supply of equity financing for SMEs in Serbia and selected countries (Bulgaria, Estonia, Hungary, and Slovenia) and identified measures to foster the development of AIFs in Serbia, to help provide local SMEs with adequate equity financing. On December 12, 2018, CFG organized a conference to raise the interest of local SMEs in equity financing as a viable option for supporting their future growth. A total of 63 Serbian SMEs attended the conference, in addition to equity fund managers, bankers, academicians, and representatives of business associations.27 27 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 4, FY 2018. Cardno 13 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov Following these efforts, CFG developed and launched a pilot program designed to demonstrate to Serbian SMEs the process of obtaining equity financing and the role it can play in generating capital for growth. Prior to the closing of the public discussion of the draft AIF law, CFG organized a roundtable with representatives of the investment funds market, the Serbian Chamber of Commerce, the World Bank, the MoF, and the Securities Commission to inform key stakeholders of the provisions of the draft laws and to give them an opportunity to provide additional suggestions and comments directly to government officials. CFG also proposed a tax incentive regime to attract greater domestic investment in private equity funds as an alternative to real estate or offshore investments. After defining a regulatory framework for digital assets, CFG published an overview of corporate income tax, personal income tax, and the VAT treatment of digital assets. The Activity also facilitated several events to introduce this new framework with the aim of educating Serbian taxpayers – both individuals and businesses – on how the new tax treatment of digital assets will function, requirements and features, and under which conditions taxpayers may benefit from the tax incentives recently introduced for investing in digital assets. CFG contributed to the enhancement of tax procedures for alternative investment funding. The Serbian Parliament adopted the AIF in 2019 and amendments to the following tax laws in December 2020: Corporate Income Tax, Personal Income Tax, VAT, Property Tax, and the Tax and Administration Procedure, all of which were proposed by CFG. According to the opinion of several key informants, the tax procedures for alternative investment funding were “eased to the great extent possible” and stakeholders can expect to “see visible results and benefits for SMEs in the future.” Positive externality: TS Venture Fund, a new local Venture Capital Fund founded in 2021 by Telekom Srbija to provide growth capital for Serbian startups, would not have happened without enactment of the new Law on Alternative Investment Funds, adopted in 2019. CFG contributed to the creation of a taxation framework to support the inclusion of digital assets in the economy. The framework provides a clear tax treatment of digital assets, removes fiscal burdens from circulating digital assets in the real economy, and provides stimuli for owners of digital assets to invest in Serbian SMEs. The CFG FY 2020 Quarter 3 Progress Report indicated that as part of the GoS working group – composed of the Prime Minister's Cabinet, the MoF, the National Bank of Serbia, the Securities Commission, the MoTTT, the Ministry of Economy (MoE), the Security Coordination Centre, the Belgrade Stock Exchange, and CFG – CFG supported the regulation of digital assets. Specifically, CFG conducted an initial analysis of regulatory solutions for the tax treatment of digital assets. As a result of the working group’s efforts, in December 2020 the Serbian Parliament adopted the complete legal and taxation regulatory framework governing digital assets in the Serbian economy, following international standards and best practices. The Law on Digital Assets provides an umbrella regulation of all aspects of digital and crypto assets. Amendments to the laws on Corporate Income Tax, Personal Income Tax, VAT, Property Tax, and the Tax and Administration Procedure provide a complete treatment of taxes on digital assets. CFG had a key role in setting up the taxation framework and led the development and design of all tax-related amendments covering digital assets. A number of key informants stated that CFG contributed significantly to the creation of a taxation framework for the inclusion of digital assets in the economy. CFG supported the MoF to reduce the tax administration compliance burden for SMEs by codifying a VAT rulebook. As mentioned in the CFG FY 2021 Quarter 2 Progress Report, as of April 22, 2021, the Single Rulebook on VAT has been adopted and is effective as of July 1, 2021. The adoption of the Single VAT Rulebook is a breakthrough for VAT taxpayers and regulators. The rulebook usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 14 provides an overview of the Law on VAT and makes it easier for taxpayers to comply with the Serbian VAT system. In addition to consolidating 27 rulebooks governing VAT, the Single Rulebook also introduces major solutions in the areas of construction, digital services, the VAT base, and real-estate services, among others. This issue was among the leading tax reform priorities presented in the White Paper of the Council of Foreign Investors. Beginning in the second quarter of 2019, CFG worked together with MoF on defining a working version of the Single Rulebook. CFG held an online public discussion of the draft of the codified VAT Rulebook. A number of key informants believe, as one informant stated, that the Single VAT Rulebook “improves business for all companies, but smaller enterprises with less capacity will have the greatest benefits.” A very significant improvement is the adoption of the new unified VAT Rulebook, which regulates in one place a matter previously regulated through 27 separate VAT rulebooks. This change will significantly facilitate the application of VAT regulations in practice. Additionally, previously prescribed amendments clarified the existing rules, which will further facilitate the application of VAT regulations. – FIC White Book 2021 CFG provided technical support in the implementation of an e-invoicing system in Serbia. The Serbian Parliament adopted the Law on Electronic Invoicing on April 28, 2021. The Law is part of the business environment structural reforms led by the MoF. CFG assisted the GoS in all technical and strategic aspects of introducing electronic invoicing into the Serbian economy. Business-to-business (B2B) and business-to-government (B2G) e-invoicing were adopted in the first draft of the law by the EU Project, following CFG's recommendation.28 Starting in the third quarter of 2019,29 CFG provided technical support to the MoF through a working group composed of the MoF, the MoE, and the MoTTT, to implement a mandatory e-invoicing system in Serbia. CFG completed a report on the best practices of the EU and other countries and provided a set of recommendations for the development of e￾invoicing.30 The Law on Electronic Invoicing is a systemic law that changes the way the Serbian economy functions, following the existing model of e-invoicing in the EU and other developed countries. CFG helped create a strong coalition and provided technical assistance for amending the Law on Foreign Exchange Operations. CFG helped create a strong coalition of 18 key Serbian business associations with the goal to advocate for amending the FX Law. All congrats to the CFG for starting this topic. It was a huge step of the CFG … connecting and gathering of business stakeholders. They all admitted that this Law [was] the biggest obstacle in development of business both for large and small companies.... Tangible result is the fact that in the Strategy of Serbia 2025 the change of the Law on Foreign Exchange Transactions was named as one of the necessities. The study has shown the increase of the GDP would be 1% just due to this potential change. 31 This included AmCham, FIC, NALED, Digital Serbia Initiative, the Chamber of Commerce, and Industry of Serbia. CFG provided technical assistance by conducting two studies that contributed evidence of the detrimental impact of the provisions of the FX Law on business daily operations and growth.32 On November 5, 2019, the coalition of business associations sent a cover letter along with CFG’s studies and recommendations.33 These studies – Business Association Representative 28 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 4, FY 2020. Cardno 29 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 3, FY 2019. Cardno 30 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 2, FY 2020. Cardno 31 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 2, FY 2019. Cardno 32 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 3, FY 2019. Cardno 33 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 4, FY 2019. Cardno 15 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov provided key evidence indicating that the liberalization of the FX Law would result in accelerated GDP growth, potentially up to 1% per year. In the cover letter, the coalition requested a meeting with the National Bank of Serbia and MoF. The coalition also sent a letter to the Cabinet of the Prime Minister and the Cabinet of the President of the Republic of Serbia to inform them about the results of the analysis. The National Bank of Serbia subsequently sent a 41-page response to the coalition's letter, elaborating on the studies and defending its current position. CFG assisted the National Bank of Serbia in structuring a response letter and developed a presentation to be given to the main stakeholders, including international organizations, donors, and embassies (USA and EU countries) once the state of emergency is lifted. CFG is in constant communication with the coalition, and the intention is to restart the advocacy after the crisis. According to CFG and other key stakeholders, including the Bank itself (which is preoccupied with measures related to COVID-19), the Bank does not have the capacity to deal with this complex reform now. Although the National Bank of Serbia did not accept the proposed changes during the Activity, CFG’s achievements provided a common platform for business associations and their further advocacy. Representatives of the business associations involved in the coalition confirmed how important the FX Law is for the development of the Serbian economy. They agreed that CFG took a great step forward, and they are very grateful for that. Stakeholders appreciate the studies that were conducted by CFG and that all business associations supported proposed amendments to the law as well as the fact that CFG established communication with the National Bank of Serbia. Stakeholders emphasized that although the Bank of Serbia didn’t accept the proposed changes during the Activity, CFG’s achievements provided a common platform for business associations to support advocacy efforts in the future. CFG leveraged human and financial resources from local partners to promote the sustainability of results. CFG cooperated with the GoS and private sector representatives to improve the business enabling environment and identify opportunities to promote sustainability of the results. Some examples of CFG collaboration with other public and private sector actors are highlighted below: • In regard to E-commerce, CFG developed constructive cooperation with the MoTTT, a key public sector partner in this area. Additionally, in cooperation with the MoTTT, ECS, MasterCard, VISA, DINA, and the National Bank of Serbia, CFG developed the E-commerce Guide for Consumers, a comprehensive tool for secure online shopping. CFG also promoted an E-commerce guide for businesses in partnership with the Serbian, Italian, Japanese, American, and German chambers of commerce and with other business associations, such as SAM, Fashion Cluster, and NALED. • During the evaluation process, the interviewed MoTTT representative pointed out that MoTTT staff closely cooperated and worked together with the CFG team; they jointly planned and implemented all activities focused on the improvement of E-commerce in Serbia. • The CFG educational workshop on E-commerce was produced in partnership with the Serbian Chamber of Commerce–Regional Chamber of Commerce Belgrade, ECS, the Serbian National Internet Domain Registry, Banca Intesa, and DHL Serbia. usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 16 • The key informants from business associations indicated that E-commerce educational activities were conducted in partnership with the local partners and indicated that these were joint efforts in terms of engagement of staff and financial resources. • Technical assistance to the MoF in implementing the taxation framework regulating AIFs and digital assets has been a joint activity with the business enabling environment. • The CFG training program that aimed to build the capacity of cold storage was developed in partnership with the MoA. • CFG partnered with IFC and AmCham to facilitate cross-border trade in the CEFTA region and introduce risk-based phytosanitary inspection checks. CFG successfully leveraged human and financial resources from international donors to support its objectives. All interviewed GoS representatives noted that CFG to a large extent leveraged human and financial resources from other donor-funded programs to improve the business enabling environment and identify opportunities to promote the sustainability of results. For example, in partnership with the Serbian Chamber of Commerce, CFG organized eight open house events across the country, co-financed by the EU IBE Project, to support dialogue between inspectors and businesses.34 During the evaluation process, an interviewed representative of the EU-IBE project stated that they “shared the costs of events’ organization with CFG, and they were very satisfied with this cooperation, especially because the meetings were very successful.” The CFG team also forged partnerships with donor partners that can scale up the impact and reputation of the two platforms and help build customers’ trust in digital financing solutions that are still new to Serbian businesses. The United Nations Development Programme and EBRD have also expressed interest in supporting Ventu and Finspot by providing them with additional capital. The World Bank Group and its business regulation sector are interested in holding a joint conference dedicated to digital services, business regulation, and RegTech. CFG’s approach to PSE, and subsequently its strategic partnerships with the public and private sector promote the sustainability and continuity of support to SMEs. CFG’s most recent Quarterly Progress Report FY 2021, Quarter 2, indicates that as the Activity is nearing its end, CFG is hoping that its assistance and initiatives will continue on beyond the life of the Activity. This is to be done through partnerships, including but not limited to the PSE mechanism. PSE allows the Activity to leverage private sector interest, expertise, and resources in building self-reliant support systems and enabling a high level of inclusiveness for SMEs. As of June 2020, CFG recorded the following contributions from the private sector as a result of PSE:35 EXHIBIT 6: CFG’S PSE CONTRIBUTIONS PSE CASE CFG CONTRIBUTION PRIVATE SECTOR CONTRIBUTION Ventu $98,000 Ventu Contribution: $400,000 34 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 4, FY 2019. Cardno 35 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 2, FY 2020. Cardno 17 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov EXHIBIT 6: CFG’S PSE CONTRIBUTIONS PSE CASE CFG CONTRIBUTION PRIVATE SECTOR CONTRIBUTION eCommerce Association Serbia $55,000 in 10 months ECS Contribution: approx. $35,000 in the first year and maintaining a self-reliant business model for years to come. Erste Bank $60,000 Erste Contribution: $29,000 Peterhof Consulting/ dobarzabanku.rs $49,800 Peterhof Consulting is contributing through initial working hours not reimbursed by the Activity, as well as by participating in a number of outreach and promotion activities. In the longer term, its contribution is estimated at approx. $10.000+ per year. Joint workshops of bankers and SMEs $28,000 CFG covers the remaining cost of technical assistance: financial management experts that develop and deliver the training; Eurobank contribution: $17,000 in the first year Eurobank is covering the costs of training venues, logistics, catering, and partial fees for trainers. WM Equity Partners $61,450 in 7 months WMEP Contribution: $15,000 in 7 months; full program price after the initial one. The success of CFG’s PSE model is also evidenced by the fact that the Activity’s private sector partners are continuing initiatives started or supported as part of its programming. As noted by CFG staff, all of the activities, highlighted in the table above, are “now self-sustainable and operate without further USAID support, funded solely by private sector partners.” Specifically, Erste Bank is continuing to finance the Step By Step mentoring program and is considering expanding the program to other countries in the region; the crowdinvesting platform Ventu is preparing a third new capital campaign; the E-Commerce Academy is developing a new training program; Pojacalo is continuing to broadcast content to SME users; Finspot is continuing to provide services and expand its user base; and Peterhof Consulting is continuing to maintain and improve the Dobar za Banku platform. CONCLUSIONS CFG contributed to strengthening Serbia’s business enabling environment, most notably in the E-commerce ecosystem, but there are still opportunities to increase access to finance for micro and SMEs. Representatives of CFG’s intended beneficiaries and key stakeholders all stated that the business enabling environment has improved over the last four years. The majority of informants noted that the greatest improvements were made in the E-commerce ecosystem due, in large part, to CFG’s significant contributions in pushing forward initiatives. CFG implemented a number of activities that contributed to the significant improvement of the E-commerce ecosystem, notably the provision of technical assistance to the GoS for the adoption of the E-commerce Action Plan and Program Development, the Strategy for Digital Skills Development, and three regulations in the E￾commerce area. In addition, CFG developed and distributed the E-commerce Guide for Businesses and promoted and disseminated useful details on how to access and benefit from online sales. Participants in the SMEs online survey also noted this achievement: the majority (86%) of respondents emphasized that they better understand the benefits and value of E-commerce today than they did three years ago. The 2021 annual Survey of 1,000 Businesses showed that the percentage of large companies taking advantage usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 18 of online sales is significantly higher than companies of other sizes (29% of large companies, compared to 17%, 21%, and 14% of medium, small, and micro companies, respectively). Based on the responses of key informants, besides E-commerce the CFG's greatest contribution was its support of the work of inspections. CFG improved policies, notably within the food retail and hospitality industries, to bring about better regulations on inspection, and provided training for inspectors and businesses in order to adapt to the changes in the regulatory environment. Respondents from businesses included in the 2021 annual Survey of 1,000 Businesses had opinions similar to those of the key informants. with the majority of survey respondents ranking the risk assessments, checklists, and advisory visits in inspections as the best and most positive aspects of the regulatory environment, second only to regulations and procedures for employees. Inspection supervision (inspection controls) was ranked positively as well. CFG provided technical support to the GoS in the development of the regulatory framework such that the Serbian regulatory system now contains improved legislation for alternative investment funds and the systematically regulated tax treatment of digital assets. CFG supported legislative reforms related to AIFs and stimuli for owners of digital assets, as well as those to reduce the tax administration compliance burdens for SMEs by codifying VAT rulebooks and implementing an e-invoicing system in Serbia. Notably, CFG provided technical assistance to the MoF and to the GoS in analyzing the tax treatment of digital assets and drafting proposals for amendments to tax laws. The Serbian regulatory system now contains a systematically regulated tax treatment of digital assets, while tax incentives additionally encourage owners of cryptocurrencies and other forms of digital assets to invest their income in the Serbian economy and thus achieve significant tax savings. In this way, an opportunity for digital assets to become an additional source of financing for companies in Serbia has been provided. Based on feedback from key informants and FGD participants, the previously mentioned CFG activities were successful because the Activity involved all key stakeholders from the public and private sector in the process, establishing a positive dialogue between public and private sector actors. CFG effectively leveraged resources from other donor-funded programs, the private sector, and the GoS, building self-reliant support systems and enabling a high level of inclusiveness for SMEs. Cooperation has proved to be the key in generating sustainable and successful support to Serbian SMEs, as pointed out by almost all interviewed key informants. The interviewed key informants, all of whom were involved in PSE initiatives, stated that they will continue to implement the initiatives established with CFG support, thus ensuring the long-term sustainability of CFG’s programming to improve the business enabling environment and related regulatory framework after its completion. Furthermore, CFG hired highly competent local experts for each intervention area. Studies and analyses developed by experts provided a good platform to define CFG’s strategies and activities. For example, two studies were developed that elaborated the need for amendments to the FX law and are, at the same time, a platform for the further advocacy of business associations involved in the coalition. EVALUATION QUESTION 3: To what extent and in what ways did CFG improve SME access to finance through expanding the variety of financial products available? Building on the findings and conclusions of EQ1, the evaluation team examined the financial as well as the non-financial products CFG created, supported, and/or enhanced to improve SMEs’ access to finance. The team also assessed the extent to which these products contributed to the diversification of 19 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov available products for SMEs. This section also discusses the extent to which these financial and non￾financial products have contributed to expanded access to financing for SMEs, including those owned by women, youth, and other marginalized groups. Finally, in response to EQ3, the evaluation team sought to assess the success of the strategies CFG implemented to identify and leverage resources from other donor-funded projects, the private sector, and the GoS to promote the sustainability of results. FINDINGS Access to finance remains the leading challenge constraining SME growth in Serbia. Based on interviews conducted with representatives from private sector businesses, the GoS, the U.S. Embassy, and AmCham, among others, “the current most important issue for SMEs is financial resources ... [because] banks in Serbia are liquid but conservative, and SMEs are risky clients. Commercial banks do not have an appetite for SMEs.”36 This presents a major constraint for SMEs seeking capital to start and grow their companies, because they often don’t have the proper documentation, such as financial and accounting statements and business plans, to be eligible for such financing. SMEs’ lack of access to capital thereby restricts job creation, employee training and innovation, and the overall growth of the Serbian economy. Additionally, the process of accessing finance is further constrained by the existing regulations, including the FX Law and the Law on the National Bank of Serbia, Article 4. These regulations were defined by CFG as “limiting and obsolete”, restricting financial institutions, excluding banks, from lending to SMEs. CFG also indicated that the National Bank of Serbia, as the main regulator, is “resisting any changes of these laws” and has been fore “more than a decade.” CFG supported the launch of two innovative alternative financial products into a historically liquid, but highly conservative and bank-centric financial market. To overcome the barriers that SMEs continue to face in accessing traditional bank financing, CFG helped develop, launch, and garner support for two new alternative financial products – Ventu and Finspot – into the Serbian market. Based on CFG reports and interviews with CFG staff, beneficiaries, USAID and US Embassy staff, and other stakeholders, these products supported SMEs’ ability to access new financing. • Ventu’s “crowdinvesting” platform demonstrated initial success in helping start-up and early-stage SMEs raise capital. This innovative platform was the first new non￾banking source of financing in Serbia in more than ten years. As previously noted, on November 3, 2020, as part of a joint venture in cooperation with BDO-Serbia and Conda, CFG supported the launch of Ventu. Ventu is the first “crowdinvesting” platform in Serbia and is focused on addressing the challenges that start-up and early-stage businesses face in raising capital. Ventu provides a way for early-stage companies, which are not generally able to obtain loans from the traditional banking sector, to raise small amounts of investment capital from friends and families as well as from their clients and customers. In its first campaign to raise capital for an early-stage specialty food business, as reported by a Ventu representative, Ventu attracted 71 investors who committed EUR 50,000 to help the business expand and develop a long-term business model. Following that initial campaign, several other SMEs expressed interest in using the Ventu platform to raise capital for their businesses. However, CFG staff explained that the onset of the COVID-19 pandemic caused many of these companies to postpone their capital campaigns due to the uncertainty of the economic environment and the potential adverse impact on their revenues, as well as the unknown 36 USG and Government Stakeholders' Representatives usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 20 willingness of potential investors to make new financial commitments. Since the completion of data collection, CFG confirmed the success of a second campaign, to support another early￾stage specialty food business. As highlighted on Ventu’s website, this second campaign attracted 39 investors who committed EUR 40,000. Ventu continues to work closely with several companies to launch campaigns to raise capital using the platform, with one more campaign anticipated before the end of CFG’s period of performance. If these campaigns proceed as expected, Ventu’s crowdfunding platform will mark a major step forward in diversifying funding opportunities for SMEs in a country where banks are still the main source of financing, with more than 90% of the market share. • Finspot’s digital invoicing platform demonstrated initial success in expanding access to finance for SMEs. In 2020, CFG helped establish Finspot, the first digital invoice financing or e-factoring platform in Serbia. Finspot allows SMEs to access financing by using their customer invoices as collateral to receive funding from the Finspot platform within a short time, typically within 24 hours. The Finspot model is an attractive financing option for SMEs in Serbia, which often face barriers, including complex, time-consuming procedures when applying for funding from traditional commercial banks. In its first three months, Finspot disbursed over EUR 1,000,000 through its e-factoring platform. Since then, the platform has been able to triple its disbursements, providing EUR 4,000,000 through more than 1100 invoice financing transactions. CFG’s self-sustaining Equity Financing Program for SMEs shows promise for expanding equity financing. Based on CFG’s recent FY 2021 Quarter 3 progress report, CFG developed an Equity Financing Program that provides training for SMEs to better prepare them to attract investors and to enable them to absorb equity financing. Additionally, the program also supports SMEs in negotiating with investors to overcome the lack of trust and improve understanding of equity financing within the SME community. This program, which was originally developed as an internal project financing tool, recently became self-sustaining, with one company signing a pre-agreement for EUR 11,000,000 in equity financing, and three more companies in the final phase of negotiation with investment funds. The total transaction value of deals closed to date include EUR 18,700,000 in new equity financing. While this is a relatively new financing tool in Serbia and is not yet being widely used, it could be a very useful financing platform for SMEs in Serbia if usage expands. CFG supported the creation and/or enhancement of a number of non-financial programs designed to improve SMEs’ access to finance. In addition to supporting the development of new financial tools, CFG also collaborated with various public and private sector actors to develop and/or enhance trainings and business services to improve SMEs’ access to and likelihood of obtaining external financing. The following summarizes the non-financial programs of most significance as identified by key stakeholders: • National Strategy on Digital Skills and Action Plan. Working closely with members of the GoS, CFG assisted the MoTTT and the MoE in developing a National Strategy on Digital Skills as well as an action plan for how to operationalize the strategy. As part of the National Strategy on Digital Skills’ Action Plan, the GoS provides critical training in coding and other related skills for SMEs, with a special effort to reach women, youth, and citizens in rural and other marginalized communities. Although implementation is still in the early stage, the activity is expected to contribute to the expansion of SMEs’ digital knowledge and skills and thus their potential to become more competitive within their industries, thereby contributing to the growth of the broader Serbian economy. 21 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov • Erste Bank, “Step by Step” Program. Erste Bank is a 200-year-old financial institution in Serbia, which was originally founded in Austria as a social enterprise to provide banking and savings services to all citizens regardless of their economic status. CFG worked closely with Erste Bank to help expand the Bank’s existing program “Step by Step,” which seeks to provide business mentoring skills to the underserved SME market sector in Serbia. “Step by Step” was designed to help SMEs prepare and apply for bank financing and to provide follow-on training and support services to the business once financing was received. Prior to CFG’s engagement, Erste Bank drew on external consultants and donor funding to provide mentors to assist SMEs. CFG assisted Erste Bank to implement a new approach using the Bank’s existing staff to directly mentor and support their SME client companies. CFG assisted Erste Bank to select and train an internal team of 20 mentors who then provide direct training and support to SMEs. This in turn allowed the Bank to gain a better understanding of the types of challenges their SME client businesses face in operating and growing their businesses. Additionally, as noted by a representative of Erste Bank, the enhanced Step by Step program provides the Bank with a “better connection, better control, [and] access to information,” providing an “early warning system” to improve the monitoring of the Bank’s portfolio. Through the Step by Step Program, Erste Bank delivered 200 mentoring sessions spanning 700 hours to 127 SMEs. Additionally, since CFG’s engagement, Erste Bank has reported that their number of SME loan clients has doubled. Building on these successes and understanding the value this program provides to SMEs, when CFG funding for the in￾house mentoring program ended, the Bank made a decision to continue implementing the program with internal resources. And thanks to the CFG … the number of approved loans doubled, after the start of cooperation with the project. – Direct Financial Beneficiary • E-Commerce Academy. The E-Commerce Academy is a specialized training program organized through the ECS. ECS provides a range of training and support services to SMEs in Serbia to increase their knowledge and skills in E-commerce so that they can operate more effectively within the growing E-commerce ecosystem. ECS recently completed the third cycle of its E-commerce Academy for Serbian businesses and announced the upcoming fourth cycle. CFG supported ECS to implement the initial two cycles of the E-Commerce Academy by connecting ECS with local and international private sector partners. Through CFG support, ECS developed partnerships with private sector partners, most notably Mastercard. As a result, the third and fourth training cycles of the E-Commerce Academy are being implemented without donor support and are considered to be fully sustainable through the financial support of private sector partnerships led by Mastercard. Private sector support demonstrates the market’s strong interest and understanding of the value add that these capacity-building programs provide to both SMEs as well as the broader Serbian economy. Another important contribution of ECS was the creation of an E-commerce Guide for consumers which was published on the MoTTT platform and promoted in the Politika newspaper and a national TV campaign conducted by the MoTTT. • Pojacalo – An Online Learning Program. CFG assisted the Pojacalo Project in creating and operating an online program to build a “virtual association” for micro and SMEs and entrepreneurs in Serbia. The goal was to create audio and video content, including weekly usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 22 podcasts to train and motivate entrepreneurs and SME owners. With the support of CFG, Pojacalo developed and produced over 130 episodes or podcasts, providing training and support services to more than 500 participants on topics of interest to SMEs such as VAT, E￾commerce, and the changing regulatory framework in Serbia. In addition to serving as a training resource, the program also provided a space for networking so the entrepreneurs and SMEs could share best practices and lessons learned to help them adapt their operations and better compete in the evolving E-commerce marketplace. • Dobar za Banku – E-Learning Program. CFG collaborated with Dobar za Banku to develop an e-Learning platform that uses videos to provide SMEs with step-by-step guidance on how to prepare and complete a loan application for bank financing. A lack of understanding of the loan application process is a barrier that many SMEs face. Through this e-learning program, SMEs are able to increase their knowledge about the processes, an important and necessary skill in helping SMEs expand their access to finance and grow their businesses. To date, as reported by Dobar za Banku, more than 200 companies have benefited from the training program provided by Dobar za Banku and supported by CFG. CONCLUSIONS CFG’s support for the development and expansion of alternative financial and non-financial products for SMEs in Serbia has increased the ability of SMEs to apply for traditional lending opportunities as well as to expand its understanding of non-traditional financial sources to grow their businesses. CFG provided and supported a number of important and innovative improvements to expand the availability of financial and non-financial resources for SMEs in Serbia, including: (1) new sources of capital, (2) ongoing training and mentoring to improve business skills, and (3) expansion of E-commerce within Serbia to address the changing economic and social context related to the COVID-19 pandemic. CFG recognized and provided the necessary resources to locally-led innovative ideas to address the constraints limiting SME growth. These important contributions by CFG have been needed and critical to assisting SMEs in Serbia to adapt and survive during this time of unprecedented external challenges. The contributions of the CFG team, in close coordination and cooperation with key financial actors in Serbia, resulted in increased availability of important new financial resources and tools, as well as enhanced business and technical skills for SMEs. Without CFG’s support, many of these initiatives would not have been realized and thus accessible to SMEs as quickly as they were. CFG’s work to expand access to finance through innovative, online platforms has expanded the availability of new financial products for SMEs as well as contributed to the broader goal of strengthening E-commerce in Serbia. The two new financial platforms developed by CFG – Ventu, and Finspot – each show promise for becoming important new sources of financing for SMEs in Serbia. The success of Ventu, the first Crowdfunding campaign and new non-banking source of financing in Serbia in more than ten years, provides an important example for other businesses and potential investors in Serbia looking to raise funds via nontraditional sources such as the internet. Similarly, Finspot’s success in providing companies with a new way to immediately finance their customers’ invoices was another important example of how new tools can be leveraged to expand access to finance and allow SMEs to run their businesses. Both tools have the potential to become recognized and trusted financial tools for SMEs. Furthermore, the initial success of both Ventu and Finspot indicates that there is an interest – on both the SME side and the investor side, including individual investors – to support and finance SMEs using these innovative platforms. 23 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov CFG supported the development of non-financial resources for SMEs to become more competitive and to adapt to and meet the evolving requirements of the business environment, thus promoting widespread economic growth. While access to financial resources is essential, many SMEs still require non-financial support in the form of training, mentorship, and technical assistance in order to grow their businesses. While outside of its mandate to diversify the types of financial products available to SMEs, CFG recognized the importance of improving SME’s awareness of the current and emerging opportunities within the financial market as well as improving their business skills to ensure that SMEs were well prepared to enter the marketplace. CFG’s work to assist Erste Bank in creating an in-house team of mentors in its “Step by Step” program who provide advisory and training services to SMEs applying for loans was a unique service, which likely contributed to the growth of the Bank’s SME clients. This program, among others including its collaboration with the new E-Commerce Academy and Pojacalo, also helped expand SMEs’ knowledge and access to business resources. While these programs have shown initial successes, continued support is critical to reaching the broad scope of SMEs in Serbia. EVALUATION QUESTION 4: How well did CFG adapt to the changing operational environment including incorporating adaptive and continuous learning in planning and delivering its assistance? The previous EQs, EQ 1–3, assessed, among other things, the CFG Activity’s response to the existing and emerging challenges and opportunities facing the SME community in Serbia. To answer EQ4, the evaluation team examined the mechanisms and approaches CFG utilized to continuously monitor project implementation as well as the contextual factors that may have influenced CFG’s programming. Additionally, the evaluation sought to understand how CFG incorporated these mechanisms into program management. Since the project was implemented during the COVID-19 pandemic, the question also examines to what extent and how CFG’s approaches and subsequent performance were affected by the pandemic, and how effectively CFG adapted programming to assist SMEs given COVID-19. FINDINGS While CFG’s AMELP did not include a learning plan, CFG’s approach did integrate collaborating, learning, and adapting (CLA) as part of its overarching monitoring and evaluation framework. In reviewing CFG’s AMELPs, the evaluation team noticed that the plan did not contain specific learning questions, although it did include a brief description of the Activity’s commitment to (1) promote strategic collaboration with internal and external stakeholders, (2) facilitate moments of pause and reflect; and (3) create an environment for adaptive management. For example, the CFG team planned to hold monthly reflect and adapt meetings as well as monthly reviews of the achievements and Activity linkages with the performance indicators. Additionally, the majority of key informants expressed great satisfaction with CFG’s approach to collaboration and adaptation based on local needs and realities. As noted under EQ1, this approach was instrumental in supporting the ongoing improvement and adaptation of CFG’s strategy to reflect ongoing learning. However, while CFG did conduct a number of relevant research activities, such as the annual Survey of 1,000 Businesses and The Effects of Foreign Exchange Liberalization Report, which were used to inform CFG’s decision-making processes, CFG did not identify and articulate their knowledge gaps and generate a plan to address them. As a result, their approach to learning was inconsistent and ad-hoc in nature which may have resulted in missed opportunities to address knowledge gaps in its theory of change and further improve its strategy and implementation. usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 24 To understand the effects of the COVID-19 pandemic on SMEs and to better support SMEs in responding to the changing operational environment, CFG supported the launch of the “Overcoming Crisis Together” survey and a series of “Specifically, for SME’’ podcasts. Due to the COVID-19 pandemic, a state of emergency was declared in the Republic of Serbia on March 15, 2020. CFG reporting shows that shortly afterward, in the last week of March (from March 27 to March 31), the CFG, in cooperation with the Serbian Chamber of Commerce and the Belgrade Chamber of Commerce, conducted a survey on the impact of the COVID-19 crisis on doing business, in which 806 businesses participated.37 CFG used the information to prepare SMEs and decision-makers for rapid recovery plans "the day after the crisis." In partnership with the ICT Hub, an online panel discussion titled “How to adjust your business in times of crisis” was held on April 8, 2020, which gathered 86 participants. Another, titled “How to make a good use of state aid measures” was held on April 12, 2020 and attracted 1,076 participants.38 A second survey cycle was conducted in April 2020 with the participation of 1,000 businesses, and a third cycle, in June 2021, with 296 business participants. CFG used research findings, for example, to match the topics of the panels with identified needs of the companies. Results from all three surveys attracted significant public attention, including two TV broadcasters with a reach of over 1.5 million viewers. This research was used as the basis for further analysis of the situation in the Western Balkans conducted by the World Bank Group and EBRD.39 The survey findings were also used in defining economic measures to mitigate the pandemic’s consequences. The GoS defined these economic measures in cooperation with the Serbian Chamber of Commerce, drawing heavily on the findings of the survey. While CFG experienced several delays in implementation due to the COVID-19 pandemic, CFG remained flexible and quickly adapted its programming to the new operating environment. CFG postponed many activities planned for the first quarter of 2020, such as practical training in retail food safety and hospitality food safety, export capacity building for cold storage, public discussion on the VAT Rulebook, training in E-commerce, activities related to developing efficient E￾commerce inspections, open days of inspections, and activities on the improvement of tourism inspections. In the second quarter, however, CFG organized 18 online trainings, 9 webinars, and 2 online public discussions/roundtables.40 The Activity’s reporting indicates CFG’s continuous intent and ability to adapt to the COVID-19 crisis and the “new normal” way of living. The CFG program continued to be implemented, following the same advice that it has offered to SMEs for resilience. CFG has strengthened its digital presence, installed and maintained online communication with stakeholders, sought feedback through new, electronic platforms, developed and conducted online surveys, and transitioned programs to e-learning courses, panels, and discussions. In addition, data in the last available quarterly progress report (Q2 of 2021) shows that of CFG’s 70 contracts with subcontractors, 69, or 98.6%, were realized, while only one was terminated. The best thing about the CFG is that they showed huge flexibility, which was showed in the COVID￾19 situation. They adjusted their approach to the COVID-19 situation and reacted promptly. – Business Association Representative 37 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 1, FY 2020. Cardno 38 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 2, FY 2020. Cardno 39 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 2, FY 2020. Cardno 40 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 2, FY 2020. Cardno 25 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov CFG helped beneficiaries to adapt to the changes in the business enabling environment. Fifty-nine percent of interviewed beneficiaries and key stakeholders pointed out that CFG helped their organizations’ ability to adapt to challenges and opportunities that arose in the business enabling environment, particularly in terms of the COVID-19 crisis. None of the interviewed key informants stated that CFG hindered their organization's ability to adapt to the challenges and opportunities that arose in the business enabling environment. The COVID-19 pandemic and the proclamation of the state of emergency made CFG’s assistance in E-commerce even more important for SMEs. According to both CFG’s research and the Serbian Government’s E-commerce development strategy, lack of knowledge and the shadow economy were the top barriers to E-commerce in Serbia. CFG’s 2019 annual Survey of 1,000 Businesses indicated that while the majority of Serbian companies strongly acknowledge the importance of E￾commerce, very few have the key features necessary for it. Only 12% of surveyed businesses have a web shop, and only 8% offer online payment options. According to key informants and CFG reporting, since the emergence of COVID-19 and the proclamation of the state of emergency, E-commerce in Serbia has increased three- to ten-fold.41 This shift to online sales is helping SMEs to generate revenue, retain their workers, and even thrive, while also providing the public with essential products they are unable to get safely and securely in person. Technology and E-commerce are providing access to markets for SMEs and access to goods for consumers, for an overall more resilient system. The findings of the 2020 annual Survey of 1,000 Businesses show that there has been a tectonic shift in businesses’ perception of the benefits of having an online store. Over 85% of respondents who were online store owners rated the online store as important or even more important than the traditional brick-and-mortar store. But almost two-thirds (66%) of businesses have not adopted a new business strategy to adjust to the effects of the pandemic. As a result, CFG has continued providing assistance to address the needs from both the demand and supply sides – training businesses on the benefits of online sales while also educating consumers and mainstreaming online shopping. COVID-19 was a booster for the CFG Project, which might sound strange. CLA methodology of the USAID is sometimes difficult to think about. But when COVID-19 happened we were doing things according to the CLA methodology. This situation pushed some changes and worked on their behalf for the development of E-commerce, for instance. – Business Association Representative 42 CFG has adapted to the new reality caused by the COVID-19 pandemic and continued to create an E-commerce community through online events. CFG continues to support businesses in overcoming the COVID-19 crisis through E-commerce. As already mentioned in EQ2, in cooperation with the MoTTT and ECS, CFG developed and published the E-commerce Guide for Businesses, which has disseminated information on how to access and benefit from online sales since 2020. Levels of adaptation to the new reality can be seen through the levels of participation in online events. CFG has seen much larger number of companies attending online events, such as the E-commerce events, which, according to half (50%) of the interviewed key informants, increased the impact of the Activity. 41 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 1, FY 2020. Cardno 42 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 1, FY 2020. Cardno usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 26 Despite numerous changes in Activity leadership, the majority of beneficiaries and key stakeholders expressed high levels of satisfaction with CFG’s programming. Over the course of its four years of implementation, CFG has had four Chiefs of Party (COP). The first COP was contracted in January 2018 but resigned the following month. The second COP started work in May 2018 but was replaced in the third quarter of 2019. In April 2020, the Activity transitioned to a local COP, promoting the Activity’s Deputy COP. Only one informant stated that it “brought some issues such as discontinuity” to the team and the Activity’s overarching implementation approach. “Since the beginning of the project CFG had a good responsive component; they asked about what is actually needed. Also, they (CFG and consultant) were aware that solutions that have some countries, such as UK, require good capacities in tax administration, and cannot be implemented in Serbia therefore.” – Business Association Representative The majority of interviewed key informants (63% of beneficiaries and key stakeholders) pointed out the professionalism and commitment of the CFG team members when asked about their engagement with CFG. They also emphasized that one of the very important features of this Activity is reflected in the “high level of expertise, both of engaged experts and developed studies,” with several citing the reports Strengthening E-commerce in the Republic of Serbia, Proposed Measures That Will Foster AIFs Development, Analysis of Obstacles for Implementing Crowdlending in Serbia, and The Effects of Foreign Exchange Liberalization. The evaluation team did not receive any unfavorable comments from key informants about the CFG team or about the activities they carried out. CFG routinely collaborated with key stakeholders and integrated feedback into its programming and adaptive management practices. For the majority of interviewed informants, one of the hallmarks of the CFG approach is nurturing public dialogue and fostering close collaboration with the business community and between the public and private sectors. Seventy-eight percent of the interviewed beneficiaries and key stakeholders pointed out that CFG addressed feedback from their organizations to a great extent as well as altering or changing strategies and Activity implementation for greater impact in response to changes in the broader environment. All interviewed GoS representatives stated that CFG addressed the feedback provided by their respective Ministries. CONCLUSIONS While CFG utilized data and beneficiary feedback to adjust program strategy and activities to better respond to SMEs’ needs and the rapidly changing operating environment, CFG did not fully leverage existing opportunities to strengthen its indicators to better systematize data-driven decision making. CFG utilized a number of studies and reports, including the annual Survey of 1,000 Businesses, the Strengthening E-commerce in the Republic of Serbia Report, the Proposed Measures That Will Foster AIFs Development Report, and the Analysis of Obstacles for Implementing Crowdlending in Serbia Report, to assess needs and inform programming. Findings from the annual Survey of 1,000 Businesses, which captures businesses’ perceptions of the existing enabling environment, access to finance, and company competitiveness, led the Activity to adapt its strategy around SME inclusion in the digital economy. Specifically, CFG helped develop an E-commerce ecosystem for SMEs to access digital markets and introduced electronic invoicing (e-invoicing) as a tool to reduce cost, improve financial management, and facilitate external financing options for SMEs. Similarly, in light of the COVID-19 pandemic, the Activity regularly sought feedback from stakeholders through online communications and surveys, adjusting activities based on such feedback, including transitioning programs to e-learning courses, panels, and discussions. Although the Activity was able to 27 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov incorporate data from reports and feedback from beneficiaries to adapt programming, as noted under EQ1, the data collected as part of the Activity’s AMELP was limited, particularly with regard to monitoring and measuring progress towards increasing access to finance. More frequent monitoring and reporting of formal AMELP and informal indicators, including those reported by CFG’s intermediaries and direct beneficiaries, could have provided USAID and the Activity with additional information to adapt and adjust programming based on progress as well as SME needs. Close collaboration and networking with other public and private actors oriented to SMEs' support helped in making strategic decisions and providing prompt responses to business community demands. Due to such an approach, the majority of key informants emphasized that CFG had adapted its strategies to the needs and goals of their organizations and that CFG respected their feedback. As an example, CFG supported the launch and administration of the “Overcoming Crisis Together” survey as well as a series of “Specifically, for SME” podcasts to better understand SMEs’ challenges in the new operating environment and, subsequently, to provide useful tactics to weather these challenges. Moreover, CFG facilitated the “How to make smart use of the Government’s support measures” event in mid-April 2020, during the lock-down, only two days after the GoS adopted a set of measures aimed at helping the private sector during the COVID-19 pandemic. As a result, more than 1,000 attendees participated in these events, which is, by far, the most attended online event since the beginning of the pandemic crisis in Serbia. This event covered practical, strategic, legal, and tax advice on how to use GoS measures to mitigate the harmful effects of the pandemic. The panel provided useful advice to SMEs about how to adequately use the policies, mitigate risk, and adapt their business to the crisis. Such prompt reaction to the business community’s needs would not have been possible if, during its previous work, CFG had not established close collaboration and networking ties with representatives of the GoS, business associations, and other key stakeholders. CFG has been extremely helpful. In formal terms, it was very professional cooperation. But it was even above that. The project team was at the disposal to the Ministry. We functioned almost as one team. It is the best project so far, in my career. Getting to the added value is how I would describe our cooperation. - Government Representative EVALUATION QUESTION 5: From the perspective of intended beneficiaries and other stakeholders, how relevant and effective has this Activity been in expanding SME growth? The purpose of EQ5 is to assess the extent to which the targeted beneficiaries and other stakeholders believe that CFG’s work over the past four years has been successful in helping to expand economic growth in Serbia. Specifically, the evaluation assessed the relevancy and effectiveness of CFG’s programming to address challenges and barriers that SMEs face to increase revenue, promote job creation, and expand exports as well as to encourage private sector investment and competitiveness. FINDINGS CFG’s programming has contributed to expanding SME growth in Serbia, but it is too early to fully measure CFG’s contributions across Serbia’s SME landscape. Results of the online survey indicate that CFG’s responding beneficiaries felt strongly that the Activity’s work in Serbia was both relevant and effective in promoting the growth of SMEs, with more than 75% responding positively, 23% being neutral, and no negative responses (refer to Exhibit 7). In addition, key informants expressed appreciation for the relevancy and effectiveness of CFG’s interventions. One informant representing a direct beneficiary noted that “[CFG’s] management had a great overview of the market needs and could react,” while a GoS representative indicated that “CFG was here to help us. They tried not to overlap with usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 28 other projects. CFG asked about our needs…. [T]he focus of the action was to improve the knowledge, access, and awareness of SMEs.” While beneficiaries indicated that CFG’s activities were both relevant and effective in expanding SME growth by addressing SMEs’ constraints and by contributing to increases in revenue, promoting job creation, and expanding exports, most stakeholders agreed that it was still “too early to properly measure these results.” Additionally, despite the overall positive response from individuals surveyed, direct attribution to CFG is not possible at this time. CFG has significantly contributed…. The work on changing the regulatory framework is complex and expensive and could not be done without this huge support, expertise and effort. It is a very slow and demanding market in Serbia and there is a huge lack of knowledge. When we take this into consideration, the results of this project were very good. – Direct Financial Beneficiary Exhibit 7: To what extent do you agree with the following statement: “CFG provided relevant and effective products and support to enterprise like mine to promote the growth of SMEs.” (n=48) 46% 25% 23% 6% 0% 0% Strongly Agreed Neutral Disagreed Strongly No Agreed Disagreed Response CFG’s work on improving the overall taxation and regulatory framework was an important step in clarifying the laws for businesses and investors in Serbia and providing an incentive to attract investment capital for Serbian SMEs and other businesses. CFG was involved in assisting the GoS to improve the body of laws and regulations impacting businesses as well as investors. As of late 2021, CFG has assisted the GoS in developing more than 80 new laws and regulations for the improvement of Serbia’s taxation and regulatory framework for businesses and investors. CFG had a positive influence on improving the ways that inspections of goods and services are conducted. One of the biggest challenges that businesses in Serbia face is how to become more competitive with similar businesses operating throughout the European community and to help Serbia continue toward its goal of becoming a member of the EU. Reducing the administrative burden on SMEs and other businesses in Serbia is a very important part of that equation, and CFG’s efforts to improve the efficiency and effectiveness of various areas relating to inspections were well received by the GoS as well as by the Serbian business community. As noted by one government stakeholder, “CFG greatly improved the communication and simplified the required interaction between businesses and GoS inspection officials, reducing the burdens of time and expense for businesses as well for the overall inspection process.” CFG’s goal was to “make inspections efficient, transparent and streamlined for businesses.”43 Over the past several years, CFG has worked with inspection departments, including Sanitary, Labor, Food, and Phytosanitary, to provide training to civil servants, to familiarize them with the issues faced by SMEs, and to help shift the strategy to one of prevention rather than extracting penalties. CFG also arranged for inspector teams to visit the UK, considered one of the best inspection services in Europe, where they received training and also attended a conference with other international inspection professionals. The Serbian attendees considered these activities to be very useful. Following CFG’s work in the inspection areas, the GoS Inspections Department plans a platform that in many cases will allow Serbian businesses to engage in required inspections of their products without the need to go in person or to deliver required paperwork in person to an inspection office. As a result of CFG’s work in this area, both sides of the process – the businesses as well as the offices of inspections – believe there has been substantial improvement in the way that inspections relating to food and health 43 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 3, FY 2021. Cardno 29 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov have been conducted and that the burden on both businesses and the inspection offices has been reduced. CFG contributed to improving the growth of SMEs in Serbia by supporting the development of E-commerce tools. One of CFG’s goals is to “support the establishment of sustainable and supportive ecosystems that will bring businesses closer to the requirements of a digital economy and the pandemic and post-pandemic realities.”44 In pursuing that important goal, CFG played a major role in helping to expand E-commerce in Serbia, with a focus on assisting SMEs to adapt to the requirements for increasing their competitiveness and adjusting to the technological developments of the growing digital economy in Serbia as well as globally. As noted by one key informant representing the SME community, “CFG gave significance to SMEs and help[ed] them to become more visible [in the market].” CFG’s strategy to engage Serbia’s private sector, specifically with respect to E-commerce, by taking a more holistic or system￾based approach, was not only seen as effective, but is also likely to contribute to greater local ownership and sustainability. CFG’s activities are structured to include close engagement with the broader private sector community, which in turn helps to leverage additional support and expertise from private sector firms and provides a greater sense of trust and inclusiveness within the SME and the overall Serbian business community, including public sector actors. Notably, CFG’s work accelerating the shift to E-Commerce was recently recognized by the Organization for Economic Co-Operation and Development (OECD) in its latest Policy Outlook report, which emphasized USAID’s role in the E-commerce revolution in Serbia through its CFG Activity in cooperation with the MoTTT to create a Program and Action Plan for the Development of E-Commerce, which contributed to expanding economic growth in Serbia. Until the start of the cooperation with USAID, [Direct Financial Beneficiary] did not have that significant and tight cooperation with the others, including [the] state and the institutions. USAID truly did this enhancement of cooperation, which still functions on the high level. The project is very significant for the development of the E-commerce in Serbia, it is very successful. – Direct Financial Beneficiary With regard to the Activity’s SME beneficiaries, specifically those participating in the E-commerce Academy, all expressed great appreciation for CFG’s work in establishing the Academy in partnership with the E-commerce Association. Specifically, one participant noted that the trainings provided were “valuable because it confirmed my thoughts and ideas for the development of my company.” Another participant commented that “the entire work of the Academy and of CFG itself are very important, especially its work in digitalization and raising the awareness of the wider audience on this. It is done in a perfect moment.” Additionally, beneficiaries indicated that these trainings were of high quality, including “the choice of lecturers … and the [modes to] transfer knowledge.” The complete Academy program is very valuable for the average user. The quality was high. Participation in the program increases the chances for success of someone who is thinking to sell online, but with external help. – SME Beneficiary As a point of feedback, several participants indicated that the training courses targeted either more advanced SMEs or, in other cases, those that were less advanced. A few participants noted, “It would be 44 USAID/Serbia, Cooperation for Growth Project Quarterly Progress Report: Quarter 3, FY 2021. Cardno usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 30 good to have a beginner’s class at the start of the program. It [the Academy] is designed for the people who are already deep in this field.” In contrast, another group of participants noted an “advanced option should have been provided.” To mitigate this difference of opinion, one participant suggested the integration of a self￾assessment tool prior to and following participation in training modules. CFG played a key role in expanding financial products for SMEs in Serbia by supporting the development of two innovative funding platforms, Ventu and Finspot. Based on the findings generated from the online survey of SMEs benefiting from one or more of CFG’s interventions – including but not exclusive to Ventu and Finspot – the evaluation team found that 41% of beneficiaries felt that CFG improved and expanded the financial products available to them, compared to 45% who felt neutral and 12% who did not agree that CFG improved or expanded the available financial products (refer to Exhibit 8). Specifically, with respect to Ventu, as previously noted under EQs 1 and 2, CFG has supported one successful campaign to date, with four more campaigns currently underway. A representative from Ventu indicated that CFG was “a crucial partner. Ventu would not have taken place without them.” CFG assisted the joint venture between BDO-Serbia and Conda to address the “complaints on the market [related to] the lack of funding. There were no instruments on the market for start-ups and newly established companies.” Through CFG’s support, Ventu was able to navigate the relevant regulations, network with key stakeholders, and identify their first client, Bubaja, a specialty food business, successfully introducing a new financial product to Serbia’s SME market. As a result of these efforts and the financing provided, Bubaja was able to expand production and open “three new, small shops.” Additionally, a representative from Bubaja indicated that they are also experiencing a shift in sales. Prior to receiving financing through Ventu, 80% of their sales came from big chain supermarkets, 10% online, and 10% in-store. Today, only 10% of sales come from the big chain supermarkets, whereas 50% and 40% come from online and in-store purchases, respectively. Exhibit 8: To what extent do you agree with the following statement: “CFG has improved and expanded the financial products available to my enterprise.” (n=49) 45% 31% 12% 10% 2% 0% Strongly Agreed Neutral Disagreed Strongly No Agreed Disagreed Response Following participation in the FinTech for Business competition and the SME in MVP forum, CFG was perceived to have provided “very relevant” support to Finspot in “the overall outreach of their brand. They [CFG] helped with publicity, such as connections with the media…. USAID has a great way of connecting various stakeholders.” Additionally, CFG “unofficially helped with human resources within the company, for instance, to choose the Chief Executive Officer.” Based on the Activity’s quarterly reporting, Finspot has provided over EUR 3,000,000 to SMEs through more than 900 transactions. Unfortunately, despite repeated efforts, Finspot’s end users were not willing to engage in the evaluation to speak about their experiences with the platform. CFG started the initiatives not only for the topics that were easy, but also opened the discussion on the difficult ones. In the A2F field, the crowd investment is very important. Even though it was a small campaign the fact they did it is so much more important. – Direct Financial Beneficiary 31 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov CFG’s contributions to non-financial support, including the Step by Step Program, Pojacalo, and Dobar za Banku, were relevant and important to increasing SMEs’ awareness and knowledge of financing options. In addition to introducing new financial products and the launch of the E-commerce Academy, as noted above, CFG contributed to expanding access to finance by increasing SMEs’ awareness of opportunities to access financing and strengthening SMEs’ capacity to pursue financing. • Step by Step Program: Leveraging the Bank’s existing program, CFG provided valuable support to Erste Bank to train a team of mentors to deliver non-financial support to SMEs. One mentor noted that CFG’s support “upgraded his skills.” He gained “a different perspective on how clients work and learned to implement a more detailed approach.” Another mentor noted, “Thanks to the very good activities they [CFG] implemented [to improve the quality of non-financial support] … the number of approved loans has grown 2 times.” He added that “to get the support to implement everything your clients need, in terms of mentoring, in the cross-cutting environment is very valuable.” Erste Bank’s clients and those that participated in the Step by Step program conveyed general levels of satisfaction with the program and its “cooperation and commitment to their growth.” Among the various types of support these clients received, particular appreciation was expressed for Erste Bank’s support in developing a business plan; providing knowledge on sales, marketing and promotion; and giving ongoing advice during financial check-ins. • Pojacalo: Through weekly podcasts utilizing both audio and video content, Pojacalo was designed to address entrepreneurs and SMEs’ “huge lack of understanding of the economic issues. There is a lot of business[es] that succeeded because of the beneficial situation on the market. But [they] could not understand how stuff happened. That is the mission of Pojacalo – knowing and explaining how something happened and how to replicate this.” Through audio and video content, Pojacalo provides information on topics of interest for entrepreneurs, such as understanding VAT, addressing issues regarding E-commerce, adapting one’s business to competing in the E￾commerce eco-system, and preparing good business plans, among other topics. As one SME beneficiary noted, “[The] Pojacalo podcast is very interesting … it presents the successful stories of entrepreneurs and increases the knowledge of the problems they encounter. Also, it gives good practice examples on overcoming the issues in business … [contributing] a lot of positive impact.” During the 12 months that Pojacalo received support from CFG, they also organized network events to provide an opportunity for SME businesses to meet and share information on issues related to managing and growing their businesses, which additionally was used to introduce local food producers to the SMEs attending. • Dobar za Banku: Micro and SME businesses generally have very low levels of knowledge regarding available financing options to support business growth. Additionally, SMEs often lack knowledge of how to prepare acceptable business plans and do not fully understand the need for collateral as security for the repayment of their loans. In efforts to address these constraints, CFG, in collaboration with Peterhof Consulting, developed the Dobar za Banku platform, which means “Good for the Bank.” Dobar za Banku is an e-Learning platform that provides SMEs with step-by-step guidance in preparing loan applications for bank financing through practical, user-friendly video content and an online tool to assess a company’s bankability. Additionally, with CFG assistance, Dobar za Banku also organizes roundtable usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 32 discussions to address existing barriers for micro businesses and SMEs in Serbia. The Dobar za Banku’s e-Learning programs are seen by CFG’s beneficiaries and key stakeholders as an “important way to better support SMEs in seeking bank financing” and as improving their “financial literacy and financial management” of their businesses. To date, over 200 companies have EVALUATION QUESTION 6: Based on the experience and knowledge from this Activity, what approaches are most likely to result in gender equity improvements in the area(s) of focus? participated in Dobar za Banku’s services. As indicated by Dobar za Banku, 87% of these participating companies have demonstrated an increased knowledge of the loan application process and loan negotiations. CONCLUSIONS CFG programming was relevant, responsive, and effective in addressing the needs of Serbian SMEs; however, it is too early to determine if these initiatives will lead to sustainable SME development and growth. CFG used a targeted approach to address key challenges facing Serbian SMEs, including limited access to traditional financial services, lack of resources to engage in E-commerce, administrative barriers associated with the inspection of goods and services, and an unfriendly regulatory environment for investment. Through the introduction of alternative funding sources, including Ventu, Finspot, and the Equity Financing Program, and increasing access and literacy around E-commerce tools and solutions, CFG helped SMEs overcome financial barriers and expand their access to the capital needed to grow their business. Additionally, by working with the GoS to reform the taxation and regulatory framework for investment and to revamp the inspections system, CFG helped remove barriers to attracting outside investors and helping Serbian SMEs compete in the regional marketplace. Although these initiatives contributed to the growth of Serbian SMEs in the near term, it is too early to determine whether they will lead to long-term, sustainable outcomes. Continued challenges and barriers remain that could affect sustainability. particularly around alternative sources of financing, SMEs’ awareness of the available financial opportunities, and SMEs’ preparation for obtaining financing, CFG addressed knowledge and capacity issues of SMEs, enhancing their ability to gain access to capital and grow their businesses. While access and regulatory barriers prevent many SMEs from capitalizing on financial resources and tools to grow their business, there is also a knowledge and capacity gap, particularly among new or micro SMEs or entrepreneurs, that limits SMEs’ access to available resources. CFG helped address these non-financial barriers through its cooperation and support for a range of training and information programs, including those provided through Erste Bank, Pojacalo, Dobar za Banku, and the E-Commerce Academy, among others. Through these programs, CFG helped SMEs gain the skills and knowledge to (1) more effectively present their companies’ financial and operating information when applying for loans with banks, (2) manage their long-term business strategy and growth, and (3) contribute to the long-term expansion and growth of the Serbian economy. These non-financial support activities are critical to sustainable, long-term growth within the SME sector. FINDINGS Women entrepreneurs and SME owners in Serbia face persistent and specific challenges in establishing and growing their businesses. Despite continuous growth in the number of women entrepreneurs and SME owners, significant gender inequalities persist in access to resources, 33 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov employment, property, and land, as well as to financial markets and training programs for business development.45 Existing patriarchal norms limit women’s participation in the labor market, as well as their growth and entrepreneurship potential. These persisting gender norms and roles hamper women from attaining leadership positions or pursuing entrepreneurship opportunities, and influence them to persist in more predominant roles as caretakers and caregivers within the household.46 Additionally, businesses owned by women receive five times less in subsidies than businesses owned by men.47 This indirect discrimination results from the fact that the majority of subsidies are awarded for processing and manufacturing activities, where women are traditionally underrepresented, and exclude sole proprietorships, where women tend to be overrepresented. 48 Women are dealing with not having enough time for networking, as a major factor of not doing as good as men. Mostly because they have more responsibilities at home. – Direct Financial Beneficiary CFG did not incorporate any contractual requirement or specific objectives that addressed gender equality or social inclusion. As a result, it did not conduct a gender equality and social inclusion (GESI) analysis during the design and inception phase, nor was a Gender Action Plan or a similar strategy developed to guide the implementation. Overall, CFG did not identify, acknowledge, or target the specific needs of women-owned SMEs through the Activity. It should be emphasized that even though it was not a specific contractual requirement, USAID policies and best practices require that a gender analysis be used as a tool for examining the potentially different roles that women and men play in the target sector, their different needs, constraints, and opportunities, and the potential impact of the Activity on their lives. These USAID policies include the USAID Gender Equality and Female Empowerment Policy (GEFE) of 2012 and the Women’s Entrepreneurship and Economic Empowerment Act of 2019. Specifically, according to the GEFE Policy, a gender analysis is a “mandatory requirement” to be integrated into strategic planning, Activity design and approval, procurement processes, and measurement and evaluation.” USAID/Serbia did complete a USAID Gender Analysis, but only in 2020, to inform the Country Development Cooperation Strategy (CDCS) 2020-2025. However, even though this analysis was completed after the CFG Activity had already begun, other available analyses, such as the EU Gender Analysis conducted in 2016, could have been used during the Activity’s early stages to inform the planning and design process. Without such a gender analysis, the Activity risked missing opportunities to advance women or further exacerbating existing inequalities. Especially, in shifting its focus to target “champion” or “high-potential” SMEs without basing it on an updated gender analysis, CFG may have reinforced existing harmful gender norms and power dynamics to further exclude SMEs owned by women, minorities, and youth from directly benefiting from CFG’s programming. However, the lack of gender-disaggregated data collected by the Activity makes it difficult to draw further conclusions. Overall, the CFG Activity has not systematically established gender-sensitive indicators or baseline measurements, and targets were not disaggregated by sex, age, economic level, urban/rural or other defining characteristics that may have marginalized population groups or put them at a disadvantage. Other than Indicator 1 under Sub-Activity 1, which includes sex disaggregation 45 Banyan Global for USAID, USAID Serbia Final Gender Analysis Report, 8 May. https://pdf.usaid.gov/pdf_docs/PA00WNPZ.pdf 46 Ibid. 47 National Alliance for Local Economic Development, Gender Analysis of Economic Programmes and Financial Measures in Serbia, 2019. https://www2.unwomen.org/- /media/field%20office%20eca/attachments/publications/2019/09/reviewing%20development%20agency%20of%20serbia%20data%2 0e.pdf?la=en&vs=415 48 Ibid. usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 34 (“Person hours of USG funded technical assistance in business enabling environment provided to counterparts or stakeholders (EG5. I)”), no other data was disaggregated or specifically collected on women, youth, or marginalized groups. Nor did CFG use quotas or affirmative measures to ensure that SMEs owned and/or led by women were equally targeted or included in the Activity. Therefore, no data was provided on how many SMEs owned by women directly benefited from the Activity, nor is there information on the level of engagement of women, youth, or any minority or marginalized group. This lack of information leaves a gap in understanding the unique GESI needs and challenges facing those populations. The only GESI-related data found was extracted manually from the disaggregated results of performance indicator I 1, under sub-Activity I, as noted above. It showed that women participated in equal or slightly higher proportions (53.1%) in trainings and other capacity building endeavors organized by the Activity up until June 2021. This information did not lead to further examination or inform implementation decisions. Women-owned companies are doing slightly worse than the average and deal with some issues, such as getting access to finance. But they do show more interest in learning. – Direct Financial Beneficiary CFG did incorporate gender-sensitive analyses in two individual studies within the Activity, which revealed the need for greater gender understanding in the sector. The 2019, 2020, and 2021 annual Survey of 1,000 Businesses and the E-Commerce Study, which were produced by the Activity to shed light on the types, locations, and characteristics of SMEs, did disaggregate data by sex and provided limited insights into the different experiences of women and men in the private sector landscape. For instance, both studies included sex-disaggregated data, and the survey also included age￾disaggregated data. The 2020 annual Survey of 1,000 Businesses contains sex-disaggregated data on women-owned and women-led businesses, such as the number of women entrepreneurs and business owners in Serbia and the number of women in management in SMEs. Evidence from that data shows that one-third of the companies surveyed have an equal or majority representation of women within their management, and the ratio of women to men in management decreases as the company size increases. Further, management competencies among women are seen as low in larger companies. There is no further exploration of women’s barriers or access to opportunities or any challenges unique to women, such as access to financing. Data from CFG’s report Strengthening E-Commerce in the Republic of Serbia, provided some statistics disaggregated by both sex and age. Some relevant findings are that women used phones and participated in E-commerce at a rate equal to and even slightly higher than that of men. This is important information, particularly as it relates to informing private sector marketing strategies. However, key areas of the study failed to explore differences between men and women that would have provided more relevant insights into their habits, needs, and access to and drivers of the utilization of E-commerce. Data related to income differences, familiarity with blockchain technologies and smart contracts, or motivations for using internet commerce were not disaggregated, which left gaps in opportunities for useful learning to inform future gender-sensitive activities in entrepreneurship and SME activities. CFG beneficiaries from women-owned and men-owned SMEs have different perspectives on opportunities available to men and women. The evaluation’s online survey revealed that the majority of respondents from men-owned SMEs (53%) believe that the CFG activities and products were equally accessible to women, youth, and enterprises owned by other marginalized groups. However, only 33% of respondents representing women-owned SMEs agreed. Only half of the respondents from women-owned SMEs agreed that CFG activities were relevant in responding to the existing and emerging opportunities and constraints for women, youth, and other enterprises owned by marginalized people, while 59% of respondents from men- or jointly-owned SMEs agreed. These 35 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov differences reveal that attention to GESI factors may have better balanced the access to and benefits from the CFG Activity. These findings are particularly relevant in the context where the majority of respondents (54%) had a neutral stance towards the statement “ Women-, youth-, and other-marginalized people-owned enterprises have greater access to financial and non-financial products than they did three years ago (Exhibit 9), and only 25% of respondents from women-owned SMEs and 34% of respondents from men-owned SMEs agreed with the statement. Exhibit 9: To what extent do you agree with the following statement: “Women-, youth-, and other￾marginalized people-owned enterprises have greater access to financial and non-financial products than they did three years ago.” (n=49) 54% 30% 4% 4% 4% 2% Strongly Agreed Neutral Disagreed Strongly No Agreed Disagreed Response KII and FGD participants were largely unaware of any approaches that CFG had used to enhance gender equality in the area of focus. Insights from key informants and FGD participants did not fully support the findings of the online survey. Perhaps due to the lack of a specific focus on gender equality by CFG, it was not always clear to partners and beneficiaries how or whether CFG has helped promote greater gender equity in the SME landscape. In fact, the vast majority of all respondents were not aware that any such approaches were used by CFG. The one issue that was highlighted by several respondents was the high participation of women entrepreneurs and SME owners in events organized and supported by CFG. This included a great number of women lecturers at the E-Commerce Academy and other events organized in cooperation with AFA (“All for All”). However, almost all respondents lacked further details regarding women’s participation in the CFG Activity. CONCLUSIONS The lack of a gender analysis in the inception phase and the absence of consistently monitored sex- and age-disaggregated data have led to a lack of awareness and missed opportunities. Opportunities have been missed to address and target competitiveness, women’s access to finance, and utilization of E-commerce, among others. Without accommodating the different needs of SMEs owned by men, women, youth, and minorities, as stated by key informants and demonstrated by CFG’s gender-blind approach, CFG operated under the assumption that all had equal access to the technical and financial assistance the Activity provided. When in fact previous and subsequent gender analyses show that heightened barriers for women exist in business development, access to assets, access to education and training, and access to finance. CFG did include gender analyses and information in some of its studies (such as the E-commerce Study and the 2020 annual Survey of 1,000 Businesses), which found that women exhibit a keen interest in E-commerce and digital technologies. However, this approach was not systematic. While CFG staff did cite a few token examples of how findings from these reports led to concrete actions to better target women in their programming; findings do not demonstrate a strategic and intentional approach to promote gender￾sensitive or gender-aware programming. CFG identified some interesting findings during its implementation, which can be used and replicated in the future or can serve as a basis for the design of future actions in the same field. For instance, CFG has identified and capitalized upon the finding that women entrepreneurs and SME owners have both a need for and an interest in taking part in capacity development initiatives. This is the case particularly when it comes to digital literacy and E-commerce. By using widespread outreach usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 36 measures, CFG has ensured that a majority (53.1%) of all training and workshop participants have been women. Offering such capacity development initiatives, particularly by focusing on the specific needs of women entrepreneurs, will likely result in gender equity improvements in the long term. Additionally, the 2020 annual Survey of 1,000 Businesses found that companies with an equal or majority share of women managers (22%) are more likely and more willing to use new Fintech solutions. This indicates that future actions that focus on empowering women entrepreneurs and SMEs owned by women will not only be beneficial for women entrepreneurs and for enhancing gender equality in Serbia, but they will also serve as drivers for increasing the use of Fintech solutions in the future. This insight on gender and Fintech solutions has not, to date, been acknowledged. 37 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov RECOMMENDATIONS Based on the previous findings and conclusions, the evaluation team makes the following recommendations for USAID consideration to support future economic growth programming under DO 2, Equitable Prosperity Increased and related IR 2.1 SME Economic Growth Accelerated.49 R1 – Continue to expand and diversify alternative access to finance for micro-business, start-ups, and SMEs broadly. It is important to introduce new non-bank financing tools such as leasing, angel capital, and microlending, as well as non-financial products and business development support services, to reduce the bank-centric structure of Serbia’s financial system. This will allow SMEs to grow and will further drive Serbia’s economic growth, building a stronger foundation for private sector investment and competitiveness. R2 – Building on CFG’s success, key areas of focus for future USAID economic growth activities could include: • Maintaining a strong focus on expanding and strengthening the recent progress in E-commerce, including continuing education in digital skills for all businesses, consumers, and citizens, including women, youth, rural residents, and other disadvantaged populations, to increase Serbia’s competitiveness in Europe and in the international marketplace. • Building on the success of CFG’s work with Erste Bank’s Social Bank Unit and its Step by Step program, USAID programs could support increased dialogue and support to other banks in Serbia to create similar units within their existing structure. This would expand lending to SMEs and provide much-needed mentoring in business and financial skills to help SMEs continue to grow. • Continuing cooperation with the MoTTT to harmonize regulatory reforms with EU standards in the areas of export promotion and the digital economy. Also important is establishing additional regulatory reform cooperation with other ministries to further enhance the existing regulatory framework for SMEs, such as the MoE, using CFG’s cooperation with the MoTTT as good model. • Drawing on CFG’s accomplishments in the food retail and hospitality industry, expand the available guidance for private sector actors to increase stakeholders’ accessibility and knowledge of the inspection process, utilizing online guides, trainings, and open days, among other methods, to address existing and emerging needs within the market. R3 – Future economic growth activities should target women, youth, and other marginalized groups. USAID and Implementing Partners could consider: • Promoting additional research to expand the existing evidence on women’s willingness to use Fintech Solutions. • Ensuring an increased usage of Fintech solutions since the evidence shows that women are more willing and more likely to use Fintech. 49 Country Development Cooperation Strategy (CDCS) - Serbia, 2020-2025 (usaid.gov) usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 38 • Capitalizing on the CFG finding that women entrepreneurs and SME owners have shown both a need for and an interest in capacity development initiatives, continue with similar trainings and workshops, particularly in the areas of digital literacy and E-commerce. • Implementing activities that specifically devise economic and entrepreneurship programs targeting women, youth, and other marginalized groups. By supporting and empowering women-owned SMEs and women entrepreneurs, future actions can simultaneously ensure an increased usage of Fintech solutions because the evidence shows that women are more willing and more likely to use Fintech. R4 – Expand implementing partner monitoring and evaluation reporting requirements and promote the continued application of CLA, drawing on CFG’s approach to adaptive management, throughout all stages of the activity cycle. USAID and Implementing Partners could consider: • Facilitating theory of change workshops during the design and launch of activities to ensure the inclusion of relevant indicators to inform all steps in the decision-making process and in future evaluation and assessment activities. For example, in activities designed to improve SMEs’ access to finance, consider requiring indicators such as the number and/or types of SMEs that participated and/or benefited from an intervention as well as the level of financing generated for SMEs. • Requiring routine reporting of performance indicators as part of ongoing activity and periodic reporting, including data tables with relevant disaggregations. • Creating CLA plans and/or Learning Agendas during the design and launch of activities to address existing knowledge gaps and promote adaptive management. • Leveraging CFG’s approach, continue to conduct relevant research and in-depth analysis to address existing knowledge gaps and better understand the dynamic SME landscape to enhance ongoing and future Activity design and implementation strategies in efforts to improve the business-enabling environment and expand broad-based economic growth. • Defining a set of indicators at the Mission level to monitor changes in the SME landscape beyond the average activity period of performance to fully understand the impact of USAID’s investments to improve Serbia’s complex regulatory environment and stimulate behavioral change. R5 – To ensure local ownership and promote the continuation and sustainability of programming, adopt a systems-based approach, working in close partnership with a range of private and public sector actors. USAID and Implementing Partners could consider: • Leveraging the learning from CFG’s PSE mechanism to strategically and systematically collaborate with the right partners at the right time to leverage local expertise and knowledge, ultimately promoting local ownership and sustainability. • Working closely with key Serbian Ministries (MoF, MoTTT, MoA, Inspections) to expand opportunities for continuous learning on both the supply and demand sides of the Serbian economy. 39 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov • Drawing on public and private sector networks to expand SMEs’ knowledge and awareness of the broader economic market or supply chain. This will increase SMEs’ compliance with standards, the implementation of new technologies, and preparation for accessing traditional financial products. R6 – Pay greater attention to outreach and communication. USAID and Implementing Partners could consider organizing activities that contribute to strengthening key stakeholder’s understanding of SMEs and the challenges that they face. Additionally, USAID and Implementing Partners could consider promoting broader dissemination of activities and learning across the SME landscape. R7 – USAID should require Implementing Partners to conduct a gender analysis during the design and launch of future activities. This will ensure that gender analyses are an integral component of the design, implementation, monitoring, and evaluation of future SME programming. usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 40 ANNEX 1: EVALUATION SCOPE OF WORK COOPERATION FOR GROWTH SUMMARY OF THE PROGRAM BEING EVALUATED TITLE COOPERATION FOR GROWTH Award Type and # Contract - Cost cost-plus-fixed-fee term-type contract: 72016918C00001 Implementer CARDNO EMERGING MARKETS USA, LTD Award value $8,772,969 Life of Award January 18, 2018 – January 17, 2022 COR Aleksandar Djureinovic C.2.1 BACKGROUND The purpose of the Activity is to improve the competitiveness of the Serbian private sector, through improvements in the business enabling environment that remove regulatory constraints to private sector growth. Activity interventions will also assist in increasing the predictability and transparency of the business enabling environment, thereby creating a stronger foundation for investment and private sector growth. The Activity must achieve the purpose through two interrelated objectives: (1) Objective 1: Improve the business environment and related administrative efficiency in selected areas; (2) Objective 2: Improve SME access to finance. Detailed background information on the SOW for this activity can be found in Attachment #2 of this RFTOP (“CFG SOW”). C.2.2 PURPOSE OF THE EVALUATION USAID/Serbia requires a full, evidence-based, and independent final performance evaluation of the USAID/Serbia Cooperation for Growth Activity, and a report that provides a qualitative as well as quantitative evaluation of the progress made towards achieving expected results per the contract. Specifically, the purpose of the evaluation is to: • Examine the effectiveness of implemented project activities; • Determine whether the project has achieved the planned results; • Identify successes and gaps in performance against targets; and • Provide information on whether the project has unintended consequences. The Contractor must not simply provide an accounting of performance against targets but provide independent analysis on why targets were realized, met, or significantly surpassed. The Contractor must 41 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov also provide an analysis of the reasons behind gaps between expected and actual performance, including an identification of the likely source(s) of these gaps (design, project management, changes in operating environment, change in context, success factors, if contractor’s approach has led to intended results and goals, etc.). This evaluation will start o/a in mid-September 2021. C.2.3 EVALUATION QUESTIONS, METHODOLOGY AND DATA SOURCES EVALUATION QUESTIONS: Evaluation questions that Contractor must address are as follows: 1. Has the activity demonstrated results that are consistent with its theory of change and to what degree it has achieved its stated objectives? 2. To what extent did CFG improve the business enabling environment and related regulatory frameworks? 3. To what extent and in what ways did CFG improve SME access to finance through expanding the variety of financial products available? 4. How well did the CFG adapt to the changing operational environment, including incorporating adaptive management and continuous learning in planning and delivering its assistance? 5. From the perspective of intended beneficiaries and other stakeholders, how relevant and effective has this activity been in expanding SME growth? 6. Based on the experience and knowledge from this activity what approaches are most likely to result in gender equity improvements in the area(s) of focus? EVALUATION METHODOLOGY: In developing an appropriate methodology, the contractor must consider potential limitations and incorporate best-practice mitigation measures. The evaluation must be supported by relevant data and information gathered from meetings with beneficiaries and all the relevant stakeholders. The evaluation team should clearly demonstrate links between the data collected and the subsequent findings, conclusions, and recommendations drawn. Where applicable, responses to the questions above and resulting data must be dis-aggregated by sex and gender. AVAILABLE DATA SOURCES: The contractor will have access to all performance data, monitoring and evaluation database, surveys, and other deliverables, that have been collected and reported by the implementing partner as part of their Performance Monitoring and Evaluation plan for CFG. The contractor must also review the following documents in preparation for the CFG evaluation: • Relevant project reports, assessments, annual and life of project work plans. usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 42 • Full data set in their formats available and collected by the implementing partner. • Annual performance reports of the CFG. • USAID/Serbia 2020-2025 CDCS. In addition, the evaluation team must consult with key informants and stakeholders, including: • Meetings with USAID/Serbia. • Meetings with current/former CFG implementing partners in Serbia. • Meeting with relevant stakeholders. • Meeting with direct project beneficiaries. • Meetings with other donors, including the EU Delegation and those donors engaged in the media sector. C.3 GENERAL CONTRACTOR REQUIREMENTS – STATEMENT OF WORK The Contractor must conduct a methodologically sound evaluation which must be fully compliant with USAID’s Evaluation Policy and Automated Directives System (https://www.usaid.gov/sites/default/files/documents/1870/USAIDEvaluationPolicy.pdf) and must utilize tested and scientific evaluation methodologies. C.4 TEAM COMPOSITION The Contractor must utilize a team with the experience and skills necessary to complete this task as specified in Section F.4. C.5 SCHEDULE AND LOGISTICS The schedule (with precise due dates for deliverables) will be defined in the work plan before the start of the field work for each evaluation. Prior to commencing the field work, all team members will review background program documents to gain a firm understanding of the situation in Serbia and the activity/ies. The Contractor will be responsible for logistical arrangements for conducting the evaluation, including scheduling meetings, making copies of documents and drafts, making travel arrangements, etc. Field visit logistics will be organized and managed by the Contractor(s). This includes travel within country and in￾country transportation. USAID/Serbia will provide the team(s) with input and guidance in setting up a schedule of interviews and site visits, but the responsibility for the schedule for the evaluation resides with the Evaluation Teams. 43 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov ANNEX 2: EVALUATION METHODOLOGY Incorporating lessons learned from USAID’s experience implementing economic growth programs in Serbia, the CFG Project, implemented by Cardno, brings together local stakeholders to facilitate improvements in the business-enabling environment, with a focus on micro enterprises and SMEs to increase private sector investment and Serbia’s competitiveness in regional and global markets. To answer the EQs, IMPAQ conducted a mixed methods evaluation integrating quantitative and qualitative data collection and analysis methods, including secondary data analysis of CFG’s internal performance monitoring data and annual Survey of 1,000 Businesses, among other external data sources; an online survey; key informant interviews (KIIs); and focus group discussions (FGDs). IMPAQ used quantitative methods to analyze secondary data sources, creating descriptive statistics, as well as to identify associations and correlations for key economic development and financial sustainability indicators. Where possible, our complementary qualitative effort provided insight into, and context for, the numbers elicited, as well as insights to better understand how and why change has or has not occurred. Our mixed methods utilization-focused approach allowed IMPAQ to present a comprehensive account of CFG’s results. Specifically, the approach: (1) strengthened the reliability and validity of the findings, conclusions, and recommendations by triangulating multiple data sources; (2) broaden and deepen understanding of the mechanisms through which Project outcomes and impacts were achieved; and (3) integrated contextual factors that affect how Project tasks were implemented. This approach also enabled the team to examine the processes, outcomes, and effectiveness of activities, and progress toward the expected results and any gaps in performance, along with any unintended consequences. We south to ensure a rigorous mixed method, utilization focused evaluation by examining the agreed￾upon EQs and combining data collection methods to triangulate findings and draw conclusions in response to each question, demonstrating strong linkages among data sources. Guided by the priorities and insights identified through a Design Workshop with USAID/Serbia, we used our mixed methods approach to answer the EQs and sub-EQs and provide USAID/Serbia and implementing partners with not only evidence on the progress CFG has made towards achieving its desired results, but also wider findings on “what has worked” and “what has not” for improving the economic prospects of SMEs in Serbia, from multiple stakeholder perspectives. Furthermore, we provided USAID/Serbia with key insights into the strategies, lessons learned, and promising practices to supporting a more enabling legal, regulatory, and enabling environment for SMEs, and strengthening the capacity of key stakeholders to enhance broad-based, inclusive economic growth. EVALUATION DESIGN MATRIX To respond to the EQs, and as a best practice, IMPAQ developed a set of sub-EQs to help guide the evaluation design and analysis, to ensure the resulting recommendations align with USAID priorities and interests. These sub-EQs, presented in Exhibit 1, were refined in collaboration with USAID during the Design Workshop. The Evaluation Design Matrix presents each EQ and sub-EQ with the relevant data source. usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 44 - EVALUATION MATRIX EVALUATION QUESTION EVALUATION SUB QUESTIONS DATA SOURCES 1. Has the Project demonstrated results that are consistent with its theory of change, and to what degree has it achieved its stated objectives? 1.1 To what extent, if any, have systems and legal frameworks designed to support SME growth improved? • Document review • Secondary data review • Online SME Beneficiary and End User Survey • KIIs • FGDs 1.2 To what extent, if any, have economic and business development services been enhanced? • Document review • Secondary data review • Online SME Beneficiary and End User Survey • KIIs • FGDs 1.3 To what extent, if any, has access to finance for SMEs improved? • Document review • Secondary data review • Online SME Beneficiary and End User Survey • KIIs • FGDs 2. To what extent did CFG improve the business enabling environment and related regulatory framework? 2.1 Which business enabling activities (i.e., strengthening of e-commerce, improving tax policies and regulatory frameworks, tax compliance, inspection and law implementation, cross-border trade, exportation, etc.) were most successful? Why? • Document review • Secondary data review • Online SME Beneficiary and End User Survey • KIIs • FGDs 2.2. How and to what extent did CFG leverage resources from other donor-funded programs, the private sector, or the Government of Serbia to improve the business enabling environment and/or identify opportunities to promote sustainability of results? • Document review • Secondary data review • KIIs 3. To what extent and in what ways did CFG improve SME access to finance through expanding the variety of financial products available? 3.1 To what extent were the available or existing financial products, approaches, and mechanisms (banking products, investment funds, equity financing, crowd investing platforms, online tools, etc.) improved by CFG’s work and associated with expanded access to financing for SMEs? • Document review • Secondary data review • Online SME Beneficiary and End User Survey • KIIs • FGDs 3.2 What new innovative financial products, approaches, and mechanisms were introduced by or in collaboration with CFG and to what extent have they expanded access to financing for SMEs? • Document review • Secondary data review • KIIs • FGDs 3.3. To what extent, with what products and in what ways did CFG improve women, youth, and other marginalized person-owned SMEs access to financing? • Document review • Secondary data review • Online SME Beneficiary and End User Survey • KIIs • FGDs 45 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov - EVALUATION MATRIX EVALUATION QUESTION EVALUATION SUB QUESTIONS DATA SOURCES 3.4. How and to what extent did CFG identify and leverage resources from other donor funded programs, the private sector, or the Government of Serbia to improve SME access to financing and promote sustainability of results? • Document review • Secondary data review • KIIs 4. How well did CFG adapt to the changing operational environment including incorporating adaptive and continuous learning in planning and delivering its assistance. 4.1 What mechanisms and approaches did CFG utilize to continuously monitor Project implementation and contextual factors? And how did CFG incorporate these mechanisms into program management? • Document review • KIIs 4.2 To what extent and how were CFG’s approaches, and subsequent performance, effected by COVID–19? How effectively did the CFG adapt programming to assist SMEs given the COVID-19 context? • Document review • Online SME Beneficiary and End User Survey • KIIs • FGDs 5. From the perspective of intended beneficiaries and other stakeholders, how relevant and effective has this Project been in expanding SME growth? 5.1. How relevant were the activities in addressing the specific challenges and barriers that SMEs face to increasing revenue, promoting job creation, and expanding exports? • Document review • Secondary data review • Online SME Beneficiary and End User Survey • KIIs • FGDs 5.2. How relevant were the activities to addressing the specific challenges and barriers that SMEs, the government, and the private sector face to increasing private sector investment and competitiveness? • Document review • Secondary data review • Online SME Beneficiary and End User Survey • KIIs • FGDs 6. Based on the experience and knowledge from this Project, what approaches are most likely to result in gender equity improvements in the area(s) of focus? 6.1 How and to what extent has CFG contributed to or hindered gender equity in Serbia’s business-enabling environment, including access to both financial and non- financial services? • Document review • Secondary data review • Online SME Beneficiary and End User Survey • KIIs • FGDs 6.2 How appropriate and effective was CFG’s GESI approach in responding to the existing and emerging gender-equity opportunities and constraints? • Document review • Secondary data review • KIIs • FGDs usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 46 DATA COLLECTION TOOLS AND INSTRUMENTS To answer the EQs and sub-EQs, the evaluation team used a mixed-method approach integrating qualitative and quantitative approaches to data collection, analysis, and interpretation. GENDER EQUALITY AND SOCIAL INCLUSION CONSIDERATION: Given IMPAQ’s commitment to integrating gender equality and social inclusion (GESI) considerations across all its work, our data collection instruments critically examined CFG’s approach through a GESI lens. Reflecting GESI best practices, our approach reflected the key concerns of women, youth, and other marginalized groups – not only by assessing the root drivers that inhibit their access to, or ability to benefit, from CFG activities; but also, through careful attention to how CFG interventions protected against unintended consequences and incorporated mechanisms that protect these vulnerable groups within the Project’s given scope of work. In addition to addressing the EQs, we assessed the extent to which CFG implementation reflects gender-aware programming, meaningfully engaging women, youth, and other marginalized groups, not only in service delivery and sensitization strategies but also in decision-making processes – thus promoting the long-term sustainability of a more equitable Serbia. Across all EQs and sub-EQs, we disaggregated data by sex and marginalized populations including age, disability status, ethnicity, and sexual orientation, when appropriate, and analyzed findings to understand the constraints and opportunities affecting men, women, and marginalized beneficiaries. Additionally, we conducted all activities within a GESI-sensitive approach, such as scheduling KIIs or FGDs that are sensitive to different groups’ availability and ensuring representation among respondents. The following data collection tools and instruments were used: DESK REVIEW: The evaluation team conducted a thorough desk review of background information and Project documentation to gain an in-depth understanding of CFG activities and to assess the availability and quality of secondary data for answering the EQs. As part of this process, the evaluation team mapped the main documents and key partners and stakeholders to ensure that all the crucial elements were considered and analyzed. The following activities were implemented as part of the desk review: • Reviewed the CFG scope of work, annual workplans, and Monitoring, Evaluation, and Learning (MEL) plan, with focus on the intervention logic and the results framework including indicators and benchmarks (baselines, targets, and sources of verification); • Analyzed CFG quarterly progress and qualitative and quantitative monitoring reports; • Analyzed CFG’s annual Survey of 1,000 Businesses, including the survey’s methodology, data collection instruments, and subsequent datasets and reports for 2019 and 2020; • Analyzed CFG’s 2020 Overcome Crisis Together Survey, including the survey’s methodology, data collection instruments, and subsequent datasets and reports; • Reviewed and analyzed other reports that the CFG team produced and provided, including: - Strengthening E-commerce in Republic of Serbia, Final Report; - Proposed Measures that will Foster Alternative Investment Funds Development; 47 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov - Analysis of obstacles for implementing crowdlending in Serbia; - Market and Social Impact of Crowdlending in Serbia; - Recommendations on and implementation plan to increase a feasible collection of accounts receivable; - Recommendations for Regulators, Intermediaries and Other Stakeholders Involved on Measures for Implementing Crowdlending in Serbia; and, - The Effects of Foreign Exchange Liberalization. • Reviewed and analyzed USAID/Serbia’s Final Gender Analysis Report; • Analyzed products (reports and analysis) of other organizations and development partners in the area of SME development, including other USAID-funded economic growth development programs in the region, and identify possible best practices or lessons learned that could apply to future USAID/Serbia SME programming or policy; • Reviewed national policies and strategies around business environment, related administrative efficiency in selected areas and SME access to finance, including: - Law on Trade50; - Law on Alternative Investment Funds51; - Law on Digital Assets52; - Inspection Oversight Law53; - Law on Beer54; - Law on Food Safety55; - Law on Foreign Exchange Operations56; - Digital Skills Strategy for 2020-202457; - Behaviour Change Strategy; and, - e-Commerce Action Plan and Program for Development of e-Commerce. 50 “Official Gazette of the Republic of Serbia” No. 52/2019 51 “Official Gazette of the Republic of Serbia” No. 73/2019 52 “Official Gazette of the Republic of Serbia” No. 153/2020 53 “Official Gazette of the Republic of Serbia” No. 36/2015, 44/2018-other law, 95/2018 54 “Official Gazette of the Republic of Serbia” No. 30/2010 55 “Official Gazette of the Republic of Serbia” No. 41/2009 and 17/2019 56 “Official Gazette of the Republic of Serbia” No. 62/2006, 31/2011, 119/2012, 139/2014 and 30/2018 57 “Official Gazette of the Republic of Serbia” No. 21/2020 usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 48 • Analyzed international practices and the international obligations of Serbia, especially considering the EU accession process for the country. SECONDARY DATA ANALYSIS OF MEL AND EXTERNAL DATA: IMPAQ reviewed the available CFG Project performance monitoring data as a starting point for assessing Project progress towards achieving the stated objectives and targets, including the extent to which gender related variables were tracked, where applicable. Using the provided reporting, the evaluation team provided an independent analysis on which targets were met or significantly surpassed and begin identifying reasons behind any performance shortfalls. Reasons for such shortfalls were further explored through qualitative data collection. Where possible, IMPAQ reviewed ad-hoc reports conducted by the CFG Project as part of the desk review to help inform if and how the Project adjusted to the changing environment, including identified challenges and opportunities. Lastly, to ensure the evaluation findings are situated within the larger SME context in Serbia and the Europe and Eurasia region, IMPAQ reviewed data from independent indices and surveys that measure changes in the business enabling environment. ONLINE SME BENEFICIARY AND END USER SURVEY: IMPAQ administered a short online survey to identify key areas of change (EQ 1, 2, and 3), perceptions around the Project’s responsiveness and relevance to SME beneficiary and end users needs and challenges they face in the economy (EQ 2, 3 and 5), if and how the Project helped SMEs adapt to the changing context and business environment (EQ 4), and whether CFG contributed to improvements in gender equity (EQ 6). The survey was administered to a sample of 798 of the Project’s direct SME beneficiaries and end users including the SMEs who participated in the eCommerce Academy, e-Commerce webinars, Open days meet-ups, and Pojacalo meet-ups as well as the SMEs registered as users of the #ersteznali educational platform, Ventu.rs - Serbia’s Crowdinvesting Platform, and Finspot.rs - Invoice Financing (e-factoring) Platform or who received support through the Step by Step Erste program. In order to address EQs 1, 2, and 3 the online survey included a selection of questions to understand how direct beneficiaries perceive improvements in the legal and regulatory environment, available business development services, and ease of accessing financing. In order to provide clear distinction between the pool of SME beneficiaries and end user groups, the online survey contained questions related to involvement of respondents in activities organized by CFG, as well as the use of products developed within the CFG Project. This allowed the evaluation team to collect insights on SME and end user’s perceptions on the relevance and effectiveness of CFG’s various interventions (EQs 2 and 5). Moreover, these questions contained information about organized activities – such as joint SME-bank training courses, events organized through Pojacalo podcasts, including meetups, etc. and developed products including – Platform, etc. In efforts to address EQ 4, the online survey also included a series of questions to understand how CFG helped or hindered SMEs and end user’s abilities to adapt to environmental challenges and opportunities, such as COVID-19 or misalignment/delays in national priorities related to SME growth. Finally, with respect to EQ 6, the survey collected data to distinguish women, youth or other marginalized-persons owned enterprises from male-owned enterprises to understand how equitable the observed changes are, if any. Please refer to Annex 3 for the Online SME Beneficiary and End User Survey. Ventu.rs, Finspot.rs KIIS: IMPAQ developed semi-structured KII guides (refer to Annex 3) tailored to each of the key stakeholder groups including US Government, the CFG Project, Government of Serbia, financial institutions and business development services, business associations and civil society organizations, 49 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov direct financial beneficiaries, and other donor-supported business reform and economic growth programs and/or donors in Serbia. The KIIs were used to delve deeper into understanding stakeholder perceptions on the success of CFG in achieving its overall expected results and lessons learned on what advanced successes as well as the constraints which may have impeded progress. The KIIs also assessed: (1) CFG’s approaches to understand effectiveness; (2) the extent to which the activities addressed the challenges within Serbia’s SME sector; (3) how well the Project adapted to the changing context, including the impact of COVID-19; and (4) how well the Project influenced the use of new approaches to enhancing SME growth. IMPAQ ensured these interviews explored the extent to which CFG facilitated dialogue between local stakeholders and focused on the root causes of the constraints facing the business environment as mapped by the CFG Project as well as those identified during the evaluation desk review and in consultation with informants. Finally, the KIIs elucidated promising approaches to address gender barriers within the economic growth sector. KIIs followed a semi￾structured approach to ensure comparability while allowing for follow up and probing on topics of interest. Protocols will be adapted as relevant to each stakeholder. FGDs: IMPAQ has developed tailored FGD guides (refer to Annex 3). FGDs were conducted with CFG direct SME beneficiaries and end users to better understand and validate how changes occurred in the business environment, what factors facilitated or hindered change, and whether the changes are sustainable. IMPAQ led discussions to unpack to what extent the Project (1) facilitated dialogue among all stakeholders engaged in, and affected by, reforms to the business environment; (2) perceived relevance of the regulatory reforms undertaken by CFG; and (3) addressed the formal and informal norms and incentives that affect the way SMEs engage in the economy. IMPAQ also led discussions to unpack to what extent the Project facilitated system improvements and legal reform, and what approaches beneficiaries felt were most successful in promoting change. To assess whether the Project addressed challenges and leveraged opportunities to promote the diversification of financial products, IMPAQ inquired about approaches or strategies CFG used to ease the accessing of finance, as well as the perceived efficacy of the different financial and non-financial products supported by the Project – with a particular focus on Ventu.rs, Finspot.rs, the E-commerce Academy, Pojacalo, and the Step-by-Step Erste program. IMPAQ also explored if and how the Project has adapted to the changing context, including whether the Project helped beneficiaries adapt to the COVID-19 context. During all FGDs, IMPAQ facilitated a discussion around specific barriers women, youth, and other marginalized groups face in the business environment, to what extent the Project helped address those barriers, and what approaches, if any, show promise for improving gender equality. FGDs followed a semi-structured approach to ensure comparability while allowing for follow up and probing on topics of interest. Protocols were adapted as relevant to each beneficiary and/or end user group. SAMPLING PLAN Based on a review of the CFG Project documents and in consultation with USAID and CFG staff during the Design Workshop and Kick-Off Meeting, the evaluation team determined an appropriate sample size and sampling strategy to identify respondents for the online SME Survey, KIIs, and FGDs. IMPAQ employed a purposive sampling approach to collect targeted information from specific stakeholders, to provide insights and perspectives that can be used to identify useful findings and develop actionable recommendations. IMPAQ used a purposive sampling approach to ensure that only necessary and usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 50 relevant data is collected and to maximize field data collection efforts. In addition to purposive sampling, the evaluation team employed a snowball sampling approach to enrich the KII sample frame. ONLINE SME BENEFICARY AND END USER SURVEY: IMPAQ administered a short online survey to a pool of 798 of CFG’s direct SME beneficiaries and end users utilizing a list provided by CFG of SMEs who participated in CFG’s E-commerce Academy, E-commerce webinars, Open days meet-ups, and Pojacalo meet-ups as well as the SMEs registered as users of the #ersteznali educational platform, Ventu - Serbia’s Crowdinvesting Platform, and Finspot - Invoice Financing (e-factoring) Platform. 55 individuals (representing 15 female-owned SMEs, 34 male-owned SMEs, 5 jointly-owned SMEs, and 1 nondisclosed – of whom, 10 were youth-owned) completed the survey, for a response rate of seven percent. Survey respondents included beneficiaries from Pojacalo (29), Finspot (7), eCommerce Webinars (6), Open Days (6), Ventu (3), eCommerce Academy (3), and #erstenzali (3).58 KEY INFORMANT INTERVIEWS: IMPAQ conducted interviews with 37 individuals (18 female and 19 male) including relevant representatives from 1) USAID, 2) the CFG Project, 3) the GoS, 4) financial institution and business development support providers, 5) business associations and civil society organizations, 6) direct financial beneficiaries, and 6) other donors. Each KII followed an interview protocol adjusted for different respondent types and were conducted in English, or Serbian, depending on the key informant’s preference. The following table provides a list of the respondents who participated in the evaluation. KII RESPONDENTS STAKEHOLDER TYPE RESPONDENTS QUANTITY USAID/Serbia and Implementing Partners • USAID/Serbia CFG Contracting Officer’s Representative (COR) • USAID/Serbia Office of Economic Growth and Trade Director • US Embassy Economic Section (2) • USAID/Serbia CFG Implementing Partner Staff (5) including Chief of Party, Deputy Chief of Party, Access to Finance Director, Business Environment Expert, and MEL Specialist 9 National Government of Serbia • Ministry of Finance and Tax Administration (3) • Ministry of Trade, Tourism and Telecommunications (2) • Ministry of Agriculture (2) • Republic Secretariate for Public Policies 8 Business Associations and Civil Society Organizations • Serbian Chamber of Commerce • AmCham • Serbian Fiscal Association • Serbian Association of Manager • Digital Serbia Initiative • National Alliance for Local Economic Development • Foreign Investors Council 7 58 Respondents were asked to select all CFG interventions which they participated in. 51 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov KII RESPONDENTS STAKEHOLDER TYPE RESPONDENTS QUANTITY Direct Financial Beneficiaries • Belgrade University of Belgrade, Faculty of Economics • Dobar za Banku • eCommerce Association Serbia (2) • Erste Bank • Finspot • ICT Hub • Partners for Democratic Change • Pojacalo • Ventu • Valor • WM Equity Partners 12 Other donors • European Union, Improving Business Environment Project 1 TOTAL 37 FOCUS GROUP DISCUSSIONS: IMPAQ has facilitated a total of 6 FGDs with a total of 16 participating (7 female and 9 male) each representing the different groups of SME beneficiaries and end users the CFG Project has engaged, including micro, SMEs, and women, youth, and other marginalized person-owned SMEs. The evaluation team would like to note, that due to the sensitive nature of sharing SMEs access to financing activities, several FGD participants preferred to provide their feedback through individual in-depth interviews rather than in a group setting. Separating FGDs by beneficiary and end user type allowed IMPAQ to delve deeper into if and how the CFG Project provided relevant and targeted support to each group and the perceived effectiveness of the different types of interventions. In collaboration with CFG’s implementing partner, the evaluation team identified a pool of over 500 SMEs and end users benefitting from Ventu, e-Commerce Academy, Pojacalo, and the Step-by-Step Erste program. Unfortunately, the evaluation team did not receive a list of Finspot beneficiaries. To avoid selection bias, the evaluation team sought to randomly select focus group participants, independent of USAID and implementing partners. However, SME beneficiaries and end users expressed a general lack of willingness to discuss their engagement with CFG’s activities as most were not aware of USAID’s investment nor of CFG’s involvement. In efforts to mitigate this challenge, the evaluation team worked directly with CFG’s direct financial beneficiaries or intermediaries to identify and schedule focus groups with their participating SMEs. While this effort was largely successful, the evaluation team was unable to organize the full number of FGDs as planned and were not able to engage Finspot’s end users. IMPAQ sought to ensure gender parity within the FGDs to the extent possible, to better understand how women and members of the LGBTQI community are affected, positively or negatively, by CFG’s interventions. Given COVID-19 restrictions and to elicit richer information during the discussions, FGDs were conducted remotely and with no more than four participants per group to allow for in￾depth discussion in a remote format. To avoid domination by the more influential and eloquent participants, the evaluation team limited the discussion time for each FGD question and ensured all participants have a chance to answer the questions. usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 52 FGD RESPONDENTS STAKEHOLDER TYPE QUANTITY Ventu.rs End Users 1 FGs comprised of 1 Respondents e-Commerce Academy 2 FGs comprised of 4 Respondents Pojacalo 1 FGs comprised of 1 Respondents Step-by-Step Erste Program 2 FGs comprised of 10 Respondents TOTAL 6 FGDs with 16 Respondents DATA ANALYSIS IMPAQ took an iterative approach to data analysis, incorporating and analyzing different data sources as the evaluation progresses, ensuring each phase of data collection is capturing the most relevant and appropriate information to answer the EQs. Data collected across each method was analyzed to understand the effects universally across the SME landscape and then by, men, women, youth, and other marginalized groups independently to explore changes in gender equity. DESK REVIEW AND SECONDARY DATA ANALYSIS: Data analysis began during the desk review as the team identified and documented key progress made towards reaching the Project’s objectives and targets. A review of MEL data revealed which indicators the Project has made progress on, as well as those where the Project may have fallen behind. A review of annual workplans and quarterly reports helped identify if and how the Project adapted to the changing context on the ground, including changes related to COVID-19. Additionally, the evaluation team analyzed the annual Survey of 1,000 Businesses and Overcome Crisis Together Survey data conducted by CFG in 2019 and 2020. Data from 2021 was not available to the evaluation team. The analysis of this secondary data assisted the team in identifying changes in SMEs’ perceptions of the business regulatory environment (its stability and predictability), availability of financing options and compare their perceptions of the efficacy of traditional and non-traditional financing products, and ability to adapt to changes in the operating environment. Initial data analysis was used to inform development of the Online SME Beneficiary and End User Survey, and KII and FGD guides. ONLINE SME BENEFICIARY AND END USER SURVEY: Using an online survey platform with integrated data analytics, IMPAQ analyzed the survey data in real time, as responses are received. IMPAQ used the survey data to identify key areas of change, perceptions around the Project’s responsiveness to beneficiary needs and challenges they face in the economy, and if and how the Project helped SMEs adapt to the changing context and business environment. Initial analysis of the online survey was used to further refine the KII and FGD guides and ensure targeted qualitative data collection, delving deeper into key themes identified, as well as potential reasons for performance shortfalls. KII AND FGD ANALYSIS: The qualitative data collected through the KIIs and FGDs was used to delve deeper into understanding how changes within the SME landscape occurred. With the informed consent 53 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov of the interviewees, KIIs and FGDs were audio-recorded for accuracy. For each KII and FGD, the team took notes using the semi-structured protocol to capture key inputs and noteworthy statements. Following transcription, the team conducted content analysis to summarize findings and analyze them for common themes based on the EQs and sub EQs using qualitative analysis software, NVIVO. The team’s preliminary analysis was incorporated into the validation session to draw out final findings and conclusions with the evaluation’s COR and alternative COR. Following the validation session, IMPAQ conducted additional data analysis on areas identified by stakeholders to round out the analysis and finalize the evaluation findings and recommendations. Following this additional data analysis by the evaluation team, the team held the Out-Briefing with USAID/Serbia. The Out-Briefing session provided USAID an opportunity to provide input to help the evaluation team to refine and prioritize its recommendations, ensuring they are useful for informing future economic growth programming and policy. usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 54 ____ ____ ____ ____ ANNEX 3: DATA COLLECTION TOOLS CFG EVALUATION ONLINE SME BENEFICIARY AND END USER SURVEY IMPAQ is conducting a final evaluation of the USAID’s Cooperation for Growth Project or CFG. The CFG Project seeks to improve Serbia’s private sector competitiveness through strengthening the business enabling environment and addressing regulatory constraints to private sector growth, ultimately to improve conditions for broad-based, inclusive economic growth. Within this context, the CFG Project is designed to engage SMEs and other stakeholders in open and effective dialogue to improve the business enabling environment, thereby increasing overall confidence in the private sector and building the foundation for increased private sector competitiveness and investment. Specifically, the CFG Project aims to generate (1) improved business environment and related administrative efficiency in selected areas and (2) improved SME access to finance. The Project, which is being implemented by Cardno Emerging Markets, was launched in January 2018, and is scheduled to end in January 2022. The purpose of the evaluation is to identify the extent to which CFG achieved its main objectives, identify key achievements, its strategies that have worked well, as well as any challenges and lessons learned. The CFG evaluation team requests your participation in the evaluation, by completing the following survey. Participation is voluntary; also, you may decline to respond to any of the survey questions. The survey itself does not require the disclosure of any identifiable information. Furthermore, the evaluation team will make every effort to protect the anonymity and confidentiality of your responses. The evaluation team hopes that you will feel comfortable providing honest feedback on your experiences and points of view. NO. QUESTION ANSWER KEY 1 Are you registered as a business organization or as an entrepreneur? 1. Business Organization 2. Entrepreneur 2 What is the total number of managers/owners in your business? (Total Male, Female) 1. Total Male 2. Total Female 3 Is any of the owners in your business under the age of 29? 1. Yes 2. No 3. Refusal 4 What is the total number of employees in your business? (Total Male, Female) 1. Total Male 2. Total Female 5 What type of products or services does your business provide? 1. Production of food and beverages 2. Transportation of goods 3. Transportation of people 4. Tourist services and accommodation 5. Health and educational institutions 6. Catering 7. Food and beverage trade 8. Trade all other non-food products 9. IT services 10. Creative products / services 11. Professional and professional services 12. Clothing and footwear 13. Chemical and pharmaceutical products 55 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov __________ NO. QUESTION ANSWER KEY 14. Other non-existential products intended for the population 15. Other industrial products intended for industry or construction 16. Other:__________ What? 17. Refusal 6 What CFG interventions have you participated in? [Select all that apply.] 1. Ventru.rs 2. Finspot.rs 3. e-Commerce Academy 4. e-Commerce webinars 5. Open Days Meetups 6. Pojacalo Meet-ups 7. #erstenzali Educational Platform 8. Step by Step Erste Program 9. Other, please specify 10. Refusal 7 How frequently did you engage in CFG interventions? 1. Regularly 2. Often 3. Sometimes 4. Rarely 5. Never 6. Refusal 8 Based on your level of engagement in CFG activities as noted in question 6, to what extent do you agree with the following statement: “CFG provided relevant and effective products and support to enterprises like mine to promote the growth of small and medium enterprises.” 1. Strong agree 2. Agree 3. Neutral 4. Disagree 5. Strongly disagree 6. Refusal 9 To what extent do you agree with the following statement: “Since 2018 the systems and legal frameworks in Serbia to support SME growth have improved.” 1. Strong agree 2. Agree 3. Neutral 4. Disagree 5. Strongly disagree 6. Refusal 10 To what extent do you agree with the following statement: “I am well informed about the legal and regulatory frameworks designed to support the growth of my enterprise.” 1. Strong agree 2. Agree 3. Neutral 4. Disagree 5. Strongly disagree 6. Refusal 11 To what extent do you agree with the following statement: “Since 2018 the available economic and business development services in Serbia have been enhanced and expanded to better align with SMEs needs.” 1. Strong agree 2. Agree 3. Neutral 4. Disagree 5. Strongly disagree 6. Refusal 12 To what extent do you agree with the following statement: “Since 2018 access to finance for SMEs has improved.” 1. Strong agree 2. Agree 3. Neutral 4. Disagree 5. Strongly disagree 6. Refusal 13 To what extent do you agree with the following statement: “Since 2018 the available financial and non￾financial products for SMEs has expanded and diversified.” 1. Strong agree 2. Agree 3. Neutral 4. Disagree 5. Strongly disagree 6. Refusal usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 56 NO. QUESTION ANSWER KEY 14 To what extent do you agree with the following statement: “I better understand the benefits and value of eCommerce today than I did three years ago.” 1. Strong agree 2. Agree 3. Neutral 4. Disagree 5. Strongly disagree 6. Refusal 15 Does your enterprise operate an online store? 1. Yes 2. No 3. Refusal 15a If yes, to what extent was CFG supportive in navigating the legal, technical, logistical, delivery, and digital aspects of operating an online store? 1. Extremely influential 2. Very influential 3. Somewhat influential 4. Slightly influential 5. Not at all influential 6. Refusal 15b If no, are you more willing to consider operating an online store today then you were three years ago? 1. Very willing 2. Somewhat willing 3. Not willing 16 To what extent do you agree with the following statement: “CFG has improved and expanded the financial products available to my enterprise.” 1. Strong agree 2. Agree 3. Neutral 4. Disagree 5. Strongly disagree 6. Refusal 16a If strongly agree or agree, please briefly describe how CFG supported your enterprises access to financing. Short answer 16b If disagree or strongly disagree, please briefly describe how CFG could have better supported your enterprises access to financing. Short Answer 17 Has your enterprise sought external financing in the past three years? 1. Yes 2. No 3. Refusal 17a If yes, was your enterprise successfully in accessing external financing? 1. Yes 2. No 3. Refusal 17b If yes, where did your enterprise receive the external financing from? Short answer 17c If no to question 17, why was your enterprise unsuccessful in obtaining external financing? Short answer 18 To what extent do you agree with the following statement: “CFG has improved and expanded the non￾financial products available to my enterprise.” 1. Strong agree 2. Agree 3. Neutral 4. Disagree 5. Strongly disagree 6. Refusal 18a If strongly agree or agree, please briefly describe how CFG supported your enterprises through the access to non-financial products. Short answer 18b If disagree or strongly disagree, please briefly describe how CFG could have better supported your enterprises access to financing. Short Answer 20 To what extent do you agree with the following statement: “CFG assisted enterprises like mine to adapt to the COVID-19 context?” 1. Strong agree 2. Agree 3. Neutral 4. Disagree 5. Strongly disagree 6. Refusal 21 To what extent do you agree with the following statement: “CFG’s interventions were relevant in 1. Strong agree 2. Agree 57 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov NO. QUESTION ANSWER KEY addressing the specific challenges and barriers to increasing revenue, promoting job creation, and expanding exports for enterprises like mine.” 3. Neutral 4. Disagree 5. Strongly disagree 6. Refusal 22 To what extent do you agree with the following statement: “CFG’s interventions were relevant in addressing the specific challenges and barriers to increasing private sector investment and competitiveness for enterprises like mine.” 1. Strong agree 2. Agree 3. Neutral 4. Disagree 5. Strongly disagree 6. Refusal 23 To what extent do you agree with the following statement: “CFG activities were relevant in responding to the existing and emerging opportunities and constraints for women, youth, and other-marginalized people owned enterprises.” 1. Strong agree 2. Agree 3. Neutral 4. Disagree 5. Strongly disagree 6. Refusal 24 To what extent do you agree with the following statement: “CFG activities and products were equally accessible to women, youth, and other-marginalized people owned enterprises?” 1. Strong agree 2. Agree 3. Neutral 4. Disagree 5. Strongly disagree 6. Refusal 25 To what extent do you agree with the following statement: “Women, youth, and other-marginalized people owned enterprises are greater access to financial and non-financial products then they did three years ago.” 1. Strong agree 2. Agree 3. Neutral 4. Disagree 5. Strongly disagree 6. Refusal usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 58 CFG EVALUATION KEY INFORMANT INTERVIEW GUIDE USAID/SERBIA AND USAID CFG PROJECT IMPLEMENTING PARTNER STAFF Thank you for taking the time to talk to me. My name is [INSERT NAME]. I am from IMPAQ International, a U.S.-based international policy research, technical assistance, and evaluation company. IMPAQ is conducting a final evaluation of the USAID’s Cooperation for Growth Project or CFG. The CFG Project seeks to improve Serbia’s private sector competitiveness through strengthening the business enabling environment and addressing regulatory constraints to private sector growth, ultimately to improve conditions for broad-based, inclusive economic growth. Within this context, the CFG Project is designed to engage SMEs and other stakeholders in open and effective dialogue to improve the business enabling environment, thereby increasing overall confidence in the private sector and building the foundation for increased private sector competitiveness and investment. Specifically, the CFG Project aims to generate (1) improved business environment and related administrative efficiency in selected areas and (2) improved SME access to finance. The Project, which is being implemented by Cardno Emerging Markets, was launched in January 2018, and is scheduled to end in January 2022. The purpose of the evaluation is to identify the extent to which CFG achieved its main objectives, identify key achievements, its strategies that have worked well, as well as any challenges and lessons learned. I would like to ask you questions related to your experience with the Project. This interview will last approximately one hour (1h). With your permission, I would like to audio record this session for report writing and analysis purposes only. The recordings will be destroyed once we complete our analysis. Is this okay with you? □ Yes □ No In the evaluation report, your name will not be mentioned except in an annex in a list of people consulted and this list will only be shared with the donor (USAID) The evaluation team will make every effort to protect the anonymity and confidentiality of our discussion and hope you will feel comfortable providing honest feedback on your experiences and points of view. Participation is voluntary; also, you may decline to respond to our questions or end the interview at any time. Do you have any questions? Can we start now? INTERVIEW INFORMATION Date Interviewer Name Primary Notetaker Name RESPONDENT INFORMATION Respondent Name Respondent Title 59 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov Respondent Organization Sex of Respondent □ Male □ Female □ Other □ Refuse 1. In what ways, if any, did the CFG Project contribute to the development or improvement of commercial laws and regulations to promote an improved business enabling environment for SMEs? Probe for specific examples. (EQ1, Sub EQ 1.1, EQ 2) 2. In what ways, if any, did the CFG Project contribute to strengthening risk assessment, tax procedures, and the e-commerce ecosystem? Probe for specific examples. (EQ1, Sub EQ 1.1, EQ 2) 3. Which CFG strategies were particularly effective or ineffective in promoting greater alignment with European Union standards? (Probe for tax policies, growth-driving SME regulations, adoption of new legislations to enable e-Commerce, Adoption of eCommerce Development Program, launch of the eCommerce Association of Serbia, etc.) (EQ 2, Sub EQs 2.1 and 2.2) Which CFG strategies were particularly effective or ineffective in enhancing existing economic and business development services? (Probe for inspection and audit systems, food and hospitality ecosystem, E-Commerce Action Plan, eCommerce Academy for businesses, online education events and trainings) (EQ 2, Sub EQs 2.1 and 2.2) ­ Describe CFG’s process for designing these strategies. To what extent did CFG engage or collaborate with key stakeholders, including end users and beneficiaries, during the design process? If so, how and among which stakeholders? ­ Describe CFG’s process for implementing and monitoring the outcomes of these strategies. To what extent did CFG engage with key stakeholders, including end users and beneficiaries, to reflect on implementation and performance? If so, how and among which stakeholders? ­ How did CFG address feedback from key stakeholders, including end users and beneficiaries, as well as changes in the broader environment to alter or change strategies and activity implementation for greater impact. Please provide a specific example. 4. In what ways, if any, did the CFG Project contribute to the improvement of available financial and non-financial products, approaches, or mechanisms for SMEs? Probe for specific examples. (EQ1, Sub EQ 1.2, EQ 3, Sub EQ 3.1) 5. In what ways, if any, did the CFG Project introduce or collaborate on new innovative financial products, approaches, and mechanisms? Probe for specific examples. (EQ1, Sub EQ 1.2, EQ 3, Sub EQ 3.2) 6. To what extent and in what ways did CFG improve women, youth, and other marginalized person owned SMEs access to financing? What financial products, approaches, and mechanisms were strengthened or introduced by CFG to support women, youth, and other marginalized person owned SMEs? What non-financial products, approaches, and mechanisms were strengthened or usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 60 introduced by CFG to support women, youth, and other marginalized person owned SMEs? (EQ 3, Sub EQ 3.3, EQ 6, Sub EQ 6.1) 7. Which CFG strategies were particularly effective or ineffective in supporting the expansion and diversification of financial and non-financial products for SMEs? (Probe for e-Invoicing, banking products and services, other financial markets, foreign financing and investment, etc.) (EQ 3, Sub EQs 3.3 and 3.4) ­ Describe CFG’s process for designing these strategies. To what extent did CFG engage or collaborate with key stakeholders, including end users and beneficiaries, during the design process? If so, how and among which stakeholders? ­ Describe CFG’s process for implementing and monitoring the outcomes of these strategies. To what extent did CFG engage with key stakeholders, including end users and beneficiaries, to reflect on implementation and performance? If so, how and among which stakeholders? ­ How did CFG address feedback from key stakeholders, including end users and beneficiaries, as well as changes in the broader environment to alter or change strategies and activity implementation for greater impact. Please provide a specific example. 8. In what ways do you believe the evolving context have affected the Project, whether positively and negatively? (EQ 4) 9. What mechanisms and approaches did the CFG Project use to continuously monitor and adapt programming based on implementation and contextual changes? Can you please give an example of how the Project adapted programming based on monitoring data during implementation or the changing context? (EQ4, Sub EQ 4.1) 10. To what extent, if any, were these mechanisms and approaches incorporated or integrated into program management? (EQ 4, Sub EQ 4.1) 11. How has the CFG Project helped or hindered SMEs and other stakeholders within the ecosystem’s ability to adapt to challenges and opportunities that arose in the business enabling environment over the past four years? (Probe for COVID-19, changes and delays in national priorities, etc.) (EQ 4, Sub EQ 4.3) 12. What specific barriers do women face in engaging and participating in the business enabling environment? (EQ6) 13. Please describe CFG’s GESI approach to responding to the existing and emerging gender-equity opportunities and constraints. Were any other approaches considered? (EQ6) 14. How has the CFG Project helped address these barriers and promote greater gender equity among SMEs? In what ways have CFG activities affected women and other marginalized populations’ access and participation in the business enabling environment? How have the CFG activities helped to address barriers and promote greater gender equity among SMEs? What gender-equity opportunity and constraints should be considered moving forward to further SME 61 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov growth? (Probe for what interventions or activities were missed, not offered, and/or not considered) (EQ6) 15. Looking ahead, what are the key challenges or needs in harmonization with European Union standards and enhanced economic and business development services related to SME growth? What opportunities should be leveraged moving forward to further SME growth? 16. Looking ahead, what are the key challenges or needs in expanding and diversifying financial and non-financial products for SMEs? What are the key challenges or needs in expanding and diversifying financial and non-financial products for women, youth, or other marginalized population-owned SMEs? What opportunities should be leveraged moving forward to further SME growth? usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 62 CFG EVALUATION DRAFT KEY INFORMANT INTERVIEW GUIDE NATIONAL GOVERNMENT OF SERBIA The evaluation team intends to review this draft interview guide following an initial round of interview with USAID and CFG representatives and update the questions, as needed, based on the initial KIIs. Any changes to this interview guide will be sent to USAID for review prior to conducting any interviews with the identified informants. Thank you for taking the time to talk to me. My name is [INSERT NAME]. I am from IMPAQ International, a U.S.-based international policy research, technical assistance, and evaluation company. IMPAQ is conducting a final evaluation of the USAID’s Cooperation for Growth Project or CFG. The CFG Project seeks to improve Serbia’s private sector competitiveness through strengthening the business enabling environment and addressing regulatory constraints to private sector growth, ultimately to improve conditions for broad-based, inclusive economic growth. Within this context, the CFG Project is designed to engage SMEs and other stakeholders in open and effective dialogue to improve the business enabling environment, thereby increasing overall confidence in the private sector and building the foundation for increased private sector competitiveness and investment. Specifically, the CFG Project aims to generate (1) improved business environment and related administrative efficiency in selected areas and (2) improved SME access to finance. The Project, which is being implemented by Cardno Emerging Markets, was launched in January 2018, and is scheduled to end in January 2022. The purpose of the evaluation is to identify the extent to which CFG achieved its main objectives, identify key achievements, its strategies that have worked well, as well as any challenges and lessons learned. I would like to ask you questions related to your experience with the Project. This interview will last approximately one hour (1h). With your permission, I would like to audio record this session for report writing and analysis purposes only. The recordings will be destroyed once we complete our analysis. Is this okay with you? □ Yes □ No In the evaluation report, your name will not be mentioned except in an annex in a list of people consulted and this list will only be shared with the donor (USAID) The evaluation team will make every effort to protect the anonymity and confidentiality of our discussion and hope you will feel comfortable providing honest feedback on your experiences and points of view. Participation is voluntary; also, you may decline to respond to our questions or end the interview at any time. Do you have any questions? Can we start now? INTERVIEW INFORMATION Date Interviewer Name Primary Notetaker Name 63 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov RESPONDENT INFORMATION Respondent Name Respondent Title Respondent Organization Sex of Respondent Male Female Other Refuse □ □ □ □ 1. Reflecting on the past four years, since 2018, what progress has been made across the Government of Serbia in strengthening systems and legal frameworks to better support SMEs? In improving SME’s access to finance? (EQ1, Sub EQ 1.1 and 1.2) 2. In what ways, if any, did the CFG Project align, support and/or contribute to the Government of Serbia’s efforts to develop and/or improve commercial laws and regulations to promote a business enabling environment for SMEs? Probe for specific examples. (EQ1, Sub EQ 1.1, EQ 2) 3. In what ways, if any, did the CFG Project align, support and/or contribute to the Government of Serbia’s efforts to strengthening risk assessment, tax procedures, and the e-commerce ecosystem? Probe for specific examples. (EQ1, Sub EQ 1.1, EQ 2) The following questions, questions 4 – 7 are for key informants representing the National Bank of Serbia, Ministry of Finance, Ministry of Economy (if applicable). For all other National Government of Serbia informants, skip to question 10. 4. In what ways, if any, did the CFG Project contribute to the improvement of available financial and non-financial products, approaches, or mechanisms for SMEs? Probe for specific examples. (EQ1, Sub EQ 1.2, EQ 3, Sub EQ 3.1) 5. In what ways, if any, did the CFG Project align, support and/or contribute to the Government of Serbia’s efforts to introduce new innovative financial products, approaches, and mechanisms? Probe for specific examples. (EQ1, Sub EQ 1.2, EQ 3, Sub EQ 3.2) 6. From your perspective, how relevant were CFG’s activities in addressing the specific challenges and barriers that the Government of Serbia faces in promoting SME growth, particularly with regards to increasing revenue, promoting job creation, and expanding exports? (EQ 5, Sub EQ 5.1) 7. From your perspective, how relevant were CFG’s activities to addressing the specific challenges and barriers that the Government of Serbia faces to increasing private sector investment and competitiveness? (EQ 5, EQ 5.2) 8. To what extent, if any, did the CFG Project collaborate with the Government of Serbia in the design, implementation, monitoring, and adapting of the Project’s overall strategic direction, work planning activities, and activity design? Probe for specific examples. (EQ 2 and 3) usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 64 ­ How did CFG address feedback from your Ministry or other Government of Serbia Ministries, as well as changes in the broader environment to alter or change strategies and activity implementation for greater impact. Please provide a specific example. 9. How has the CFG Project helped or hindered the Government of Serbia’s ability to adapt to challenges and opportunities that arose in the business enabling environment over the past four years? (Probe for COVID-19, etc.) (EQ 4, Sub EQ 4.3) 10. In what ways have CFG activities affected women’s access and participation in the business enabling environment? In what ways have CFG activities affected marginalized-populations access and participation in the business enabling environment? How have the CFG activities helped to address barriers and promote greater gender equity among SMEs? (EQ 6, Sub EQ 6.1) 11. Looking ahead, what are the key challenges or needs in achieving harmonization with European Union standards and enhanced economic and business development services related to SME growth? What opportunities should be leveraged moving forward to further SME growth? 12. Looking ahead, what are the key challenges or needs in expanding and diversifying financial and non-financial products for SMEs? What are the key challenges or needs in expanding and diversifying financial and non-financial products for women, youth, or other marginalized population-owned SMEs? What opportunities should be leveraged moving forward to further SME growth? 65 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov CFG EVALUATION DRAFT KEY INFORMANT INTERVIEW GUIDE FINANCIAL INSTITUTIONS AND BUSINESS DEVELOPMENT SERVICES The evaluation team intends to review this draft interview guide following an initial round of interview with USAID and CFG representatives and update the questions, as needed, based on the initial KIIs. Any changes to this interview guide will be sent to USAID for review prior to conducting any interviews with the identified informants. Thank you for taking the time to talk to me. My name is [INSERT NAME]. I am from IMPAQ International, a U.S.-based international policy research, technical assistance, and evaluation company. IMPAQ is conducting a final evaluation of the USAID’s Cooperation for Growth Project or CFG. The CFG Project seeks to improve Serbia’s private sector competitiveness through strengthening the business enabling environment and addressing regulatory constraints to private sector growth, ultimately to improve conditions for broad-based, inclusive economic growth. Within this context, the CFG Project is designed to engage SMEs and other stakeholders in open and effective dialogue to improve the business enabling environment, thereby increasing overall confidence in the private sector and building the foundation for increased private sector competitiveness and investment. Specifically, the CFG Project aims to generate (1) improved business environment and related administrative efficiency in selected areas and (2) improved SME access to finance. The Project, which is being implemented by Cardno Emerging Markets, was launched in January 2018, and is scheduled to end in January 2022. The purpose of the evaluation is to identify the extent to which CFG achieved its main objectives, identify key achievements, its strategies that have worked well, as well as any challenges and lessons learned. I would like to ask you questions related to your experience with the Project. This interview will last approximately one hour (1h). With your permission, I would like to audio record this session for report writing and analysis purposes only. The recordings will be destroyed once we complete our analysis. Is this okay with you? □ Yes □ No In the evaluation report, your name will not be mentioned except in an annex in a list of people consulted and this list will only be shared with the donor (USAID) The evaluation team will make every effort to protect the anonymity and confidentiality of our discussion and hope you will feel comfortable providing honest feedback on your experiences and points of view. Participation is voluntary; also, you may decline to respond to our questions or end the interview at any time. Do you have any questions? Can we start now? INTERVIEW INFORMATION Date Interviewer Name Primary Notetaker Name usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 66 RESPONDENT INFORMATION Respondent Name Respondent Title Respondent Organization Sex of Respondent Male Female Other □ Refuse □ □ □ 1. In what ways, if any, have systems and legal frameworks within Serbia evolved over the past four years (since 2018) to support financial institutions’ abilities to promote SME growth and expand SMEs access to finance? Probe for alignment with European Union standards, economic and business development services (risk, tax compliance, and eCommerce) (EQ1, Sub EQ 1.1) 2. In what ways, if any, did the CFG Project contribute to strengthening the e-commerce ecosystem? Probe for specific examples. (EQ1, Sub EQ 1.1, EQ 2) 3. Which CFG strategies were particularly effective or ineffective in enhancing existing economic and business development services? (Probe for food and hospitality ecosystem, E-Commerce Action Plan, eCommerce Academy for businesses, online education events and trainings) (EQ 2, Sub EQs 2.1 and 2.2) 4. In what ways, if any, did the CFG Project align, support and/or contribute to your institution’s efforts to strengthen and improve the existing financial products, approaches, and mechanisms offered to SMEs in Serbia? Probe for specific examples. (EQ1, Sub EQ 1.2, EQ 3, Sub EQ 3.1) 5. In what ways, if any, did the CFG Project align, support and/or contribute to your institution’s efforts to introduce new innovative financial products, approaches, and mechanisms to expand SME financing within Serbia? Probe for specific examples. (EQ1, Sub EQ 1.2, EQ 3, Sub EQ 3.2) 6. Which CFG strategies or actions were particularly effective or ineffective in supporting the expansion and diversification of financial and non-financial products for SMEs? (Probe for e-Invoicing, banking products and services, other financial markets, foreign financing and investment, etc.) (EQ 3, Sub EQs 3.3 and 3.4) 7. From your perspective, how relevant were CFG’s activities in addressing the specific challenges and barriers that the Government of Serbia faces to promoting SME growth, particularly with regards to increasing revenue, promoting job creation, and expanding exports? (EQ 5, Sub EQ 5.1) 8. In what ways, if any, did the CFG Project align, support, and/or contribute to your institution’s efforts to improve women, youth, and other marginalized person owned SMEs access to financing? What financial products, approaches, and mechanisms does your institution currently offer to address the challenges and barriers facing women, youth, and other marginalized person owned SMEs? Did the CFG Project provide support or hinder your institution’s ability to provide these 67 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov products or services to women, youth, and other marginalized person owned SMEs? (EQ 3, Sub EQ 3.3, EQ 6) 9. To what extent, if any, did the CFG Project collaborate with your institution through the design, implementation, monitoring, and adapting of the Project’s strategies and activity design? Probe for specific examples. (EQ 2 and 3) 10. From your perspective, how relevant were CFG’s activities to addressing the specific challenges and barriers that your organization, as well as other financial institutions, face to increasing private sector investment and competitiveness? (EQ 5, EQ 5.2) 11. Looking ahead, what are the key challenges or needs in increasing harmonization with European Union standards and enhanced economic and business development services related to SME growth? What opportunities should be leveraged moving forward to further SME growth? 12. Looking ahead, what are the key challenges or needs in expanding and diversifying financial and non-financial products for SMEs? What are the key challenges or needs in expanding and diversifying financial and non-financial products for women, youth, or other marginalized population-owned SMEs? What opportunities should be leveraged moving forward to further SME growth? usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 68 CFG EVALUATION DRAFT KEY INFORMANT INTERVIEW GUIDE BUSINESS ASSOCIATIONS AND CIVIL SOCIETY ORGANIZATIONS The evaluation team intends to review this draft interview guide following an initial round of interview with USAID and CFG representatives and update the questions, as needed, based on the initial KIIs. Any changes to this interview guide will be sent to USAID for review prior to conducting any interviews with the identified informants. Thank you for taking the time to talk to me. My name is [INSERT NAME]. I am from IMPAQ International, a U.S.-based international policy research, technical assistance, and evaluation company. IMPAQ is conducting a final evaluation of the USAID’s Cooperation for Growth Project or CFG. The CFG Project seeks to improve Serbia’s private sector competitiveness through strengthening the business enabling environment and addressing regulatory constraints to private sector growth, ultimately to improve conditions for broad-based, inclusive economic growth. Within this context, the CFG Project is designed to engage SMEs and other stakeholders in open and effective dialogue to improve the business enabling environment, thereby increasing overall confidence in the private sector and building the foundation for increased private sector competitiveness and investment. Specifically, the CFG Project aims to generate (1) improved business environment and related administrative efficiency in selected areas and (2) improved SME access to finance. The Project, which is being implemented by Cardno Emerging Markets, was launched in January 2018, and is scheduled to end in January 2022. The purpose of the evaluation is to identify the extent to which CFG achieved its main objectives, identify key achievements, its strategies that have worked well, as well as any challenges and lessons learned. I would like to ask you questions related to your experience with the Project. This interview will last approximately one hour (1h). With your permission, I would like to audio record this session for report writing and analysis purposes only. The recordings will be destroyed once we complete our analysis. Is this okay with you? □ Yes □ No In the evaluation report, your name will not be mentioned except in an annex in a list of people consulted and this list will only be shared with the donor (USAID) The evaluation team will make every effort to protect the anonymity and confidentiality of our discussion and hope you will feel comfortable providing honest feedback on your experiences and points of view. Participation is voluntary; also, you may decline to respond to our questions or end the interview at any time. Do you have any questions? Can we start now? INTERVIEW INFORMATION Date Interviewer Name Primary Notetaker Name 69 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov □ □ □ □ RESPONDENT INFORMATION Respondent Name Respondent Title Respondent Organization Sex of Respondent Male Female Other Refuse 1. Please describe and give a brief history of your [company, organization, or institution] and its goals in Serbia, particularly as they relate to SME economic growth and building a durable foundation for increased private sector investment and competitiveness. 2. Please describe your level of engagement or collaboration with the Cooperation for Growth Project. Which CFG interventions did you participate in and/or review support through? (Probe specifically for e-Commerce ecosystem, Single VAT rulebook, Finspot.rs, inspections and audit systems training, inspection policy and digital platform for the food sector, financial management training, business community coalition, etc.) 3. In what ways, if any, have systems and legal frameworks evolved over the past four years (since 2018) to promote SME growth? Probe for alignment with European Union standards, economic and business development services (risk, tax compliance, and eCommerce) (EQ1, Sub EQ 1.1) 4. In what ways, if any, did the CFG Project contribute to the development or improvement of commercial laws and regulations to improve the existing business enabling environment for SMEs? Probe for specific examples. (EQ1, Sub EQ 1.1, EQ 2) 5. In what ways, if any, did the CFG Project contribute to strengthening the risk mitigation, tax compliance, and e-commerce ecosystem? Probe for specific examples. (EQ1, Sub EQ 1.1, EQ 2) 6. Which CFG strategies were particularly effective or ineffective in enhancing existing economic and business development services? (Probe for food and hospitality ecosystem, E-Commerce Action Plan, eCommerce Academy for businesses, online education events and trainings) (EQ 2, Sub EQs 2.1 and 2.2) 7. In what ways, if any, did the CFG Project align, support and/or contribute to your organization’s efforts to strengthen and improve the existing financial and non-financial products, approaches, and mechanisms offered? Probe for specific examples. (EQ1, Sub EQ 1.2, EQ 3, Sub EQ 3.1) 8. In what ways, if any, did the CFG Project align, support and/or contribute to your organization’s efforts to introduce new innovative financial and non-financial products, approaches, and mechanisms? Probe for specific examples. (EQ1, Sub EQ 1.2, EQ 3, Sub EQ 3.2) 9. Of the CFG interventions your [company, organization, or institution] participated in, were CFG’s strategies particularly effective or ineffective in supporting the expansion and diversification of financial and non-financial products for SMEs? Why or why not? (Probe for e-Invoicing, banking usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 70 products and services, other financial markets, foreign financing and investment, etc.) (EQ 3, Sub EQs 3.3 and 3.4) 10. From your perspective, how relevant were CFG’s activities in addressing the specific challenges and barriers that the Civil Society community faces to promoting SME growth, particularly with regards to increasing revenue, promoting job creation, and expanding exports? (EQ 5, Sub EQ 5.1) 11. How does your institution currently seek to address the challenges and barriers facing women, youth, and other marginalized person owned SMEs? Did the CFG Project provide support or hinder your institution’s ability to provide these products or services to women, youth, and other marginalized person owned SMEs? (EQ 3, Sub EQ 3.3, EQ 6) 12. To what extent, if any, did the CFG Project collaborate with your organization through the design, implementation, monitoring, and adapting of the Project’s strategies and activity design? How did CFG address feedback from your organization as well as changes in the broader environment to alter or change strategies and activity implementation for greater impact. Please provide a specific example. Probe for specific examples. (EQ 2 and 3) 13. How has the CFG Project helped or hindered your organization’s ability to adapt to challenges and opportunities that arose in the business enabling environment over the past four years? (Probe for COVID-19, delays in national priorities, etc.) (EQ 4, Sub EQ 4.3) 14. From your perspective, how relevant were CFG’s activities to addressing the specific challenges and barriers that your organization, among others, face to increasing private sector investment and competitiveness? (EQ 5, EQ 5.2) 15. Looking ahead, what are the key challenges or needs in harmonization with European Union standards and enhanced economic and business development services related to SME growth? What opportunities should be leveraged moving forward to further SME growth? 16. Looking ahead, what are the key challenges or needs in expanding and diversifying financial and non-financial products for SMEs? What are the key challenges or needs in expanding and diversifying financial and non-financial products for women, youth, or other marginalized population-owned SMEs? What opportunities should be leveraged moving forward to further SME growth? 71 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov CFG EVALUATION DRAFT KEY INFORMANT INTERVIEW GUIDE DIRECT FINANCIAL BENEFICARIES The evaluation team intends to review this draft interview guide following an initial round of interview with USAID and CFG representatives and update the questions, as needed, based on the initial KIIs. Any changes to this interview guide will be sent to USAID for review prior to conducting any interviews with the identified informants. Thank you for taking the time to talk to me. My name is [INSERT NAME]. I am from IMPAQ International, a U.S.-based international policy research, technical assistance, and evaluation company. IMPAQ is conducting a final evaluation of the USAID’s Cooperation for Growth Project or CFG. The CFG Project seeks to improve Serbia’s private sector competitiveness through strengthening the business enabling environment and addressing regulatory constraints to private sector growth, ultimately to improve conditions for broad-based, inclusive economic growth. Within this context, the CFG Project is designed to engage SMEs and other stakeholders in open and effective dialogue to improve the business enabling environment, thereby increasing overall confidence in the private sector and building the foundation for increased private sector competitiveness and investment. Specifically, the CFG Project aims to generate (1) improved business environment and related administrative efficiency in selected areas and (2) improved SME access to finance. The Project, which is being implemented by Cardno Emerging Markets, was launched in January 2018, and is scheduled to end in January 2022. The purpose of the evaluation is to identify the extent to which CFG achieved its main objectives, identify key achievements, its strategies that have worked well, as well as any challenges and lessons learned. I would like to ask you questions related to your experience with the Project. This interview will last approximately one hour (1h). With your permission, I would like to audio record this session for report writing and analysis purposes only. The recordings will be destroyed once we complete our analysis. Is this okay with you? □ Yes □ No In the evaluation report, your name will not be mentioned except in an annex in a list of people consulted and this list will only be shared with the donor (USAID) The evaluation team will make every effort to protect the anonymity and confidentiality of our discussion and hope you will feel comfortable providing honest feedback on your experiences and points of view. Participation is voluntary; also, you may decline to respond to our questions or end the interview at any time. Do you have any questions? Can we start now? INTERVIEW INFORMATION Date Interviewer Name Primary Notetaker Name usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 72 RESPONDENT INFORMATION Respondent Name Respondent Title Respondent Organization Sex of Respondent □ Male □ Female □ Other □ Refuse 1. Please describe and give a brief history of your [company, organization, or institution] and its goals in Serbia, particularly as they relate to SME economic growth and building a durable foundation for increased private sector investment and competitiveness. 2. Please describe your level of engagement or collaboration with the Cooperation for Growth Project. 3. In what ways, if any, have systems and legal frameworks evolved over the past four years (since 2018) to promote SME growth? Probe for alignment with European Union standards, economic and business development services (risk, tax compliance, and eCommerce) (EQ1, Sub EQ 1.1) 4. In what ways, if any, did the CFG Project contribute to enhancing existing economic and business development services? Probe for specific examples. (EQ1, Sub EQ 1.1, EQ 2) 5. From your perspective, how relevant were CFG’s activities in addressing the specific challenges and barriers that the private sector faces in promoting SME growth, particularly with regards to increasing revenue, promoting job creation, and expanding exports? (EQ 5, Sub EQ 5.1) 6. Which CFG strategies or actions were particularly effective or ineffective in supporting the expansion and diversification of financial and non-financial products for SMEs? (Probe for e￾Invoicing, banking products and services, other financial markets, foreign financing and investment, etc.) (EQ 3, Sub EQs 3.3 and 3.4) 7. To what extent, if any, did the CFG Project collaborate with your [company, organization, or institution] through the design, implementation, monitoring, and adapting of the Project’s strategies and activity design? How did CFG address feedback from your organization as well as changes in the broader environment to alter or change strategies and activity implementation for greater impact. Please provide a specific example. Probe for specific examples. (EQ2 and 3) 8. How has the CFG Project helped or hindered your [company, organization, or institution]’s ability to adapt to challenges and opportunities that arose in the business enabling environment over the past four years? (Probe for COVID-19, delays in national priorities, etc.) (EQ 4, Sub EQ 4.3) 9. From your perspective, how relevant were CFG’s activities to addressing the specific challenges and barriers that your [company, organization, or institution], among others, face to increasing private sector investment and competitiveness? (EQ 5, EQ 5.2) 73 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov 10. Looking ahead, what are the key challenges or needs in achieving greater harmonization with European Union standards and enhanced economic and business development services related to SME growth? What opportunities should be leveraged moving forward to further SME growth? 11. Looking ahead, what are the key challenges or needs in expanding and diversifying financial and non-financial products for SMEs? What are the key challenges or needs in expanding and diversifying financial and non-financial products for women, youth, or other marginalized population-owned SMEs? What opportunities should be leveraged moving forward to further SME growth? usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 74 CFG EVALUATION DRAFT KEY INFORMANT INTERVIEW GUIDE OTHER DONORS The evaluation team intends to review this draft interview guide following an initial round of interview with USAID and CFG representatives and update the questions, as needed, based on the initial KIIs. Any changes to this interview guide will be sent to USAID for review prior to conducting any interviews with the identified informants. Thank you for taking the time to talk to me. My name is [INSERT NAME]. I am from IMPAQ International, a U.S.-based international policy research, technical assistance, and evaluation company. IMPAQ is conducting a final evaluation of the USAID’s Cooperation for Growth Project or CFG. The CFG Project seeks to improve Serbia’s private sector competitiveness through strengthening the business enabling environment and addressing regulatory constraints to private sector growth, ultimately to improve conditions for broad-based, inclusive economic growth. Within this context, the CFG Project is designed to engage SMEs and other stakeholders in open and effective dialogue to improve the business enabling environment, thereby increasing overall confidence in the private sector and building the foundation for increased private sector competitiveness and investment. Specifically, the CFG Project aims to generate (1) improved business environment and related administrative efficiency in selected areas and (2) improved SME access to finance. The Project, which is being implemented by Cardno Emerging Markets, was launched in January 2018, and is scheduled to end in January 2022. The purpose of the evaluation is to identify the extent to which CFG achieved its main objectives, identify key achievements, its strategies that have worked well, as well as any challenges and lessons learned. I would like to ask you questions related to your experience with the Project. This interview will last approximately one hour (1h). With your permission, I would like to audio record this session for report writing and analysis purposes only. The recordings will be destroyed once we complete our analysis. Is this okay with you? □ Yes □ No In the evaluation report, your name will not be mentioned except in an annex in a list of people consulted and this list will only be shared with the donor (USAID) The evaluation team will make every effort to protect the anonymity and confidentiality of our discussion and hope you will feel comfortable providing honest feedback on your experiences and points of view. Participation is voluntary; also, you may decline to respond to our questions or end the interview at any time. Do you have any questions? Can we start now? INTERVIEW INFORMATION Date Interviewer Name Primary Notetaker Name 75 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov RESPONDENT INFORMATION Respondent Name Respondent Title Respondent Organization Sex of Respondent □ Male □ □ Female Other □ Refuse 1. Please describe and give a brief history of your [company, organization, or institution] and its goals in Serbia, particularly as they relate to SME economic growth and building a durable foundation for increased private sector investment and competitiveness. 2. Please describe your level of engagement or collaboration with the Cooperation for Growth Project. 3. In what ways, if any, have systems and legal frameworks evolved over the past four years (since 2018) to better support SMEs? (EQ1, Sub EQ 1.1) ­ To what extent are the legal frameworks, specifically commercial laws and regulations that are related to SME growth, better aligned with European Union or international standards? ­ To what extent, if any, have economic and business development services, which are specifically designed to address risk, tax compliance, and e-commerce, been enhanced to better support SME growth? 4. In what ways, if any, did the CFG Project contribute to the development or improvement of commercial laws and regulations to promote an improved business enabling environment for SMEs? Probe for specific examples. (EQ1, Sub EQ 1.1, EQ 2) 5. In what ways, if any, did the CFG Project contribute to strengthening risk assessment, tax procedures, and the e-commerce ecosystem? Probe for specific examples. (EQ1, Sub EQ 1.1, EQ 2) 6. In what ways, if any, have SMEs’ access to finance improved over the past four years (since 2018)? (EQ1, Sub EQ 1.2, EQ 3) ­ To what extent, have the financial products available to SMEs been expanded and diversified over the past four years (since 2018)? Probe for specific examples. 7. In what ways, if any, did the CFG Project contribute to the improvement of available financial and non-financial products, approaches, or mechanisms for SMEs? Probe for specific examples. (EQ1, Sub EQ 1.2, EQ 3, Sub EQ 3.1) 8. In what ways, if any, did the CFG Project introduce or collaborate on new innovative financial products, approaches, and mechanisms? Probe for specific examples. (EQ1, Sub EQ 1.2, EQ 3, Sub EQ 3.2) usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 76 9. To what extent, if any, did the CFG Project collaborate with the donor funded community through the design, implementation, monitoring, and adapting of the Project’s strategies and activity design? To what extent did the CFG Project communicate progress and lessons learned with the donor funded community? (EQ 2 and 3) 10. In what ways might the evolving context have affected the Project, whether positively and negatively? (EQ 4) 11. Reflecting on the past four years, since 2018, what progress has been made to address the barriers for women, youth, and other marginalized person owned SMEs access to financing? What financial products, approaches, and mechanisms were strengthened or introduced to support women, youth, and other marginalized person owned SMEs? (EQ 3, Sub EQ 3.3) What non-financial products, approaches, and mechanisms were strengthened or introduced by CFG to support women, youth, and other marginalized person owned SMEs? (EQ 6, Sub EQ 6.1) 12. From your perspective, how relevant were CFG’s activities in addressing the specific challenges and barriers that the private sector faces to promoting SME growth, particularly with regards to increasing revenue, promoting job creation, and expanding exports? (EQ 5, Sub EQ 5.1) 13. In what ways have CFG activities affected women’s access and participation in the business enabling environment? In what ways have CFG activities affected marginalized-populations access and participation in the business enabling environment? How have the CFG activities helped to address barriers and promote greater gender equity among SMEs? (EQ 6) 14. Looking ahead, what are the key challenges or needs in achieving greater harmonization with European Union standards and enhanced economic and business development services related to SME growth? What opportunities should be leveraged moving forward to further SME growth? 15. Looking ahead, what are the key challenges or needs in expanding and diversifying financial and non-financial products for SMEs? What are the key challenges or needs in expanding and diversifying financial and non-financial products for women, youth, or other marginalized population-owned SMEs? What opportunities should be leveraged moving forward to further SME growth? 77 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov CFG EVALUATION DRAFT FOCUS GROUP DISCUSSION GUIDE CFG BENEFICARIES AND END USERS The evaluation team intends to review this draft focus group discussion guide following an initial round of interview with USAID and CFG representatives and update the questions, as needed, based on the initial KIIs. Any changes to this guide will be sent to USAID for review prior to facilitating any FGDs with the identified participants. Thank you for taking the time to talk to participate in this focus group discussion. My name is [INSERT NAME] and I will be facilitating our discussion today. With me are my colleagues [INSERT NAME] and [INSERT NAME] who may also participate by asking questions and taking notes. We are from IMPAQ International, a U.S.-based international policy research, technical assistance, and evaluation company. IMPAQ is conducting a final evaluation of the USAID’s Cooperation for Growth Project or CFG. The CFG Project seeks to improve Serbia’s private sector competitiveness through strengthening the business enabling environment and addressing regulatory constraints to private sector growth, ultimately to improve conditions for broad-based, inclusive economic growth. Within this context, the CFG Project is designed to engage SMEs and other stakeholders in open and effective dialogue to improve the business enabling environment, thereby increasing overall confidence in the private sector and building the foundation for increased private sector competitiveness and investment. Specifically, the CFG Project aims to generate (1) improved business environment and related administrative efficiency in selected areas and (2) improved SME access to finance. The Project, which is being implemented by Cardno Emerging Markets, was launched in January 2018, and is scheduled to end in January 2022. The purpose of the evaluation is to identify the extent to which CFG achieved its main objectives, identify key achievements, its strategies that have worked well, as well as any challenges and lessons learned. Through this focus group discussion, we hope to learn about your experiences with CFG, including which types of CFG support were effective or ineffective, the challenges you experienced (if any), as well as the outcomes your collaboration with the Project. This discussion will last approximately two hours (2h). With your permission, we would like to audio record the session for report writing and analysis purposes only. The recordings will be destroyed once we complete our analysis. Is this okay with you? □ Yes □ No In the evaluation report, your name will not be mentioned except in an annex in a list of people consulted and this list will only be shared with the donor (USAID) The evaluation team will make every effort to protect the anonymity and confidentiality of our discussion and hope you will feel comfortable providing honest feedback on your experiences and points of view. Participation is voluntary; also, you may decline to respond to our questions or end the interview at any time. Do you have any questions? Can we start now? Informed Consent: The FGD facilitator will explain the purpose of the meeting and administer the informed consent protocol (which includes information about CFG, the purpose of evaluation, and explains that their participation is voluntary, and that information shared will be kept confidential. The usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 78 facilitator will also request that participants in the group to respect the confidentiality of their co￾participants by not discussing what was discussed with others outside the group). Introductions: We will ask each participant to introduce herself/himself and briefly how his/her institution collaborated with CFG. Sign-in Sheet: At the start of the discussion, participants will be prompted to complete anonymous questionnaire to provide their name, title, enterprise, size of enterprise, sex, age, and disability and LGBTQI status. This information will only be available to the facilitators. INTERVIEW INFORMATION Date Interviewer Name Primary Notetaker Name PARTICIPANT INFORMATION NAME TITLE ENTERPRISE NO. OF EMPLOYEES SEX AGE DISABILITY STATUS Male Female Other Refuse □ < or = 29 years of age > 29 years of age Refuse □ □ Yes □ □ No □ □ □ Refuse □ □ Male Female Other Refuse □ < or = 29 years of age > 29 years of age Refuse □ □ Yes □ □ No □ □ □ Refuse □ □ Male Female Other Refuse □ < or = 29 years of age > 29 years of age Refuse □ □ Yes □ No Refuse □ □ □ □ □ □ Male Female Other Refuse □ < or = 29 years of age > 29 years of age Refuse □ □ □ □ □ □ □ □ Yes No Refuse □ 1. To what extent and how has the CFG Project been effective in strengthening systems and legal frameworks to promote SME growth? What CFG-supported commercial laws and regulations have improved the existing business enabling environment to date and why? (Probe for e-Commerce ecosystem, tax compliance - Single VAT rulebook, inspection and audit systems, risk mitigations, etc.) Follow-up questions: ⋅ What are some success stories? What factors contributed to success? ⋅ What are examples of less successful initiatives? What factors limited results? 2. To what extent and how has the CFG Project improved SMEs access to a diverse pool of financial products? What CFG-supported approaches and/or products have been most useful and effective to date and why? In what ways, if any, could CFG assistance have been more effective? (Probe for usefulness of Ventru.rs, Finspot.rs, block chain, digital finance, etc.) 79 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov Follow-up questions: ⋅ What are examples of success stories? What factors contributed to these successes? ⋅ What are examples of less successful initiatives? What factors limited results? 3. To what extent and how has the CFG Project improved SMEs access to a diverse pool of non￾financial products or business development services? What CFG-supported approaches and/or products have been most useful and effective to date and why? In what ways, if any, could CFG assistance have been more effective? (Probe for usefulness of eCommerce Academy and webinars, Open Days Meetups, Pojacaol Meet-ups, @erseznali Educational Platform, Step by Step Erste Program, etc.) Follow-up questions: ⋅ What are examples of success stories? What factors contributed to these successes? ⋅ What are examples of less successful initiatives? What factors limited results? 4. To what extent did CFG helped or hindered your enterprise’ ability to adapt to challenges and opportunities that arose in the business enabling environment over the past four years? (Probe for COVID-19, delays in national priorities, etc.) 5. To what extent and how did CFG foster audience outreach and engagement activities to underserved groups (women, minority groups, youth, people living in rural areas, LGBTQI) and/or integrate gender concerns into their programming? ⋅ What are examples of successful partnerships/collaborations? What were the success factors? ⋅ What are examples of less successful initiatives? What factors limited results? 6. In what ways, if any, did CFG identify and capitalize on lessons learned to improve its support to SMEs during the implementation period? (probe for if/how CFG tested, assessed, and documented the results of different approaches? Created opportunities for partners to provide feedback?)? 7. In what ways, if any, has CFG Project helped promote greater gender equity in the SME landscape, specifically as it relates to: o Representation of women, youth, and other marginalized person owned SMEs in CFG’s interventions? o Addressing the challenges and constraints to women, youth, and other marginalized person owned SMEs access to financial products? In what ways, if any, do you think the CFG Project could have addressed these issues more? usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 80 ANNEX 4: LIST OF DOCUMENTS REVIEWED Oliver-Burgess, Angela, Babović, Marija, Popovicki, Ana, Scippa, Dina. USAID/Serbia Gender Analysis Report. Prepared by Banyan Global. 2020. • CFG scope of work • CFG annual workplans • CFG Monitoring, Evaluation, and Learning (MEL) plan • CFG quarterly progress and qualitative and quantitative monitoring reports • CFG’s Annual Serbia Business Survey, including the survey’s methodology, data collection instruments, and subsequent datasets and reports for 2019 and 2020 as well as the anticipated 2021 Business Survey • CFG’s 2020 Overcome Crisis Together Survey, including the survey’s methodology, data collection instruments, and subsequent datasets and reports • Strengthening e-commerce in Republic of Serbia, Final Report • Proposed Measures that will Foster Alternative Investment Funds Development • Analysis of obstacles for implementing crowdlending in Serbia • Market and Social Impact of Crowdlending in Serbia • Recommendations on and implementation plan to increase a feasible collection of accounts receivable • Recommendations for Regulators, Intermediaries and Other Stakeholders Involved on Measures for Implementing Crowdlending in Serbia • The Effects of Foreign Exchange Liberalization • USAID/Serbia’s Final Gender Analysis Report • Online SMS Beneficiary and End User Survey results 81 | Final Performance Evaluation of CFG Project, Final Evaluation Report usaid.gov ANNEX 5: ANALYSIS OF CFG KEY PERFORMANCE INDICATORS CFG PERFORAMNCE AGAINIST OBJECTIVE 1 PERFORMANCE INDICATORS INDICATOR BASELINE TARGETS PERFORMANCE 2018 2019 2020 LOP 2020 Person hours of USG funded technical assistance in business enabling environment provided to counterparts or stakeholders (EG5.1-1) 0 (2018) 1,000 5,000 8,000 10,000 14,527 Number of commerce laws and regulations developed/changed in accordance with international standards as a result of USG assistance 0 (2018) 5 12 20 25 58 Number of inspections based on risk assessments (%) 65% (2018) 65% 68% 69% 70% 19% Number of firms involved in preventive activities of inspection 2,140 (2018) 2,140 3,643 4,000 4,500 2,974 E-commerce Ecosystem Strengthening Index 28% (2019) N/A 28 35 40 37 Decrease of % firms in Serbia reporting TAX procedures having negative impact on their doing business 24% (2019) N/A 24% 23% 21% 22% Number of self-reliant systems established through project assistance 0 (2018) 0 0 2 4 2 Percentage of financing coming from own funds 84% (2017) N/A 81% 80% 79% 79.4% Number of new SME financial products introduced 0 (2018) 0 1 2 3 2 Growth of new SME loans -7.3% (2018) -7.3% 2.6% 4% 4% 3% usaid.gov Final Performance Evaluation of CFG Project, Final Evaluation Report | 82