MONITORING AND EVALUATION SUPPORT ACTIVITY (MEASURE-BIH) PERFORMANCE EVALUATION OF THE USAID/BIH DIASPORA INVEST ACTIVITY Final Report July 2019 This document is made possible by the generous support of the American people through the United States Agency for International Development (USAID). The contents are the responsibility of the authors and do not necessarily reflect the views of USAID or the United States Government. MONITORING AND EVALUATION SUPPORT ACTIVITY (MEASURE-BIH) PERFORMANCE EVALUATION OF THE USAID/BIH DIASPORA INVEST ACTIVITY Final Report July 2019 Prepared under the USAID’s Bosnia and Herzegovina Monitoring and Evaluation Support Activity (MEASURE-BiH), Contract Number AID-168-C-14-00003 Submitted to: USAID/Bosnia Herzegovina, July 2019 Contractor: IMPAQ International, LLC 3 CONTENTS CONTENTS................................................................................................................................................. 3 TABLE OF EXHIBITS................................................................................................................................. 4 ACKNOWLEDGMENTS............................................................................................................................ 6 ACRONYMS ................................................................................................................................................ 7 EXECUTIVE SUMMARY ............................................................................................................................ 9 PURPOSE, EVALUATION QUESTIONS, AND PRIMARY AUDIENCE............................................................................9 EVALUATION METHODS AND LIMITATIONS ....................................................................................................................9 BRIEF BACKGROUND INFORMATION..................................................................................................................................9 FINDINGS AND CONCLUSIONS............................................................................................................................................10 RECOMMENDATIONS ................................................................................................................................................................12 I EVALUATION PURPOSE, QUESTIONS, DESIGN, AND LIMITATIONS ................................ 15 PURPOSE...........................................................................................................................................................................................15 EVALUATION QUESTIONS .......................................................................................................................................................15 DESIGN AND DATA SOURCES ...............................................................................................................................................16 LIMITATIONS ..................................................................................................................................................................................17 II BACKGROUND INFORMATION ON DIASPORA INVEST ....................................................... 19 III FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS...................................................... 24 EVALUATION QUESTION 1......................................................................................................................................................24 EQ1 FINDINGS ...............................................................................................................................................................................24 EQ1 CONCLUSIONS....................................................................................................................................................................28 EQ1 RECOMMENDATIONS ......................................................................................................................................................29 EVALUATION QUESTION 2......................................................................................................................................................29 EQ2 FINDINGS ...............................................................................................................................................................................30 EQ2 CONCLUSIONS....................................................................................................................................................................47 EQ2 RECOMMENDATIONS ......................................................................................................................................................48 EVALUATION QUESTION 3......................................................................................................................................................50 EQ3 FINDINGS ...............................................................................................................................................................................50 EQ3 CONCLUSIONS....................................................................................................................................................................58 EQ3 RECOMMENDATIONS ......................................................................................................................................................58 EVALUATION QUESTION 4......................................................................................................................................................59 EQ4 FINDINGS ...............................................................................................................................................................................59 EQ 4 CONCLUSIONS ..................................................................................................................................................................62 EQ4 RECOMMENDATIONS ......................................................................................................................................................62 ANNEXES.................................................................................................................................................. 63 ANNEX I: DOCUMENTS REVIEWED.................................................................................................... 64 4 ANNEX II: LIST OF KEY INFORMANTS............................................................................................... 66 ANNEX III: EVALUATION STATEMENT OF WORK ......................................................................... 68 ANNEX IV: DRAFT DATA COLLECTION INSTRUMENTS FOR DIASPORA INVEST EVALUATION........................................................................................................................................... 85 ANNEX V: DI ACTIVITY AND RESULTS ........................................................................................... 110 ANNEX VI: TECHNICAL ASSISTANCE BENEFICIARIES................................................................ 112 ANNEX VII: ANALYSIS OF FINANCIAL PERFORMANCE OF DI GRANTEES ............................. 115 ANNEX IX: GRANTS AND MAIN INDICATORS .............................................................................. 135 ANNEX X: THE EVALUATION TEAM RESPONSES TO COMMENTS FROM THE IMPLEMENTING PARTNER ................................................................................................................................................ 136 TABLE OF EXHIBITS Exhibit 1. Online survey samples and response rates .................................................................................................................17 Exhibit 2. Main information about the Diaspora Invest Activity...............................................................................................19 Exhibit 3. DI Beneficiaries...................................................................................................................................................................24 Exhibit 4. DI Performance Indicators, 2017 and 2018.................................................................................................................25 Exhibit 5. DI Performance Indicators under Component 2.......................................................................................................27 Exhibit 6. How beneficiaries valued different types of DI’s assistance ....................................................................................31 Exhibit 7. How beneficiaries rated the effects of different types of DI’s assistance on their business performance..32 Exhibit 8. Would you have invested if you had not received a grant? ....................................................................................33 Exhibit 9. Would you have hired new employees if you had not received a grant?................................................................ Exhibit 10. DI Beneficiary Investments, 2017 and 2018 (self-reported and AFIP/FIA data) .................................................. Exhibit 11. Did you receive grants from other donors? .............................................................................................................35 Exhibit 12. Did you receive grants/subsidies from BiH governments?....................................................................................35 Exhibit 13. Did you receive a loan from commercial banks?.....................................................................................................36 Exhibit 14. Sources of financing MSME needs in BiH...................................................................................................................36 Exhibit 15. Access to finance..............................................................................................................................................................37 Exhibit 16. Classification of grant value added ..............................................................................................................................37 Exhibit 17. Grant Added Value and Contractual Targets...........................................................................................................38 Exhibit 18. Is your company a recipient of any type of DI technical assistance?..................................................................39 Exhibit 19. Are you satisfied with the quality of technical assistance received? ...................................................................40 Exhibit 20. Have you attended/participated in conferences (RECONNECT, BHDiaFor), info sessions or other events organized by the Diaspora Invest project for the purpose of networking across the BiH diaspora?......40 Exhibit 21. Do you think that these conferences, info sessions, or any other events organized by Diaspora Invest, were beneficial for your business?.............................................................................................................................41 Exhibit 22. Did you use DBC services? ...........................................................................................................................................43 Exhibit 23. Would your company be willing to pay for the following services? ....................................................................... 5 Exhibit 24. Have you used the information and services provided by the website www.diasporainvest.ba? ...............45 Exhibit 25. Are you a member of any diaspora business network?.........................................................................................46 Exhibit 26. Did you notice any improvements or changes in cooperation among diaspora business networks in the last two years?.................................................................................................................................................................47 Exhibit 27. DI Requests for Applications........................................................................................................................................50 Exhibit 28. Grantees according to classification of economic activities..................................................................................51 Exhibit 29. DI’s beneficiaries and their sectors of economic activity, self-reported ...........................................................51 Exhibit 30. Grant Application Results, RFAs 1, 2, and 3.............................................................................................................52 Exhibit 31. Grant approval and rejection rates................................................................................................................................. Exhibit 32. Survey results about transparency of grant application process .........................................................................54 Exhibit 33. Grant approval process..................................................................................................................................................55 Exhibit 34. Grant purpose ..................................................................................................................................................................57 Exhibit 35. Were grant funds sufficient to cover your business development needs?........................................................57 Exhibit 36. Are diaspora investors facing specific challenges/barriers compared to other investors (domestic and foreign)?................................................................................................................................................................................ Exhibit 37. Business enabling environment obstacles – WB......................................................................................................61 Exhibit 38. What are the greatest challenges for your company in terms of legal, regulatory, and institutional framework, or current policies?.................................................................................................................................61 6 ACKNOWLEDGMENTS Guided by the U.S. Agency for International Development (USAID) Evaluation Policy, USAID/Bosnia and Herzegovina (BiH) commissioned IMPAQ International within the USAID/BiH Monitoring and Evaluation Support Activity (MEASURE-BiH) to conduct the Performance Evaluation of USAID/BiH DIASPORA INVEST Activity. The USAID Diaspora Invest Activity is a $5.19 million activity with an expected implementation period from April 3, 2017 until March 30, 2022, implemented by Financial Markets International, Inc. (FMI). The evaluation team members include Davorin Pavelic (Team Leader and MEASURE-BiH Chief of Party), Anesa Hadzic (evaluation team member and MEASURE-BiH Senior Research Analyst), Jasmina Mangafic (evaluation team member and SME Financing Expert), Nermin Oruc (evaluation team member and Bosnian diaspora expert), and Amina Smajovic (evaluation team member and MEASURE-BiH Analyst). The authors wish to thank those who contributed to the process of developing this evaluation report. At USAID/BiH, the following people were instrumental in designing the evaluation: Elma Bukvic Jusic, MEASURE-BiH Contracting Officer’s Representative (COR); Andrew Boegel, Economic Development Office (EDO) Director; and Dobrila Vukmanovic, Diaspora Invest Activity COR, who helped us understand the design of Diaspora Invest Activity and its main tasks, results and management. The authors also wish to thank Amy Kracker Selzer, MEASURE-BIH Project Director and Managing Director at IMPAQ International, for providing technical guidance and advice throughout the process of preparing the report. We also thank the DI Activity management team for their assistance providing Activity documentation and databases. Any errors in this report are the sole responsibility of the authors. 7 ACRONYMS APIF B2B BAM BD BDM BiH BNP COP COR D4D DBC DEP DI DIAC DO DQA EQ EU FA FARMA II FBiH FIPA FIA GDP GM ICT ILDP IP KI KII LoA M&E MEASURE-BiH MEL MOFTER MoU MSME OSS REC RFA ROA ROE ROTA RS SDC SME SOW TA Agency for Intermediary, IT and Financial Services in RS Business-to-Business Bosnian Mark Brcko District BiH Diaspora Marketplace Bosnia and Herzegovina Business Network Platform Chief of Party Contracting Officer’s Representative Diaspora for Development Diaspora Business Center Directorate for Economic Planning Diaspora Invest Diaspora Investors Advisory Council Development Objective Data Quality Assessment Evaluation Question European Union Fixed Assets Fostering Agricultural Markets Activity II Federation of Bosnia and Herzegovina Foreign Investment Promotion Agency Financial Information Agency Gross Domestic Product Grant Manager Information and Communications Technology Integrated Local Development Project Implementing Partner Key informant Key informant interview Life of Activity Monitoring and evaluation USAID/BiH Monitoring and Evaluation Support Activity Monitoring, evaluation, and learning BiH Ministry of Foreign Trade and Economic Relations Memorandum of understanding Micro, small, and medium enterprise One-Stop-Shop Review and Evaluation Committee Request for Application Return on assets Return on equity Return on total assets Republika Srpska Swiss Agency for Development and Cooperation Small and medium enterprise Statement of work Technical assistance 8 USAID USAID/BiH USG VPS WB U.S. Agency for International Development U.S. Agency for International Development in Bosnia and Herzegovina U.S. government Virtual Private Server World Bank 9 EXECUTIVE SUMMARY PURPOSE, EVALUATION QUESTIONS, AND PRIMARY AUDIENCE IMPAQ International (IMPAQ) was commissioned by USAID/Bosnia and Herzegovina (BiH) through the USAID/BiH Monitoring and Evaluation Support Activity (MEASURE-BiH) to conduct a performance evaluation of USAID/BIH’s Diaspora Invest (DI) Activity in Bosnia and Herzegovina, implemented by Financial Markets International, Inc., under Contract No. AID-168-C-17-00001. The purpose of the performance evaluation of the USAID/BiH Diaspora Invest Activity (DI) is to assess the progress towards achievement of the Activity’s contractual obligations to date and to provide recommendations for adjustments to the Activity, if needed. The four evaluation questions (EQs) address DI’s design, management, implementation, and results. The evaluation considers DI’s progress to date in reaching contract targets; implementation of its technical approach; beneficiaries’ perception of the added value of grants and various forms of technical assistance (TA); management, implementation, monitoring, and coordination of the grant component of the Activity; and perceptions of public sector partners and stakeholders of the value of DI’s assistance with removal of the institutional and policy barriers to diaspora investments. USAID/BiH is the primary audience for this evaluation, and its findings, conclusions, and recommendations will be used to assess the achievements of DI and inform further diaspora-related interventions. This report reflects data collected by the evaluation team during May 2019. EVALUATION METHODS AND LIMITATIONS To conduct this evaluation, we used a mixed-methods approach that included an extensive desk review of implementation documents, databases and reports; analysis of beneficiaries’ financial reports; 69 semi￾structured key informant interviews (KIIs); an online survey of DI beneficiaries; and an online mini-survey of non-beneficiary diaspora companies. The main limitations of the evaluation include potential recall bias and the staggered implementation of DI’s approach to delivery of TA. To address these limitations, the evaluation team triangulated data from multiple sources including documentation, KIIs, and surveys. BRIEF BACKGROUND INFORMATION DI is a $6.2 million Activity whose implementation began in April 2017 and is expected to be completed in April 2022. The purpose of the Activity is to catalyze the economic contributions of the BiH diaspora to foster the socio-economic development of BiH. USAID/BiH’s DI is designed to support USAID’s broader economic growth goal of achieving a competitive, market-oriented economy providing better economic opportunities for all its citizens (Development Objective 2), as expressed in USAID’s Country Development Cooperation Strategy. The development hypothesis specifies that, if linkages are made between domestic investment opportunities and diaspora capital and knowledge, if the obstacles in the business environment 10 are reduced; and if the diaspora has access to finance, technical assistance, and grants to start and grow their businesses, then the diaspora investment in BiH will increase, which will contribute to job creation.1 The Activity is divided into three components: Component 1 aims to identify obstacles to the business￾enabling environment in the country and devise policy recommendations to address them; Component 2 is a grant scheme combined with TA, provided to eligible early-stage micro, small and medium enterprises (MSMEs). Under Component 3, the Activity is expected to develop a sustainable local platform to provide business services and facilitation to potential diaspora investors. The main implementation mechanisms used by DI include a small grants fund, which awards grants to eligible diaspora investors (grantees), a web portal (an online business matchmaking network) and diaspora map, investment conferences and business￾to-business (B2B) events, the diaspora business center (DBC)/one-stop-shop (OSS), and a pool of experts and consultants who, together with the Activity’s staff, provide TA to eligible diaspora investors. FINDINGS AND CONCLUSIONS Component 1: Policy Support After the Government of Republika Srpska withdrew from the working groups for development of the Framework Diaspora Cooperation Strategy, USAID/BiH halted all DI policy-related activities. These policy-related interventions were restarted in August 2018, after the BiH Council of Ministers’ decision to establish a working group for the development of the Framework Diaspora Cooperation Strategy for 2019–2023. DI submitted policy measures for the strategy to the BiH Ministry of Human Rights and Refugees, of which more than 70 percent were eventually adopted by the working group. However, addressing key challenges identified by investors, such as high tax burden, a complicated administrative system, underdeveloped infrastructure and lack of a skilled labor force, are outside the aim and scope of the Framework Diaspora Cooperation Strategy. The only issues that DI can tackle through the Strategy are support activities that can make the investment process less burdensome, such as the provision of information and assistance in obtaining documents or licenses, as well as support in networking, but any success in removing these obstacles can only have a limited effect on attracting diaspora investments compared to other challenges. In relation to Component 1, the DI did not report any achievements in its 2017 and 2018 annual reports. However, the policy component target related to the percentage of policy recommendations generated through structured dialogue with diaspora investors and adopted by the relevant BiH institutions, could be considered achieved, and additional efforts await the outcome of the Framework Diaspora Cooperation Strategy adoption process. Component 2: Provision of Grants and TA The success of the DI Activity’s Component 2 is measured against contractually set targets related to the value of new direct investments (originally $2 million, subsequently increased to $7 million in the latest contract modification) and direct new job creation (250) and achievements of these targets greatly 1 Contract modification no. 7 11 depends on the identification eligible beneficiary companies. Unlike other USAID/BiH activities, such as the Workforce and Access to Higher Markets (WHAM) and Fostering Agricultural Markets Activity II (FARMA II), DI’s specific focus on the diaspora greatly reduces the pool of potential beneficiary companies (DI identified not more than 200). Very explicit eligibility criteria shrinks this pool even further, making DI’s implementation more challenging than the implementation of other USAID activities. In spite of this impediment, DI had already, in its second year of implementation, achieved the life of the Activity (LOA) target related to the number of new formal full-time jobs and the value of direct investments in fixed assets for Component 2. This achievement was made possible by the grantee selection process and the RfA selection criteria that allowed for the prioritization of applicants who proposed to make substantively larger investments. However, an analysis of grantees’ financial statements shows that, in six out of 14 cases, the ratio between the amount of the DI grant award and grantee investment from internal resources ranged from 2 to 10 percent, which reduced the grants’ value-added by more than 30 percent for the entire DI portfolio of closed grants. For comparison, the average ratio of DI grant to grantee investment for the 14 closed grants, excluding one outlier, was 31 percent. The reduction in the grants’ value-added2 was also confirmed through KIIs and the survey of DI beneficiary companies, with 23 percent of respondents stating that they would have invested the same amount in the same year even they had not received the DI grant. An assessment of DI beneficiaries’ access to external sources of financing also calls into question the added value of some awarded grants, as about 24 percent of DI grantees had access to financing from the formal financial system (commercial banks, BiH governments and other donors). In the grantee selection process, DI introduced several steps at various points in the process, including onsite visits and direct negotiations with the applicant, before and after the Technical Evaluation Committee (TEC) and Review and Evaluation Committee (REC) review of applications. Under DI procedures, the REC only reviews applications shortlisted by DI implementers, and USAID/BiH’s involvement in the selection process starts with the REC. While the Small Grants Fund Manual does not clearly describe these intermediary steps, they are not new practices and are regularly used by other USAID activities. The evaluation team believes that standardizing and describing these intermediary steps would provide more clarity about decision making in the grant award process. Even taking into account these new steps, 88 percent of the respondents of the online beneficiaries’ survey were of opinion that the grant application and grant award decision processes were transparent, although 32 percent of the online survey respondents stated they did not receive feedback with their scores and evaluation of their application. 2 There is no formal definition/method of how the value added of grants should be measured in order to achieve a developmental impact. In order to assess the value added by grants, the evaluation team translated the formulation “Source of direct investment that may not have otherwise occurred” from the contractual definition of the expected results into the question “would you have invested in BiH, even if there had been no DI assistance?” 12 Component 3: Platform for Business Services and Diaspora Investors The DI-implemented DBC provides a range of business advisory services to diaspora investors that target two broad groups of beneficiaries: incoming investors mainly in need of information and/or support to facilitate their investment, and already established diaspora companies. During the first two years of DI implementation, the DBC primarily focused on assisting the new, incoming investors. The desk review and DI’s annual reports, KIIs and the online beneficiaries’ survey confirm that a few current DI grant beneficiaries (already established diaspora companies) used DBC services, while many of them were never offered these services. Possible reasons for this limited use of DBC services include the fact that established diaspora companies no longer need such services, and that the competitive advantage of the DBC compared to lawyers or consultants, who provide the same services to the larger market, is not clear. In addition, developing a set of services for existing diaspora3 companies is essential to ensure the sustainability of the DBC, which is one of the Activity’s objectives. Besides increasing the visibility of business advisory services for existing diaspora companies, DBC could improve its profile by extending its services to local businesses interested in cooperation with diaspora investors. To emphasize its comparative advantage over other local service providers, such as consulting companies, the DBC could emphasize connections with potential diaspora investors. However, if the DBC’s services continue to be aimed only at potential incoming diaspora investors, then the DBC/one￾stop shop can operate as an online service. The DBC connections with diaspora incoming investors, existing diaspora companies and local businesses could serve as a good platform for advocacy interventions that could also improve the DBC’s profile and visibility. This would allow for the possibility of establishing an advocacy mechanism (Diaspora Business Council) linked to the DBC. The current location of the DBC, with a single office located in Sarajevo, may be an important obstacle to the sustainability and growth of the DBC. The DI attempted to resolve this issue by signing Memorandums of Understandings (MoUs) with eight municipalities to establish partnerships with local investor services and thereby increase its presence in the field. RECOMMENDATIONS Given the outcomes of DI implementation under C1 to date, as exemplified in our findings, USAID should consider reformulating the development hypothesis to reflect the scope of DI by specifying that the Activity will address diaspora business support services and not obstacles to creating a better business￾enabling environment. The project’s development hypothesis should be rephrased to read that “more diaspora investments will be attracted if support services are provided”—and not “if business enabling environment obstacles are reduced”. 3 Although the existing DBC’s Business Plan list the set of services to be provided to existing diaspora companies, the majority of these services were not yet offered, and according to DI staff, the DBC’s Business Plans needs to be thoroughly revised. 13 Bearing in mind the importance of grants in USAID/BiH’s assistance to BiH, USAID should consider providing a formal definition/method of how grants’ value added should be measured in order to achieve a developmental impact. The formal definition should be provided in the design phase of any new activity. This addition would affect the design itself, the types of beneficiaries selected, and the expected results and contract targets. USAID should also consider, to the extent possible, applying a unique definition of grant value-added for all of its Activities to ensure a unique approach to the grant component. Such a definition would increase comparability across the Activities. The fact that about 23 percent of beneficiary companies would have invested in BiH even without DI grants suggests that the grantee selection criteria need to be revised. USAID should consider changing the grantee selection criteria and targeting only companies for which the grants will add value. In parallel, TA beneficiary selection criteria should be aligned with the new grantee selection criteria to avoid potential cases where ineligible grant applicants receive substantial financial support under TA. In turn, contractual targets will have to be revised. The anticipated revision of selection criteria will reduce the size of the pool of potential DI beneficiaries, but it would thereby facilitate the desired increase in the added value of the grants. Bearing in mind that the main LOA targets under Component 2 related to the number of new formal full￾time jobs and the value of direct investments in fixed assets have been achieved, the grantee selection process could be less risk-averse. This shift should ensure that the focus of the grantee selection process shifts from the amount of investments in fixed assets and the number of jobs created toward well￾structured business proposals and business models that have the potential to be profitable and sustainable. To address these items, USAID/BiH should simultaneously adjust the contractual grantee eligibility criteria to deemphasize the size of investment as the target (which has been already reached anyhow), and work with the IP to align grant evaluation processes and grantee selection with the Small Grants Fund Manual. Although the latest contract modification related to Component 2 set the new cost-share investment target that recognizes working capital expenditures, workforce development expenditures and in-kind contributions as investments, rather than just fixed assets, the evaluation team is of opinion that for Component 2 and the small grants fund, USAID/BiH should consider keeping the same grant purpose as in the second and third requests for grant applications (investments into fixed assets). The BDM small grants should be dedicated entirely to financing purchases of fixed assets with a useful life of more than one year. The duration of the grant agreement implementation ranges from 12 to 18 months and therefore current assets should not be considered eligible for financing from this grant scheme since their useful life is, by definition, less than one year. Also, verification of presently recognized elements of cost-share investments through external data sources might prove to be a challenge. Current DI grant beneficiaries should be one of the main target markets for the DBC in the short run. DBC services should be extended to local businesses interested in cooperation with diaspora investors and extended beyond BiH by networking with potential investors abroad. This can be done in cooperation with the Foreign Investment Promotion Agency of BiH (FIPA), one of the likely places for the DBC in case of its institutionalization, as envisaged by the Framework Diaspora Cooperation Strategy. Also, this approach can strengthen the DBC’s position with local businesses, emphasizing its connections with businesses abroad. 14 One possible measures to raise the profile of the DBC would be for USAID to consider supporting the establishment of a new DIAC (Diaspora Business Council) 4 associated with the DBC. The advocacy interventions of this mechanism, aimed at supporting diaspora investors and entrepreneurs in formulating reform recommendations, and promoting an enabling environment for diaspora investment could be extended over the entire duration of the Activity. 4 This is one of DI’s submitted measures adopted by working groups - ”1.2 Establishing the BiH Diaspora Business Council to bring together representatives of business diaspora to provide advice and input relating to policies and procedures relevant for increasing diaspora investment” 15 I EVALUATION PURPOSE, QUESTIONS, DESIGN, AND LIMITATIONS PURPOSE USAID/BiH commissioned IMPAQ International (IMPAQ) through the USAID/BiH Monitoring and Evaluation Support Activity (MEASURE-BiH) to conduct a performance evaluation of USAID/BiH’s Diaspora Invest (DI) Activity in Bosnia and Herzegovina (BiH). DI is being implemented by Financial Markets International, Inc., under Contract No. AID-168-C-17-00001. DI started on April 3, 2017, and will continue until April 2, 2022. This draft report reflects data collected by the evaluation team during May 2019. This DI performance evaluation covers the period from April 2017 to June 15, 2019, when the evaluation team completed the data collection phase of the evaluation process. The performance evaluation provides USAID/BiH with an evidence-based and independent review of DI, including qualitative and quantitative analyses of DI’s design, implementation, management, and performance. The DI performance evaluation examined progress to date in reaching contract targets; implementation of the technical approach; beneficiaries’ perception of the added value of grants and various forms of technical assistance; management, implementation, monitoring, and coordination of the grant component of the Activity; and perceptions of public sector partners and stakeholders of the value of DI’s assistance related to removing the institutional and policy impediments to diaspora investments. DI’s goal is to create new job opportunities for BiH citizens through increased diaspora investment. To meet its goal, DI delivers assistance through three principal components. Component 1 focuses on leveraging diaspora input into the development of an institutional policy framework for diaspora investment. Component 2 focuses on expanding diaspora direct investment by providing technical assistance and grants to eligible early-stage small and medium-sized enterprises (SMEs) 5 and start-ups. Component 3 focuses on developing a sustainable local platform to provide business services and facilitation to potential diaspora investors. EVALUATION QUESTIONS The evaluation team conducted this evaluation using a rigorous design and methodological approach that addresses the specified evaluation questions and provides insights into DI’s progress in achieving the expected results. The evaluation questions (EQs) address DI’s design, management, implementation, and results. EQ1. What progress has been achieved to date in reaching contract targets (new full-time registered jobs, value of direct investments, number of firms receiving DI’s funded technical assistance for improving 5 According to the definition in the Federation of BiH (FBiH) law on incentives for small businesses, small enterprises are companies with fewer than 50 employees and an annual turnover of less than BAM 19.5 million; medium-sized companies have fewer than 250 employees and an annual turnover of less than BAM 97.5 million. 16 business performance, percentage of female participants in DI’s assisted programs to increase access to productive economic resources, percentage of policy recommendations generated through structured dialogue with diaspora investors adopted by the BiH relevant institutions, number of diaspora eligible early-stage small and medium-sized enterprises [SMEs] and start-ups that graduated from the BiH Diaspora Marketplace [BDM], number of BiH diaspora organizations / business networks registered on the diaspora map, number of potential diaspora investors who used information and services provided by the DI one￾stop shop)? EQ2. How has DI’s technical approach been implemented, and how is it perceived by beneficiaries in terms of added value of grants and different forms of technical assistance? This includes technical assistance and business networking/linkages through the Diaspora Business Networking Platform (BNP) and the Diaspora Business Center (DBC) one-stop shop. How has DI contributed to improved cooperation among diaspora business networks? EQ3. How has DI managed, implemented, monitored, and coordinated the grant component of the Activity? EQ4. How do public sector partners/stakeholders (at different government tiers) perceive and value DI’s assistance related to removing the institutional and policy impediments to diaspora investments, crafting reform proposals and their implementation? How do DI’s interventions fit within the broader diaspora policy efforts? DESIGN AND DATA SOURCES To answer the research questions, the evaluation team used a mixed-methods approach that incorporated both quantitative and qualitative data from multiple sources. These included: DI implementation documentation, data, and records; the actual values of expected results reported by the IP to date; available official financial records for DI beneficiaries that report to FIA/APIF; official documents from donors and state institutions; 69 semi-structured key informant interviews (KIIs); an online survey of DI beneficiaries; and an online mini-survey of non-beneficiary diaspora companies. The evaluation team collected primary data through KIIs and online surveys. The 69 KIIs involved seven broad types of stakeholders:  USAID/BiH staff (1)  DI implementers and partners (6)  DI grantees (24)  Recipients of DI technical assistance (TA) (10)  DI public sector beneficiaries or stakeholders (15)  Non-beneficiary SMEs (6)  Representatives of other donors and international organizations (7) In addition to qualitative data collection, the evaluation team also conducted web surveys of DI beneficiary and non-beneficiary SMEs. The objective of the surveys was to gather information and data on 17 respondents’ perceptions of the technical approach applied by DI and of the added value of grants and various forms of TA. The beneficiary survey was sent to all identified SME beneficiaries for whom we were able to obtain an email address. The survey questionnaire is provided in Annex III. The data collected through this survey allow MEASURE to assess whether the perceptions of DI among respondents align with the themes that emerge from the qualitative data. The sample of non-beneficiary SMEs was drawn from DI’s database of all identified diaspora-owned or diaspora-managed companies and from its assessment of diaspora investors currently operating in BiH. For the purposes of the online survey, all 53 diaspora companies assisted by DI were sent the link with the invitation, and 36 responded. For the purposes of the online mini-survey of non-beneficiary diaspora companies, the link with the invitation was sent to 112 diaspora companies, and 24 responded. The surveyed beneficiary and non-beneficiary populations and their survey response rates are shown in Exhibit 1. Exhibit 1. Online survey samples and response rates Type Total Respondents Response Rate All BiH enterprises that meet DI’s diaspora investor criteria 165 60 36.36% DI beneficiary SMEs 53 36 67.29% Non-beneficiary SMEs 112 24 21.42% When possible, the team triangulated data from all data sources to address the evaluation questions and sub-questions from multiple perspectives. Comparing data enabled the team to more fully understand the issues and provide more reliable findings. LIMITATIONS The main limitations of the evaluation include recall bias and the staggered implementation of the DI approach to delivery of TA to SMEs and start-ups. Limitation 1: Recall bias In providing data for this evaluation, it is possible that informants inaccurately represented the implementation or outcomes of the DI activities. For example, some beneficiaries could not recall signing the technical agreement with DI or receiving technical assistance although the document review confirms they did. To address this concern, as noted previously, the evaluation team triangulated data across sources to provide reliable results and recommendations. 18 Limitation 2: Staggered implementation of DI approach to delivery of TA to SMEs and start￾ups. DI has implemented activities in a staggered manner throughout the period of performance. For example, the DBC and BNP approaches, which we examine under EQ 2, began implementation relatively recently and it is expected that DBC’s sustainability will only be achieved at the end of the fourth year of the Activity implementation. Additionally, only a small portion of the DBC’s business advisory services are being provided to SMEs. As a result, there is not sufficient data to analyze progress of the implementation of the DBC’s business plan and to what extent to which the services planned to be offered will contribute to DBC’s sustainability. Thus, the evaluation team examined how SMEs and stakeholders value these channels for service delivery and the services provided to date. More information on the evaluation purpose, design, and limitations is presented in Annex III. 19 II BACKGROUND INFORMATION ON DIASPORA INVEST DI (contract number AID-168-C-17-00001) is a $6.19 million activity with an expected implementation period from April 3, 2017, until April 2, 2022. It is being implemented by Financial Markets International. Exhibit 2. Main information about the Diaspora Invest Activity Activity to be evaluated Diaspora Invest Implementer Financial Markets International, Inc. Contract # AID-168-C-17-00001 Total estimated cost $6,191,337 Life of activity April 3, 2017 – April 2, 2022 Period to be evaluated April 3, 2017 – May 30, 2019 CDCS Development Objective DO 2, Economic Growth: Bosnia and Herzegovina is a competitive, market-oriented economy providing better economic opportunities for all its citizens. The purpose of the Activity, as specified in the award, is “to catalyze the economic contributions of the BiH diaspora to foster the socio-economic development of BiH.” The goal of the USAID/BiH DI Activity is to create new job opportunities for BiH citizens through increased diaspora investment. The development hypothesis for this activity is that, “if linkages are made between domestic investment opportunities and diaspora capital and knowledge; if the obstacles in the business environment are reduced; and if the diaspora has access to finance, technical assistance, and grants to start and grow their businesses, then the diaspora investment in BiH will increase, which will contribute to job creation.”6 The DI contract specifies three objectives to accomplish the Activity’s purpose and goal, along with eight expected results.7 The expected results associated with Objective 1 include conducting a diaspora investment assessment that will:  Identify existing obstacles and barriers to stronger diaspora involvement in the BiH economy  Analyze government institutions, regulations, policies, and laws relevant to barriers to diaspora investments in BiH  Identify the intersections between the needs of diaspora investors and policies that will create a more favorable business-enabling environment  Measure the diaspora’s perceptions of the barriers to involvement (such as lack of information, scarcity of financial products, business registration services) 6 Contract modification no. 7 7 Source: Contract number AID-168-C-17-00001; USAID Bosnia and Herzegovina; April 11, 2019 20  Provide recommendations for the BiH diaspora investment policy framework One of the Activity’s main outputs under Objective 1 is the Diaspora Investment Assessment, which was completed in June 2017. The assessment aimed to determine, identify, and/or cover: (1) obstacles and barriers to economic involvement by the diaspora and an analysis of government institutions, regulations, policies, and laws relevant to barriers to diaspora investments in BiH; (2) intersections between the needs of diaspora investors and policies that create a more favorable business environment; (3) barriers to diaspora involvement; (4) stock, remittance, and overall economic potential of the diaspora, along with diaspora direct investments and policy and institutional framework. DI also carries out regular analytical work (surveys and information and data analysis) to ensure that diaspora input is properly articulated and mainstreamed into government policies. After the diaspora assessment was approved, DI initiated the creation of the Diaspora Investors Advisory Council (DIAC). DIAC was intended to serve as an advocacy platform for introducing business environment reforms, with an emphasis on diaspora investment. DI facilitated DIAC’s activities and helped formulate the first set of recommendations based on diaspora investors’ input. However, in October 2017, the Ministry of Human Rights and Refugees proposed a “modular” approach to the entity and Brcko district governments, whereby three strategic documents developed by the entities and Brcko district would be brought together under a state-level framework diaspora cooperation strategy. As this new approach was forwarded to USAID/BiH for consideration, DI supported the establishment of an informal DIAC. In March 2018, only the Brcko district working group was established. In the end of Year 1, USAID/BiH informed DI of its decision to postpone all policy-related activities until further notice. This decision was based on the decision of the government of Republika Srpska to withdraw from the working groups for development of the Framework Diaspora Cooperation Strategy. On August 1, 2018, the BiH Council of Ministers adopted a decision to establish a working group for the development of the Framework Diaspora Cooperation Strategy for 2019–2023. This step restarted the DI policy activities. The DI submitted input for the situation analysis and a policy options document that outlined some possible approaches, measures, and instruments to be included in the strategy to the BiH Ministry of Human Rights and Refugees. In January 2019, DI undertook the first full survey of diaspora investors currently active in BiH, which recorded a total of 200 diaspora companies. The survey was conducted to ensure that inputs from private sector stakeholders were channeled into the policy-making process in a structured way and to quantify and validate the key inputs received from diaspora investors through DIAC. At the end of the second year, Pillar 3 of the Framework Diaspora Cooperation Strategy, including detailed explanations of the priorities and measures to be included, was adopted at a joint meeting by the three working groups. In future years, the focus of DI will shift to providing TA in developing action plans for the strategies and support the advocacy process to adopt the strategies. 21 Objective II. Expand diaspora direct investment by providing TA and grants to eligible early￾stage SMEs and start-ups. To achieve Objective II, the IP has been conducting the following activities:  Establish a small matching grant program for eligible early-stage SMEs and start-ups;  Organize a business plan competition to spur BiH diaspora investment and use of the BiH Diaspora Marketplace (BDM);  Award grants and provide business advisory services to BDM grantees. The Objective II activities are associated with the following expected results:  At least 70 eligible early-stage diaspora SMEs and start-ups graduate from the BDM by the end of Year 5.  Assisted enterprises create at least 250 new jobs.  Diaspora direct investment increased (source of direct investment that may not have otherwise occurred) by at least $7 million, as evidenced by the amount of equity contributed by diaspora entrepreneurs and other metrics established to capture qualitative effects, such as diaspora knowledge transfer, introduction of new technologies, investment in job creation. Upon graduation from the BDM, qualifying grantees that expressed needs for further financing were expected to be referred to Nova Banka to be considered for lending under the USAID/Sweden-supported Loan Portfolio Guarantee program for diaspora SMEs. However, contract modification 1 removed the obligation for FMI to partner with a specific commercial bank to establish and manage the grant program because of the reluctance of the Nova Banka management to participate. DI also provides grantees with tailored business advisory services in three phases of the award process: pre-award, award, and post-award. Pre-award services are delivered through an online information sharing tool,8 which is integrated into the online BNP. During the award period, grantees can receive advisory services on, for example, market analysis or financial projections, to support successful implementation of the business plans. Upon graduation from the BDM, DI links grantees with the one-stop shop for any potential post-award services, such as matchmaking and business intelligence. Objective III. Develop a sustainable local platform to provide business services and facilitation to potential diaspora investors. Achieving Objective III will require the following four activities to be implemented:  Establish a one-stop shop to provide information and services to diaspora investors to facilitate their investment in BiH. 8 List of pre-award services: (i) interactive checklists for starting a business, registering property, obtaining permits, etc.; (ii) online repository of useful business-related information on sectors of priority, including the main trends, players, news and business intelligence; (iii) common start-up mistakes, industry/market analysis, successful projects, business register; (iv) links and contacts with relevant governmental institutions; and (v) frequently asked questions (FAQ) addressing the most common inquiries of new investors in BiH. 22  Establish a diaspora online business network to serve as a database for investment opportunities, networking events, and investment conferences and to facilitate matchmaking between diaspora investors and local businesses.  Conduct outreach activities and organize diaspora investment conferences and business-to￾business (B2B) events that will bring together diaspora investors, local business communities, and government representatives.  Within the first quarter of the operation of the one-stop shop, develop a strong, detailed business plan of provision of high-impact services as the fee-based core offering, in order to achieve sustainability. The Objective III activities are expected to lead to the following results:  BiH diaspora investment increases by $15 million as shown by a comparative analysis of data on investment levels evidenced by the amount of equity contributed by diaspora entrepreneurs and other metrics established to capture qualitative impacts, such as diaspora knowledge transfer, introduction of new technologies, and investment in job creation.  Cohesion among diaspora business networks improves through an online BiH diaspora map, built into the web portal, which collects and visualizes BiH diaspora business organizations’ presence and influence. Map users will be able to search diaspora organizations by sector, countries of registration, and other characteristics. The map will register at least 100 BiH diaspora organizations worldwide by the end of Year 2.  A fully self-sustaining one-stop shop and web portal is established by the end of Year 4 to provide fee-based services and information to BiH diaspora investors and associations. Under Component 3, DI provides short-term business services focused on investment facilitation, improvement of manufacturing or service processes, product development and prototyping, organizational and financial management, market research, market strategy development, and more. TA is provided under the TA guidelines, based on the agreements signed with diaspora companies. The TA is limited to companies from the following sectors: manufacturing, agriculture, tourism, information and communications technology (ICT), metal, wood, and energy. The DBC one-stop shop was envisaged as the main mechanism for facilitating and institutionalizing diaspora investment to ensure sustainability of DI results beyond the life of the Activity. The sustainability plan and the detailed DBC business plan are the key documents that guide the institutionalization processes of DI interventions. The fully functional and equipped DBC was established in Sarajevo in November 2017. The services include, but are not limited to, assistance and advice on setting up a business, including the business registration paperwork; industry-specific consultations; advice on investment opportunities and market linkages and on legal and regulatory requirements; and liaison with local institutions, lawyers, bankers, and others. In November 2017, in partnership with the local NGO Restart, DI launched the BNP, an online forum for the diaspora that serves as a database for investment opportunities, networking events, and investment conferences and that facilitates partnerships between diaspora investors and local businesses. The BNP provides opportunities for virtual knowledge and technology transfer, mentoring, and volunteerism. 23 Because of a loss of data on the DI’s virtual private server, DI relaunched an upgraded BNP in September 2018. The structured and systematic outreach and awareness-raising interventions (public campaigns in local media and through media established by diaspora associations) aim to reach diaspora business communities, prospective diaspora investors, and the local private sector to guide them toward services implemented under DBC and BNP. The DI set up Facebook, Twitter, YouTube, and LinkedIn accounts, which have generated a large following. Outreach is a continuing effort and an integral part of the DI approach. During the second year of implementation, DI supported several events, such as the Ninth Sarajevo Business Forum, the conference "BiH Excellence: The Role of the Diaspora in Strengthening Economic Capacity of Bosnia and Herzegovina," 10th Days of the Bosnian-Herzegovinian American Academy of Arts and Sciences, the Fifth Diaspora Business Forum, Reconnect 2018, Diaspora Investment Day, and the development event Venture and Start a Business. 24 III FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS EVALUATION QUESTION 1 What progress has been achieved to date in reaching contract targets (new full-time registered jobs, value of direct investments, number of firms receiving DI’s funded technical assistance for improving business performance, percentage of female participants in DI’s assisted programs to increase access to productive economic resources, percentage of policy recommendations generated through structured dialogue with diaspora investors adopted by the BiH relevant institutions, number of diaspora eligible early-stage SMEs and start-ups that graduated from the BDM, number of the BiH diaspora organizations / business networks registered on the diaspora map, number of potential diaspora investors who used information and services provided by the Activity one-stop shop)? To address the first evaluation question, the evaluation team examined DI’s work under all three components in order to verify the work and provide a detailed analysis of DI’s progress toward each of the contractual targets outlined in EQ1. The evaluation team compared the contractual targets to the actual values reported by the IP to date. For 2017 and 2018, available official financial records for DI beneficiaries that report to FIA/APIF were collected and analyzed. All reported contractual targets and results were initially verified through review of the IP’s documentation and analysis of secondary data. They were then compared with data collected through the KIIs and beneficiary surveys. The triangulation of data from different sources enabled the evaluation team to develop comprehensive, detailed, and reliable insights into progress made by the IP in reaching the contractual targets. EQ1 FINDINGS Finding 1. During the implementation period covered by this evaluation, DI served 118 potential investors and beneficiary companies, out of which 75 unique beneficiary companies9 received BAM 1,173,054 worth of grants (BAM 1,071,496) and/or TA (BAM 101,558) as shown in Exhibit 3. Exhibit 3. DI Beneficiaries Type of Beneficiary Grant and TA Agreements Signed Grant and TA Agreements Closed Total Value of Awards (BAM) Grantees (with TA) 33 17 GA closed out 23 TA final report 10 TA final report not completed 1,071,496 Grant Amount committed 19,436 TA Value TA recipients 21 17 TA final reports 82,122 TA Value Type of Beneficiary BiH diaspora organizations/ business networks registered on the diaspora map 102 9 Beneficiary companies that received only one type of DI assistance 25 Potential investors who used information provided by DBC/OSS 5310 Source: DI reports, documents and databases Finding 2. The summary of the DI performance indicators for 2017 and 201811 shown in Exhibit 4, suggests that DI exceeded targets on all performance indicators in Year 1 and Year 2, except for two indicators: value of direct investments under Component 3 and percent of policy recommendations generated through the structured dialogue with diaspora investors submitted12 to the relevant BiH institutions. Exhibit 4. DI Performance Indicators, 2017 and 2018 Indicators 2017 Actual (Target) 2017 (FIA/AFIP) 2018 Actual (Target) 2018 (FIA/AFIP) Number of new full-time registered jobs 44 (15) 48 214 (40) 211 Value of direct investments (fixed assets) (Objective 2) 1.1 mil USD (0 mil USD) 1.1 mil USD 1.8 mil USD (0.2 mil USD) 1.8 mil USD Value of direct investments (fixed assets) (Objective 3) 0.6 mil USD (1.5 mil USD) 0.6 mil USD 2.5 mil USD (3.8 mil USD) 2.3 mil USD # of firms receiving DI’s funded TA 17 (12) - 37 (25) - % of female participants in DI’s assisted programs 22 (21) - 27 (22) 28 % of policy recommendations generated through the structured dialogue with diaspora investors submitted to the relevant BiH institutions 0 (0) - 0 (15) - # of SMEs graduated from BDM 0 (0) - 17 (10) - # of the BiH diaspora organizations/business networks registered on the diaspora map 41 (40) - 61 (60) - # of potential investors who used information provided by OSS 23 (15) - 30 (20) - Source: DI’s reports, documents and databases Note: “-“ the data was unavailable Finding 3. Although DI exceeded targets for the number of companies that received DI-funded TA under Component 3, the achieved values of direct investments were almost two times lower than targets. Finding 4. The DI’s 2018 annual report cites no results related to the indicator measuring the percent of policy reform proposals submitted to the relevant BiH institutions. However, more than 70 percent 10 There is a disagreement in the reported number of potential investors and companies assisted by DBC. In the main body of the 2019 Annual Report, DI mentioned 41 beneficiaries assisted by DBC but in the annex it provided a list of 30 assisted individuals and companies. 11 Data on the DI performance indicators are also provided in more detailed exhibits in Annex V. 12 The previous definition of policy indicators was ““% of policy recommendations generated through the structured dialogue with diaspora investors adopted by the relevant BiH institutions” and the latest 2019 modification of DI MEL Plan replaced word “adopted” with word “submitted.” 26 of policy measures submitted by DI were adopted by the BiH government working group for the development of the Framework Diaspora Cooperation Strategy. Thus, the policy component targets, as defined by the Activity’s Monitoring, Evaluation and Learning (MEL) Plan, could be considered achieved, and additional efforts are pending the outcome of the Framework Diaspora Cooperation Strategy adoption process. Finding 5. Based on data from both DI reports and FIA/AFIP, DI performed particularly well on indicators under Component 2 related to the level of direct investments and new jobs created (see Exhibit 5). Actual verified values for these performance indicators were several times higher than targets for 2017 and 2018. In fact, DI already achieved the life of the Activity target related to the number of new formal full-time jobs and the value of direct investments into fixed assets under Component 2. The LOA for the number of new full-time registered jobs is 250 and by the end of Year 2, DI reported, and the evaluation team verified, 259 new full-time registered jobs generated by DI grantees. The LOA target value of direct investment into fixed assets was $2.1 million after the latest contract modification ($2 million before), and DI reported, and the evaluation team verified, $2.9 million in direct investment into fixed assets, was achieved. According to the Small Grants Fund Manual, “grants will be used to facilitate mobilization of diaspora capital into high-growth potential, but underfunded SMEs” 13 and grant applicants are required to match the grant requested at the rate of 1:1, at a minimum. However, the manual also specifies that “cost sharing is encouraged and expected to the greatest extent possible” and that “grantees with higher leverage amounts (cash to in-kind ratio) are viewed more favorably in the grant evaluation process.” The DI’s 2018 annual report noted that “each dollar invested by USG was leveraged by more than 10 dollars of private investment during the year.” Also, according to the DI’s 2018 annual report, in the metal processing sector, which is the core activity of many of DI’s beneficiaries, each grant dollar was leveraged by more than 24 dollars of grantees investments. The latest contract modification from March 2019 and the MEL Plan modification that reduced the LOA target related to the value of direct investments under Component 3 from $20 million to $15 million, also increased the LOA target in terms of the value of direct investments under Component 2 from $2 million to $7 million. The new Component 2 LOA target of $7 million is disaggregated into two sub-indicators: the old indicator of value of direct investments into fixed assets, which has already been achieved (LOA of $2.1 million), and the new additional indicator for the value of direct investment measured as grantees’ cost-share in implementation of grant agreements (LOA of $4.9 million). This investment (cost-share) may again include acquisition of fixed assets, but it is also recognizing as an investment if grantees invest in working capital and in new employment. Grantees’ in-kind contributions, as defined by the Grants Manual, can be also recognized as cost-sharing. The Year 2 target value of direct investment measured as grantees’ cost-share was $1.5 million, and DI reported $1.4 million (LOA of $4.9 million). The evaluation team could not verify the reported result through the TRON database, although this mode of verification was possible with the investments into fixed assets. 13 The Small Grants Fund Manual; Diaspora Invest Activity; June 2, 2017. 27 Exhibit 5. DI Performance Indicators under Component 2 Indicators 2017 Target 2017 Actual 2018 Target 2018 Actual LoA Target Number of new full-time registered jobs 15 48 40 211 250 Value of direct investments (cost share) (Objective 2) - - 1.5 mil USD 1.4 mil USD 4.9 mil USD Value of direct investments (fixed assets) (Objective 2) 0 1.1 mil USD 0.2 mil USD 1.8 mil USD 2.1 mil USD Source: DI’s reports, documents and databases and TRON database Finding 6. Under Component 2, all grantees received TA. During the first year of implementation, DI provided TA for five beneficiaries, with eight other TA interventions ongoing at the time the annual report was produced. DI indicated that 74 working days (592 hours) of assistance were provided to the beneficiaries, of which 56 days were provided by external short-term TA providers and 18 days by DI staff. Expected new investment by these beneficiaries in Year 1 was $750,000. In terms of TA service providers, in Year 2 DI made a shift from using external experts to provide TA to using the Activity’s staff. According to the second annual report, 23 working days (184 hours)14 of assistance were provided to the beneficiaries: 24 hours by external providers and 160 hours by DI staff. The value of TA provided by the Activity’s staff was not monetized, and was mainly related to media promotion, assistance in selecting business partners and supplier and business intelligence. In Year 2, 10 BDM program beneficiaries successfully completed the TA interventions and signed the TA final report. The remaining nine TA agreements for BDM beneficiaries stemming from RFA 2 and RFA 3 are expected to be finalized during the first two quarters of Year 3. Under Component 3, DI signed four TA agreements with diaspora firms in Year 1 and another 17 TA agreements with diaspora firms in the course of Year 2. In comparison to the grant public calls, DI public calls to interested diaspora firms for technical assistance, generated more limited interest, with only 22 inquiries and applications. The investment resulting from the first four TA agreements was expected to reach $835,000; for the other 17 non-grantee beneficiaries, the amount was expected to reach $2,580,000. Finding 7. According to the DI report data at the end of Year 1, DI provided assistance to 17 companies, out of which 8 (47%) were managed or owned by women. In Year 2, DI exceeded the target for participation of women in DI-assisted programs and reported that 27 percent of participants were women. The evaluation team verified these data using the TRON database and found that 28 percent of participants assisted by DI were women. In Year 2, DI provided assistance to 37 companies, out of which 12 were managed or owned by women. Also, DI provided information to 30 potential investors through DBC/OSS, out of which 7 were women. The evaluation team was not able to verify DI’s report of 22 percent female participation in Year 1 since the data on assistance provided to companies and potential investors provided with information by DBC/OSS reported in the 2017 Annual Report were not disaggregated by gender. 14 When reporting about the intensity of technical assistance provided to beneficiary companies, DI sometimes measures duration of technical assistance in hours and sometimes in days. 28 Finding 8. According to the data from both DI annual reports, at the end of Year 1, 41 BiH diaspora organizations registered on the online diaspora map built into the diasporainvest.ba online platform, one more than the target. In Year 2, DI registered 61 BiH diaspora organizations on the diaspora map, again one more than the target. By December 31, 2018, DI reported 490 registered users and at the end of Year 2, the BNP reached a total of 662 registered users. A majority of respondents to the online beneficiaries’ survey (70%) had used the web portal and its services. Finding 9. DI divides DBC’s support services and clients into two categories: “provision of information and investment facilitation support (‘light touch’ interventions) to new, incoming investors, and provision of direct technical assistance to already established diaspora firms (‘full service’ interventions)15.” However, the majority of beneficiaries assisted by the DBC did not come from the pool of DI beneficiaries receiving grants and TA, but are instead, in most of cases new incoming investors. Out of 39 interviewed beneficiary and non-beneficiary companies, 26 stated they did not use DBC/OSS services. According to the data collected through the online survey, only about 6 percent of DI grantees and TA recipients used DBC services. Based on data from both DI reports, in the first year of implementation, the DBC provided information and investment facilitation assistance to 23 potential investors or companies that visited the DBC. In Year 2, the DBC provided assistance to 41 potential investors or companies (as verified by the signed affidavits of diaspora investors). EQ1 CONCLUSIONS The evidence collected for the DI performance evaluation supports the following conclusions in relation to EQ1:  The fact that DI already achieved the life of the Activity target related to the number of new full-time registered jobs and the value of direct investment in fixed assets under component 2, indicates that in some cases the number of new jobs and investments were much larger in size than the Mission staff anticipated in design documents. 16 This may be because, the IP is focusing on large investments or beneficiary selection favored those who would invest larger amounts in fixed assets and hire a larger number of new workers. This approach to selecting applicants aligns with the Small Grants Fund Manual and eligibility criteria specified in RFAs.  The expected contributions of Component I related to “supporting diaspora investors and entrepreneurs in formulating reform recommendations to promote an enabling environment for diaspora investment” and “providing assistance to relevant institutions in Bosnia and Herzegovina in developing a Diaspora Investment Document,” although not reported by DI, have been achieved. Since the additional efforts are pending the outcome of the Framework Diaspora Cooperation Strategy adoption process.  Under Component 3, one of the largest challenges facing DI is the identification of beneficiaries. The aggregate expected target for the value of direct investments in fixed assets, facilitated through DI funded TA, was $5.2 million, and DI reported $3.1 million in direct 15 Both DI’s annual reports describe division of DBC’s services and clients in the same way. 16 Also, please see finding #17 regarding the number of new full-time registered jobs and the value of direct investments in fixed assets. 29 investments. The evaluation team confirmed that the total cumulative direct investment after Year 2 was $2.9 million. The underperformance might be explained by the small pool of potential beneficiaries who are interested in receiving DI-funded TA and meet the DI’s eligibility criteria, and by the limited budget available for DI TA. The companies that were able to invest more in fixed assets were mainly assisted under Component 2. This was confirmed through an interview with IP staff, who stated that, based on current trends, the beneficiaries in Component 3 will not generate the targeted investment DI will need to increase the number of beneficiaries to reach its targets. Previously, DI focused on one-to-one tailored TA interventions. Now, with a larger pool of companies and more knowledge of their potential, DI intends to provide TA through group interventions. That approach could enable DI to cover a larger number of beneficiaries who are provided with DI-funded TA under the same available TA budget. To some extent, the latest contract modification from March 2019 (contract modification no. 7) and the MEL Plan modification reduced the pressure on the expected value of the direct investment target under Component 3 by reducing the LOA target from $20 million to $15 million.  Although the DI’s DBC managed to reach its targets related to the number of potential investors who used their information, there is no clear information about the DBC’s path toward financial sustainability. Information that only a small portion of beneficiaries used DBC services indicates that there are additional opportunities to develop the DBC’s client base and increase its chances to be financially sustainable. EQ1 RECOMMENDATIONS The analysis of progress achieved in reaching contract targets is closely linked with the other three evaluation questions and provides inputs for the analysis of the implementation of DI’s technical approach, management of the grant component and results of the policy component. Therefore, any recommendations related to the achievement of contract targets, their relevance and possible changes will be addressed through recommendations resulting from the analysis of the other three evaluation questions. EVALUATION QUESTION 2 How has DI’s technical approach been implemented, and how is it perceived by beneficiaries in terms of added value of grants and different forms of technical assistance? This includes technical assistance and business networking / linkages through the Diaspora Business Networking Platform (BNP) and the Diaspora Business Center (DBC) one-stop shop. How has DI contributed to improved cooperation among diaspora business networks? Information related to SMEs’ access to finance, which was collected through the document review, and information related to grantees’ direct investments, obtained from grantees’ financial reports, were compared with the data collected from beneficiaries and stakeholders on how they value the grant assistance and TA provided by DI. The KII guides and surveys include specific questions to compare how beneficiaries perceive the value of grants with how they value various types of TA. The instruments also include questions that permitted the assessment of cooperation among diaspora 30 business networks and gauged key informants’ perceptions of DI’s contribution to building cooperation among these networks. Non-beneficiary KIIs and survey questions assessed this group’s perceptions of the value of the types of grants and TA offered by DI, as well as their perceptions of changes in cooperation among diaspora business networks. This information allows us to compare beneficiaries’ and non-beneficiaries’ perceptions. EQ2 FINDINGS Finding 10. The majority of interviewed companies that are DI beneficiaries (26 out of 34), stated that they were satisfied with DI and its technical approach. In addition, 21 beneficiary companies noted that there were some areas where the Activity management and implementation can be improved. Most of these remarks concern extending the duration of the grant implementation process, simplifying reporting requirements and increasing the limit of the grant amount that can be awarded. Finding 11. Out of 18 interviewed donors and public sector institutions, 16 stated that they were familiar with the goal and objectives of DI, 15 were familiar with the availability of the BDM Small Grant Fund, and 12 were familiar with the availability of DI TA (2 did not know about the TA and another did not answer). Finding 12. When asked to compare the added value of grants and TA, out of 39 beneficiary and non-beneficiary interviewees, 17 stated that grants were the most important type of assistance with the highest added value. Thirteen said that TA had a higher added value, and eight said that the added value of these two types of assistance could not be compared because they produce good results only when combined. Finding 13. The online DI beneficiary survey asked beneficiaries17 to compare the value added of grants and of other types of DI assistance in terms of improving their business performance. As shown in Exhibit 6, grants were rated as more important than non-financial types of assistance. The percentage of respondents (32 percent) stating that the grants significantly contributed to their overall business performance was almost equal to the percentage of those who found this statement only partially true (35 percent). About 13 percent of respondents (4 respondent) stated that they would not achieve business results without DI grants. After grants, information sharing, and business networking were perceived to be the most valuable support provided. Business networking was noted to contribute significantly to overall business performance in 26 percent of cases and partially in 53 percent of cases. 17 Respondents were allowed to select more than one option offered by this multiple-choice question 31 Exhibit 6. How beneficiaries valued different types of DI’s assistance Source: MEASURE-BiH online beneficiaries’ survey Finding 14. However, when respondents were asked about the contribution of different types of assistance to specific results of their business operations, grants remained the most important type of assistance. The survey also asked grantees to rate types of DI assistance in enhancing the value of sales and exports, new jobs, and new investment. The DI grants were rated by the respondents as contributing the most to new job creation, new investments and sales and export. The grants were rated third among types of DI assistance in enhancing the value of sales and exports, after information sharing and institutional support in acquiring permits and licenses. The technical assistance from production experts received the lowest ratings in terms of contribution to sales and exports. The market linkages assistance (B2B, promotion and marketing, research and analysis) received the lowest rating in contributing to the new jobs generation. The DBC support received the lowest ranting in contributing to new investments. The results are included in Exhibit 7. 0% 20% 40% 60% 80% 100% Small grants Technical assistance from production expert Market linkages (B2B, research&analysis, promotion &… Workforce development Information sharing Institutional support in acquiring different permits and… Business Networking Did not contribute to achieving business results Slightly contributed Partially contributed Significantly contributed Would not achieve business results without the help of DI activity 32 Exhibit 7. How beneficiaries rated the effects of different types of DI’s assistance on their business performance SALES/EXPORT N/A 1 2 3 4 5 Total Grant funds 31% 8 0% 0 4% 1 15% 4 27% 7 23% 6 26 TA from a manufacturing expert 50% 9 6% 1 6% 1 6% 1 17% 3 17% 3 18 Market linkages and market penetration (B2B, research and analysis, marketing and promotion) 40% 8 0% 0 5% 1 15% 3 25% 5 15% 3 20 Worker training 37% 7 5% 1 0% 0 11% 2 21% 4 26% 5 19 Various business information 35% 7 5% 1 0% 0 25% 5 10% 2 25% 5 20 Support of the Diaspora Business Center 47% 9 0% 0 0% 0 16% 3 5% 1 32% 6 19 Business Networking 33% 6 11% 2 0% 0 17% 3 17% 3 22% 4 18 NEW EMPLOYMENT N/A 1 2 3 4 5 Total Grant funds 16% 4 0% 0 4% 1 16% 4 20% 5 44% 11 25 TA from a manufacturing expert 55% 11 10% 2 0% 0 5% 1 15% 3 15% 3 20 Market linkages and market penetration (B2B, research and analysis, marketing and promotion) 44% 8 0% 0 17% 3 17% 3 22% 4 0% 0 18 Worker training 39% 7 6% 1 0% 0 17% 3 6% 1 33% 6 18 Various business information 41% 7 6% 1 6% 1 24% 4 12% 2 12% 2 17 Support of the Diaspora Business Center 59% 10 0% 0 6% 1 12% 2 6% 1 18% 3 17 Business Networking 41% 7 12% 2 12% 2 6% 1 18% 3 12% 2 17 NEW INVESTMENT N/A 1 2 3 4 5 Total Grant funds 23% 6 4% 1 4% 1 4% 1 27% 7 38% 10 26 TA from a manufacturing expert 56% 10 11% 2 0% 0 6% 1 17% 3 11% 2 18 Market linkages and market penetration (B2B, research and analysis, marketing and promotion) 42% 8 11% 2 0% 0 21% 4 21% 4 5% 1 19 Worker training 35% 6 6% 1 6% 1 18% 3 12% 2 24% 4 17 Various business information 39% 7 6% 1 6% 1 17% 3 11% 2 22% 4 18 Support of the Diaspora Business Center 59% 10 0% 0 6% 1 12% 2 18% 3 6% 1 17 Business Networking 47% 8 6% 1 6% 1 12% 2 18% 3 12% 2 17 Note: 1.00 is the minimum and 5.00 is the maximum. Source: MEASURE-BiH online beneficiaries’ survey 33 Finding 15. In order to assess the value added by grants, the evaluation team translated the formulation “Source of direct investment that may not have otherwise occurred” from the contractual definition18 of the expected results into the question “would you have invested in BiH, even if there had been no DI assistance?” Of the 34 beneficiaries interviewed, 20 grantees and four recipients of DI TA stated that they would have invested in BiH even if there had been no DI assistance. The survey of DI beneficiary companies found that 88 percent of respondents would have invested even without DI assistance. About 12 percent of respondents stated that they would not have invested if they had not received a grant and in an additional 65 percent of cases, DI grants influenced the investment decisions of beneficiaries, either by accelerating their decision to invest or encouraging them to plan larger investments (see Exhibit 8). Exhibit 8. Would you have invested if you had not received a grant? Source: MEASURE-BiH online beneficiaries’ survey Finding 16. The survey also asked DI beneficiaries whether they would have hired new employees if they had not received a grant. In 46% of cases, DI beneficiaries would not have hired new employees if there had been no DI assistance (see Exhibit 9). 18 Source: Contract number AID-168-C-17-00001; USAID Bosnia and Herzegovina; April 11, 2019 23% 19% 46% 12% I would have invested the same amount, in the same business year I would have invested the same amount, but in the following business year I would have invested a smaller amount, in the same business year I would not have invested at all Source: MEASURE-BiH online beneficiaries’ survey 54% 46% Yes No Exhibit 9. Would you have hired new employees if you had not received a grant? 34 Finding 17. When looking at DI beneficiaries’ investment patterns, the survey results (self-reported amount of investment) show that the largest proportion of beneficiary respondents (30 percent) invested between BAM 100,000 and 200,000 during DI implementation in the last two years. Another 15 percent of DI beneficiaries invested more than BAM 1 million in the last two years (see Exhibit 10). Source: MEASURE-BiH online beneficiaries’ survey More precise data on actual investments are available through the financial reports for all DI beneficiaries in FIA/APIF data. According to this publicly available data, cumulative investment during 2017 and 2018 among the 60 DI beneficiaries receiving assistance as of June 30, 2018 is as follows: 35 beneficiaries (58%) invested up to 50,000 KM, 4 beneficiaries (7%) invested between 50,001 and 100,000 KM, 8 beneficiaries (13%) invested between 100,001 and 500,000, 6 beneficiaries (10%) invested between 200,001 and 500,000 KM, 6 beneficiaries (10%) invested between 501,000 and 1 million KM, 1 beneficiary (2%) invested between 1m and 2m KM, and none of the beneficiaries invested more than 2 million KM. Finding 18. To understand the value added by DI grants, the evaluation team assessed whether DI grantees had access to and whether they used other sources of financing. We focused specifically on financial assistance from other international organizations, BiH government institutions, and commercial banks. The evaluation team also analyzed how and where DI grantees obtained financial resources and examined their access to financing. Out of 24 DI grantees and 10 TA recipients interviewed, eight grantees and six TA recipients applied for grants from programs funded by other international organizations (UNDP, Sweden, SDC, and Netherlands). Three grantees and two TA recipients received grants from these programs. Also, 12 percent of surveyed beneficiaries received grants from other donors, and 9 percent received grants from another donor prior to receiving a DI grant (Exhibit 11). 11% 22% 30% 7% 15% 7% 7% Up to 50,000 BAM Up to 100,000 BAM Between 100,000 BAM and 200,000 BAM Between 200,001 BAM and 500,000 BAM Between 500,001 BAM and 1 million BAM Between 1 million BAM and 2 million BAM Over 2 million BAM 58% 7% 13% 10% 10% 2% Up to 50,000 BAM 50,001 to 100,000 BAM 100,001 to 200,000 BAM 200,001 to 500,000 BAM 500,001 to 1 mill BAM 1 mill to 2 mill BAM Exhibit 10. DI Beneficiary Investments, 2017 and 2018 (self-reported and AFIP/FIA data) 35 Exhibit 11. Did you receive grants from other donors? Source: MEASURE-BiH online beneficiaries’ survey Finding 19. Of the 34 DI beneficiaries interviewed, eight grantees and four TA recipients applied for subsidies and/or grants from programs financed by BiH government institutions. Six grantees and three TA recipients received financial assistance from various tiers of the BiH government. Also, 24 percent of surveyed beneficiaries received grants from the BiH governments, and 12 percent received grants from the BiH governments prior to receiving a DI grant. Another 58 percent stated that they had never applied for any BiH government grant program and 18 percent did apply for such assistance but were rejected (see Exhibit 12). The purposes of these BiH government grants include subsidies for the purchase of equipment, employment bureau support for subsidized employment, subsidies for start-ups and export and agricultural subsidies. Exhibit 12. Did you receive grants/subsidies from BiH governments? Source: MEASURE-BiH online beneficiaries’ survey 9% 3% 30% 58% Yes, we received assistance from another donor/international organization, prior to receiving Diaspora Invest support Yes, we received assistance from another donor/international organization, after receiving Diaspora Invest support No, although we have applied. No, we have never applied. 12% 12% 18% 58% Yes, we received a subsidy or a grant from BiH institutions, prior to receiving Diaspora Invest support Yes, we received a subsidy or a grant from BiH institutions, after receiving Diaspora Invest support No, although we have applied No, we have never applied 36 Finding 20. Of the 34 DI beneficiaries interviewed, 17 (12 grantees and five TA recipients) stated that they needed a loan from a commercial bank. Seven grantees and three TA recipients applied for loans, and four of the grantees and all three TA recipients were approved. Also, 52 percent of survey respondents, stated that they did not need a loan from a commercial bank. The remaining 49 percent had at some point applied for a loan, 15 percent received a loan prior to receiving a DI grant, and 9 percent received a loan after a DI grant (see Exhibit 13). Exhibit 13. Did you receive a loan from commercial banks? Source: MEASURE-BiH online beneficiaries’ survey Finding 21. According to the World Bank Group Access to Finance Survey from 2018,19 the main sources of external financing of MSMEs in BiH are commercial banks, which cover up to 75 percent of the financing needs of medium size companies and 91 percent of the financing needs for small companies (Exhibit 14). BiH governments, through various subsidies, provide the second largest source of external financing for micro and medium sized companies. As shown in exhibits 11, 12 and 13, commercial banks and the BiH government are equally important sources of external financing for DI’s grantees; 24 percent of DI’s grantees received financing from each of these two sources. A somewhat lower percent (12 percent) of DI’s grantees also received grants from other donors in addition to DI’s grants. Exhibit 14. Sources of financing MSME needs in BiH Micro Small Medium Commercial bank 80% 91% 75% A government body within BiH 11% 2% 11% Microfinance institutions 8% 7% 0% Other finance sources 8% 2% 4% Leasing companies 5% 0% 4% Venture capital funds 7% 0% 11% Source: The World Bank Group Gender MSME Access to Finance Survey; BiH 2018 19 The World Bank Group Gender MSME Access to Finance Survey; BiH 2018 15% 9% 24% 52% Yes, we received a bank loan, prior to Diaspora Invest support Yes, we received a bank loan, after Diaspora Invest support No, although we had applied for a bank loan No, we did not need a loan 37 Finding 22. Data collected through the online DI beneficiaries survey conducted by the evaluation team indicate that the financing gap among DI grantees was about 24 percent, meaning that, out of all DI grantees, about 24 percent could use funds productively to grow their businesses and enhance business results, but could not obtain financing from the formal financial system. The remaining DI grantees either accessed financing from the formal financial system (24 percent), or stated they did not need a bank loan. (see Exhibit 15). Exhibit 15. Access to finance Micro Small Medium DI grantees % of firms that have a loan 34% 47% 58% 24% % of firms with no need for bank loan 18% 32% 26% 52% Financing gap 48% 21% 16% 24% Source: The World Bank Group Gender MSME Access to Finance Survey; BiH 2018, and MEASURE-BiH online survey. Finding 23. To assess the value added of grants, we calculated the grant value as a percentage of the beneficiaries’ investment into fixed assets. This measure is particularly relevant, since the grants were intended for the purchase of fixed assets. To gather the relevant information, we analyzed beneficiary financial statements to calculate DI grants as a proportion of beneficiaries’ investment in fixed assets. The rationale for this analysis is based in RFAs 2 and 3, which explicitly defined the purpose of requested funding. Funds requested from the BDM Small Grant Fund were required to be spent only on the purchase of fixed assets. Fixed assets were defined as assets purchased for long-term use and not likely to be converted into cash within 12 months, such as land, buildings, and equipment. Equipment was defined as either tangible assets (such as machinery, computers, vehicles, etc.) or intangible assets (such as a trademark or patent, goodwill, etc.) with a useful life of more than one year and an acquisition cost of $1,000 or more per unit. According to the grant purpose specified in the RFAs, the added value of a grant can be observed in the beneficiary’s balance sheet as the value of fixed assets. We also explored whether there was a relationship between the grants awarded to diaspora-owned eligible early-stage SMEs and start-up companies and their financial performance and looked for any difference between the two years in the observed closeout companies. Based on the ratio of grant value to fixed assets, three groups of grantees were distinguished: grantees with high, medium, and low grant value added, as shown in Exhibit 16. For six cases, the ratio of grant value to fixed assets was below 10 percent, which makes these grants' added value to the beneficiaries' investments into fixed assets minimal. In three cases, this ratio is below 4 percent. Exhibit 16. Classification of grant value added Grant value to fixed assets ratio Number of companies Grant value added 0–10% 6 Low 11–50% 4 Medium 51–100% 4 High Source: DI's documents and FIA/AFIP financial reports 38 Exhibit 17 provides an overview of the percentage of the grant received in comparison to the value of fixed assets beneficiaries reported for 2018. The analysis includes only the grantees that completed the grant agreement, as stated in the grant closeout report, and for whom we have financial statements for both 2017 and 2018. This resulted in a sample of 14 companies. Exhibit 17 also shows how the grantees in these three categories (low, medium and high contribution) contributed to the achievement of the other contractual targets, such as new full time jobs. Although the value added by the grants in terms of the value of fixed assets tends to be low, the six companies from Exhibit 17 contributed significantly to the DI target indicators. The grants to these six beneficiaries accounted for 49 percent of the total value of the grant portfolio under analysis, but these six beneficiaries accounted for 72 percent of the total value of new direct investments, and for 81 percent of the total number of newly created full-time jobs reported by DI. The value added of grants in terms of the fixed assets was assessed as high in the last four companies from Exhibit 17. These four grantees account for 21 percent of the total value of the grant portfolio under analysis, and they account for only 3 percent of the total value of new direct investments, and for 5 percent of the total number of newly created full-time jobs. More detailed data about value of grants, investment in fixed assets and contribution of each of 14 grantees to DI targets is provided in the annex IX. Exhibit 17. Grant Added Value and Contractual Targets % Grant value in fixed assets Frequency Grant value added to the fixed assets % of individual grant in grant portfolio Difference in absolute value of FA (BAM) Difference in absolute value of FA (%) Difference in absolute number of employees Difference in absolute number of employees (%) 0-10 6 Low value added 49.04 793,288 72.59 115 81.85 11-50 4 Low to medium contribution 29.53 264,852 24.23 18,5 13.17 51-100 4 High contribution 21.43 34,756 3.18 7 4.98 100 1,092,896 100 140.5 100.00 Source: DI's documents and FIA/AFIP financial reports Finding 24. Under Component 2, all grantees also received TA. Although all 34 interviewed beneficiary companies signed TA agreements with DI, only 26 recalled signing20 the TA agreements and receiving some type of TA (more details about TA recipients are provided in Annex VI). Out of 39 DI beneficiary and non-beneficiary companies interviewed, only one grantee and two non-beneficiary companies were not aware that DI also provides TA. Two TA recipients stated that they were not aware that DI offered a broad spectrum of different types of TA. The total value of the TA provided to the 23 grantees that signed and completed activities under their TA agreements was BAM 19,436. Of these 23, 14 grantees signed TA agreements in which the TA value was not monetarized (BAM 0.00). For the remaining eight TA agreements, the TA value ranged between BAM 960 and BAM 4,000. 20 This might be attributed to recall bias. 39 Finding 25. Out of the 39 beneficiary and non-beneficiary companies interviewed, 28 (including all 26 that recalled signing TA agreements) stated that they were aware of DI’s TA component, and 19 stated that they were familiar with the different types of TA offered by the DI. The specific TA provided to companies were noted to be based on either direct company requests (ten cases) or developed jointly by DI and beneficiary companies. As shown in Exhibit 18, survey results indicate that over 30 percent of DI beneficiaries did not receive technical assistance and that over 20 percent of them were not aware of the possibility of applying for TA. Exhibit 18. Is your company a recipient of any type of DI technical assistance? Source: MEASURE-BiH online beneficiaries’ survey Finding 26. Under Component 3, 17 non-grantees signed and completed activities under TA agreements and the total value of TA provided was BAM 71,122. The TA value for each beneficiary ranged between BAM 1,200 and BAM 13,690. Under this component, DI provided short-term business services focused on, among other things, investment facilitation, improvement of manufacturing or service processes, product development and prototyping, organizational and financial management, market research, and market strategy development (more information on the types of TA and their value is provided in Annex VI). Finding 27. The specific types of TA provided to DI beneficiaries was determined in two ways. Nine beneficiary companies (seven grantees and two TA recipients) contacted DI on their own and asked for a particular type of TA. Seven grantees and seven TA recipients were approached by DI and defined their TA needs jointly through discussion and brainstorming with DI staff. Finding 28. Both KIs and survey respondents expressed their satisfaction with the quality of TA received. Out of 34 DI beneficiary companies that received TA, 20 (11 grantees and nine TA recipients) stated that they were satisfied with the TA received. As Exhibit 19 shows, almost 90 percent of the survey respondents 18% 45% 3% 6% 6% 21% Yes, we received technical assistance, because that was what we applied for Yes, we received technical assistance after applying for the grant No, we have applied, but we have not received technical assistance No, we have not applied, although we are eligible and we are acquainted with the option of applying for technical assistance No, we have not applied, we are ineligible, but we are acquainted with the option of applying for technical assistance No, we have not applied, and we are not aware of the option to apply for technical assistance 40 who received TA were satisfied with the quality of the assistance, a result that is aligns with the findings from the KIIs. Exhibit 19. Are you satisfied with the quality of technical assistance received? Source: MEASURE-BiH online beneficiaries’ survey Finding 29. Through its activities, DI became the major organizer of networking events and activities among the diaspora community. In the 69 KIIs, 21 respondents were familiar with DI-organized conferences (Reconnect, BHDiaFor) and other DI networking events. Of those 21 interviewees, nine represented public sector institutions and donors, 11 represented DI beneficiaries (nine grantees and two TA recipients), and one was from a non-beneficiary company. Finding 30. Out of 39 interviewed beneficiary and non-beneficiary companies, 12 participated in the investment conferences (Reconnect and BHDiaFor) and in B2B events; 10 (all grantees) stated that these events were beneficial to their businesses. As Exhibit 20 shows, the majority of survey respondents had not participated in DI events. Exhibit 21 shows that those who had participated generally found them valuable. Exhibit 20. Have you attended/participated in conferences (RECONNECT, BHDiaFor), info sessions or other events organized by the Diaspora Invest project for the purpose of networking across the BiH diaspora? Source: MEASURE-BiH online beneficiaries’ survey 90% 10% 0% Yes In part No 45% 55% Yes No 41 Exhibit 21. Do you think that these conferences, info sessions, or any other events organized by Diaspora Invest, were beneficial for your business? Source: MEASURE-BiH online beneficiaries’ survey Finding 31. One of the three key subcomponents of the local platform is the DBC (one-stop shop) facility, which provides information and business support services to diaspora investors to facilitate their establishment in BiH. The one-stop shop facility was created in collaboration with a local nongovernmental organization, Naša Perspektiva, whose personnel, management, and organizational structure would provide the initial framework for the DBC. The Diaspora Business Center (one-stop-shop) was envisaged as the main institutionalization mechanism for diaspora investment facilitation and was intended to ensure sustainability of Diaspora Invest results beyond the life of the Activity. The sustainability plan and the detailed OSS business plan are the key documents that guide the localization processes of the Activity’s interventions. The fully functional and equipped OSS was established in November 2017. The OSS services include but are not limited to: assistance and advice on setting up a business, including the business registration paperwork, industry-specific consultations, advice on investment opportunities and market linkages, legal and regulatory requirements, liaison with local institutions, lawyers, and bankers. Finding 32. Currently, the activities of the DBC are predominantly focused on potential incoming investors. This shows that the potential for cooperation and synergies among DI components is being realized; however, other DBC target markets, such as local businesses interested in cooperation with diaspora investors and companies, should also be targeted. 56% 63% 38% 50% 63% 19% 25% 6% 0% 0% Yes, I received information about investment opportunities Yes, I received information about company support programs Yes, I received information about financing available to support company projects Yes, I received information about other companies' operations and about their products Yes, I exchanged contact information with other companies Yes, I established a business contact which didn’t lead to a business cooperation/commercial transaction Yes, I established a business contact which led to further discussions about potential business cooperation Yes, I established a business contact that led to business cooperation No, I had not benefitted from attendance at these events Yes, but none of the above, but (please state how you benefitted from attendance at these events) 42 It is not clear how potential investors, will be reached. Currently, outreach is being conducted through a passive approach, in which services are provided to potential investors who visit the DBC or its events. Recently the DBC has established cooperation with local authorities through memoranda of understanding (MoUs), although there is not yet clear evidence on the quality of the cooperation. It should be noted that this is a recent effort, and the cooperation has already yielded at least five referrals of diaspora companies seeking assistance. Also, there are no outreach activities to potential investors abroad. One of the differences identified between the DI and the Swiss-funded Diaspora for Development (D4D) project is that D4D also has events and outreach activities outside BiH, which is not the case with DI. Finding 33. The evaluation team could not determine any specifically tailored actions that DI was taking to reach local businesses interested in connecting with the BiH diaspora (a potential third group of DBC’s beneficiaries). This group is particularly important for the sustainability of the DBC, which, according to DBC staff, currently covers around 20 percent of its expenses by selling services to diaspora investors. Inclusion of this group could considerably expand the size of the DBC’s potential market, which would otherwise remain rather limited. The DBC can develop its profile among local businesses by facilitating their connections with diaspora businesses or investors abroad. Finding 34. The DI MEL Plan covers activities targeted to the first two groups (the new incoming investors and existing diaspora companies), 21 but there are no indicators or evidence in DI reports about efforts to target the third group (local businesses interested in connection with diaspora investors and diaspora companies abroad). This omission aligns with the statement of work, according to which the DBC is expected to provide information and services to diaspora investors and DI is expected to develop a business plan that would support the DBC’s sustainability. Finding 35. When asked whether they were familiar with the DBC, 12 out of 18 interviewed donor organizations and public sector institutions, 26 beneficiaries (19 grantees and seven TA recipients), and one non-beneficiary confirmed that they were. However, only 6. 5 percent of online survey respondents stated that they used DBC services (see Exhibit 22). Respondents were not clear about the competitive advantage of the DBC compared to lawyers or consultants who provide the same services and serve the larger market, rather than focusing on diaspora investment only. 21 According to the DI 2019 Annual Report, DBC assisted 30 potential investors (individuals) and diaspora companies. Out of 30, 14 were individuals and 16 were registered companies. No such disaggregation was provided for DBC’s beneficiaries in the DI 2018 Annual Report. 43 Exhibit 22. Did you use DBC services? Source: MEASURE-BiH online beneficiaries’ survey Finding 36. When asked whether they would be willing to pay for DBC services if the quality is high, 16 out of 34 interviewed beneficiary companies (11 grantees and 5 TA recipients) stated that they would. Only four representatives of donor and public sector institutions felt that private sector companies would pay for these services. However, respondents did not express clear consensus on whether potential diaspora investors would be willing to pay for such services, as willingness to pay would depend on the types of services, their quality and prices. Finding 37. According to some business interviewees and the subcontractor in charge of implementing the DBC, the fact that the DBC services to diaspora investors are being provided as a TA grant scheme may have a negative influence on the willingness of potential future customers to pay for services that are now being provided free of charge. A possible solution for this would be a TA arrangement in which DI would refund beneficiaries for all or part of the price of the DBC services provided. Finding 38. Although respondents felt that the provision of information to potential investors was the most valuable service of the DBC, the data in Exhibit 23 show that they expect this service to be provided free of charge. Many respondents said that they were willing to pay for other services, but the information collected through KIIs suggests that they would need to be convinced that the quality of the DBC’s services is better than that of competitors’. 6% 29% 65% Yes No, I was unaware of this Center's existence No, I had no need of the services offered by this Center 44 Source: MEASURE-BiH online beneficiaries’ survey Finding 39. DI representatives are aware that the current location of the DBC may be an important obstacle to its sustainability and growth, as one office located in Sarajevo cannot reach and cover all of BiH efficiently. The DI signed eight Memorandums of Understandings (MoUs) with municipalities in an attempt to increase its presence in the field. The KIIs with the six municipalities that signed MoUs did not suggest that DBC has managed to increase its presence in the field so far. Out of 15 interviewees representing government institutions seven felt that the DBC should be institutionalized and not managed by a nongovernmental organization. Two donor and public sector interviewees stated that the DBC should not be integrated with BiH institutions since they lack capacity to manage this type of facility. A few donor and public sector interviewees expressed the opinion that DBC should have better communication with local communities. However, KIIs offered no clear recommendations on how such institutionalization could be achieved. The responses depended on the opinion of interviewees about the format of the DBC: whether it should be a single one-stop center or a network of local offices. Those who supported a single center generally thought that the best place for such an office would be within the Foreign Investment Promotion Agency of Bosnia and Herzegovina (FIPA). Those who saw the DBC as a network thought that local offices should be connected to the departments 11% 56% 52% 26% 56% Provision of general business information Business support with investments (market research, feasibility studies, site identification support, organization of business visits, support with company registration, assistance in acquiring required licences) Business linkage services (organization of business meetings, events or visits, organization of attendance/participation in trade fairs/sector network meetings in BiH and the region) Business planning and consulting services for the diaspora (development strategies, business plans, marketing and communication plans) Business consulting services (strengthening corporate governance, product development, support in the management of transformation processes, cost-effectiveness and cost optimization, expert assistance in development of business proposals and grant applic Exhibit 23. Would your company be willing to pay for the following services? 45 for economic growth within municipal administrations and to the diaspora coordinating offices established in several municipalities through the UNDP’s Integrated Local Development Program (ILDP). Regarding the current DI efforts to sign MOUs with municipalities, few respondents expressed clear opinions about the need of the DBC to communicate more fully with local communities. Moreover, some interviewees mentioned that the DBC is probably not necessary because there is already a similar one-stop shop in their municipality. There is no evidence that DI and UNDP have coordinated their activities targeted to municipalities. Finding 40. In November 2017, in partnership with Restart, DI launched the BNP, an online forum for the diaspora that serves as a database for investment opportunities, networking events, investment conferences, and facilitating partnerships between the diaspora and local businesses. At the end of Year 1, 450 diaspora organizations, professionals, and local private sector representatives were registered on the platform and 10 potential investment projects were published by registered users with assistance from the Activity’s staff. Because of a loss of data on the Activity’s virtual private server (VPS), the Activity relaunched the upgraded BNP version in September 2018, and in December 31, 2018, reported 1,014 registered users. Finally, DI’s structured and systematic outreach and awareness-raising interventions (public campaigns in local media and media established by diaspora associations) aim to reach diaspora business communities, prospective diaspora investors and the local private sector and guide them towards services implemented under DBC and BNP. The Activity set up Facebook, Twitter, YouTube, and LinkedIn accounts, which have generated a significant following. Finding 41. When asked whether they were familiar with the DI web portal, 35 of 69 KII respondents confirmed that they were. These included 14 representatives of donors and public institutions, 19 beneficiary companies (14 grantees and five TA recipients), and two non-beneficiary companies. Exhibit 24 shows that the majority of survey respondents had used the web portal and its services. Exhibit 24. Have you used the information and services provided by the website www.diasporainvest.ba? Source: MEASURE-BiH online beneficiaries’ survey 70% 3% 27% Yes No, I was not aware of this website No, I had no need for the type of information and services offered by this website 46 The overall perception of interviewees is that DI’s and D4D’s web portals22 are complementary and not overlapping. Respondents were generally not convinced that the DI and D4D portals can be merged, since they have different purposes and audiences; respondents recommended keeping the two portals separate. The predominant opinion with regard to institutionalization of the web portals was that hosting the web portal at the Ministry of Human Rights and Refugees may be riskier than hosting and maintaining it through a nongovernmental organization due to the capacities and timeliness of work of governmental organizations. Finding 42. Regarding DI networking activities, interviewees from both government institutions and businesses agreed that these activities are important in harnessing diaspora potential for the development of BiH. Access to information and improved networking were considered by seven donor and public sector interviewees to be priority issues that should be addressed in order to ensure stronger diaspora involvement in the BiH economy. Almost all agree that the BiH diaspora is, despite recent efforts such as the establishment of a diaspora representative body in Germany, still not well connected or organized. According to interviewees, this lack of connection particularly applies to businesses in the diaspora. Most KII respondents from the business sector stated that they are not members of any diaspora organization. Out of 39 beneficiary and non-beneficiary interviewees, only 10 were members of a diaspora business network or association. Also, the majority of surveyed beneficiaries were not members of any diaspora network (Exhibit 25). This finding, which aligns with the information collected through KIIs, further emphasizes the importance of and need for increased efforts to improve diaspora networking. Exhibit 25. Are you a member of any diaspora business network? Source: MEASURE-BiH online beneficiaries’ survey 22 An interactive web portal for BiH Emigration (www.dijaspora.mhrr.gov.ba) created under the "Diaspora for Development" (D4D) project was the first interactive portal for the diaspora from Bosnia and Herzegovina. It is owned by the Ministry of Human Rights and Refugees and established for the purpose of implementing the Policy on Cooperation with Diaspora, i.e. improving communication and exchange of information between BiH emigration and Bosnia and Herzegovina. 24% 76% Yes No 47 Finding 43. Exhibit 26 shows that the majority of survey respondents saw improvements in cooperation among diaspora business networks over the last two years. DI has included some of the major diaspora events organized before the project (BHDiaFor, Reconnect) in its activities and has consequently become the major organizer of networking events and activities among the diaspora community. Therefore, improvements in diaspora business networking can be associated with DI. Still, there is a lot to be done, since the vast majority of businesses are not yet members of any diaspora network. Exhibit 26. Did you notice any improvements or changes in cooperation among diaspora business networks in the last two years? Source: MEASURE-BiH online beneficiaries’ survey Finding 44. KII respondents said that they perceived diaspora portals to be an important tool for improving networking with and among diaspora organizations. The list of more than 140 organizations that registered on the DI web portal (BNP) and its more than 4,000 visitors were seen as important indicators of the success of the portal. Also, KII respondents from 21 companies (14 grantees, 5 TA recipients, and 2 non-beneficiaries) stated that they were familiar with the BNP or the DI diaspora map. Several interviewees were not sure whether their businesses were listed on the map. Even the ones who knew they were listed said that they so far had never or almost never used the map. Interviewees perceived the web portal and the BNP as sites that have different roles, so that they do not necessarily need to be merged into one. Since the main focus of the DI diaspora map is business and the main target users are potential investors, the map should retain its specific purpose and be distinguished from other currently available web portals with information and services for the diaspora community, such as the one developed by the D4D project and the one developed by the Bosnia International Forum 23. EQ2 CONCLUSIONS According to KIIs and survey results, many grantees would have invested in BiH and hired new employees even without DI investment. However, DI grants often increased the size of these investments or allowed businesses to make them earlier than they would have otherwise. In some cases, the amounts invested by grantees from their own internal resources were substantial, dwarfing the share of the grant in the beneficiaries’ total investment. 23 https://bif.ba/index.php/bs/features-3/137-servis-za-dijasporu 82% 18% Yes No 48 DI grantees used the following sources of external financing, in order of frequency: commercial banks, BiH government institutions, and other international organizations. Analysis of the financing gap among DI beneficiaries indicates that grants added value in 24 percent of cases. Grantees with low value added by grants (in terms of their share and contribution to total grantees investment in fixed assets) contributed substantially to the fulfilment of DI targets related to new direct investments and new full-time jobs. Regarding perceptions of the value added by DI grants and TA, the evidence suggests the following conclusions:  Beneficiaries felt that DI's technical approach, comprising both grants and TA, was adequate and needed.  The vast majority of respondents were satisfied with the quality of TA provided.  After the grants, the provision of information and business networking were perceived to be the most valuable supports provided.  Although TA services were envisaged to be provided by the DBC, the majority of beneficiaries perceived the services as being provided by DI, not by the DBC. Few current DI grant beneficiaries used DBC services, and many of them were never offered the services.  The location of the DBC is perceived as its major weakness. It is yet to be seen whether networking with local authorities will help the DBC cover the market effectively. An alternative would be changing the focus of the DBC toward investors abroad, which would bring a different set of activities and partners (e.g. FIPA, MFA, MoFTER). The findings suggest that portal contents should be regularly reassessed and updated. There is still a large share of respondents who did not find the information or services they needed on the diasporainvest.ba web portal. EQ2 RECOMMENDATIONS Bearing in mind the importance of grants in USAID/BiH’s assistance to BiH, USAID should consider providing a formal definition/method of how the value added of grants should be measured in order to achieve a developmental impact. The formal definition should be provided in the design phase of any new activity. This addition would affect the design itself, the types of beneficiaries, and the expected results and contract targets. USAID should also consider, to the extent possible, applying a unique definition of grant value-added for all of its Activities to ensure a unique approach to the grant component. Such a definition would increase comparability across the Activities. The fact that about 23 percent of beneficiary companies would have invested in BiH even without DI grants suggests that the grantee selection criteria need to be revised. USAID should consider changing the grantee selection criteria and targeting only companies for which the grants will add value. In parallel, TA beneficiary selection criteria should be aligned with the new grantee selection criteria to avoid potential cases where ineligible grant applicants receive substantial financial support under TA. In turn, contractual targets will have to be revised. The anticipated revision of selection criteria will reduce the size of the 49 pool of potential DI beneficiaries, but it would thereby facilitate the desired increase in the added value of the grants. Bearing in mind that the main LOA targets under Component 2 related to the number of new formal full￾time jobs and the value of direct investments in fixed assets have been achieved, the grantee selection process could be less risk-averse. This change should ensure that the focus of the grantee selection process shifts from the amount of investments in fixed assets and the number of jobs created toward well￾structured business proposals and business models that have the potential to be profitable and sustainable. These two prominent causes of concern should be addressed jointly, with USAID/BiH adjusting the contractual grantee eligibility criteria to deemphasize the size of investment as the target (which has been already reached anyhow), and working with the IP to reflect evolving practices and align it with the actual process of evaluating grant applications and grantee selection, and harmonize it with the Small Grants Fund Manual. Current DI grant beneficiaries should be one of the main target markets for the DBC in the short run. DBC services should be extended to local businesses interested in cooperation with diaspora investors. The DBC should extend its activities beyond BiH by networking with potential investors abroad. This can be done in cooperation with FIPA, one of the likely places for the DBC in case of its institutionalization, as envisaged by the Framework Diaspora Cooperation Strategy. Also, this approach can strengthen the DBC’s position with local businesses, emphasizing its connections with businesses abroad. One of the possible measures to increase the profile of the DBC would be for USAID to consider supporting the establishment of a new DIAC (Diaspora Business Council) 24 associated with the DBC. The possibility of having the DBC assembly, or another group affiliated with the DBC, take the role of DIAC to be a channel for and initiator of policy reforms on diaspora investments should be explored. This role would strengthen the position of the DBC among its stakeholders. The advocacy interventions of this mechanism aimed to supporting diaspora investors and entrepreneurs in formulating reform recommendations to promote an enabling environment for diaspora investment could be extended over the entire duration of the Activity. To promote the sustainability of the DBC, revenues and the number of diaspora and local businesses served on a commercial basis should be included as monitoring indicators. Organization of events and outreach activities outside of BiH could be considered under a possible extension of DI or included in the second round of DI, if envisaged. In the effort to produce long-term impact, DI should consider supporting the DBC in all its activities, including provision of services to local businesses. Some of the services are already provided indirectly by connecting diaspora investors with local partners. These efforts can be included in the monitoring indicators related to supporting the sustainability of the DBC. 24 This is one of DI’s submitted measures adopted by working groups - ”1.2 Establishing the BiH Diaspora Business Council to bring together representatives of business diaspora to provide advice and input relating to policies and procedures relevant for increasing diaspora investment” 50 Since, there is still a large share of respondents who did not find the information or services they needed on the diasporainvest.ba portal, DI could consider more frequent update and reassessment of the portal contents. EVALUATION QUESTION 3 How has DI managed, implemented, monitored, and coordinated the grant component of the Activity? EQ3 FINDINGS Finding 45. DI used various channels to recruit beneficiaries. Of the 34 beneficiary companies, five were directly approached by the DI, nine learned about the existence of DI from its web portal, seven established contact with DI through DI information sessions, and six found information about DI through different channels (such as the Internet, or friends) and initiated contact with DI on their own initiative. Finding 46. The small grant program, implemented through a business plan competition, serves several purposes: providing an incentive to attract direct diaspora investment and leverage other sources of investment, offering a tool for tapping into the diaspora’s know-how, and utilizing market outreach and business networks. Design and adoption of the Small Grants Fund Manual in June 2018 enabled DI to publish four requests for applications (RFAs), two per year, to invite applications for grant funding. Exhibit 27 provides basic information on results of RFAs published so far. Exhibit 27. DI Requests for Applications RFA Publication date Applications received Grants awarded Total amount awarded (BAM) 1 August 25, 2017 54 13 386,600 2 February 26, 2018 72 10 371,250 3 August 24, 2018 30 10 358,646 4 February 27, 2019 42 To be determined* To be determined* Source: DI's documents and databases Note: Intuitiva (RFA 2), Amal (RFA 3), and Fibraworld (RFA 3) withdrew their applications. * Selection was ongoing at the time of data collection. Finding 47. DI grantees come from diverse sectors, as specified in the RFAs, though proposals from additional sectors could be considered on a case-by-case basis. More than one-third of grantees are in the manufacturing sector, 25 which is the most capital and labor intensive. The sectors of the 33 grantees are outlined in Exhibit 28. 25 The manufacturing sector was added to the list of DI’s eligible sectors by contract modification no. 6. 51 Exhibit 28. Grantees according to classification of economic activities Frequency Percent (%) Accommodation and food services activities 1 3.03 Agriculture, forestry and fishing 5 15.15 Arts, entertainment and recreation 1 3.03 Information and communication 5 15.15 Manufacturing 12 36.36 Professional, scientific and technical activities 7 21.21 Wholesale and retail trade; repair of motor vehicles and motorcycles 2 6.06 Total 33 100.0 Source: DI's documents and databases Survey respondents represented a similar range of industry sectors. According to the survey results (Exhibit 29), 29 percent of DI’s beneficiaries were in the metal processing sector and a further 9 percent were in manufacturing. The agricultural and information and communications technology (ICT) sectors each accounted for about 14 percent of DI’s beneficiaries. These results align with official data about the statistical classification of economic activities of grantees presented in Exhibit 29. Exhibit 29. DI’s beneficiaries and their sectors of economic activity, self-reported Source: MEASURE-BiH online beneficiaries’ survey The DI has a diversified portfolio of grantees, not only in terms of business sectors, but also in terms of financial results, value of fixed assets, value of business revenue, and number of employees. All companies are organized as limited liability companies. Almost half of the grantees (16) are micro enterprises with up to 5 employees, 11 grantees are small companies with up to 25 employees, and the remaining grantees are medium-sized or large companies. In 2018, out of the sample of 33 companies, 26 finished the year with financial gains: 14 reported profits in the range of BAM 10,000-100,000, and four reported profits higher than BAM 100,000. The fixed assets of 19 grantees in 2018 were worth less than BAM 100,000, and two reported fixed assets worth more than BAM 1 million. The remaining grantees (12) reported 29% 14% 3% 3% 6% 9% 14% 23% 52 fixed assets worth between BAM 100,000–BAM 1,000,000. More than 50 percent of all investments originated from four countries (Germany, USA, Sweden and Croatia). Finding 48. The grant application evaluation process is not fully harmonized with the approved Small Grants Fund Manual. The screening, evaluation, shortlisting, and scoring of small grant applications is performed by the technical team and general manager of DI. The DI Review and Evaluation Committee (REC)26 reviews applications that have been already shortlisted by the IP. The minutes from a meeting of the Small Grants Fund evaluation committee contain descriptive information about the selection process but no information about evaluation criteria ranking. In contrast, the approved Small Grants Fund Manual requires that “all business proposals will be reviewed and scored by REC.” Applicants must score a minimum of 70 points to be eligible for further consideration. In accordance with ADS 303.3.6.3, committee members consider and discuss applications in terms of the evaluation criteria defined in the RFAs. According to documents provided by DI, not all applications that were scored 70 or more points were forwarded for the REC review (Exhibit 30). In the case of the first RFA, more than 50 percent of successful applicants were not reviewed by the REC. The evaluation team, as well as the DI’s COR, have no information as to why that was the case. One potential explanation is that DI introduced several additional steps in the evaluation of received applications that are not clearly described by the Small Grants Fund Manual, such as site visits and direct negotiations with the applicants. Exhibit 30. Grant Application Results, RFAs 1, 2, and 3 RFA Publication date Applications received Applications not meeting eligibility criteria Applications scored below 70 points Applications scored 70 or more points Applications sent to REC Applications sent to the Mission for approval Grants awarded 1 August 25, 2017 54 10 10 34 15 13 13 2 February 26, 2018 72 20 35 17 13 11 10 (1 approved by the Mission and withdrew their application (Intuitiva) 3 August 24, 2018 30 4 13 13 11 13 (2 from 2nd RfA) 10 (2 approved by the Mission and withdrew their applications (Fibra and Amal) Storsen rejected. Finding 49. The DI rejected about 40 percent of all grant applications that passed the TEC’s review and received a score of more than 70 points. Also, 46 percent of surveyed non-beneficiaries stated that their application was rejected, although they think they met all of the required criteria (Exhibit 31). Of the 39 26 The core REC will be made up of the Technical Management Team (COP; DCOP; Business Advisor), USAID COR (non-voting, observing member), representative(s) of other grant fund contributor(s) and a representative from the Activity partners (Naša perspektiva, Restar, and/or others) and GM (non-voting). 53 beneficiary and non-beneficiary companies interviewed, 33 applied for a grant and nine of them were rejected (seven TA recipients and two non-beneficiaries). Source: MEASURE-BiH online beneficiaries’ and non-beneficiaries’ surveys Finding 50. Of 24 grantees interviewed, only three stated that the application process was difficult. Five grantees and two TA recipients hired external consultants to complete the application. Out of a total of 34 beneficiaries, 14 (12 grantees and two TA recipients) stated that the DI staff was helpful in preparing the application. One interviewee from a non-beneficiary company that applied for a DI grant stated that the DI staff was helpful in preparing the application. Finding 51. The grant application process and grant awarding decision were perceived to be transparent. According to the results of the online beneficiaries’ survey, 88 percent of respondents were of the opinion that grant application and grant awarding decision processes were transparent. None of them assessed it as non-transparent, though 13% stated that they did not know. Asked whether they were familiar with the method and criteria for scoring the grant application, 77 percent of grantees said yes, 16 percent said no, and 6 percent said they did not know. The criteria for scoring the grant application were evaluated as adequate by 65 percent of respondents. None stated that the criteria were inadequate; 35 percent said they did not know. Asked whether they had received feedback on the number of points and grade assigned to their application, 45 percent said yes, 32 percent said no, and 23 percent stated they did not know (Exhibit 32). In KIIs, 19 of 24 grantees stated that they were satisfied with the grant application and implementation processes. 79% 15% 3% 3% Yes No, we applied, but we did not receive a grant No, we never applied, but we are aware of the opportunity to apply for a Diaspora Invest grant No, we never applied, and we have not been aware of the opportunity to apply for a Diaspora Invest grant 46% 8% 13% 33% Yes, we applied and although we met the criteria, we did not receive a grant Yes, we applied, but we were informed that we do not meet the criteria (years of existance, share in ownership by diaspora member, etc.) No, we never applied, although we were aware of DI grant opportunities No, we never applied and were not aware of DI grant opportunities Exhibit 31. Grant approval and rejection rates 54 Exhibit 32. Survey results about transparency of grant application process YES NO I don’t know Grant application process and grant awarding decision was transparent 88% 0% 13% Applicants were familiarized with application scoring method and scoring criteria 77% 16% 6% Application scoring and scoring criteria were adequate 65% 0% 35% Applicants received information (feedback) about score their application received and overall assessment of their application 45% 32% 23% Source: MEASURE-BiH online beneficiaries’ survey Finding 52. Grant amounts can differ considerably in the grant application, grant agreement, and closeout report for the same beneficiary (Exhibit 33). During the application process, the IP technical team visited each applicant. These site visits were not a standard procedure during the first RFA and they became regular during the second RFA. Based on their assessment, in most cases, they negotiated reduced grant amounts. There are no written procedures on how to conduct due diligence or negotiate grant amounts. Of the 24 DI grantees interviewed, 13 stated that the grant amount received was different from the amount requested in the grant application. One interviewed grantee that withdrew after its grant application was approved said that it had been asked to change the grant amount requested during the application process. Also, four beneficiary companies (three grantees and one TA recipient) were asked during the grant evaluation process to change the grant purpose. 55 Exhibit 33. Grant approval process Grant app Assistance overview form Grant agreement Close Out Company Grant value requested (BAM) In-kind contribution (BAM) Employees (women) Employees to hire (women) DI assistance contribution (BAM) Diaspora Investment (BAM) Job created Grant amount (BAM) Diaspora Investme nt (BAM) Employees to hire Used grant amount (BAM) Hired employees (women) ADV PAX 45,000.00 1,000,000.00 26 (1) 10 (7) 45,000.00 1,173,170.00 26+10 40,000.00 89,000.00 10 (min 5) 21,683.00 26 (12) BMT Engineering 50,000.00 27,998.00 8 (1) 4 (0) 25,000.00 620,000.00 9+4 25,000.00 49,090.00 4 25,000.00 5 Code Line Solution 60,000.00 102,970.00 9 (2) 3 12,600.00 102,970.00 11+3 12,600.00 17,500.00 1 12,000.00 1 FANA Arifagic 50,000.00 118,000.00 7 (0) 7 (2) 35,000.00 118,000.00 7+7 35,000.00 178,530.00 7 35,000.00 3 FREUND Elektronika 50,000.00 408,600.00 6 (1) 9 (3) 40,000.00 408,600.00 6+9 40,000.00 361,600.00 3 31,164.00 1 Master Inzinjering 49,760.00 101,000.00 2 (1) 2 (0) 15,800.00 101,000.00 2+2 15,800.00 27,580.00 1 15,800.00 1 Movida 50,000.00 100,000.00 2 (1) 16 (13) 20,000.00 150,000.00 2+9 20,000.00 121,940.00 3 14,334.00 8 Probosing 28,619.00 29,052.11 6 (3) 2 (1) 23,200.00 29,052.00 6+2 23,200.00 38,360.00 1 21,697.00 1 Lumitic 54,350.00 232,550.00 0 8 (3) 30,000.00 232,550.00 0+8 30,000.00 116,380.00 4 30,000.00 4 Sitex 40,000.00 150,700.00 5 (3) 7 (5) 30,000.00 150,700.00 5+7 30,000.00 77,900.00 8 19,822.00 7 Suco 50,000.00 64,000.00 14 (1) 50 (3) 40,000.00 74,000.00 15+40 40,000.00 66,900.00 10 35,000.00 11 Talbot (Zendev) 65,000.00 240,000.00 8 (1) 10 (5) 35,000.00 274,000.00 8+10 35,000.00 129,000.00 5 35.000.00 5 Te Ora 0 40,000.00 206,150.00 0+13 40,000.00 207,000.00 4 40,000.00 4 Bonatura 42,260.00 44,150.00 1 (0) 2 (1) 34,000.00 44,150.00 1+2 34,000.00 34,026.00 2 34,000.00 2 Delta Plus BH 50,000.00 197,572.00 36 (2) 6 (0) 45,000.00 147,660.00 36+6 45,000.00 166,540.00 8 43,940.00 19 Ekonomik Group 50,000.00 183,650.00 3 (0) 20 (1) 40,000.00 132,550.00 3+10 40,000.00 112,500.00 10 FITS doo 19,250.00 19,260.00 6 (4) 2 (1) 19,250.00 19,260.00 6+2 19,250.00 22,110.00 2 19,250.00 2 Galop Digital doo 50,000.00 40,000.00 2 (1) 4 (2) 28,000.00 50,000.00 2+4 28,000.00 46,055.00 3 H2I Balkans doo 50,000.00 80,400.00 2 (2) 2 (2) 35,000.00 80,400.00 2+2 35,000.00 56,378.00 1 KMC doo 41,700.00 100,000.00 0 15 (4) 30,000.00 410,000.00 0+15 30,000.00 159,800.00 15 MRD Engineering doo 50,000.00 128,050.00 10 (0) 15 (3) 45,000.00 133,050.00 10+5 45,000.00 102,000.00 5 SB Laser 50,000.00 2,081,170.00 0 5 (1) 50,000.00 2,081,170.00 0+5 50,000.00 163,000.00 5 56 Grant app Assistance overview form Grant agreement Close Out Company Grant value requested (BAM) In-kind contribution (BAM) Employees (women) Employees to hire (women) DI assistance contribution (BAM) Diaspora Investment (BAM) Job created Grant amount (BAM) Diaspora Investme nt (BAM) Employees to hire Used grant amount (BAM) Hired employees (women) Agroplus 124,000.00 50,000.00 4 (1) 2 (1) 50,000.00 74,000.00 4+2 50,000.00 74,000.00 2 BioFood BH doo 42,620.00 277,040.00 3 (2) 5 (3) 41,306.00 277,040.00 3+5 41,306.00 277,040.00 5 Eko Jasmina 26,620.00 26,620.00 1 (1) 1 (0) 26,620.00 86,620.00 1+1 26,620.00 86,620.00 1 Empress 31,200.00 112,390.00 15 (13) 10 (10) 30,500.00 74,770.00 15+5 30,500.00 74,770.00 5 Fam Farms 48,055.00 97,055.00 4 (1) 2 (1) 48,060.00 57,400.00 4+2 48,060.00 57,405.00 2 Octopus 50,000.00 560,000.00 0 9 (5) 50,000.00 509,900.00 5 50,000.00 512,500.00 5 Propeller 10,200.00 52,540.00 9 (4) 4 (2) 10,020.00 52,540.00 9+4 10,020.00 52,540.00 4 Supersoft doo 48,840.00 347,910.00 9 (8) 3 (3) 44,590.00 124,070.00 9+3 44,590.00 124,070.00 3 TNT Soft 10,000.00 5,000.00 8 (0) 8 (0) 8,250.00 13,070.00 8+4 8,250.00 13,070.00 4 8,250.00 4 WBR Zona 49,500.00 164,300.00 6 (1) 3 (2) 49,300.00 177,970.00 6+2 49,300.00 177,970.00 2 Amal 13,250.00 26,550.00 1 (0) 1 (1) 13,250.00 22,800.00 1+1 13,250.00 23,600.00 1 CNC Lab 45,000.00 216.850,00 6 45,000.00 223.053,00 6 Source: DI's documents and databases 57 Finding 53. Three grantees confirmed in interviews that DI, in addition to financing the purchase of fixed assets, also accepted payments for salaries, rents, and similar ongoing expenses as eligible costs. These grants were awarded under RFA 1, which did not include specific requirements regarding the costs for which the grant could be used. This was supported by survey results (Exhibit 34) where 19 percent of beneficiary respondents stated they used grant funds for other purposes, such as the purchase of inventories, licenses, ISO certification and knowledge transfer, license purchase for software and equipment. Exhibit 34. Grant purpose Source: MEASURE-BiH online beneficiaries’ survey Finding 54. The DI supported many start-up companies, either by grants or with technical assistance. Of 33 companies that received grants, the FIA/APIF data on 2017 financial statements (balance sheets and income statements) were available for 23 companies. For 2018, we have financial statements for all 33 grantees. The difference in available financial statements in 2017 and 2018, implies that seven new companies were established in 2018. Analysis of DI documents confirmed that in 2018, seven DI grantees and four TA recipients registered their companies. Only one company was registered in 2019 (TA recipient). Finding 55. Beneficiaries were asked whether the maximum grant amount of BAM 50,000 offered by DI was sufficient to cover business development needs. Exhibit 35 shows that most respondents stated that BAM 50,000 could only partially cover their business development needs. Only 18 percent of respondents thought that it was sufficient. Exhibit 35. Were grant funds sufficient to cover your business development needs? Source: MEASURE-BiH online beneficiaries’ survey 81% 0% 0% 19% Procurement of equipment Investment in plant facilities Salaries and other employee outlays Other 18% 61% 21% Yes In part No 58 Finding 56. DI representatives noted that the duration of the grant approval process sometimes interfered with applicants’ business activities. The time from the application deadline to the grant award was four months for RFA 1, nearly five months for RFA 2, and three months for RFA 3. According to DI, there are situations where businesses cannot wait, or cannot put on hold their business activities until DI completes the grant approval process. For example, a company might plan to purchase a machine and it would apply for a grant. In the meantime, while the approval process was ongoing, they were forced to buy that machine. Then DI would negotiate with the applicant a new purpose of grant, and try to identify another piece of equipment and redefine milestones, if needed. Finding 57. When it comes to coordination of the grant component of the Activity, DI regularly exchanged information about grant applicants and grantees with the FARMA II, WHAM and D4D projects in order to avoid overlaps. However, the survey results, as well as interviews with other international organizations, indicate that there were some cases of overlap in provision of grants to companies. The D4D project representative stated that recently, there was one case where both projects provided grants to the same company. EQ3 CONCLUSIONS Of the three DI RFAs issued during the period evaluated, the first did not specify that the funding was designated for purchase of fixed assets. Beneficiary KIIs revealed that grant funds were used for other purposes, such as financing of salary expenses. The failure of RFA 1 to specify the grant purpose was an omission that resulted in grant money being spent on costs other than fixed assets. Having identified this weakness, DI has specified the allowable uses of grant funds in RFAs 2 and 3. The grant application evaluation process does not seem to be fully harmonized with the approved Small Grants Fund Manual. The REC only reviews applications shortlisted by DI implementers, and USAID/BiH’s involvement in the selection process starts with the REC. More information about the evaluation process between the TEC and REC reviews could provide better insight into the grantees’ selection. The grantees were generally satisfied with the way DI implemented the grant scheme and with the transparency of the grant application process. However, DI could do more in informing the applicants about outcomes of the applications evaluation. EQ3 RECOMMENDATIONS USAID should request that RFAs in all cases clearly describe allowable uses of grants. RFAs 2 and 3 clearly defined the requirement that the grant be used only to finance fixed assets and USAID should keep the same grant purpose as in the second and third requests for grant applications. USAID should consider harmonizing the Small Grant Fund Manual to reflect evolving practices and align it with the actual process of evaluating grant applications and grantee selection. 59 EVALUATION QUESTION 4 How do public sector partners/stakeholders (at different government tiers) perceive and value the DI’s assistance related to removing the institutional and policy impediments to diaspora investments, crafting reform proposals and their implementation? How do DI’s interventions fit within the broader diaspora policy efforts? EQ4 FINDINGS Finding 58. The DI supported diaspora investors and entrepreneurs in formulating policy recommendations. This support took the form of a survey of about 200 diaspora companies in BiH of obstacles to diaspora investments, through the establishment of the Diaspora Investors Advisory Council - DIAC (USAID decided to discontinue this intervention after one meeting), and continued assistance to relevant institutions through support to working groups for development of the Framework Diaspora Cooperation Strategy. Finding 59. Companies that participated in the DI survey intended to identify the most important obstacles to stronger diaspora involvement in the economic life of BiH were not sure about the purpose of the survey and out of 34 interviewed beneficiaries, eight was familiar with the policy recommendations derived from that survey. When conducting the survey of diaspora investors, the DI sent out the questionnaire to 197 companies and received back 87 completed questionnaires. According to KIIs, 20 interviewed beneficiary companies (14 grantees and six TA recipients) confirmed that they participated in the DI survey, but most stated they were not sure about the purpose of the survey. Finding 60. Of the 18 donors and public sector institutions interviewed, nine affirmed that working groups received high-quality support and inputs from DI consultants. However, only three interviewees were familiar with the process of formulating recommendations proposed by the DI for designing the Framework Diaspora Cooperation Strategy. Finding 61. Of 39 interviewed companies (beneficiaries and non-beneficiaries), 32 (21 grantees, six TA recipients, and five non-beneficiaries) agreed that there are institutional and regulatory barriers to stronger diaspora involvement in the BiH economy. The main obstacles emphasized by investors are the lack of information on investment opportunities, complicated administrative procedures and related lack of information and guidance, together with the shortage of skilled labor. Only five companies stated they noticed substantial changes in policies or institutions aimed at enhancement of the enabling business environment for diaspora investment in BiH. 60 The survey findings, shown in Exhibit 36, are aligned with the information collected through interviews: 14 percent of respondents perceived specific barriers to diaspora investors. Source: MEASURE-BiH online beneficiaries’ survey Finding 62. According to the survey of diaspora investors conducted by the DI, the major obstacles for diaspora investments include high tax burden, political instability, legal and regulatory requirements (procedures), corruption, underdeveloped infrastructure, and the shortage of skilled labor. The World Bank Group Gender MSMEs Access to Finance survey also identified high taxes, tax administration and corruption as some of the major enabling business environment issues27 (Exhibit 37). Given the scope of the DI Activity, none of these can be directly tackled by the DI alone. Other important obstacles mentioned in the survey, such as the lack of information about procedures, lack of available financing, and lack of reliable investment partners, fall within the DI’s scope. However, any success in removing these obstacles can have only limited impact on attracting diaspora investments when the major obstacles are outside the Activity’s scope. Some policy measures proposed by DI were discussed but rejected, 28 due to possible introduction of preferential treatment for diaspora investors, which is strictly prohibited by BiH laws. 27 “MSMEs view the government situation as the major business obstacle, especially when it comes to tax rates, tax administration, and corruption. Other issues were raised about cost of finance and crime, theft, and disorder. As illustrated in Figures 43 and 44, 43–61 percent of surveyed enterprises complained about tax rates, followed by 37–59 percent about tax administration, and 31–54 percent corruption. A second tier of other governmental issues were assessed in terms of political environment (31–41 percent), labor regulations (29–37 percent), and customs and trade regulations (23–38 percent). Regarding financing, cost of finance was seen as the strongest impediment (27–46 percent) and for the external environment, crime, theft, and disorder were at the forefront with 28–46 percent, followed by macroeconomic with 31–44 percent.”; The World Bank Group Gender MSME Access to Finance Survey; BiH 2018. 28 “Measure dropped by the working group members: (i) Remove general business environment obstacles and speed up structural reforms - there’s a whole plethora of different obstacles which we would like to narrow down with the help of the survey of diaspora investor; and ii) Address present obstacles relating to citizenship and property rights – key measures include introducing flexible economic citizenship/golden visa for diaspora investors above a certain threshold;” Source: Annual Summary Report For the Period April 2, 2018 – April 3, 2019; USAID Diaspora Invest Activity; May 2, 2019. 14% 86% Yes No Exhibit 36. Are diaspora investors facing specific challenges/barriers compared to other investors (domestic and foreign)? 61 Exhibit 37. Business enabling environment obstacles – WB Micro Small Medium Lowering of taxes 54% 55% 60% Availability of cheaper financing 28% 30% 15% Simplification of regulation 10% 8% 9% Source: The World Bank Group Gender MSME Access to Finance Survey; BiH 2018 According to the results of the DI beneficiaries’ survey, as shown in Exhibit 38, 68 percent and 52 percent of respondents, respectively, found administrative and regulatory barriers and the absence of institutional support for diaspora investors to be the major obstacles to diaspora investments. Exhibit 38. What are the greatest challenges for your company in terms of legal, regulatory, and institutional framework, or current policies? Source: MEASURE-BiH online beneficiaries’ survey Finding 63. The lack of information about investment opportunities was not mentioned as one of the major obstacles for diaspora investments (DI survey). This finding may call into question the proposal to establish a one-stop shop to provide information about investment opportunities, as specified in the materials for the diaspora strategy working groups. Still, the one-stop shop can serve other purposes that were identified as important to investors, such as the provision of information about legal procedures. Finding 64. In KIIs, only four donors and public sector interviewees and six DI beneficiaries (five grantees and one TA recipient) had heard about DIAC. This finding is understandable, given that DIAC, as an advocacy mechanism for channeling diaspora investors’ reform proposals to the BiH government, existed only briefly. The respondents who had participated in DIAC did not have clear information about why its activities ceased. 45% 35% 68% 52% 10% Unfavorable economic situation in BiH Lack of reliable information on conditions for doing business in BiH Administrative and regulatory obstacles (a complicated system) Absence of institutional support for diaspora investors Other 62 Finding 65. Two government institutions clearly expressed hope that the Framework Diaspora Cooperation Strategy, as it is currently structured without RS, would be accepted at the BiH level, although they expressed concern that RS could block it. The timing of the strategy’s adoption is unpredictable. Finding 66. The overall perception of interviewees is that DI and other projects (such as D4D) 29 are complementary and not overlapping. EQ 4 CONCLUSIONS The DI development hypothesis states that more diaspora investments will be attracted if obstacles to the business-enabling environment are reduced. The reduction of obstacles was not addressed in the policy recommendations included in the Framework Diaspora Cooperation Strategy. The list of policy recommendations is of secondary importance, according to the survey of diaspora investors conducted during the process of their development. The most pressing issues are in the domain of enabling business environment and are outside the scope of this project. EQ4 RECOMMENDATIONS USAID should consider reformulating the development hypothesis to reflect the scope of DI by specifying that the Activity will address support services and not barriers. The project’s development hypothesis should be rephrased to read that “more diaspora investments will be attracted if support services are provided” — and not “if business enabling environment obstacles are reduced.” The only issues the DI can tackle are support activities that can make the investment process less burdensome, such as provision of information and assistance in obtaining documents or licenses, as well as support in networking. These support services can be provided by the DBC. USAID should consider continuing relevant policy activities by providing ongoing support to diaspora investors and entrepreneurs in formulating reform recommendations for promoting an enabling environment for diaspora investment. This type of support could be offered continuously for the entire duration of the Activity. 29 The Ministry of Human Rights and Refugees of Bosnia and Herzegovina (MHRR), the Swiss Government and the United Nations Development Programme in Bosnia and Herzegovina (UNDP BiH) implemented a pilot project for policy development and mainstreaming the concept of migration and development into development plans. Through the project, the Government of Switzerland is supporting BiH to create an enabling environment for the inclusion of the human and financial capital of the diaspora. 63 ANNEXES 64 ANNEX I: DOCUMENTS REVIEWED 1. Activity Contract AID-168-C-17-00001, April 11, 2016 2. The Activity’s Contract modification no. 7: AID-169-C-17-00001/07, March 24, 2017 3. Diaspora Invest Year I Annual Report 4. Diaspora Invest Year II Annual Report 5. USAID Diaspora Invest – Year I First Quarterly Report 6. USAID Diaspora Invest – Year I Second Quarterly Report 7. USAID Diaspora Invest – Year I Third Quarterly Report 8. USAID Diaspora Invest – Year I Fourth Quarterly Report 9. USAID Diaspora Invest – Year II First Quarterly Report 10. USAID Diaspora Invest – Year II Second Quarterly Report 11. USAID Diaspora Invest – Year II Third Quarterly Report 12. USAID Diaspora Invest – Year II Fourth Quarterly Report 13. USAID Diaspora Invest – Year I Work Plan 14. USAID Diaspora Invest – Year II Work Plan 15. USAID Diaspora Invest – Year III Work Plan 16. FIA/AFIP financial reports 17. USAID Diaspora Invest Monitoring and Evaluation Plan 18. USAID Diaspora Invest Sustainability Plan 19. USAID Diaspora Invest Outreach and communications report 20. Assessment of Diaspora Investment in Bosnia and Herzegovina 21. Diaspora Investors Advisory Council (DIAC) Concept Note 22. Diaspora Investors Advisory Council (DIAC) – Founding meeting 23. Documents from Working Group sessions (participant lists, meeting minutes, proposals of strategy text etc.) 24. Draft of the economic development within the Framework Strategy on Cooperation within Diaspora 25. Situational analysis 26. Questionnaires for diaspora survey 27. Survey of diaspora investors: overview of results 28. Diaspora Invest List of all diaspora owned/managed companies surveyed 29. Small grants fund manual (draft) – Bosnian Diaspora Marketplace 30. Diaspora Invest Requests for Applications 1, 2, 3, 4 31. Diaspora Invest Grant Applications 32. USAID Approvals of the Grants under Contracts applications 33. Invitation letter to BDM REC 34. Review of shortlisted small grant applications – Small Grant Fund Review and Evaluation Committee 35. Diaspora Invest Grant Activity Overview Forms 36. Diaspora Invest environmental review checklists for identifying potential environmental impacts of project activities and processes 37. Diaspora Invest Grant Agreements 65 38. Diaspora Invest Grant Close Outs 39. Attendance Sheets from B2B Events 40. Diaspora Invest Technical Assistance tracking forms 41. Diaspora Invest Public Call for Technical Assistance Registration 42. Diaspora Invest Technical Assistance Requests 43. Diaspora Invest Technical Assistance Agreements 44. Diaspora Invest Technical Assistance Final Reports 45. Memorandum of Cooperation signed with Bosanska Krupa, Bosanski Petrovac, East Ilidza, Municipality of Buzim, Municipality of Kljuc, Municipality of Maglaj and Sanski Most 46. Diaspora Business Center Business Plan for the period 2018- 2021 47. Diaspora Invest Technical Assistance guidelines 48. Registration and Financial Statements – SME beneficiaries 66 ANNEX II: LIST OF KEY INFORMANTS Key informant Type of key informant Date 1 USAID USAID 4/23/2019 2 DI team Implementers 4/24/2019 3 H2I Balkans Grantee 4/25/2019 4 Freund Grantee 5/6/2019 5 PI Consulting Implementers 5/6/2019 6 Nasa Perspektiva Implementers 5/7/2019 7 Orea Bazaar Non-grantee 5/7/2019 8 Everest Consulting Non-grantee 5/7/2019 9 MDG International Non-grantee 5/7/2019 10 Factoring Insolation Technical Solutions (FITS) Grantee 5/8/2019 11 Te Ora Grantee 5/8/2019 12 Silicon Constellations (Lumitic) Grantee 5/8/2019 13 Symphony d.o.o. Sarajevo Non-beneficiary 5/9/2019 14 Intuitiva Non-beneficiary 5/9/2019 15 Movida Grantee 5/10/2019 16 DI team Implementers 5/10/2019 17 Restart Implementers 5/10/2019 18 Kenn Art Non-grantee 5/10/2019 19 Storsen Non-grantee 5/10/2019 20 EKO Jasmina Grantee 5/13/2019 21 Supersoft Grantee 5/13/2019 22 B.M.T. Engineering Grantee 5/13/2019 23 Amal Grantee 5/14/2019 24 Talbot (Zendev) Grantee 5/14/2019 25 Transform Art Non-grantee 5/15/2019 26 State Level Working Group Working groups 5/15/2019 27 Fam Farms Grantee 5/16/2019 28 Octopus Grantee 5/16/2019 29 Lopare MOU 5/16/2019 30 Brcko District Working group Working groups 5/16/2019 31 Probosing Grantee 5/17/2019 32 Suco Grantee 5/17/2019 33 FARMA II International Organizations 5/20/2019 34 Softhouse Balkans Non-beneficiary 5/20/2019 67 Key informant Type of key informant Date 35 FBIH Working group Working groups 5/20/2019 36 BiH Central Bank Public stakeholder 5/21/2019 37 BiH Ministry of Foreign Affairs Public stakeholder 5/21/2019 38 BiH Ministry of Foreign Trade and Economic Relations Public stakeholder 5/21/2019 39 Foreign Investment Promotion Agency Public stakeholder 5/21/2019 40 EIA International organizations 5/22/2019 41 IOM International organizations 5/22/2019 42 Netherlands Embassy - The Dutch Good Growth Fund International organizations 5/22/2019 43 SDC International organizations 5/23/2019 44 UNDP International organizations 5/23/2019 45 East Ilidza MOU 5/23/2019 46 HUB 387 Non-beneficiary 5/23/2019 47 Code Line Solutions Grantee 5/24/2019 48 Propeller doo Grantee 5/24/2019 49 Industrial Non-Grantee 5/24/2019 50 Master Inzinjering Grantee 5/28/2019 51 DVC Solutions Non-beneficiary 5/28/2019 52 Sekretarijat RS Public stakeholder 5/28/2019 53 MRD Engineering Grantee 5/29/2019 54 Fana – Arifagic Grantee 5/29/2019 55 SB Laser Grantee 5/29/2019 56 AgroPlus Grantee 5/30/2019 57 BK Oaza Non-beneficiary 5/30/2019 58 Edna Metalworking Non-grantee 5/30/2019 59 WHAM International organizations 5/31/2019 60 DI team Implementers 6/7/2019 61 Directorate for Economic Planning Public stakeholder 6/7/2019 62 TNT Soft Grantee 6/12/2019 63 Bosanska Krupa MOU 6/12/2019 64 AKM Technology Non-grantee 6/12/2019 65 Delta Plus Grantee 6/13/2019 66 Bosanski Petrovac MOU 6/13/2019 67 HIK-SM Non-grantee 6/13/2019 68 Kljuc MOU 6/14/2019 69 Sanski Most MOU 6/14/2019 68 ANNEX III: EVALUATION STATEMENT OF WORK EVALUATION PURPOSE The purpose of the performance evaluation of the USAID/BiH Diaspora Invest Activity (DI) is to assess the progress towards achievement of the Activity’s contractual obligations to date and to provide recommendations for adjustments to the Activity, if needed. The primary audience is USAID/BiH, who will use the evaluation findings, conclusions, and recommendations to assess the achievements of DI and adjust the specific components of the Activity, such as the grant component, if needed. A variety of stakeholder groups will play a significant role in this evaluation process; stakeholders from USAID/BiH will provide insights into the Activity design and implementation, and DI implementers will provide their own insights into the design and implementation of specific assistance/support components. Finally, the stakeholders/beneficiaries of implemented activities and relevant public/government institutions will provide information on their perceptions of the design and implementation of the Activities’ interventions/assistance. In addition, we will gather data from other international organizations/donors relevant to diaspora involvement in BiH. Finally, non-beneficiary, diaspora-owned companies will be interviewed/surveyed to provide a perspective on BiH diaspora investment from a group that did not benefit from DI assistance. Findings, conclusions, and recommendations of this evaluation will contribute to knowledge and learning of several groups of stakeholders: I USAID/BiH will be able to assess the role of DI in increasing direct diaspora investment and creation of new job opportunities. The evaluation will help USAID/BiH achieve a better understanding of the Activity’s implementation and lessons learned, which can inform adjustments of current interventions if needed, as well as future funding and programming decisions that involve combination of grants and technical assistance. II DI implementers will learn about their strengths and potential areas for improvement. III Other stakeholders, such as BiH governing institutions and the public, may also benefit from USAID’s contribution to public knowledge of development efforts to engage the diaspora in BiH economic development. EVALUATION DESIGN In this Evaluation Work Plan, we described the MEASURE team’s approach to the implementation of this performance evaluation. We described the evaluation’s key questions, explained the evaluation methodology, and presented our detailed work plan and timeline. The MEASURE team will assess the implementation of the DI and the extent to which the Activity’s contract targets have been achieved to date. Specifically, the MEASURE team will:  Verify and provide detailed information on DI’s results for the set of indicators listed under the evaluation question #1. The evaluation team will also review the Activity’s data collection and analysis methods used to track progress and report results. 69  Pay particular attention to the grant component of the Activity and TA and to networking assistance provided to beneficiaries through the Activity’s service delivery channels. Regarding the grant assistance, the evaluation team will thoroughly review how the Activity manages and monitors the grant component. We will examine transparency of the grant award process and whether this process was in compliance with the grant manual. The evaluation will also provide information and assess grants processing, monitoring of implementation of grant agreements, grant monitoring tools, monitoring of beneficiaries’ grant expenditures, verification of fulfillment of beneficiaries’ contributions, etc. The evaluation team will also explore whether grants helped or facilitated beneficiaries’ own investments, and, based on beneficiaries’ perceptions, compare the added value of grants and other types of assistance provided by the Activity to beneficiaries’ business operations. The evaluation team will also explore whether there are potential overlaps between interventions and assistance provided by DI and other donors and activities / projects. For example, the evaluation team will determine whether the DI and Diaspora for Development (D4D) Project financed by the Swiss Development Cooperation (SDC) and implemented by the United Nations Development Programme (UNDP) work with the same beneficiaries.  Regarding analysis of technical assistance provided by the DI, the evaluation team will explore transparency of beneficiaries’ selection and transparency of the DI’s selection and contracting processes in terms of hiring external technical assistance providers. Also, the evaluation team will, based on beneficiaries’ perception, analyze added value of the business to business (B2B) and networking services provided by DI. The evaluation team will make an effort to identify examples of successful B2B and networking assistance and provide detailed description of these cases.  Explore how public sector stakeholders perceive the DI’s assistance and how DI’s interventions fit within the broader diaspora policy efforts. This evaluation team will ask KIs from the Framework Diaspora Cooperation Strategy development working groups (WG) about the DI’s contribution to crafting reform proposals and recommendations that will become integral part of the Framework Diaspora Cooperation Strategy. To do this, we will conduct a review of relevant Activity documentation and policy documents. We will also conduct a variety of primary data collection tasks, including key informant interviews (KIIs) with Activity beneficiaries and stakeholders to understand their experiences with DI and the Activity’s perceived results. This performance evaluation will also conduct a web survey of beneficiaries and a sample of non-beneficiaries to assess progress in DI implementation and results to date. DATA COLLECTION AND ANALYSIS DATA COLLECTION The Evaluation Team’s proposed methodology will ensure systematic and efficient data collection from the following sources to answer each of the evaluation questions:  DI design and implementation documentation and databases, including award and award modification, work plans, quarterly reports, annual reports30, MEL documentation, the Activity’s 30 (i) The Activity’s Contract: Contract number AID-168-C-17-00001; April 11, 2016. (ii) The Activity’s Contract modification no. 7: AID-169-C-17-00001/07; March 24, 2017. (iii)Monitoring and Evaluation Plan for the period April 2017 – September 70 different records and documentation resulting from the DI’s direct surveying of different types of beneficiaries, and deliverables within the scope of DI’s work with the public sector.  Secondary data and documentation relevant to DI, such as FIA/APIF data on financial statements, Activity databases, documentation from international organizations, as well as documentation from relevant BiH government/public institutions.  76 semi-structured key informant interviews (KIIs) with USAID/BiH staff, implementers, public sector stakeholders, beneficiaries, non-beneficiaries and other donors / international organizations. Speaking with non-beneficiaries will allow the MEASURE team to compare perceived results against an absence of treatment.  Online survey of DI beneficiaries.  Mini online survey of a sample of non-beneficiaries. In selecting KIs, we will pay particular attention to ensuring geographic diversity and collecting sufficient data across entities (please see exhibit 3). The semi-structured KIIs will be audio recorded and transcribed. This will support a thematic analysis for each evaluation question and coding categories will be applied when reviewing the interview transcripts. The qualitative analysis of interview transcripts will follow a process of consolidating multiple responses related to a similar theme that are mentioned by different categories of respondents and analyzing them for general findings. In this manner, we will be able to identify common themes. 2022; July 17, 2017 and the M&E plan revisions, (ii) Annual Work Plan for the period April 3, 2017 – April 2, 2018; June 2, 2017; (iii) Annual Work Plan for the period April 3, 2018 – April 2, 2019; March 9, 2018. (iv) Quarterly Performance Report for the period April 3 – June 30, 2017; July 17, 2017. (v) Quarterly Performance Report for the period July 1 – September 30, 2017; October 13, 2017. (vi) Quarterly Performance Report for the period October 1 – December 31; January 15, 2018; (vii) Quarterly Performance Report for the period January 1 – March 31, 2018; April 16, 2028; (viii) Quarterly Performance Report for the period April 1 – June 30, 2018; July 16, 2018. (ix) Quarterly Performance Report for the period July 1 – September 30, 2018; October 15, 2018, etc. 71 Exhibit 3. Geographic Distribution of the DI’s Beneficiaries To answer each research question, we will analyze each data source independently and then compare our findings across sources to triangulate information. The relationship between these data sources, methods, and research questions is illustrated in Exhibit 4. 72 Exhibit 4. Evaluation Matrix EVALUATION QUESTIONS DATA SOURCES DATA COLLECTION INSTRUMENT METHODOLOGY 1. What progress has been achieved to date in reaching contract targets (new full-time registered jobs, value of direct investments, number of firms receiving DI’s funded technical assistance for improving business performance, percentage of female participants in DI’s assisted programs to increase access to productive economic resources, percent of policy recommendations generated through the structured dialog with diaspora investors adopted by the BiH relevant institutions, number of diaspora eligible early stage SMEs and start-ups that graduated from BDM, number of the BiH diaspora organizations / business networks registered on the diaspora map, number of potential diaspora investors who used information and services provided by the Activity/ One-Stop-Shop)? DI design and implementation documentation/databases review and secondary documentation from international organizations/donors and BiH authorities FIA/APIF data on financial statements Key informant interviews with USAID/BiH staff; DI implementers and partners and OSS staff; DI’s beneficiaries (grantees, TA recipients; DI public sector stakeholders; non-beneficiary SMEs; and other donors/international organizations. Results of the online survey of DI beneficiaries Results of the mini online survey of a sample of non-beneficiaries Desk review Desk review Key informant interview protocol Online survey questionnaire Online survey questionnaire Mixed Methods Triangulation 2. How has the DI’s technical approach been implemented and how is it perceived by beneficiaries in terms of added value of grants and different forms of technical assistance? This includes technical assistance and business networking / linkages through the Diaspora Business Networking Platform (BNP) and the Diaspora Business Center (DBC) / One-Stop-Shop. How has DI contributed to improved cooperation among diaspora business networks? DI design and implementation documentation/databases review and secondary documentation from international organizations/donors and BiH authorities Key informant interviews with USAID/BiH staff; DI implementers and partners and OSS staff; DI’s beneficiaries (grantees, TA recipients; DI public sector stakeholders; non-beneficiary SMEs); and other donors/international organizations. Results of the online survey of DI beneficiaries Desk review Key informant interview protocol Online survey questionnaire Mixed Methods Triangulation 73 3. How has the DI’s managed, implemented, monitored and coordinated the grant component of the Activity? DI design and implementation documentation/databases review and secondary documentation from international organizations/donors and BiH authorities Key informant interviews with USAID/BiH staff; DI implementers and partners; DI’s beneficiaries (grantees); other donors and international organizations Desk review Key informant interview protocol Mixed Methods Triangulation 4. How do public sector partners / stakeholders (at different government tiers) perceive and value DI’s assistance related to removing the institutional and policy impediments to diaspora investments, crafting reform proposals and their implementation? How do DI’s interventions fit within the broader diaspora policy efforts? DI design and implementation documentation/databases review and secondary documentation from international organizations/donors and BiH authorities Key informant interviews with USAID/BiH staff; DI Implementers and partners; public sector stakeholders; other donors and international organizations Desk review Key informant interview protocol Mixed Methods Triangulation 74 DOCUMENTS AND REPORTS A thorough review of relevant Activity and policy documents will allow the evaluation team to develop a comprehensive understanding of the intended goals of DI, implementation progress, the Activity’s relationship to policy progress, and progress on results. This document review will provide a foundational understanding for answering all four evaluation research questions. As an initial step in the evaluation, the MEASURE team will conduct a thorough review of relevant documentation including:  Activity Award31;  Annual Work Plans32;  Monitoring and Evaluation Plan33;  Quarterly reports34;  Annual Report35;  Activity internal documents and deliverables o RfAs, approved and rejected business plans, grant agreements, technical assistance requests, technical assistance agreements, STTA contracts; o internal activity records (attendance sheets, records on the value of investments in assisted private enterprises, records on female participants, technical assistance tracking forms, etc.); o analyses o reports (technical assistance final reports) and o beneficiary surveys;  FIA/AFIP financial reports;  SME beneficiaries’ documents and records;  Relevant legislation and policy documents (recommendations) such as draft of the Framework Diaspora Cooperation Strategy;  Other documents. For each research question, we will identify relevant sections of documentation. We will then document themes that emerge from the documents with the research questions. The preliminary themes identified through this process will provide a foundation for identifying themes that emerge from the KIIs. 31 The Activity’s Contract: Contract number AID-168-C-17-00001; April 11, 2016; The Activity’s Contract modification no. 7: AID-169-C-17-00001/07; March 24, 2017. 32 Annual Work Plan for the period April 3, 2017 – April 2, 2018; June 2, 2017.; Annual Work Plan for the period April 3, 2018 – April 2, 2019; March 9, 2018. 33 Monitoring and Evaluation Plan for the period April 2017 – September 2022; July 17, 2017 and the M&E plan revision from February 26, 2019 and March 29, 2019. 34 Quarterly Performance Report for the period April 3 – June 30, 2017; July 17, 2017.; Quarterly Performance Report for the period July 1 – September 30, 2017; October 13, 2017.; Quarterly Performance Report for the period October 1 – December 31; January 15, 2018.; Quarterly Performance Report for the period January 1 – March 31, 2018; April 16, 2028.; Quarterly Performance Report for the period April 1 – June 30, 2018; July 16, 2018.; Quarterly Performance Report for the period July 1 – September 30, 2018; October 15, 2018. 35 Annual Summary Report for the period April 3. 2017 – April 2, 2018; May 3, 2018. 75 KEY INFORMANT INTERVIEWS KIIs will be conducted with selected individuals to gather data relevant to each of the research questions. The relationships between respondents and research questions as well as sampling strategies are outlined in Exhibit 3. Based on initial implementation documentation/databases review, the evaluation team has identified seven groups from which we will select participants. Overall, the evaluation team conduced 69 key informant interviews (KIIs) with seven broad types of stakeholders: 1. USAID/BiH staff: 1 interview 2. DI implementers and partners: 6 interviews 3. DI assisted beneficiaries – grantees: 24 interviews 4. DI assisted beneficiaries – recipients of technical assistance: 10 interviews 5. DI public sector beneficiaries/stakeholders (including municipalities): 15 interviews 6. Non-beneficiary SMEs: 6 interviews 7. Other donors/international organizations: 7 interviews; Separate KII interview protocols will be used for the six groups delineated as data sources. These protocols will ensure that similar issues are addressed in all KIIs while tailoring the questions to the experiences of particular types of respondents. All KIIs will be conducted with two evaluation team members present, both of whom will take notes independently. Drafts of the interview protocols are provided in the Annex of this document. Because of the large number of KIIs, the MEASURE team will continually identify and refine themes. This will allow for simultaneous data collection and analysis. After each KII, the team members will complete a form summarizing the key points of the KII, relating this information to other KIIs, and identifying any emerging themes. This process will allow the team to identify and track themes as they emerge throughout data collection and build upon the themes identified through the document review described previously. Additionally, this process will allow for ongoing triangulation of information across the documents and qualitative data. BENEFICIARY SURVEY In addition to our qualitative data collection efforts, the evaluation team will also conduct a web survey of SME beneficiaries of DI and non-beneficiary SMEs. The objective of the survey is to gather information and data on contracting targets achieved so far by the Activity, data on financial results estimates for 2018, and information on beneficiaries’ perceptions on the technical approach applied by DI and added value of grants and different forms of technical assistance. This survey will ask SME beneficiaries to report information such as new jobs and investments. This information will be used to measure progress toward targets and contribute to answering research questions 1. If the FIA/AFIP data on SMEs business results for 2018 are not available until after the completion of the evaluation the evaluation team will use the self￾reported estimate data collected through this web survey as a substitute. The survey will be sent to all assisted SME beneficiaries that have been identified and for whom we are able to obtain an email address. These individuals will be asked about progress on targets as well as 76 perceptions of DI. A draft of the survey questionnaire is provided in the Annex. The data collected through the web survey will allow MEASURE to estimate progress toward targets, compare progress across beneficiaries and assess whether the perceptions of DI among respondents align with the themes that emerge from the qualitative data collection described previously. The relationship between each data source and the research questions is illustrated in Exhibit 5. Exhibit 5. Data sources and Research Questions DATA SOURCE CONTRIBUTION TO RESEARCH QUESTION 1 CONTRIBUTION TO RESEARCH QUESTION 2 CONTRIBUTION TO RESEARCH QUESTION 3 CONTRIBUTION TO RESEARCH QUESTION 4 Documents and Reports Provide information about progress toward targets Provide information about technical approach design and implementation mechanisms Provide information about management, implementation, monitoring and coordination the grant component of the Activity. Information on progress toward design of recommendations for the Framework Diaspora Cooperation Strategy and cooperation with the BiH Ministry for Human Rights and Refugees, the Strategy design Working Group and Municipalities. KIIs Perceptions of results of DI, Activity successes and challenges Perceptions of added value of grants and technical assistance Activity successes and challenges, in technical approach implementation Perception of SME beneficiaries and other representatives of other donor organizations about management, implementation, monitoring and coordination the grant component of the Activity. Perception of public sector partners / stakeholders (at different levels of government) about value of DI’s assistance. Perception of public sector stakeholders and other donors and international organizations about the fit of Di’s interventions within the broader diaspora policy efforts. Online Surveys Perceptions of results of DI, reported progress on targets, services received, Reported progress on targets, services received, perceptions of added value of grants and technical assistance N/A N/A DATA ANALYSIS To address the first evaluation question, related to DI’s progress toward achieving targets, the IMPAQ team will examine DI’s work under all three components and verify and provide a detailed analysis of the Activity’s progress towards reaching each of the contractual targets. This includes those related to the creation of new full-time registered jobs, the value of direct investments, the number of firms receiving DI’s funded technical assistance for improving business performance, the percentage of female participants in DI’s assisted programs to increase access to productive economic resources, the percent of policy recommendations generated through the structured dialog with diaspora investors adopted by relevant BiH institutions, the number of diaspora eligible early stage SMEs and start-ups that graduated from BDM, the number of the BiH diaspora organizations/business networks registered on the diaspora map, and the number of potential diaspora investors who used information and services provided by the Activity/ One￾Stop-Shop. 77 The evaluation team will examine DI’s above-mentioned contractual targets and compare them to the actual values reported by the IP to date. We will verify the reported results as described in Exhibit 6. Exhibit 6. Process for Verifying DI Reported Results Indicator Verification Process Number of direct full time officially registered jobs in Diaspora Invest￾assisted enterprises and other private sector partners/beneficiaries a. Review the IP’s documentation used to confirm and monitor the employment status of new hires; b. Verify that the reported number of direct full time officially registered jobs is reported for beneficiaries that satisfy eligibility criteria (private enterprises with legally registered and recognized status in BiH) c. Verify the number of enterprises and other private sector partners / beneficiaries that fall under the “assisted” category; d. Identify these enterprises in the official FIA/AFIP databases and verify the IP￾reported number of direct full-time registered jobs. e. Review IP’s survey data to measure new jobs by gender. f. During the field visits and interviews with beneficiaries, with the permission of beneficiaries, perform a brief spot checks with employees to determine whether they are the new hires in the reported period or ask beneficiary to present employment contracts and / or confirm payment of social contributions for the reported new hires. g. Review the IP’s documentation (surveys, Technical Assistance Request, Technical Assistance Agreement, Technical Assistance Tracking Form, Technical Assistance Final Report) to verify IP-reported percentage of female participants in DI – assisted programs designed to expand the ability of female entrepreneurs and investors to pursue economic opportunities, invest capital, hire employees, and grow their businesses. h. Randomly select female participants for KIIs. Number of firms receiving DI￾funded technical assistance for improving business performance a. Review IP’s documentation which was used to select beneficiaries of technical assistance, define types and intensity of technical assistance to be provided to firms (technical assistance agreements, etc.), monitor implementation of technical assistance, and track results of technical assistance; b. Compile a final list of the “assisted” firms; c. Randomly select firms to be interviewed by the evaluation team, and adjust (if needed) data collection instruments (beneficiary interview questionnaire, survey questionnaire). Value of investment in assisted private enterprises, disaggregated by direct investment resulting from BiH Diaspora Marketplace and investment resulting from DI One￾Stop Shop a. Review the IP’s exiting documentation36 such as Technical Assistance Agreement, Grant Agreement, Assistance Overview Form, Technical Assistance Final Report, Grant Agreement Final Report, etc. b. Identify enterprises that received assistance from DI’s two components (component 2 – BH Diaspora Marketplace and component 3 – One-Stop-Shop) and compare the reported values of investments with the data from official FIA / APIF databases37 and verify IP-reported values of investments.38 c. Randomly select firms to be interviewed by the evaluation team. 36 Investment data for firms receiving financial assistance (Component 2) will be sourced from the grant agreement and grant agreement final report, and from the standardized forms the IP uses to collect data once the financial assistance is completed. 37 Financial reports from the FIA/APIF databases of firms receiving both grant and technical assistance (Components 2 and 3) for previous calendar year will be reviewed. 38 Investment includes but is not limited to: (i) Investment generated by grant assistance beneficiaries recognized as cost share in the implementation of Grant Agreements signed under the Bosnia and Herzegovina Diaspora 78 Percent of policy recommendations generated through the structured dialogue with diaspora investors submitted to the relevant BiH institutions a. Review the IP’s existing documentation such as strategies for cooperation with diaspora at the state-level, FBiH level and the Brcko District level (especially recommendations and action plans related to the Diaspora Investment and Economic Engagement component of the above-mentioned strategies); b. Review survey results collected by the IP survey instruments; c. Select KIs from the three working groups developing the strategies and receiving the IP’s assistance and support for interviews. Number of diaspora-eligible, early stage SMEs and start-ups that graduated from BDM39 a. Review the IP’s existing documentation such as internal records that contain information about name of the beneficiary, basic beneficiary info, municipality/city, tax ID or registration document. b. Randomly select firms to be interviewed by the evaluation team. Number of BiH diaspora organizations/business networks registered on the Diaspora Map40 a. Review the IP’s existing documentation and the IP’s internal records such as the registration forms devised by the IP; b. Select KIs to be interviewed by the evaluation team. Verify the reported number of potential diaspora investors who used information and services provided by the Activity/One￾Stop-Shop41 a. Review the IP’s existing documentation and internal records. b. Review results of IP survey instruments; c. Select KIs to be interviewed by the evaluation team. For 2017, available official financial records for DI beneficiaries that report to FIA/APIF will be collected and analyzed. If the actual 2018 official financial results are not available through FIA/APIF, the evaluation team will gather estimates from beneficiaries via an online survey, which will be sent to all beneficiaries whose email contacts are provided to the evaluation team (the survey will also include questions on 2016 and 2017 business results). All the reported contractual targets/results initially verified through review of the IP’s existing documentation and analysis of secondary data, will be compared with data/information collected through the key informant interviews and beneficiaries’ surveys. The triangulation of data from different sources Marketplace (Component 2). This investment may include acquisition of fixed assets, working capital, investment in generating new employment and in-kind contributions, as defined by the grant manual, and resulting from assistance provided by DI. (ii) Increase in fixed assets (tangible or intangible) as recorded in the balance sheets under accounts 01 through 098. This investment will be measured for beneficiaries assisted under Components 2 and 3. 39 Number of early stage SMEs and start-ups with verifiable partnership with BiH diaspora member that graduated from Bosnia and Herzegovina’s Diaspora Marketplace (BDM). Early stage SME is defined as a business with five or less years of operational history. Start-up is defined as a business with two or less years of operational history. Diaspora member is defined as a member of the BiH diaspora who are “emigrants and their descendants who live outside the country of their birth or ancestry, either on a temporary or permanent basis, yet still maintain significant ties to their countries of origin.” Diaspora eligible SME is defined as an SME seeking to expand or introduce new goods or services with at least 25% equity stake by a diaspora member and/or a verifiable business partnership with a diaspora member residing in a country other than BiH. Verifiable business partnership is proven by a contractual relationship. 40 Any formal or informal private organization, business club or cooperative registered or operated/co-operated by BiH Diaspora member(s) with the objective of business networking. Diaspora member is defined as a member of the BiH diaspora who are “emigrants and their descendants who live outside the country of their birth or ancestry, either on a temporary or permanent basis, yet still maintain significant ties to their countries of origin”. Diaspora Map is defined as a geographic presentation of BiH Diaspora organizations. 41 Investment facilitation services include, but are not limited to: general information on the investment process and starting a business, business intelligence, market intelligence and research; location services, business-to-business (B2B) matching via meetings, events, and networking; basic consulting services on business planning and operations; administrative, legal, accounting paperwork; staffing recruitment and operational logistics; advice on business processes and management; identification of viable projects and connection with investors. 79 will enable the evaluation team to develop comprehensive, detailed and reliable insight into progress made by the IP in reaching the contractual targets from multiple data sources. To answer the second evaluation question, related to how DI was implemented and perceived, the evaluation team will first review the Activity’s available documentation to gather information on implementation successes and challenges. This information will be compared with data collected from beneficiaries and stakeholders on how they value grant assistance and technical assistance provided by DI. The data collection instruments (key informant interview guides and survey questionnaires) will include specific questions to compare how beneficiaries perceive the value of grants relative to different types of technical assistance. The key informant interview guides and survey questionnaires will also include questions to assess cooperation among diaspora business networks before and after DI’s assistance and key informants’ perceptions of DI’s contribution to building cooperation among diaspora business networks. In interviews and an online survey of non-beneficiaries, this group’s perceptions of the value of the types of grants and technical assistance offered by DI, as well as their perception of changes in cooperation among diaspora business networks, will also be assessed. This information will allow us to compare beneficiaries and non-beneficiaries’ perceptions. Non-beneficiary SME population will be drawn from the DI’s database of all identified diaspora owned / managed companies and the DI’s survey of diaspora investors currently operating in BiH. Once the non-beneficiary SME population is established, the evaluation team will use the stratified random sampling method to divide the population into smaller groups of non-beneficiary SME which share similar characteristics with the DI’s group of beneficiaries. To address the third evaluation question, related to management of DI’s grant component, the evaluation team will review and assess the methods, tools, and data used by the IP to ensure grant resources are used and spent for the agreed upon purposes and ensure that beneficiaries’ contributions are adequately recorded and tracked. The review of the IP’s documentation related to the grant component management and implementation will be complemented by information gathered though semi-structured interviews with SME grantees and non-grantees who applied for grants, an online surveys of DI’s SME grantees, and a mini online survey of a sample of non-beneficiary SMEs that applied but did not receive grants. The evaluation team will also interview all relevant international donors and organizations involved in similar grant activities in order to better understand the level of coordination between them and the Activity. To address the fourth evaluation question, related to public sector perceptions of DI and DI’s integration into policy efforts, we will examine DI’s work with diaspora investors’ associations and other diaspora investors groups that participated in the structured dialogue facilitated by the DI, and speak with public sector stakeholders involved in diaspora policy making. In order to determine how the public sector stakeholders at local, entity, and state levels perceive and value DI’s technical assistance in removing policy and institutional impediments to diaspora investments, the evaluation team will conduct semi-structured key informant interviews with diaspora investors, relevant public sector stakeholder institutions, and international organizations and donors. These interviews will be supplemented with a review of DI’s documentation, including specific investors’ recommendations developed through DI technical assistance and submitted to the relevant institutions, strategic documents and action plans related to diaspora investments and economic engagement and secondary documentation from international organizations/donors and BiH authorities. The evaluation team will examine the relevance of DI’s policy recommendations for creating an enabling business environment for diaspora investors. 80 LIMITATIONS OF THE EVALUATION Limitation 1: Recall bias In collecting data for this evaluation, it is possible that informants may inaccurately represent the implementation or outcomes of the DI activities. As noted previously, the MEASURE team will triangulate data across sources to ensure we are providing reliable results and recommendations. Limitation 2: Staggered implementation of DI technical approach to delivery of technical assistance to SMEs and start-ups. In particular, the OSS and BNP approach, which is to be examined under Evaluation Question 2, has relatively recently begun its implementation. Thus, the evaluation team will need to examine SMEs and stakeholders’ perceptions on how they value these channels for services delivery, as well as SMEs and stakeholders’ feedback on how they value the services provided under BNP and OSS to date. EVALUATION WORK PLAN TIMELINE AND DELIVERABLES Exhibit 7 provides an overview of the evaluation timeline. Exhibit 7. Tentative Evaluation Timeline Tentative Dates Tasks and Deliverables April 17, 2019 Submission of the Work Plan to USAID April 22- April 26, 2019 Logistical preparation, contracting, scheduling for KIIs interviews, online survey preparation April 23, 2019 Initial meetings/interviews with USAID/BiH April 24, 2019 Initial meetings/interviews with implementing partner April 25, 2019 Interview with 2 beneficiaries to test data collection instruments May 6 - June 15, 2019 Data collection through KIIs and online survey Interview transcribing Initial data analysis June 19, 2019 Additional interview / consultations with the USAID/BiH and / or IP prior to report drafting for final clarifications needed from the Mission, if needed June 27, 2019 Recommendations workshop discussion with USAID/BiH July 3, 2019 Holding the preliminary findings and recommendations presentation for USAID/BiH June 17 – July 15, 2019 Continue and finalize transcribing Continue and finalize data analysis Report drafting July 23, 2019 Submission of Draft Evaluation Report to USAID The deliverables for this evaluation will include: 1. Detailed evaluation work plan and data collection instrument(s) The evaluation work plan will include: (1) a detailed evaluation design matrix (including the key questions, methods, and data sources used to address each question and the data analysis plan for each question); (2) draft questionnaires and other data collection instruments or their main 81 features; (3) the list of potential interviewees and sites to be visited; (4) known limitations to the evaluation design; and (5) a dissemination plan. The work plan will include: (1) the anticipated schedule and logistical arrangements; and (2) a list of the members of the evaluation team, delineated by roles and responsibilities. 2. Presentation of preliminary findings Presentation of preliminary findings to USAID/BiH shall discuss the summary of preliminary findings and recommendations to USAID/BiH. 3. Draft evaluation report Draft evaluation report will be consistent with the USAID Evaluation Report Requirements provided in ADS REFERENCE 201MAH ( https://www.usaid.gov/ads/policy/200/201mah ) and take into account criteria to ensure the quality of the evaluation report specified in ADS REFERENCE 201MAA ( https://www.usaid.gov/ads/policy/200/201maa ). Once the initial draft evaluation report is submitted, USAID/BiH will have 10 calendar days in which to review and comment on the initial draft, and submit the consolidated comments to the evaluation team. The evaluation team will address the consolidated comments and submit a revised final draft report in 10 days hence. 4. Final evaluation report The evaluation team will be asked to take no more than 10 calendar days to respond/incorporate the final comments from USAID/BiH. The evaluation team leader will then submit the final report. EVALUATION TEAM For the DI evaluation, we have assembled a team with methodological and subject matter expertise, including two external consultants and MEASURE-BiH’s Research Fellows, who are leading experts in BH diaspora and SME financing, and have published peer reviewed research in these fields. The team composition and team members’ key qualifications are described in Exhibit 8 and the overall level of effort (LOE) by task is described in Exhibit 9. Exhibit 8. The team composition and team members’ key qualifications and LoE Position LoE (days) Key Qualifications Team Lead (TL) Davorin Pavelic 34 Team and project management skills; Adequate level of seniority to conduct interviews with highly ranked government officials, subject matter expertise in SMEs and start-ups development and expertise in program evaluation and evaluation methodologies Diaspora Expert (DE) Nermin Oruc 51 Subject matter expertise in BiH diaspora investment and ongoing business enabling environment reforms SME Financing Expert Jasmina Mangafic 51 Subject matter expertise in SME financing and access to finance Senior Research Analyst (SRA) Anesa Hadzic 54 Expertise in program evaluation, qualitative and quantitative research methods Research Analyst (RA) Amina Smajovic 45 Experienced in data collection and analysis 82 Position LoE (days) Key Qualifications Office Manager (OM) Taisa Zecic 4 Logistical preparation for field work, contracting, and payments Transcribers 40 Experience transcribing audio recordings from KIIs Editor 4 Draft Evaluation report editing HO/FO support 10 HO review of deliverables/general oversight Evaluation Team Lead: Davorin Pavelic, MEASURE-BiH’s Chief of Party. Davorin has over 20 years of experience in a wide range of donor funded initiatives (USAID, World Bank / IFC, EU, UNDP, IMG, OECD, etc.) related to labor markets, job generation, local economic development, SME and entrepreneurship development, access to finance, and enabling business environment throughout BiH, Serbia, Montenegro, Macedonia, Albania, Kosovo and Croatia. As a member of the evaluation team, Mr. Pavelic worked on a meta-evaluation of assistance of the Norwegian government to the SEE region from 1991 to 2001. He was also Team Leader in a performance evaluation of the USAID BiH Excellence in Innovation intervention. He is a MEASURE-BiH Research Fellow and earned Certificate of the Evaluation Practice at the Evaluator’s Institute at Claremont Graduate University in 2017. He was evaluation lead for the 2010-2013 USAID BiH Development Grants Program, evaluation lead for the Whole-of-Project Evaluation of USAID/BiH Project 2.2, team lead for the Assessment of the BiH E-Governance and E￾Administration, evaluation co-lead for the performance evaluation of USAID/SWEDEN FARMA II Activity and the performance evaluation of USAID’s Justice Activity, and team leader and MEL expert for the UNDP intervention aimed to build municipal capacities for monitoring and evaluation of integrated development strategies in BiH. Davorin has an undergraduate degree from the High Military Academy in Zagreb, a MSc degree from the Sarajevo Economic Faculty, and an MBA degree from the University of Delaware. He is currently a PhD candidate at the Faculty of Economy and Management from Zagreb. Diaspora Expert: Nermin Oruc is Director of CDESS and Coordinator of the Western Balkans Migration Network (WB-MIGNET). He has 8 years of teaching experience and more than 10 years of experience in leading and participating in research projects. His research interests include migration, labor market analysis, poverty and inequality. Nermin holds PhD degree in Economics from Staffordshire University, UK. His PhD thesis was exploring relationships between conflict and human capital, with focus on BiH. Until now, Nermin led and participated more than 20 research projects in migration, including first online survey of BiH diaspora in 2007, analysis of IOM survey of diaspora and returnees in 2011, survey of diaspora in 2012 for a project exploring options for harnessing potential of diaspora for rural development in BiH, first mapping of BiH diaspora in ten destination countries in 2017, technical support to BiH authorities in improving migration statistics in 2011-2012, and several project analyzing effects of migration and remittances on poverty, inequality, entrepreneurship, labor force participation, etc. SME Financing Expert: Dr. Jasmina Mangafic (PhD in Management and Business) is Assistant Professor at the AACSB accredited School of Economics and Business, University of Sarajevo. Her research interests include microeconomics, economics of enterprise, and financial management. She earned her Bachelor, MSc, and PhD degree from the Sarajevo School of Economics and Business, as well 83 as MBA degree from the University of Delaware. She is USAID MEASURE-BiH Research Fellow and in this Fellows Program she has earned the Certificate of the Evaluation Practice at the Evaluators’ Institute at the Claremont Graduate University in 2017. As an experienced freelance consultant, she has over 10 years of experience in donor-funded programs (USAID, WB, UNDP, IOM), government- as well as private budget-funded programs, working on business development, financial modelling, data analysis and processing, as well as project management. She also has extensive experience in development evaluation. Inter alia, she participated as evaluation specialist in the following four USAID/BiH funded evaluations: (i) Performance evaluation of the 2010-2013 USAID BiH Development Grants Program (ii) Impact evaluation of the Fostering Agricultural Markets Activity (FARMA) (iii) Performance evaluation of the Component I of the Partnership for Advancing Reforms in the Economy (PARE) and (iv) Performance evaluation of the Excellence in Innovation (EI) Activity in Bosnia and Herzegovina. Currently, she performs the role of Evaluation Specialist in the Mid-term performance evaluation of the Fostering Agricultural Markets Activity II (FARMA II), a USAID/BiH and Sida-funded $16.28 million activity with an expected implementation period from January 1st, 2016 till December 31st, 2020. The purpose of this evaluation is to assess the progress in fulfilling the Activity’s contractual obligations to date and to provide recommendations for adjustments to the Activity, if necessary. Evaluation Team Member: Anesa Hadzic, MEASURE-BiH’s Senior Research Analyst. Anesa has worked with MEASURE-BiH for four years. As a student, she worked on several projects focused on economic competitiveness, youth employment, international student mobility, IT, and management. She was a part of the MEASURE-BiH evaluation team that conducted the impact evaluation of the USAID-BiH FIRMA Activity, performance evaluations of the Civil Society Sustainability Project, FARMA II, and the Whole-of-Project Evaluation of the USAID/BiH Project 2.2. Also, she worked on the National Survey of Citizens Perceptions in BiH Findings Report for 2015, 2016, 2017, and 2018, the National Youth Survey in BiH, and Assessment of the BiH E-Governance and E-Administration in BiH. She holds a Bachelor Degree in management and is currently working towards a Master’s degree in the same field. Evaluation Team Member: Amina Smajovic, MEASURE-BiH’s Analyst. Amina has worked for MEASURE-BiH since November 2018. She was a member of the MEASURE-BiH’s team working on the Basic Education in Bosnia and Herzegovina: the Follow-on Assessment in 2018. She also worked on the 2018 National Survey of Citizens Perceptions in BiH Findings Report. As a student, she worked on several projects focused on youth employment and activism, project management, entrepreneurship, and international student mobility. For many years she was an active member of the European Youth Parliament in BiH. She holds a Bachelor’s Degree in Management and is currently working towards a Master’s Degree in Marketing. 84 Exhibit 9. Evaluation Team Level of Effort Tasks and Deliverables Responsible Team Member Level of Effort (days) Logistical preparation, scheduling for KIIs interviews, online survey preparation Team Leader (TL) Diaspora Expert (DE) SME Financing Expert Research Analyst (RA) Admin staff (AS) 2 (TL) 1 (DE) 4 (FE) 5 (RA) 4 (AS) Initial meetings/interviews with USAID/BiH and IP Team Leader (TL) Diaspora Expert (DE) Senior Research Analyst (SRA) 2 (TL) 2 (DE) 2 (SRA) Data collection through KIIs and online survey Interview transcribing Initial data analysis Team Leader (TL) Diaspora Expert (DE) SME Financing Expert (FE) Senior Research Analyst (SRA) Research Analyst (RA) Transcribers (TR) 10 (TL) 30 (DE) 29 (FE) 30 (SRA) 30 (RA) 40 (TR) Additional interview with USAID/BiH and IP prior to report drafting for final clarifications needed from the Mission Team Leader (TL) Diaspora Expert (DE) SME Financing Expert (FE) Senior Research Analyst (SRA) 1 (TL) 1 (DE) 1 (FE) 1 (SRA) Preparing and holding a recommendations workshop discussion with USAID/BiH Team Leader (TL) Diaspora Expert (DE) SME Financing Expert (FE) Senior Research Analyst (SRA) 3 (TL) 3 (DE) 3 (FE) 3 (SRA) Preparing and holding the preliminary findings and recommendations presentation for USAID/BiH Team Leader (TL) Diaspora Expert (DE) SME Financing Expert (FE) Senior Research Analyst (SRA) 3 (TL) 3 (DE) 3 (FE) 3 (SRA) Continue and finalize transcribing Continue and finalize data analysis Report drafting Team Leader (TL) Diaspora Expert (DE) SME Financing Expert (FE) Senior Research Analyst (SRA) Research Analyst (RA) 8 (TL) 8 (DE) 8 (FE) 10 (SRA) 10 (RA) Draft Evaluation Report editing Editor (E) 4 (E) Addressing received comments on Draft Evaluation Report Team Leader (TL) Diaspora Expert (DE) SME Financing Expert (FE) Senior Research Analyst (SRA) 5 (TL) 3 (DE) 3 (FE) 5 (SRA) 85 ANNEX IV: DRAFT DATA COLLECTION INSTRUMENTS FOR DIASPORA INVEST EVALUATION INTERVIEW PROTOCOL – DI EVALUATION (USAID/ BIH MISSION) This interview is being conducted for the purpose of the mid-term performance evaluation of USAID/BiH’s DI activity. Our purpose in meeting with you today is to learn about your thoughts and experiences with the DI activity. Your insights will help us understand how the DI activity and DI’s technical approach has been implemented; what progress has been achieved so far in terms of reaching contract targets of the Activity; and how the DI’s grant component is implemented, managed, monitored and coordinated. We are also interested in learning about DI’s assistance related to removing the institutional and policy impediments to diaspora investments, crafting reform proposals and their implementation, and understanding how DI’s interventions fit within the broader diaspora policy efforts. All your comments are confidential and you won’t be identified by name in any report. [NOTE: Make sure you have name, position, and office organization] Would it be alright if I record this interview? BACKGROUND  Can you describe your role with the DI Activity? 

 CONTRACTING TARGETS How successful is the Activity in achieving contracting targets to date? o PROBE: New jobs; o PROBE: Disbursement of grants; o PROBE: Value of investments; o PROBE: Improving beneficiaries’ business performance; o PROBE: Assisting female business owners; o PROBE: Providing services to SME through the One-Stop-Shop o PROBE: Creating business linkages and networks o PROBE: Crafting and submitting reform oriented policy recommendations; In your opinion, what are prospects of reaching the LoA contractual targets? DI MANAGEMENT Please describe the management of the DI Activity. o PROBE: What are the roles of the Activity’s partner organizations (PI Consulting, Restart, Nasa Perspektiva? o PROBE: How would you assess capacities of these partner organizations to implement DI’s interventions? What are their strengths and weaknesses? o PROBE: What are the main management challenges? 86  Is there overlap in terms of beneficiaries between DI and other USAID/BiH activities (WHAM, FARMA II) and other donors and projects implemented by other international organizations (UNDP)?  How does DI seek linkages and collaboration with other diaspora related programs? In what areas has DI established linkages and collaboration with other diaspora related programs?42  How would you assess the quality of the DI Activity’s M&E system? Do you receive information in a timely manner? Do you receive relevant, valid and reliable data and information about progress of the Activity in achieving contracted results? o PROBE: How do they track achievement of predefined Activity indicators (jobs, investments, beneficiaries’ contribution, grant expenditures, etc.)?  How does the DI activity communicate / cooperate with relevant stakeholders and partners within the Activity? Outside the Activity? o PROBE: How would you rate the quality of work (and capacities) of relevant partners and stakeholders within the Activity? Outside the activity?  Has USAID/BiH experienced any challenges in cooperating with DI IP and DI partners?  In your opinion, how do SME beneficiaries value grant assistance provided by DI?  In your opinion, how SME beneficiaries value different types of technical assistance provided by DI? DI ACTIVITIES  Can you describe the types of assistance DI provides (i.e. grants, technical assistance, etc)?  Can you briefly describe the selection process of DI grantees? o PROBE: Is that process fully transparent? o PROBE: Is that process in line with the Activity’s purpose, goal and objectives?  How successful is the IP in management and implementation of the grant component of the Activity? What are the main management challenges in the grant component of the Activity?  Do you know how DI identifies the types of technical assistance to provide to beneficiaries?  What kinds of changes do you think assisted SME firms saw as a result of DI’s assistance? What kind of feedback have you received from them, if any?  Would you say the technical approach (combination of grants, technical assistance and services and loans) has facilitated the achievement of contracted targets? How?  How is the DI’s technical approach affected by the withdrawal of the commercial banks from the Activity and lack of commercial loans in the planned package of services?  What were the most useful parts of DI’s assistance? What were the least useful parts? How have SMEs used One-Stop-Shop services?  As a result of DI assistance, do you know if assisted SMEs saw substantial changes in any of the following: o employment (new jobs)? o investments? 42 Questions in red are contributed by the Swiss Development Cooperation. 87 o business performance?  How have these changes varied by the type of service SMEs receive (grants, TA)? POLICY AND INSTITUTIONAL FRAMEWORK Next, I would like to ask you more specifically about DI’s effects on general policy and institutions relevant to the diaspora investments in Bosnia and Herzegovina.  What kinds of changes do you think relevant government institutions saw as a result of DI’s assistance? What kind of feedback have you received from them, if any? What were the most useful parts of this assistance? What were the least useful parts?  What were the biggest institutional and policy impediments to diaspora investments prior to DI (i.e. in 2015/2016)?  What have been the most substantial changes related to removal of the institutional and policy impediments to diaspora investments, in the last two years? Are these changes related to DI’s efforts? What are the specific reform proposals crafted by DI which contributed to these changes? Have any of these reform proposals been implemented so far? If yes, how?  How have these changes affected diaspora investments?  What are the most important policy and institutional impediments preventing diaspora investments? DI IMPLEMENTATION  What are the main challenges the IP has faced in implementing the DI Activity? Have they resolved these challenges? If so, how?  Has the Activity adapted to what has worked, and what has not? If so, how?  Have there been any unintended effects (either positive or negative) of the DI Activity? If so, how has the IP adjusted its activities to either take advantage of these or address them? SUSTAINABILITY  What is your perception of the sustainability of the One-Stop-Shop? How likely is it that this kind of assistance will be integrated with the formal BiH institutions? 88 INTERVIEW PROTOCOL – DI EVALUATION (IMPLEMENTING PARTNER AND ASSOCIATED ORGANIZATIONS) This interview is being conducted for the purpose of the mid-term performance evaluation of USAID/BiH’s DI activity. Our purpose in meeting with you today is to learn about your thoughts and experiences with the DI activity. Your insights will help us understand how the DI activity and DI’s technical approach has been implemented; what progress has been achieved so far in terms of reaching contract targets of the Activity; and how the DI’s grant component is implemented, managed, monitored and coordinated. We are also interested in learning about DI’s assistance related to removing the institutional and policy impediments to diaspora investments, crafting reform proposals and their implementation, and understanding how DI’s interventions fit within the broader diaspora policy efforts. All your comments are confidential and you won’t be identified by name in any report. [NOTE: Make sure you have name, position, and office organization] Would it be alright if I record this interview? BACKGROUND  Can you describe your role with the DI Activity? 

 CONTRACTING TARGETS How successful has the Activity been in achieving contracting targets to date? o PROBE: New jobs; o PROBE: Disbursement of grants; o PROBE: Value of investments; o PROBE: Improving beneficiaries’ business performance; o PROBE: Assisting female business owners; o PROBE: Providing services to SME through the One-Stop-Shop o PROBE: Creating business linkages and networks o PROBE: Crafting and submitting reform oriented policy recommendations; In your opinion, what are prospects of reaching the LoA contractual targets? DI MANAGEMENT Please describe the management of the DI Activity. o PROBE: What are the roles of the Activity’s partner organizations (PI Consulting, Restart, Nasa Perspektiva? o PROBE: How do you assess the capacities of these partner organizations to implement DI’s interventions? What are their strengths and weaknesses? o PROBE: What are the main management challenges?  Is there overlap in terms of beneficiaries between DI and other USAID/BiH activities (WHAM, FARMA II) and other donors and projects implemented by other international organizations (UNDP)? 89  How does DI seek linkages and collaboration with other diaspora related programs? In what areas DI has established linkages and collaboration with other diaspora related programs?43  How do you track achievement of predefined Activity indicators (jobs, investments, beneficiaries’ contribution, grant expenditures, etc.)?  How does the DI activity communicate / cooperate with relevant stakeholders and partners within the Activity? Outside the Activity? o PROBE: How would you rate the quality of work (and capacities) of relevant partners and stakeholders within the Activity? Outside the activity?  In your opinion, how do SME beneficiaries value DI’s grant assistance?  In your opinion, how SME beneficiaries value different types of technical assistance? DI ACTIVITIES  Can you describe the types of assistance DI provided (i.e. grants, technical assistance, networking, information sharing, outreach, etc)?  Can you briefly describe the grantee selection process? o PROBE: How do you ensure that the process is fully transparent? o PROBE: Is that process in line with the Activity’s purpose, goal and objectives? o PROBE: In your opinion, would grantees invest even if they have not been awarded grants? o PROBE: In your opinion, would grantees hire new employees even if they have not been awarded grants?  What are the main management challenges in the grant component of the Activity?  How does DI identify the types of technical assistance to provide to beneficiaries? PROBE: Do you approach SMEs directly and offer TA, do SMEs apply for TA, or it is a combination of both approaches?  In your opinion, how do SME beneficiaries value DI’s grant assistance?  In your opinion, how SME beneficiaries value different types of technical assistance?  What kinds of changes do you think assisted SME firms saw as a result of DI’s assistance? What kind of feedback have you received from them, if any?  Would you say the technical approach (combination of grants, technical assistance and services and loans) has facilitated the achievement of contracted targets? How?  How is the DI’s technical approach affected by the withdrawal of the commercial banks from the Activity and the lack of commercial loans in the planned package of services?  What were the most useful parts of DI’s assistance for beneficiaries? What were the least useful parts?  How have SMEs used One-Stop-Shop services? 43 Questions in red are contributed by the Swiss Development Cooperation. 90  As a result of DI assistance, do you know if assisted SMEs saw substantial changes in any of the following: o employment (new jobs)? o investments? o business performance?  As a result of DI, how have these changes varied by the type of service SMEs receive (grants, TA)? POLICY AND INSTITUTIONAL FRAMEWORK I would like to ask you more specifically about DI’s effects on general policy and institutions relevant to diaspora investments in Bosnia and Herzegovina.  What kinds of changes do you think relevant government institutions saw as a result of DI’s assistance? What kind of feedback have you received from them, if any? What were the most useful parts of this assistance? What were the least useful parts?  What were the biggest institutional and policy impediments to diaspora investments prior to DI (i.e. in 2015/2016)?  What have been the most substantial changes related to removal of the institutional and policy impediments to diaspora investments, in the last two years? Are these changes related to DI’s efforts? How?  What are the specific reform proposals crafted by DI which contributed to these changes? Are any of these reform proposals implemented so far? If yes, how?  How have these changes affected diaspora investments?  What are the most important policy and institutional impediments preventing diaspora investments? DI IMPLEMENTATION  What are the main challenges you have faced in implementing the DI Activity? How have you resolved these challenges?  Has the Activity evolved in response to what has worked and what has not? If so, how?  Have there been any unintended effects (either positive or negative) of the DI Activity? If so, how have you adjusted your activities to either take advantage of these or address them? SUSTAINABILITY  What is your perception about the sustainability of the One-Stop-Shop? How likely is it that this kind of assistance will be integrated with the formal BiH institutions? 91 INTERVIEW PROTOCOL – DI EVALUATION (SME BENEFICIARIES) This interview is being conducted for the purpose of the mid-term performance evaluation of USAID/BiH’s DI activity. Our purpose in meeting with you today is to learn about your thoughts and experiences with the DI activity. Your insights will help us understand how the DI activity and DI’s technical approach has been implemented; what progress has been achieved so far in terms of reaching contract targets of the Activity; and how the DI’s grant component is implemented, managed, monitored and coordinated. We are also interested in learning about DI’s assistance related to removing the institutional and policy impediments to diaspora investments, crafting reform proposals and their implementation, and understanding how DI’s interventions fit within the broader diaspora policy efforts. All your comments are confidential and you won’t be identified by name in any report. [NOTE: Make sure you have name, position, and office organization] Would it be alright if I record this interview? BACKGROUND 1. Please describe your company and its main business activities? a. Where is your business registered? Is your business affiliated through common ownership and/or centralized management with other business/businesses (in BiH and abroad)? b. How long has your business been operating? Please provide information on the industry classification in which your business operates. c. What is the size of your business (# of employees, annual turnover)? d. How many full-time does the company currently employ? How many full-time employees were employed in 2017 (or at the time you commenced your business operations)? e. How many new employees has your company hired in the period since you signed the cooperation agreement with DI? What is the employment status of these employees to date? How many of them have employed for more than 6 months? How many of them are women?44 INVOLVEMENT IN DI ACTIVITIES 2. How did you learn about the DI Activity and how did you start your collaboration with DI? GRANTS 3. Have you applied for a grant? If yes, can you describe the process? Was there a public invitation to apply? Were you able to contact DI for information regarding the documentation needed for applying? 44 Questions in red are contributed by the Swiss Development Cooperation. 92 a. Have you received grant funds from DI, and, if yes, please describe the purposes for which the grant is used? What was the grant amount you received from DI? What was the amount you needed to contribute to realize your business plan for which you received the grant? b. Can you please describe the process of being selected as a grant recipient? How would you assess the business plan competition process in terms of transparency, clarity of criteria and scoring methodology and fairness? Did you have any external assistance while developing the business plan you submitted? Did you received any assistance directly from DI while developing your business plan? c. What was the amount of your investments in business operations during the last two years? When (date of investment)? What was the purpose of investment? d. Are your investments linked with DI activity? e. What is the value of investments in your business after you signed the cooperation agreement with DI? Please explain the form / type (grant / loan / equity) investment, (what source), and purpose of investment. f. Have you received any type of financial assistance from other donors / international organizations? g. Have you received any type of financial assistance or support from BiH institutions?45 h. Would you invest in your business operations anyway, even if there was no cooperation with DI? TECHNICAL ASSISTANCE 4. Have you received any DI services/assistance which resulted in the transfer of knowledge and / or new technologies that are relevant to your business operations? 5. Have you signed a TA agreement? a. Did DI directly approach you and offer TA or did you apply for a particular type of TA with DI? b. If you applied for TA with DI, would you say that application process was transparent? c. Are you satisfied with the quality of technical assistance received from DI? d. Would you say that the service providers and experts provided by DI have a sufficient level of knowledge and expertise in the areas in which they are providing advice? e. What needs of your business operations were addressed by that technical assistance? Did they improve your business and business operations? If so, how? 6. Have you received B2B assistance from DI? If yes, could you please provide details and explain results of the assistance? 7. Are you a member of or are you familiar with any BH diaspora business organizations / business networks? Have you received any service or advice from these business networks? 8. Are you familiar with the Diaspora Invest map of diaspora business organizations / business networks? Are these mapped organizations / networks active in advising companies? 9. Are you familiar with any diaspora online business network platform? How would you rate the usefulness of the diaspora online business network platform? 10. Have you participated in / used any of these DI services: a. Database of investment opportunities? When? How? What was the result of this type of assistance? b. Networking events? When? How? What was the result of that type of assistance? c. Investment conferences? When? How? What was the result of that type of assistance? 45 Questions in red are contributed by the Swiss Development Cooperation. 93 d. Business to business (B2B) events? Matchmaking between diaspora and local businesses? When? How? What was the result of that type of assistance? 11. Are you aware of the existence of the DI-supported One-Stop-Shop? a. Are you familiar with information and services provided by OSS? If yes, could you please explain the type of services provided by OSS? b. Have you used any of these services? How would you rate the usefulness and quality of these services? c. Would you be willing to pay for these services? In your opinion, is the OSS a commercially viable and sustainable mechanism? DIASPORA INVEST ACHIEVEMENTS 12. What worked well in terms of the assistance you received from DI? What could be improved in terms of the assistance you received from DI? 13. What trends in business results (new full-time jobs and investments) did your organization experience in the last two years? What was the amount of equity investment you made? a. How has the assistance your organization received from DI contributed to these results (especially to the new full-time jobs and new investments in your business)? b. Would these new investments and the new full-time jobs occur without DI activities? 14. How relevant is DI’s assistance to your industry in general (wood, metal processing, tourism, ICT)? POLICY AND INSTITUTIONAL FRAMEWORK 15. Are you familiar with the existence of the Diaspora Investors’ Advisory Council (DIAC)? Are you familiar with the advocacy activities of DIAC? 16. In your opinion, what are the most important obstacles to stronger diaspora (or your) involvement in the BiH economy (i.e. lack of information, financial products, business registration, etc.)? Why? 17. Are you familiar with the recommendations DI proposed to governments in order to remove obstacles / barriers for diaspora economic engagement? If yes, how important do you feel these recommendations are? 18. How do you think the environment for diaspora investments will improve if these recommendations are implemented? 46 19. Have you noticed any changes in policy or the institutional framework relevant to your industry or diaspora investments in BiH, in the last two years? If yes, please describe the changes. a. How have these changes affected your organization? Are these changes related to DI efforts? OTHER 20. What are your priority needs in the coming years? What type of assistance would be the most important for your organization? 21. What type of institutional support would be most relevant and effective for SMEs and investors from diaspora in general? 46 Questions in red are contributed by the Swiss Development Cooperation. 94 INTERVIEW PROTOCOL – DI EVALUATION (NON-BENEFICIARY SME) This interview is being conducted for the purpose of the mid-term performance evaluation of USAID/BiH’s DI activity. Our purpose in meeting with you today is to learn about your thoughts and experiences with the DI activity. Your insights will help us understand how the DI activity and DI’s technical approach has been implemented; what progress has been achieved so far in terms of reaching contract targets of the Activity; and how the DI’s grant component is implemented, managed, monitored and coordinated. We are also interested in learning about DI’s assistance related to removing the institutional and policy impediments to diaspora investments, crafting reform proposals and their implementation, and understanding how DI’s interventions fit within the broader diaspora policy efforts. All your comments are confidential and you won’t be identified by name in any report. [NOTE: Make sure you have name, position, and office organization] Would it be alright if I record this interview? BACKGROUND 1. Please describe your company and its main business activities. a. Where is your business registered? Is your business affiliated through common ownership and/or centralized management with other business/businesses (in BiH and abroad)? b. How long has your business been operating? Please provide information on the industry classification in which your business operates? c. What is the size of your business (# of employees, annual turnover)? d. How many full-time employees does the company currently employ? How many full-time employees were employed in 2017 (or at the time you commenced your business operations)? e. How many new employees did your company hire in 2018? What is their employment status to date? How many of them have been employed for more than 6 months? How many of them are women?47 FAMILIARITY WITH DIASPORA INVEST 2. Are you with DI? If yes, have you been contacted, or have you considered using DI assistance? GRANTS 3. Are you familiar with grant opportunities provided by DI? 4. Have you applied for grant funds from DI? a. Have you submitted your business plan to DI? If yes, how would you assess the business plan competition process in terms of transparency, clarity of criteria and scoring methodology and fairness? b. Did you have external assistance while developing your submitted business plan? c. Have you received any assistance directly from DI while developing your business plan? 5. How would you assess the importance and value of small grants? 47 Questions in red are contributed by the Swiss Development Cooperation. 95 a. Could grants of this size improve business results for you (new jobs, and new investment)? b. If you were to receive the types of grants offered by DI, how might you use them? TECHNICAL ASSISTANCE 6. Did you apply for technical assistance with DI? 7. Have you received any technical assistance from DI? 8. Are you familiar with other types of assistance and services provided by DI? PROBE: Are you familiar with DI’s Diaspora Business Marketplace/Business Advisory Services, Diaspora Business Network Platform (BNP) and One-Stop-Shop interventions? PROBE: Have you received a B2B assistance from DI? If yes, could you please provide details and explain results of the assistance? 9. How would you assess the importance of technical assistance for your organization in terms of meeting priority needs? What type of technical assistance do you think would be the most valuable to your business? PROBE: How likely do you think these types of assistance would be in improving business results (jobs, and investment)? 10. How would you assess the value of grant assistance versus technical assistance? POLICY AND INSTITUTIONAL FRAMEWORK 11. Have you noticed any changes in policies or institutions relevant to creation of a business enabling environment for diaspora investment in BiH in the last two years? If yes, please describe any such changes. a. How have these changes affected your organization? Do you know whether these changes are related to DI efforts? b. What are the biggest challenges for your organization and industry in terms of the legal, regulatory, and institutional framework or current policies? 12. What are the main priority needs of your organization in the coming years? a. What type of institutional support would be most relevant and effective for SMEs and investors from diaspora in general? OTHER 13. What trends in business results (direct new full-time jobs and investment) has your organization experienced in the last two years? 96 INTERVIEW PROTOCOL – DI EVALUATION (PUBLIC SECTOR STAKEHOLDERS) This interview is being conducted for the purpose of the mid-term performance evaluation of USAID/BiH’s DI activity. Our purpose in meeting with you today is to learn about your thoughts and experiences with the DI activity. Your insights will help us understand how the DI activity and DI’s technical approach has been implemented; what progress has been achieved so far in terms of reaching contract targets of the Activity; and how the DI’s grant component is implemented, managed, monitored and contracted. We are also interested in learning about DI’s assistance related to removing the institutional and policy impediments to diaspora investments, crafting reform proposals and their implementation, and understanding how DI’s interventions fit within the broader diaspora policy efforts. All your comments are confidential and you won’t be identified by name in any report. [NOTE: Make sure you have name, position, and office organization] Would it be alright if I record this interview? BACKGROUND  Are you familiar with the DI’s purpose, goal and objectives?  How did you start your collaboration with DI? o In what DI interventions have you participated?  Can you describe your role with the DI Activity? 

 CONTRACTING TARGETS In, your opinion, does the Activity contribute to new jobs generation and increase of diaspora investments? PROBE: Would you say the technical approach (combination of grants, technical assistance and services and loans (DCA)) could facilitate the generation of new jobs and increased diaspora investments? How? PROBE: What parts of the assistance package offered by DI do you find to be the most valuable? Why? DI MANAGEMENT  How does the DI activity communicate / cooperate with relevant stakeholders and partners? PROBE: Have you experienced any challenges in cooperation with DI IP and DI partners?  Is there overlap in terms of beneficiaries between DI and other USAID/BiH activities (WHAM, FARMA II) and other donors and projects implemented by other international organizations (UNDP)?  How does DI seek linkages and collaboration with other diaspora related programs? In what areas DI has established linkages and collaboration with other diaspora related programs?48 48 Questions in red are contributed by the Swiss Development Cooperation. 97  In your opinion how valuable to diaspora investors are grants versus technical assistance, information sharing, institutional support and networking? DI ACTIVITIES  Are you familiar with the existence of the One-Stop-Shop and its services? How likely is it that this kind of assistance will be integrated with the formal BiH institutions? POLICY AND INSTITUTIONAL FRAMEWORK I would like to ask you more specifically about DI’s effects on general policy and institutions relevant to the diaspora investments in Bosnia and Herzegovina.  What kinds of changes do you think relevant government institutions saw as a result of DI? What kind of feedback have you received from them, if any? What were the most useful parts of this assistance? What were the least useful parts?  What were the biggest institutional and policy impediments to diaspora investments prior to DI (i.e. in 2015/2016)?  What have been the most substantial changes related to removal of the institutional and policy impediments to diaspora investments, in the last two years? Are these changes related to DI’s efforts? What are the specific reform proposals crafted by DI which contributed to these changes? Have any of these reform proposals been implemented so far? If yes, how?  How have these changes affected diaspora investments?  What are the most important policy and institutional impediments preventing diaspora investments? DI IMPLEMENTATION  How relevant was DI’s assistance in terms of meeting your priority needs and addressing priority obstacles to diaspora investments in BiH?  How likely is it that DI’s policy and institutional reform recommendations will result in adoption and implementation of key regulations and improved institutional framework for diaspora investment?  What worked well in terms of cooperation with DI? What were the challenges in terms of cooperation with DI, if any? OTHER  What are the main priority needs of governments in BiH in terms of connecting and networking with diaspora investors and higher involvement of diaspora in BiH economic development in next year of two? o What types of assistance would be most relevant in meeting these needs? o What would be the most effective in supporting diaspora related policy adoption and implementation? 98 INTERVIEW PROTOCOL – DI EVALUATION (DONORS AND INTERNATIONAL ORGANIZATIONS) DIASPORA RELATED PROGRAMS AND DI ACTIVITY  Can you describe diaspora related programs supported by your organization and the types of assistance provided by these programs (i.e. grants, technical assistance, networking, information sharing, outreach, etc)?  Are you familiar with the DI’s purpose, goal and objectives, and types of assistance and services provided by DI?  In your opinion, how similar / different is the DI Activity design from other donor supported diaspora related programs?  How does DI seek linkages and collaboration with other diaspora related programs? In what areas DI has established linkages and collaboration with other diaspora related programs? Is there overlap in terms of beneficiaries between DI and other donors and projects implemented by other international organizations (UNDP)?  How does the DI activity communicate / cooperate with relevant stakeholders (public institutions)?  Has your organization experienced any challenges in cooperating with DI Activity? GRANTS AND TA AND FAMILIARITY WITH DIASPORA INVEST  In your opinion, how do SME beneficiaries value grant assistance?  In your opinion, would SME grantees hire new employees and invest even if they have not been awarded grants?  How do you ensure that the grantee selection process in programs supported by your organization is fully transparent?  In your opinion, how SME beneficiaries value different types of technical assistance?  Are you familiar with grant opportunities provided by DI?  Are you familiar with other types of assistance and services provided by DI?  Would you say the technical approach (combination of grants, technical assistance and services and loans) facilitates new investments and jobs creation?  In your opinion how valuable to diaspora investors are grants versus technical assistance, information sharing, institutional support and networking?  Are you familiar with any diaspora online business network platform? In general, how would you rate the usefulness of the diaspora online business network platforms?  Are you familiar with the Diaspora Invest map of diaspora business organizations / business networks? Are these mapped organizations / networks active in advising companies?  Are you familiar with the existence of the One-Stop-Shop and its services? How likely is it that this kind of assistance will be integrated with the formal BiH institutions? POLICY AND INSTITUTIONAL FRAMEWORK I would like to ask you more specifically about DI’s effects on general policy and institutions relevant to the diaspora investments in Bosnia and Herzegovina. 99  Have you noticed any changes in policies or institutions relevant to creation of a business enabling environment for diaspora investment in BiH in the last two years? If yes, please describe any such changes.  Are you familiar with the existence of the Diaspora Investors’ Advisory Council (DIAC)? Are you familiar with the advocacy activities of DIAC?  In your opinion, what are the most important obstacles to stronger diaspora (or your) involvement in the BiH economy (i.e. lack of information, financial products, business registration, etc.)? Why?  Are you familiar with the recommendations DI proposed to governments in order to remove obstacles / barriers for diaspora economic engagement? If yes, how important do you feel these recommendations are?  How do you think the environment for diaspora investments will improve if these recommendations are implemented?  How likely is it that DI’s policy and institutional reform recommendations will result in adoption and implementation of key regulations and improved institutional framework for diaspora investment?  What are the main priority needs of governments in BiH in terms of connecting and networking with diaspora investors and higher involvement of diaspora in BiH economic development in next year or two? OTHER What type of institutional support would be most relevant and effective for SMEs and investors from diaspora in general? 100 ONLINE SURVEY QUESTIONNAIRE FOR DI SME BENEFICIARIES 1. Name of your company: _____________________________________________________ 2. In which of the sectors listed below is your company active (primary activity): a. Wood-processing industry b. Metalworking industry c. Agriculture d. Energy e. Textile industry f. ICT g. Manufacturing h. Cross-cutting. Please explain: ___________________________________________ 3. When did you start participating in DI activities (month and year): ______________________ 4. Was your company a beneficiary of Diaspora Invest grant funds: a. Yes b. No, we applied, but we did not receive a grant c. No, we never applied, but we are aware of the opportunity to apply for a Diaspora Invest grant d. No, we never applied, and we have not been aware of the opportunity to apply for a Diaspora Invest grant 5. If you have been a beneficiary of a Diaspora Invest grant funds, please state for what the funds were expended: a. Procurement of equipment b. Investment in plant facilities c. Salaries and other employee outlays d. Other (please specify) 6. If you invested in your company/enterprise in the last two years, what was the size if the investment? a. Up to 50,000 BAM b. Up to 100,000 BAM c. Between 100,000 BAM and 200,000 BAM d. Between 200,001 BAM and 500,000 BAM e. Between 500,001 BAM and 1 million BAM f. Between 1 million BAM and 2 million BAM g. More than 2 million BAM 7. Would you have invested if you had not received a grant? a. I would have invested the same amount, in the same business year b. I would have invested the same amount, but in the following business year c. I would have invested a smaller amount, in the same business year I would not have invested at all 8. Would you have hired new employees if you had not received a grant? a. Yes b. No 101 9. Would you have invested and created new jobs even if you had not received a grant? a. Yes b. No 10. In your view, were the grant application and award processes transparent? a. Yes b. No c. I don't know 11. In the process of grant application, were you informed about the scoring methodology and criteria? a. Yes b. No c. I don't know 12. Have you received feedback with your score and total evaluation of your application? a. Yes b. No c. I don't know 13. In your view, were the criteria and scoring of your application adequate? a. Yes b. No c. I don't know 14. What is the share (in %) of Diaspora Invest grant funds in your proposed investment? a. Up to 20% b. From 21% to 40% c. From 41% to 50% d. More than 50% 15. Was the grant amount offered by Diaspora invest (up to 50,000 KM) sufficient to meet your business expansion needs? a. Yes b. In part c. No 16. Have you received grants or any other form of financial assistance from any international organization/donor? a. Yes, we received assistance from another donor/international organization, prior to receiving Diaspora Invest support b. Yes, we received assistance from another donor/international organization, after receiving Diaspora Invest support c. No, although we have applied. d. No, we have never applied. e. If your answer was Yes, please state from which institutions_____________________ 17. Have you received any subsidies or grants from BiH institutions (the government, government bodies, agencies, etc.)? 102 a. Yes, we received a subsidy or a grant from BiH institutions, prior to receiving Diaspora Invest support b. Yes, we received a subsidy or a grant from BiH institutions, after receiving Diaspora Invest support c. No, although we have applied. d. No, we have never applied. e. If your answer was Yes, please state from which institutions_____________________ 18. If the answer was YES, please state which form of financial assistance (you may choose more than one answer): a. Co-financing of equipment purchases b. Employment subsidies c. Business start-up subsidies d. Incentives for agriculture, export promotion, etc. e. Other, please specify: 19. Have you received a loan from any commercial bank in BiH? a. Yes, we received a bank loan, prior to Diaspora Invest support b. Yes, we received a bank loan, after Diaspora Invest support c. No, although we had applied for a bank loan d. No, we did not need a loan. 20. Did your company receive any form of technical assistance? a. Yes, we received technical assistance, because that was what we applied for b. Yes, we received technical assistance after applying for the grant c. No, we have applied, but we have not received technical assistance d. No, we have not applied, although we are eligible and we are acquainted with the option of applying for technical assistance e. No, we have not applied, we are ineligible, but we are acquainted with the option of applying for technical assistance f. No, we have not applied, and we are not aware of the option to apply for technical assistance 21. Are you satisfied with the quality of technical support you are receiving or you have received from Diaspora Invest project? a. Yes b. In part c. No 22. Please describe specific technical support your company received from Diaspora Invest project (write N/A if your company did not receive technical support from Diaspora Invest project). _______________________________________________________________________ 23. For each type of assistance provided by Diaspora Invest project, how much did it contribute to your business results (sales/exports, new jobs and investments)? 103 Not effective at all Somewhat effective Moderately effective Very effective Have not received this type of assistance a. Grant funds 1 2 3 4 0 b. TA from a manufacturing expert 1 2 3 4 0 d. Market linkages and market penetration (B2B, research and analysis, marketing and promotion) 1 2 3 4 0 e. Worker training 1 2 3 4 0 f. Various business information 1 2 3 4 0 g. Institutional support in obtaining permits and licenses 1 2 3 4 5 h. Business linkages (business conferences and events) 1 2 3 4 0 i. Other: Please explain (add): 1 2 3 4 0 24. In your view, to what extent, on the scale from 1 to 5, did the type of TA you received from Diaspora Invest project contribute to achievement of your business results? Sales/Exports New jobs New investments a. Grant funds 1 2 3 b. TA from a manufacturing expert 1 2 3 d. Market linkages and market penetration (B2B, research and analysis, marketing and promotion) 1 2 3 e. Worker training 1 2 3 f. Various business information 1 2 3 g. Institutional support in obtaining permits and licenses 1 2 3 h. Business linkages (business conferences and events) 1 2 3 i. Other: Please explain (add): 1 2 3 25. Did you take advantage of information and services provided by the website www.diasporainvest.ba? 104 a. Yes b. No, I was not aware of this website c. No, I had no need for the type of information and services offered by this website 26. Have you made use of the services of the Diaspora Business Center? a. Yes b. No, I was unaware of this Center's existence c. No, I had no need of the services offered by this Center d. If your answer was Yes, please state which types of services you received from this Center? 27. Did you attend/participate in conferences (RECONNECT, BHdiaFOR), info sessions or any other events organized by Diaspora Invest project for the purposes of networking with BiH diaspora? a. Yes b. No 28. Do you find that these conferences, info sessions or any other events organized by Diaspora Invest project were beneficial for your business (you may choose more than one answer) a. Yes, I received information about investment opportunities b. Yes, I received information about company support programs c. Yes, I received information about financing available to support company projects d. Yes, I received information about other companies' operations and about their products e. Yes, I exchanged contact information with other companies f. Yes, I established a business contact which didn’t led to a business cooperation/commercial transaction g. Yes, I established a business contact which led to further discussions about potential business cooperation h. Yes, I established a business contact that led to business cooperation i. No, I had not benefitted from attendance at these events j. Yes, but none of the above, but (please state how you benefitted from attendance at these events) 29. Are you a member of any diaspora business network (association, NGO, alliance, foundation, etc.)? a. Yes b. No 30. In your view, were there any improvements in business linkages and networking of BiH diaspora in the last two years? a. Yes b. No 31. Would your company be willing to pay for the following services (you may choose more than one answer)? a. Provision of general business information 105 b. Business support with investments (market research, feasibility studies, site identification support, organization of business visits, support with company registration, assistance in acquiring required licenses) c. Business linkage services (organization of business meetings, events or visits, organization of attendance/participation in trade fairs/sector network meetings in BiH and the region) d. Business planning and consulting services for the diaspora (development strategies, business plans, marketing and communication plans) e. Business consulting services (strengthening corporate governance, product development, support in the management of transformation processes, cost-effectiveness and cost optimization, expert assistance in development of business proposals and grant application, etc.) 32. Which are the major challenges your company faces in terms of the legal, regulatory and institutional framework? (you may select more than one answer) a. Unfavorable economic situation in BiH b. Lack of reliable information on conditions for doing business in BiH c. Administrative and regulatory obstacles (a complicated system) d. Absence of institutional support for diaspora investors e. Other, please specify: 33. Are diaspora investors facing somewhat different challenges/barriers relative to other (domestic and foreign) investors? a. Yes b. No c. If your answer was Yes, please state what challenges are different for diaspora investors relative to other (domestic and foreign) investors: 34. Please fill in the blanks with information about estimated increase/decrease in your capital investments in the coming period (by year, in %) PROJECTED ANNUAL % CHANGE 2019 2020 2021 Capital Investment Fixed assets Number of employees 35. If you have any other comments, please let us know: ____________________________________________________________________ ____________________________________________________________________ WE SINCERELY THANK YOU FOR YOUR TIME IN FILLING OUT THIS SURVEY! 106 ONLINE SURVEY QUESTIONNAIRE FOR SME NON-BENEFICIARIES 1. In which of the sectors listed below is your company active (primary activity): a. Wood-processing industry b. Metalworking industry c. Agriculture d. Energy e. Textile industry f. ICT g. Manufacturing h. Cross-cutting. Please explain: ___________________________________________ 2. Are you able to access loans from commercial banks or other commercial sources of financing? a. Yes, we received a bank loan b. No, although we had applied for a bank loan c. No, we did not need a loan 3. Did you receive grants or other types of financial assistance from international organizations / donors or any government subsidy/grant (any government tier, agency)? a. Yes, we received assistance from other donor or International organization b. No, even we applied c. No, we never applied If the answer is Yes, please provide information which organization provided the subsidy/grant in question: __________________________ 4. If the answer to the previous question is “yes”, please select the type of financial assistance received from the government (you can select more than one answer): a. Co-financing of equipment purchases b. Employment subsidies c. Business start-up subsidies d. Incentives for agriculture, export promotion, etc. e. Other (please explain): ________________________________________________ 5. Did your company apply for DI grants: a. Yes, we applied, even we met the criteria, we didn’t receive the grant b. Yes, we applied, but we were informed that we do not meet the criteria (years of existence, share in ownership by diaspora member, etc.) c. No, we never applied, although we were aware of DI grant opportunities d. No, we never applied and were not aware of DI grant opportunities 6. If you applied for a DI grant, in your opinion, was the grant application and award process transparent? a. Yes b. No c. I do not know 7. Would the grant amount of up to 50,000 KM be sufficient to cover your current business / investment and development needs? a. Yes 107 b. Only a small portion of my current investment and business development needs c. A larger portion of my current investment and business development need d. No 8. If you invested in your company/enterprise in the last two years, what was the size if the investment? a. Up to 50,000 BAM b. Up to 100,000 BAM c. Between 100,000 BAM and 200,000 BAM d. Between 200,001 BAM and 500,000 BAM e. Between 500,001 BAM and 1 million BAM f. Between 1 million BAM and 2 million BAM g. More than 2 million BAM 9. Did you attend/participate in conferences (RECONNECT, BHdiaFOR), info sessions or any other events organized by Diaspora Invest project for the purposes of networking with BiH diaspora? a. Yes b. No 10. Do you find that these conferences, info sessions or any other events organized by Diaspora Invest project were beneficial for your business (you may choose more than one answer) a. Yes, I received information about investment opportunities b. Yes, I received information about company support programs c. Yes, I received information about financing available to support company projects d. Yes, I received information about other companies' operations and about their products e. Yes, I exchanged contact information with other companies f. Yes, I established a business contact which did not lead to a business cooperation/ commercial transaction g. Yes, I established a business contact which led to further discussions about potential business cooperation h. Yes, I established a business contact that led to business cooperation i. No, I had not benefitted from attendance at these events j. Yes, but none of the above, but (please state how you benefitted from attendance at these events) 11. Are you a member of any diaspora business network (association, NGO, alliance, foundation, etc.)? a. Yes b. No 12. In your view, were there any improvements in business linkages and networking of BiH diaspora in the last two years? a. Yes b. No 13. What type of assistance would be most important for diaspora investors? _______________________________________________________________________ _______________________________________________________________________ 108 14. Would your company be willing to pay for the following services (you may choose more than one answer)? a. Provision of general business information b. Business support with investments (market research, feasibility studies, site identification support, organization of business visits, support with company registration, assistance in acquiring required licenses) c. Business linkage services (organization of business meetings, events or visits, organization of attendance/participation in trade fairs/sector network meetings in BiH and the region) d. Business planning and consulting services for the diaspora (development strategies, business plans, marketing and communication plans) e. Business consulting services (strengthening corporate governance, product development, support in the management of transformation processes, cost-effectiveness and cost optimization, expert assistance in development of business proposals and grant application, etc.) 15. Did you take advantage of information and services provided by the website www.diasporainvest.ba? a. Yes b. No, I was not aware of this website c. No, I had no need for the type of information and services offered by this website 16. Have you made use of the services of the Diaspora Business Center? a. Yes b. No, I was unaware of this Center's existence c. No, I had no need of the services offered by this Center d. If your answer was Yes, please state which types of services you received from this Center? 17. Which are the major challenges your company faces in terms of the legal, regulatory and institutional framework? (you may select more than one answer) a. Unfavorable economic situation in BiH b. Lack of reliable information on conditions for doing business in BiH c. Administrative and regulatory obstacles (a complicated system) d. Absence of institutional support for diaspora investors e. Other, please specify: 18. Are diaspora investors facing somewhat different challenges/barriers relative to other (domestic and foreign) investors? a. Yes b. No c. If your answer was Yes, please state what challenges are different for diaspora investors relative to other (domestic and foreign) investors: 19. Please fill in the blanks with information about estimated increase/decrease in your capital investments in the coming period (by year, in %) 109 PROJECTED ANNUAL % CHANGE 2019 2020 2021 Capital Investment Fixed assets Number of employees 20. If you have any other comments, please let us know: ____________________________________________________________________ ____________________________________________________________________ WE SINCERELY THANK YOU FOR FILLING OUT THIS QUESTIONNAIRE. 110 ANNEX V: DI ACTIVITY AND RESULTS Exhibit. DI Activities and Results, April 2017–March 2018 Level of result Narrative summary Indicator name Year 1 target Year 1 actual Life of activity target Activity goal 2. A competitive, market-oriented economy providing better economic opportunities for all its citizens corresponds to New job opportunities created for the citizens of BiH Total investment as % GDP MEASURE-BiH tracks this indicator Activity purpose 2.1. Improved capacity of private sector to compete in market economy Number of new full time officially registered jobs in Diaspora Invest-assisted enterprises and other private sector partners/beneficiaries 15 48 250 Number of firms receiving Diaspora Invest-funded TA for improving business performance 12 17 140 Percentage of female participants in Diaspora Invest-assisted programs designed to increase access to productive economic resources 21 22 25 Activity sub￾purpose I Diaspora direct investment increased corresponds to 2.1.2. Investment Growth: Increased investment into private sector Value of investment in assisted POs, disaggregated by direct investment resulting from BiH Diaspora Marketplace and investment registered on an online catalogue $ 1,500,000 $ 1,627,157 $ 22,000,000 0 $ 1.049.151 $ 2,000,000 $ 1,500,000 $ 578.006 $ 20,000,000 Activity outcome/o utput 1.1 Institutional policy framework for diaspora investment adopted Percent of policy recommendations generated through the structured dialogue with the diaspora investors adopted by the BiH relevant institutions 0 0 50 Activity outcome/o utput 1.2 Technical assistance and grants provided to diaspora eligible early-stage SMEs and start-ups Number of diaspora eligible early-stage SMEs and startups that graduated from the BDM 0 0 70 Activity outcome/o utput 1.3 A sustainable local platform developed to provide business services and facilitation to potential diaspora investors Number of the BiH diaspora organizations/ business networks registered on the diaspora map 40 41 100 Number of (potential) diaspora investors who used information and services provided by the Activity/One Stop Shop 15 23 100 Exhibit. DI Activity and Results, April 2018–March 2019 Level of result Narrative summary Indicator name Life of activity target Year 2 target Year 2 actual Activity goal 2. A competitive, market-oriented economy providing better economic opportunities for all its citizens corresponds to New job opportunities created for the citizens of BiH Total investment as % GDP MEASURE-BiH tracks this indicator Activity purpose 2.1. Improved capacity of private sector to compete in market economy+ Number of direct full time officially registered jobs in USAID-assisted enterprises and other private sector partners/beneficiaries 40 211 250 Number of firms receiving Diaspora Invest-funded TA for improving business performance 25 37 140 Percentage of female participants in Diaspora Invest – assisted programs designed to expand the ability of women entrepreneurs and investors to pursue economic opportunities, invest capital, hire employees, and grow their businesses 22 27 25 Activity sub￾purpose I Activity outcome/o utput 1.1 Diaspora direct investment increased corresponds to 2.1.2. Investment Growth: Increased investment into private sector Value of investment in assisted private enterprises, disaggregated by direct investment resulting from BiH Diaspora Marketplace and Diaspora Invest One-Stop Shop $ 5,500,000 $ 5,559,707 $ 22,000,000 $ 1,500,000 $ 1,421,428 $ 4,900,000 $ 200,000 $ 1,823,454 $ 2,100,000 $ 3,800,000 $ 2,314,825 $ 15,000,000 Activity outcome/o utput 1.2 Institutional policy framework for diaspora investment adopted Percent of policy recommendations generated through the structured dialogue with diaspora investors articulated and submitted to the BiH relevant institutions in the process of developing Strategy for cooperation with diaspora 15 0 50 Activity outcome/o utput 1.3 Technical assistance and grants provided to diaspora eligible early-stage SMEs and start-ups Number of diaspora eligible, early stage SMEs and startups that graduated from the BDM 10 17 70 Activity Outcome/ Output 1.3 A sustainable local platform developed to provide business services and facilitation to potential diaspora investors Number of the BiH diaspora organizations/ business networks registered on the diaspora map 60 61 100 Number of (potential) diaspora investors who used information and services provided by the Activity/One Stop Shop 20 30 100 112 ANNEX VI: TECHNICAL ASSISTANCE BENEFICIARIES Exhibit: Technical assistance provided to grantees (TA under component 2) Company Technical assistance Start date End date Value (BAM) Status ADV PAX 20.12.2017 18.07.2018 0 completed Agroplus 03.10.2018 27.11.2018 1,200 completed BioFood BH doo 14.12.2018 30.09.2019 0 to do BMT Engineering 20.12.2017 30.03.2018 0 completed Bonatura 01.08.2018 31.01.2019 0 completed CNC Lab 08.08.2018 04.10.2018 0 completed Code Line Solution 12.01.2018 17.08.2018 0 completed Delta Plus BH 23.08.2018 28.03.2019 0 completed Eko Jasmina 11.12.2018 31.08.2019 0 to do Ekonomik Group 05.09.2018 31.12.208 0 to do Empress 12.07.2018 05.02.2019 0 completed Fam Farms 13.12.2018 30.06.2019 0 to do FANA Arifagic 01.04.2018 20.07.2018 2,500 completed FITS doo 24.08.2018 02.04.2019 0 completed FREUND Elektronika 12.01.2018 31.12.2018 0 completed Galop Digital doo 31.08.2018 02.04.2019 0 to do H2I Balkans doo 31.08.2018 02.04.2019 0 to do KMC doo 13.08.2018 30.06.2019 0 to do Lumitic 10.02.2018 31.03.2018 0 completed Master Inzinjering 01.02.2018 01.04.2018 2,500 completed Movida 01.02.2018 01.04.2018 2,400 completed MRD Engineering doo 31.08.2018 28.11.2018 0 completed Octopus 13.12.2018 30.06.2019 0 to do Probosing 31.01.2018 31.03.2018 0 completed Propeller 11.12.2018 18.03.2019 0 completed SB Laser 22.05.2018 06.08.2018 0 completed Sitex 20.12.2017 22.03.2018 2,400 completed Suco 20.12.2017 15.04.2018 2,500 completed Supersoft doo 20.12.2018 30.09.2019 0 to do Talbot (Zendev) 20.12.2017 30.10.2018 0 completed 113 Te Ora 12.01.2018 26.07.2018 4,000 completed TNT Soft 08.12.2018 28.02.2019 0 completed WBR Zona 18.12.2018 30.09.2019 0 to do Exhibit. Technical assistance under component 3 Company Sector TA AGREEMENT Assistance Title Invest. (BAM) # of new empl. LoE (hrs) Results AKM Technology Metal industry Software education for AKM Technology employees 200,000 2 80 3 new employees CNC Proizvodnja Bajric Metal Industry Software education to operate with CNC machines 200,000 2 112 developed approved and fully functional website Eagle Technology Metal industry Product certification 12,800 / / payment for segments of the certification process Edna Metalworking Metal industry Software education- Solid works, CAD/CAM 300,000 / 96 software education of EPRIT , social and marketing promotion Eko Farma Borje Agriculture Analysis of the Eko Farma Borje's herd 400,000 / / / Emar Agriculture Analysis and education plan for raising broilers 400,000 / / / Everest Consulting ICT Delivery of a training for the development of the Super-sensitive flourescent light detector 20,000 1 20 days designed and delivered technical training for six engineers FibraWorld Manufacturing Search engine optimization for Fibraworld's website 200,000 / 64 SEO solutions for Fibraworld's website HIK SM Manufacturing Education for employees (financial DI assistance) 300,000 / 100 prepared ISO certification 9001 and 14001 Industrial Manufacturing Marketing activities, certification 475,000 / / / Izvor Blaga Agriculture Review of lighting and heating 100,000 / / / Kenn Art Other Social and marketing promotion 15,000 / 16 completed social and marketing promotion Mandoob Agriculture Financial aid 200,000 / / / Maric Projekti Metal industry Financial aid 300,000 10 / / MDG International Metal industry / / / provided the training in implementing and operating the ERP system MP Pandurevic Metal industry Software education 300,000 / 75 completed software education of TEKLA 114 Company Sector TA AGREEMENT Assistance Title Invest. (BAM) # of new empl. LoE (hrs) Results Orea Bazaar ICT Graphical design solution for online promotion 36,000 2 8 days provided solution for improved online visual identity Storsen Other Marketing and promotional services 50,000 5 16 completed social and marketing promotion TAZ Global Other design and implementation of Single Sign On 12,000 3 100 supported the design of the software architecture for SSO and oAuth2 Transform Art Wood industry audit and preparation for QMS certification 30,000 1 120 completed all activities required as a pre￾condition to obtain certification Webo Metal industry ISO certification 400,000 / 120 completed preparation for ISO certification 115 ANNEX VII: ANALYSIS OF FINANCIAL PERFORMANCE OF DI GRANTEES The small grant program, implemented through a business plan competition, serves several purposes: providing an incentive to attract direct diaspora investment and leverage other sources of investment, offering a tool for tapping into the diaspora’s know-how, and utilizing market outreach and business networks. Design and adoption of the Small Grants Fund Manual in June 2018 enabled DI to publish four requests for applications (RFAs), two per year, to invite applications for grant funding. This section provides an overview of characteristics of the companies that have received grants so far. We analyzed the following firm characteristics: business sector, financial results, value of fixed assets (absolute and average), value of business revenues (absolute and average), number of employees based on hours worked, and value of export revenues (absolute and average). The grantees come from diverse sectors, as specified in the RFAs, though proposals from other sectors could be considered on a case-by-case basis. More the one third of grantees are from the manufacturing sector49, which is the most capital and labor intensive. The sectors of the 33 grantees are outlined in Exhibit 1. Exhibit 1. Grantees according to statistical classification of economic activities Frequency Percent (%) Accommodation and food services activities 1 3.03 Agriculture, forestry and fishing 5 15.15 Arts, entertiment and recreation 1 3.03 Information and communication 5 15.15 Manufacturing 12 36.36 Proffesional, scientific and technical activities 7 21.21 Wholesale and retail trade; repair of motor vehicles and motorcycles 2 6.06 Total 33 100.0 DI has a highly diversified portfolio of grantees. The targets of the grantee scheme are companies belonging to diverse sectors that are classified as SMEs.50 As a result, DI has a highly diversified portfolio of grantees in terms of business sectors, financial results, value of fixed assets, value of business revenue, and number of employees. This finding was confirmed by the results of descriptive statistics when profitability analysis was performed. The standard deviation is much higher than average, which implies a huge variety among companies. 49 The manufacturing sector was added to the list of DI’s eligible sectors by the Contract modification no. X 50 E.g. according to the definition applied in the Federation BiH (FBiH Law on Incentives to Small Business), small enterprises are companies with fewer than 50 employees, and an annual turnover of less than 19.5 million BAM; the medium sized companies are the ones with fewer than 250 employees, and an annual turnover of less than 97.5 million BAM. 116 All companies are organized as limited liability companies. More than 50 percent of all investments originate from four countries (Germany, USA, Sweden and Croatia), as shown in Exhibit 2. Exhibit 2. Sample structure according to the origin of investments Investment origination Frequency Investment origination Frequency Germany 6 Serbia 2 USA 4 Norway 1 Sweden 4 Denmark 1 Croatia 4 Singapore 1 Switzerland 3 Macedonia 1 Netherlands 3 Austria 1 United Kingdom 2 Total 33 In 2018, out of the sample of 33 companies, 26 finished the year with financial gains: 14 reported profits within the range BAM 10,000-100,000, and 4 reported profit higher than BAM 100,000. We also compared the absolute value of fixed assets in 2017 and 2018, as shown in Exhibit 3. Exhibit 3. Comparison of the fixed assets’ absolute value Fixed assets (BAM) 2017 2018 Up to 100,000 13 19 Between 100,000-1,000,000 7 12 Over 1,000,000 3 2 Total 23 33 Since the numbers of companies in 2017 and 2018 are different, we calculated the average value of fixed assets and compared the result (Exhibit 4). Exhibit 4. Comparison of the fixed assets average value Year Fixed assets - all companies (BAM) Number of companies Average Fixed assets (BAM) 2017 7,110,430 23 309,149.1 2018 10,279,685 33 311,505.6 There was 0.76 percent growth in the average value of fixed assets from 2017 to 2018. The numbers of employees come from company income statements. This analysis relied on numbers of employees based on hours worked. 117 Exhibit 5. Number of employees based on hours worked Year Number of employees based on hours worked (all companies) Number of companies Average number of employees based on hours worked 201751 157.4 21 7.49 201852 371.5 31 11.98 There is a 59.87 percent increase in the average number of employees. Exhibit 6. Structure according to number of employees based on hours worked Number of employees based on hours worked 2017 2018 Up to 5 employees 10 16 Between 6-25 employees 10 11 More than 25 employees 1 4 Total 21 31 An analysis of data on the 33 grantees sought to determine the effect of the grants on the financial performance of these mainly diaspora-owned early-stage SMEs and start-ups. The data for this analysis were collected from secondary sources. The data on the grants’ values were obtained from grant agreements and, if available, grant closeout reports. The secondary data from annual financial reports, including balance sheets and income statements, were obtained from the FIA/APIF database from the DI IP and TRON database. Using these data, the evaluation team developed a database that includes the following information: (i) Basic information: company name, investment location, legal form, date of registration, statistical code, statistical classification of economic activities, investment origination, RFA number (ii) Balance sheet and income statement covering the three-year period 2016–2018 (iii) Information about the grant: grant agreement data including initially signed grant value and contractual obligations of the grantee in terms of diaspora investment and employees to hire; grant closeout report data included the grant amount used, employees hired, and grantee cost share; grant status; grant start and end dates (iv) Information about TA provided to grantees: TA start and end dates, TA value, TA status Quantitative data from secondary sources were analyzed through descriptive statistics. According to DI’s annual and quarterly reports, 33 companies have received grants. The FIA/APIF data on financial statements in 2017 provide balance sheets and income statements for 23 companies. For 2018, we have financial statements for 33 companies. The difference implies that seven new companies were established in 2018. The analysis covers 2017, the year before the grant was received, and 2018, the year when the grant was approved and implemented. 51 There is no information for Octopus and Galop-digital for 2017. 52 There is no information for Octopus and Fam-farms for 2018. 118 In contrast to RFA I, RFA 2 and RFA 3 explicitly describe the purpose of requested funding as follows: “Funds requested from the BDM Small Grant Fund must be dedicated entirely to the purpose of financing the purchase of fixed assets. Fixed assets are defined as assets which are purchased for long-term use and are not likely to be converted into cash within at least 12 months, such as land, buildings, and equipment. Equipment is defined as either tangible assets (such as machinery, computers, vehicles, etc.) or intangible assets (trademark, patent, goodwill, etc.) that have a useful life of more than one year, and an acquisition cost of $1,000 or more per unit.” DI’s documents contain data from closeout completion statements for 17 companies. The closed-out grants originated from 13 grants under RFA 1, three under RFA 2, and one under RFA 3. The progress achieved to date in reaching grant contract targets is presented in Exhibit 7. Exhibit 7. Completed grant agreements No. Company Grant agreement Closeout completion statement Grant amount Diaspora Investment Employees to hire Used grant amount Grantee cost share Hired employees 1 ADV PAX 40.000,00 89.000,00 10 21.683,00 48,538 26 2 BMT Engineering 25.000,00 49.090,00 4 25.000,00 82,641 5 3 Bonatura 34.000,00 34.026,00 2 34.000,00 41,135 2 4 Code Line Solution 12.600,00 17.500,00 1 12.000,00 87,230 1 5 Delta Plus BH 45.000,00 166.540,00 8 43.940,00 161,820 19 6 FANA Arifagic 35.000,00 178.530,00 7 35.000,00 752,500 3 7 FITS doo 19.250,00 22.110,00 2 19.250,00 24,554 2 8 FREUND Elektronika 40.000,00 361.600,00 3 31.164,00 251,015 1 9 Lumitic 30.000,00 116.380,00 4 30.000,00 177,069 4 10 Master Inzinjering 15.800,00 27.580,00 1 15.800,00 27,677 1 11 Movida 20.000,00 121.940,00 3 14.334,00 121,940 8 12 Probosing 23.200,00 38.360,00 1 21.697,00 34,329 1 13 Sitex 30.000,00 77.900,00 8 19.822,00 60,151 7 14 Suco 40.000,00 66.900,00 10 35.000,00 113,153 11 15 Talbot (Zendev) 35.000,00 129.000,00 5 34.917,00 130,193 5 16 Te Ora 40.000,00 207.000,00 4 40.000,00 248,589 4 17 TNT Soft 8.250,00 13.070,00 4 8.250,00 18,314 4 TOTAL 493.100,00 1.716.526,00 77 441.857,00 2,380,848.00 104 The analysis of the profitability indicators of the 17 companies whose grants have closed out includes only the grantees that completed the grant agreement as stated in the grant closeout report and for which we have financial statements for both observed years. Thus, we obtained a sample of 14 companies. 119 The best statistical tool for this kind of analysis would be multiple regression, with an indicator as dependent variable (e.g. profitability) and with the grant and firm size (number of employees, fixed assets and total revenue) as independent variables (as shown in Annex 1I). However, our sample is very small and we need at least five companies for each new independent variable. This analysis could be performed during the final performance evaluation, when there will be data available for all companies included in the intervention and all grant agreements would be closed, yielding more meaningful results. What we can assess here is whether there was a difference between the two compared years in the observed 14 companies. Exhibit 8 presents information on average values and then we test whether those differences are significant or not. Exhibit 8. Paired samples statistics Mean N Std. Deviation Std. Error Mean Pair 1 Net profit margin 1 2017 10.89 11 14.12 4.26 Net profit margin 1 2018 1.00 11 24.60 7.42 Pair 2 Net profit margin 2 2017 5.14 13 20.93 5.80 Net profit margin 2 2018 -3.83 13 32.23 8.94 Pair 3 ROTA 2017 4.64 14 253.98 67.88 ROTA 2018 17.83 14 35.44 9.47 Pair 4 ROA 2017 14.57 14 28.72 7.68 ROA 2018 14.89 14 36.23 9.68 Pair 5 ROE 2017 35.95 11 54.26 16.36 ROE 2018 56.87 11 36.29 10.94 Pair 6 Number of employees based on hours worked 2017 7.93 14 7.99 2.14 Number of employees based on hours worked 2018 17.96 14 23.72 6.34 Pair 7 Fixed assets and long term investments 2017 332784.21 14 579100.60 154771.15 Fixed assets and long term investments 2018 410848.21 14 714686.40 191007.97 Pair 8 Business revenues 2017 401081.14 14 524808.96 140261.09 Business revenues 2018 1119767.07 14 1673199.66 447181.42 For comparison, we applied nonparametric statistic tests due to the small size of the sample and considerable variability. DI grantees' financial statements analysis found no significant differences between the profitability ratios, but significant differences were found in the number of employees, value of fixed assets, and business revenue. Exhibit 9 shows the results of tests of the significance of differences between 2017 and 2018. 120 Exhibit 9. Results of nonparametric Wilcoxon signed ranks test Z P value net profit margin 1 2018 - net profit margin 1 2017 -.267 .790 net profit margin 2 2018 - net profit margin 2 2017 -.454 .650 ROTA 2018 - ROTA 2017 -1.538 .124 ROA 2018 - ROA 2017 -.345 .730 ROE 2018 - ROE 2017 -.711 .477 number of employees based on hours worked 2018 - number of employees based on hours worked 2017 -3.186 .001 fixed assets and long term investments 2018 - a. fixed assets and long term investments 2017 -3.296 .016 business revenues 2018 - business revenues 2017 -.267 .001 In Exhibit 9, bolded values show significant differences between 2017 and 2018. No significant differences can be found among the profitability ratios, but they can be found in numbers of employees, the value of fixed assets, and business revenues. Due to the small sample size, we cannot provide statistical evidence to verify that DI grants are the cause of this difference. Keeping in mind that DI started only two years ago, the evaluation can assess only whether there are statistically significant differences before and after the grant was received. Due to the limited sample size, we cannot determine whether there is a relationship between the grants and beneficiaries' financial performance. We also tested the correlation between the used grant amount and rates of change. The correlation between the used amount of the grant and the rate of change for the number of employees is significant and positive (43.8%), so we can conclude that a higher amount of grant relates to the higher increase in the number of employees. Exhibit 10 compares the value of the grants received with the three variables of interest: employees, fixed assets, and business revenues. Exhibit 10. Comparison between the values of grant received with the three variables (employees, fixed assets and business revenues) Name Used grant amount from close out Based on hours worked 2018 Fixed assets 2018 Business revenues 2018 ADV PAX 21,683.00 49 605,233 643,467 BMT Engineering 25,000.00 11 858,044 1,128,953 Bonatura 34,000.00 3 212,099 1,303,149 Code Line Solution 12,000.00 10 177,903 716,899 121 Delta Plus BH 43,940.00 87 2,721,209 6,473,913 FANA Arifagic 35,000.00 12 524,799 465,570 FITS doo 19,250.00 7 49,679 589,038 FREUND Elektronika 31,164.00 5 49,769 244,859 Master Inzinjering 15,800.00 2,5 55,716 434,132 Movida 14,334.00 4 258 63,283 Probosing 21,697.00 6 24,087 189,657 Suco 35,000.00 30 348,204 2,606,493 Talbot (Zendev) 34,917.00 21 117,197 688,702 TNT Soft 8,250.00 4 7,678 128,624 The correlation between used grant amount and the change in the number of employees is significant and positive. The changes in fixed assets, business revenues, and numbers of employees for each company individually are shown in Exhibit 11. This overview provides summaries the changes each company experienced after receiving the grant. Exhibit 11. Changes in significant determinants, 2017 to 2018 Company Change in fixed assets Change in business revenues Change in number of employees* based on hours worked ADV PAX -42.20 183.40 250.00 BMT Engineering 54.01 71.30 37.50 Bonatura 196.69 699.74 200.00 Code Line Solution 248.80 55.55 25.00 Delta Plus BH 32.98 234.15 180.65 FANA Arifagic 7.36 395.77 33.33 FITS doo 248.18 739.76 133.33 FREUND Elektronika 18.72 10.39 66.67 Master Inzinjering 89.80 17.52 25.00 Movida -42.92 80.77 33.33 Probosing 1,017.73 176.58 0.00 Suco 38.32 151.43 114.29 Talbot (Zendev) 114.14 152.97 133.33 TNT Soft 206.26 * Full-time equivalents based on the number of hours worked. The correlation between the amount of the DI grant used by beneficiaries and the change in the number of employees is significant and positive (43.8 percent). 122 Exhibit 12. Grant app Assistance overview form Grant agreement Close Out Company Grant value requested (BAM) In-kind contribution (BAM) Employees (women) Employees to hire (women) DI assistance contribution (BAM) Diaspora Investment (BAM) Job created Grant amount (BAM) Diaspora Investme nt (BAM) Employees to hire Used grant amount (BAM) Hired employees (women) ADV PAX 45,000.00 1,000,000.00 26 (1) 10 (7) 45,000.00 1,173,170.00 26+10 40,000.00 89,000.00 10 (min 5) 21,683.00 26 (12) BMT Engineering 50,000.00 27,998.00 8 (1) 4 (0) 25,000.00 620,000.00 9+4 25,000.00 49,090.00 4 25,000.00 5 Code Line Solution 60,000.00 102,970.00 9 (2) 3 12,600.00 102,970.00 11+3 12,600.00 17,500.00 1 12,000.00 1 FANA Arifagic 50,000.00 118,000.00 7 (0) 7 (2) 35,000.00 118,000.00 7+7 35,000.00 178,530.00 7 35,000.00 3 FREUND Elektronika 50,000.00 408,600.00 6 (1) 9 (3) 40,000.00 408,600.00 6+9 40,000.00 361,600.00 3 31,164.00 1 Master Inzinjering 49,760.00 101,000.00 2 (1) 2 (0) 15,800.00 101,000.00 2+2 15,800.00 27,580.00 1 15,800.00 1 Movida 50,000.00 100,000.00 2 (1) 16 (13) 20,000.00 150,000.00 2+9 20,000.00 121,940.00 3 14,334.00 8 Probosing 28,619.00 29,052.11 6 (3) 2 (1) 23,200.00 29,052.00 6+2 23,200.00 38,360.00 1 21,697.00 1 Lumitic 54,350.00 232,550.00 0 8 (3) 30,000.00 232,550.00 0+8 30,000.00 116,380.00 4 30,000.00 4 Sitex 40,000.00 150,700.00 5 (3) 7 (5) 30,000.00 150,700.00 5+7 30,000.00 77,900.00 8 19,822.00 7 Suco 50,000.00 64,000.00 14 (1) 50 (3) 40,000.00 74,000.00 15+40 40,000.00 66,900.00 10 35,000.00 11 Talbot (Zendev) 65,000.00 240,000.00 8 (1) 10 (5) 35,000.00 274,000.00 8+10 35,000.00 129,000.00 5 35.000.00 5 Te Ora 0 40,000.00 206,150.00 0+13 40,000.00 207,000.00 4 40,000.00 4 Bonatura 42,260.00 44,150.00 1 (0) 2 (1) 34,000.00 44,150.00 1+2 34,000.00 34,026.00 2 34,000.00 2 Delta Plus BH 50,000.00 197,572.00 36 (2) 6 (0) 45,000.00 147,660.00 36+6 45,000.00 166,540.00 8 43,940.00 19 Ekonomik Group 50,000.00 183,650.00 3 (0) 20 (1) 40,000.00 132,550.00 3+10 40,000.00 112,500.00 10 FITS doo 19,250.00 19,260.00 6 (4) 2 (1) 19,250.00 19,260.00 6+2 19,250.00 22,110.00 2 19,250.00 2 Galop Digital 50,000.00 40,000.00 2 (1) 4 (2) 28,000.00 50,000.00 2+4 28,000.00 46,055.00 3 H2I Balkans 50,000.00 80,400.00 2 (2) 2 (2) 35,000.00 80,400.00 2+2 35,000.00 56,378.00 1 KMC 41,700.00 100,000.00 0 15 (4) 30,000.00 410,000.00 0+15 30,000.00 159,800.00 15 MRD Engineering BAM 50,000.00 BAM 128,050.00 10 (0) 15 (3) BAM 45,000.00 133,050.00 10+5 45,000.00 102,000.00 5 SB Laser 50,000.00 2,081,170.00 0 5 (1) 50,000.00 2,081,170.00 0+5 50,000.00 163,000.00 5 123 Agroplus 124,000.00 50,000.00 4 (1) 2 (1) 50,000.00 74,000.00 4+2 50,000.00 74,000.00 2 BioFood BH 42,620.00 277,040.00 3 (2) 5 (3) 41,306.00 277,040.00 3+5 41,306.00 277,040.00 5 Eko Jasmina 26,620.00 26,620.00 1 (1) 1 (0) 26,620.00 86,620.00 1+1 26,620.00 86,620.00 1 Empress 31,200.00 112,390.00 15 (13) 10 (10) 30,500.00 74,770.00 15+5 30,500.00 74,770.00 5 Fam Farms 48,055.00 97,055.00 4 (1) 2 (1) 48,060.00 57,400.00 4+2 48,060.00 57,405.00 2 Octopus 50,000.00 560,000.00 0 9 (5) 50,000.00 509,900.00 5 50,000.00 512,500.00 5 Propeller 10,200.00 52,540.00 9 (4) 4 (2) 10,020.00 52,540.00 9+4 10,020.00 52,540.00 4 Supersoft 48,840.00 347,910.00 9 (8) 3 (3) 44,590.00 124,070.00 9+3 44,590.00 124,070.00 3 TNT Soft 10,000.00 5,000.00 8 (0) 8 (0) 8,250.00 13,070.00 8+4 8,250.00 13,070.00 4 8,250.00 4 WBR Zona 49,500.00 164,300.00 6 (1) 3 (2) 49,300.00 177,970.00 6+2 49,300.00 177,970.00 2 Amal 13,250.00 26,550.00 1 (0) 1 (1) 13,250.00 22,800.00 1+1 13,250.00 23,600.00 1 CNC Lab 45,000.00 216.850,00 6 45,000.00 223.053,00 6 124 ANNEX VIII: BENEFICIARY SURVEY RESULTS 125 126 127 128 129 130 For each type of assistance provided by Diaspora Invest project, how much did it contribute to your business results (sales/exports, new jobs and investments)? It did not contribute to our business results. Somewhat effective Moderately effective Very effective Would not achieve my business results without the help of DI project Have not recevied this type of assistance Grant funds 3% 0% 35% 32% 13% 16% TA from a manufacturing expert 0% 0% 33% 20% 3% 43% Market linkages and market penetration (B2B, research and analysis, marketing and promotion) 3% 10% 33% 17% 0% 37% Worker training 3% 7% 13% 23% 3% 50% Various business information 3% 13% 33% 23% 3% 23% Institutional support in obtaining permits and licences 3% 20% 10% 3% 0% 63% Business linkages (business conferences and events) 3% 10% 32% 23% 3% 29% In your view, to what extent, on the scale from 1 to 5, did the type of TA you received from Diaspora Invest project contribute to achievement of your business results? SALES/EXPORT N/A 1 2 3 4 5 Grant funds 31% 0% 4% 15% 27% 23% TA from a manufacturing expert 50% 6% 6% 6% 17% 17% Market linkages and market penetration (B2B, research and analysis, marketing and promotion) 40% 0% 5% 15% 25% 15% Worker training 37% 5% 0% 11% 21% 26% Various business information 35% 5% 0% 25% 10% 25% Support of the Diaspora Business Center 47% 0% 0% 16% 5% 32% Business Networking 33% 11% 0% 17% 17% 22% In your view, to what extent, on the scale from 1 to 5, did the type of TA you received from Diaspora Invest project contribute to achievement of your business results? NEW EMPLOYMENT N/A 1 2 3 4 5 Grant funds 16% 0% 4% 16% 20% 44% TA from a manufacturing expert 55% 10% 0% 5% 15% 15% Market linkages and market penetration (B2B, research and analysis, marketing and promotion) 44% 0% 17% 17% 22% 0% Worker training 39% 6% 0% 17% 6% 33% Various business information 41% 6% 6% 24% 12% 12% Support of the Diaspora Business Center 59% 0% 6% 12% 6% 18% Business Networking 41% 12% 12% 6% 18% 12% 131 In your view, to what extent, on the scale from 1 to 5, did the type of TA you received from Diaspora Invest project contribute to achievement of your business results? NEW INVESTMENT N/A 1 2 3 4 5 Grant funds 23% 4% 4% 4% 27% 38% TA from a manufacturing expert 56% 11% 0% 6% 17% 11% Market linkages and market penetration (B2B, research and analysis, marketing and promotion) 42% 11% 0% 21% 21% 5% Worker training 35% 6% 6% 18% 12% 24% Various business information 39% 6% 6% 17% 11% 22% Support of the Diaspora Business Center 59% 0% 6% 12% 18% 6% Business Networking 47% 6% 6% 12% 18% 12% 132 Yes, I received information about investment opportunities 56% Yes, I received information about company support programs 63% Yes, I received information about financing available to support company projects 38% Yes, I received information about other companies' operations and about their products 50% Yes, I exchanged contact information with other companies 63% Yes, I established a business contact which didn’t lead to a business cooperation/commercial transaction 19% Yes, I established a business contact which led to further discussions about potential business cooperation 25% Yes, I established a business contact that led to business cooperation 6% No, I had not benefitted from attendance at these events 0% Yes, but none of the above, but (please state how you benefitted from attendance at these events) 0% Do you find that these conferences, info sessions or any other events organized by Diaspora Invest project were beneficial for your business (you may choose more than one answer) 133 134 135 ANNEX IX: GRANTS AND MAIN INDICATORS Exhibit % of grant value to the fixed assets value in comparison with main indicators Name Used grant amount from close out % of individual grant in grant portfolio % Grant value in fixed assets (FA) 2018 Fixed assets and long term investments 2017 Fixed assets and long term investments 2018 Differenc e in absolute value of FA (BAM) Based on hours worke d 2017 Based on hours worke d 2018 Difference in absolute number of employees Delta Plus BH 43,940.00 12.48 1.61 2,046,322 2,721,209 674,887 31 87 56 BMT Engineering 25,000.00 7.10 2.91 557,123 858,044 300,921 8 11 3 ADV PAX 21,683.00 6.16 3.58 1,047,088 605,233 -441,855 14 49 35 FANA Arifagic 35,000.00 9.94 6.67 488,834 524,799 35,965 9 12 3 Code Line Solution 12,000.00 3.41 6.75 51,004 177,903 126,899 8 10 2 Suco 35,000.00 9.94 10.05 251,733 348,204 96,471 14 30 16 Bonatura 34,000.00 9.66 16.03 71,488 212,099 140,611 1 3 2 Master Inzinjering 15,800.00 4.49 28.36 29,355 55,716 26,361 2 2.5 0.5 Talbot (Zendev) 34,917.00 9.92 29.79 54,728 117,197 62,469 9 21 12 FITS doo 19,250.00 5.47 38.75 14,268 49,679 35,411 3 7 4 FREUND Elektronika 31,164.00 8.85 62.62 41,922 49,769 7,847 3 5 2 Probosing 21,697.00 6.16 90.08 2,155 24,087 21,932 6 6 0 TNT Soft 8,250.00 2.34 107.45 2,507 7,678 5,171 0 4 4 Movida 14,334.00 4.07 5,555.81 452 258 -194 3 4 1 TOTAL 352,035.00 100.00 1,092,896 140.5 Source: DI's documents and FIA/AFIP financial reports 136 ANNEX X: THE EVALUATION TEAM RESPONSES TO COMMENTS FROM THE IMPLEMENTING PARTNER Date: September 25, 2019 To: Elma Bukvic Jusic Development Assistance Specialist / MEASURE-BiH COR USAID/BiH Subject: THE EVALUATION TEAM’S RESPONSES TO COMMENTS FROM THE IMPLEMENTING PARTNER ON THE PERFORMANCE EVALUATION OF USAID DIASPORA INVEST (DI) REPORT Dear Ms. Bukvic-Jusic, Below, we have provided the evaluation team’s responses to comments received from the implementing partner on the Performance Evaluation of USAID Diaspora Invest Activity (DI) report. RESPONSES TO THE IMPLEMENTING PARTNER’S COMMENTS IP’s Comment #1: Finding 3 (pg 26, paragraph 3), in discussing the recording of cost-share by DI grantees, notes that: “The evaluation team could not verify the reported result through the TRON database, although this mode of verification was possible with investments into fixed assets.” While this statement regarding the TRON database is accurate, Activity records offer detailed means of verification of cost-share, including invoices for expenses incurred, employment data, payroll information, payment slips, bank statements and other records related to grant implementation. These are available for all grantees, as required by the USAID Diaspora Invest Small Grants Manual (Grant Manual) and are easily verifiable. The Evaluation Team Response #1: Regarding the IP’s comment regarding finding #3, please note that a related discussion can be found in finding #5. The evaluation team did not have the opportunity to verify the total amount of reported cost￾share value since we were provided with hard copy examples of the Activity’s records for three grantees. Although willing to share information, the IP suggested that it would be impractical to share the documentation given the sheer volume. The evaluation team did not have access to the Activity’s management and information system which, as explained by the IP, contains electronic versions of verification documentation. IP’s Comment #2: Finding 9 (pg 28) notes that: “According to the data collected through the online survey, only about 6 percent of DI grantees and TA recipients used DBC services.” We want to clarify that all (i.e., 100 percent) Diaspora Invest beneficiaries have used DBC services, as each of the financial and technical assistance beneficiaries signed a TA agreement and was provided assistance through the DBC. We do agree however, that given the multiple service lines implemented through the DBC (“light touch” interventions handled solely by DBC staff, “full service” interventions implemented by the Activity staff and/or external 137 consultants, B2B and Networking events, DiaWorks knowledge sharing sessions, etc.) it might be confusing for a beneficiary to differentiate whether an intervention was implemented directly by DBC or by other parts of the Activity. The Evaluation Team Response #2: The evaluation team welcomes the IP’s explanation about potential reasons for the beneficiaries’ confusion. In finding #9, the evaluation team only presented data collected through key informant interviews and surveys. IP’s Comment #3: Finding 15 (pg 33) notes that “the evaluation team translated the formulation ‘source of direct investment that may not have otherwise occurred’ from the contractual definition of the expected results into the question ‘would you have invested in BiH, even if there had been no DI assistance?’” According to this survey, more than 3/4 (77%) of beneficiaries stated that had there not been DI grant support they would not have invested at all, would have invested a smaller amount, or would have invested at a later date. Accordingly, we recommend that the wording of this paragraph be reformulated to reflect this fact. The Evaluation Team Response #3: We feel that the proposed combination of categories masks nuance and important variation across the spectrum of the beneficiaries who would have changed their investment patterns but would still have invested in the absence of DI. We would therefore like to keep the analysis categories as presented. IP’s Comment #4: Finding 16 (pg 34) notes that roughly half (46%) of beneficiaries would not have hired new employees if there had been no DI assistance. This framing of the question misses the point in terms of assessing grant value added for employment. A much more revealing question would have been whether the company would hire the same number of employees without DI grant funding. In almost all of the grant interventions, DI grants were used to increase employment expansion for companies that were already hiring or planning to hire staff. The Evaluation Team Response #4: The questions were reviewed and approved by the Mission, and the evaluation team can only comment on survey results for the question that was asked. Also, it should be noted that when reporting the number of new jobs generated, DI was not distinguishing between new employees that beneficiaries planned to hire, even if there had been no DI assistance, and additional new employees that the beneficiary hired due to grant support. The DI reported the aggregated number of new jobs generated between two reporting periods. IP’s Comment #5: Finding 17 (pg 34) discusses the self-reported investment of DI beneficiaries and categorizes the beneficiaries per investment size. As company managers/owners usually overestimate investment, and given the relatively small number of respondents (sample size) of the online survey conducted by the evaluation team, a much more precise source of data for actual investment are the financial reports for all DI beneficiaries. These are available through public sources (FIA/APIF). According to this publicly available 138 data, cumulative investment during the previous two years (2018 and 2017) of the 60 DI beneficiaries receiving assistance as of June 30, 2018 is as follows: 35 beneficiaries (58%) invested up to 50,000 KM; 4 beneficiaries (7%) invested between 50,001 and 100,000 KM; 8 beneficiaries (13%) invested between 100,001 and 500,000; 6 beneficiaries (10%) invested between 200,001 and 500,000 KM; 6 beneficiaries (10%) invested between 501,000 and 1 million KM; 1 beneficiary (2%) invested between 1m and 2m KM; and none of the beneficiaries invested more than 2 million KM. This differs significantly from the self￾reported investment of the small pool of survey respondents presented in the report. Moreover, contrary to the conclusion of the evaluation, it reveals that an overwhelming majority of DI beneficiaries are micro and small businesses that have invested relatively modest sums over the previous two years. The Evaluation Team Response #5: The evaluation team agrees with this comment and the IP’s conclusion that micro and small companies constitute an overwhelming majority of DI beneficiaries. Also, the evaluation team agrees that AFIP/FIA data provide more accurate insight into the size of DI beneficiaries’ investments than self-reported data. The evaluation team added a new exhibit and explanation based on data provided by the IP. It should be noted that the evaluation team did not focus on the size of the company but rather the size of fixed assets among selected grantees and their influence on contractual results under component 2. IP’s Comment #6: Finding 22 (pg 37) relating to the financing gap, calculated as a difference between the total number of surveyed firms and those reporting as either having a loan or reporting no need of a loan. This ignores company age as a major determinant to finance and is therefore considerably depressed. Startups and early-stage companies rarely have access to commercial bank lending and as such, most of the company founders do not rely on this type of financing at such an early stage of development. This is particularly true in Bosnia and Herzegovina as commercial banks are not willing to lend to companies less than two years old without an established firm as a loan co-guarantor. The finding further notes that “more than half of DI grantees (52%) use internal sources to finance working capital and fixed assets” and that as such ”this group of DI grantee MSMEs was overrepresented in the DI grantee population relative to the share of this group in the total MSME population in BiH” when comparing the results of DI’s beneficiaries survey with The World Bank Group’s Gender MSME Access to Finance Survey (2018). This comparison is wholly flawed because it fails to recognize that DI beneficiaries are a considerably younger population of firms compared to those in the World Bank Group’s survey. The Evaluation Team Response #6: All survey responses related to the need for loans are provided by startups and early-stage companies assisted by DI. The evaluation team tried to acquire data about the World Bank survey sample, but the World Bank was unable to provide them, since the survey was conducted by an external contractor. Therefore, we agree with the IP’s comment and the part of the finding related to overrepresentation was deleted from the report. IP’s Comment # 7: Finding 23 (pg 38) establishes the ratio of grant to fixed assets as a proxy for grant value added, which is at best overly simplistic and at worst misleading. The size of fixed assets alone reveals little in terms of the future capacity of the firm to invest. Operating margin or earnings before interest, depreciation and amortization is probably a better indicator of the company’s ability to expand the business through 139 additional debt or other methods. When analyzed in this way, it is clear that the grant financing constituted a significant share of the beneficiaries’ investment capacity. However, even looking at the grant-to-fixed asset ratio, it is clear that the sample used in the report is skewed towards larger companies as they were simply the first to graduate from the BDM program. Looking at the total of 20 grant closeouts at June 30, 2019, the grant value to fixed assets ratio looks much different: 6 companies with grant value to fixed assets ratio of 0-10%, 4 companies with grant value to fixed assets ratio of 11-50%, and 10 companies with grant value to fixed assets ratio of 51-100% The Evaluation Team Response #7: Please note that this DI performance evaluation covers the period from April 2017 to June 15, 2019, when the evaluation team completed the data collection phase of the evaluation. Until June 15, 2019, the IP submitted data for 14 grantees through the grant closeout reports to the evaluation team. With that in mind, the evaluation team did not apply any sampling of the submitted data and used the entire population of grantees based on data provided by the IP. The grant-to-fixed assets ratio is one of the four types of data used to assess grant value added (grant-to-fixed assets ratio, interviews, surveys, the financing gap). However, it should be noted that six companies accounted for the bulk of the FY2 report results, comprising 78% of fixed assets and 45% of new jobs generated. Also, similar to the IP's comments related to the size of the assisted companies (micro, small, medium and large companies), the evaluation team never considered the size of the assisted companies as a relevant variable, but instead focused on the grant-to-fixed assets ratio. IP’s Comment #8: Findings 24 and 25 (pg 39) indicate that some of the beneficiaries interviewed or surveyed could not recall receiving technical assistance. As all of the grant and TA beneficiaries signed TA agreements with Diaspora Invest and have signed off on the TA final reports (for completed interventions), we can attribute this only to recall bias. Evaluation Team Response #8: The evaluation team agrees with the IP’s comment that this could be attributed to recall bias. A related footnote was added. IP’s Comment #9: Finding 32 (pg 42 and 43) notes the need for the DBC to target other markets, such as “local businesses interested in cooperation with diaspora investors and companies”. While we agree with the assessment, it is necessary to note that providing assistance to local firms (non-diaspora related) would require changes to the current scope of the Activity. Furthermore, the finding notes the need for a more active approach and outreach to potential investors abroad. Again, while we agree with the assessment, the Activity was envisioned as serving primarily diaspora investors who have either already made a decision to invest or are at a mature stage of the investment decision-making process. Proactive approach would require the Activity to reach out to potential investors in their host countries with a better-resourced promotion effort (road shows, pitching sessions in target countries, etc.), typical for this type of investment promotion. However, given that Diaspora Invest was envisioned as a pilot to test the development hypothesis of diaspora investment impact on economic development, this proactive approach would probably be excessive at this stage. However, any possible follow-on efforts to incentivize diaspora investment should take this into consideration and design and resource these types of instruments. Finally, 140 the finding also notes the lack of evidence of the quality of cooperation with local communities established through the MoU’s signed. While this is a rather recent effort, the cooperation has already yielded at least five referrals of diaspora companies seeking assistance, so we do not concur. The Evaluation Team Response #9: The evaluation team agrees with the IP’s comment that "any possible follow-on efforts to incentivize diaspora investment should take this into consideration and design and resource these types of instruments.” Please note that recommendations were primarily for USAID/BiH’s consideration and assumed the availability of additional funds that could alter the Activity’s scope of work. As per the IP comment regarding “the lack of evidence of the quality of cooperation with local communities established through the MoUs signed,” the evaluation team will add a note, that “this is a rather recent effort, the cooperation has already yielded at least five referrals of diaspora companies seeking assistance.” IP’s Comment #10: Finding 33 (pg 43) notes that “The evaluation team could not determine what actions DI was taking to reach local businesses interested in connecting with the BiH diaspora”. While this was not the key focus of DI’s activities, this finding is particularly confusing given that we have so far mobilized hundreds of local firms taking part in the networking and B2B events, such as Reconnect, BHDiaFor, DiaWorks and others, connecting local businesses with diaspora entrepreneurs and professionals. Furthermore, one of the key functionalities of the diasporainvest.ba website is the networking and virtual matchmaking between the local and diaspora businesspeople. While this is an area that holds a lot of promise for economic development and trade and investment integration of BiH firms, to be able to scale up these types of activities, an amended Activity scope would be needed to better link these types of activities with desired outcomes, results and indicators. The Evaluation Team Response #10: Please note that in the quoted text, the evaluation team was primarily referring to the lack of tailored activities. IP’s Comment #11: Finding 39 (pg 45) discusses the best positioning of the DBC in regard to the state-level institutions and its relationship with the local level. It is worth noting here that the State-level Working Group for the development of the Strategy for cooperation with diaspora has determined that neither FIPA, MoFTER, nor MHRR have the competences to host a state-level diaspora center and that this role could be performed by other actors, thus opening up the space for the Diaspora Invest DBC. The Evaluation Team Response #11: The evaluation team agrees with the IP’s statement regarding the position expressed by the WG. Please note that the evaluation team also took into account the view of other stakeholders interviewed. 141 IP’s Comment #12: Finding 41 (pg 46) indicates that “there is still a large share of respondents who did not find the information or services they needed on the portal. This finding suggest that portal contents should be regularly reassessed and updated”. We would like to note that 70% of respondents reported using the information and services provided by the web portal, which is a very significant share of companies. The web portal content is updated regularly, at least two times per week, thus ensuring that information contained is up to date and relevant for most potential users targeted by its content. The Evaluation Team Response #12: The evaluation team agrees with the IP’s statement that the web portal content is being updated regularly. Please note that the evaluation team was primarily referring to the key informant interviews, which suggested that many companies did not find the information they were looking for there. IP’s Comment #13: Finding 42 (pg 47) notes that “the majority of surveyed beneficiaries were not members of any diaspora network. This finding…further emphasizes the importance of and need for increased efforts to improve diaspora networking”. As all of the DI beneficiaries are included in the direct DI mailing lists, are distributed monthly newsletters and are regularly invited to all of the DI events, they are by definition part of the DBC diaspora network. The Evaluation Team Response #13: The evaluation team is of the opinion that diaspora networking consists of more than newsletter mailing lists. Please note that this suggestion that further support for diaspora networking is needed is directed toward USAID/BiH and not to DI. IP’s Comment #14: Finding 47 (pg 52, paragraph 2) discusses the characteristics and structure of DI beneficiaries, including their size. our beneficiaries we used the EU’s Small Business Act (). According to this categorization, out of the 68 beneficiaries served by the Activity as of August 31, 2019, 49 companies are categorized as micro (<10 staff and ≤€2m turnover), 19 companies are categorized as small (<50 staff and ≤€10m turnover), with no medium-sized (50<250 staff & 10m≤€50m turnover) or large companies. DI beneficiary companies are thus entirely micro (72%) and small (28%) firms, thus it is inaccurate to suggest that the selection criteria for assisted companies is skewed towards larger firms. The Evaluation Team Response #14: Similar to other IP comments, it should be noted that the evaluation team did not focus on the size of the company but rather on the size of fixed assets among selected grantees and the influence of this on contractual results under component 2. IP’s Comment #15: Findings 48 and 49 (pg 53) note that the grant evaluation process is not fully harmonized with the approved Small Grants Fund Manual, as the grant evaluation process, as implemented by DI, included site visits to 142 applicants that were not described in the Grant Manual. This was remedied in the interim period with the latest amendments to the Grant Manual submitted on August 16 and approved on September 6, 2019. The Evaluation Team Response #15: The IP’s comment does not require any response from the evaluation team. IP’s Comment #16: Finding 52 (pg 55) indicates that “there are no written procedures on how to conduct due diligence or negotiate grant amounts”. In implementing grant negotiations, DI’s Grant Manager has followed the procedures laid out in the Grant Manual and has documented the negotiations in the Grant Negotiation Memo prepared for each of the grant agreements. However, should this be deemed necessary by USAID, we stand ready to develop additional written procedures outlining the due diligence process and other criteria used in negotiating grants. The Evaluation Team Response #16: The evaluation team concurs with the IP’s comment and suggests that USAID consider asking for additional clarification and procedure outlining the due diligence process and criteria used in negotiating grants. IP’s Comment #17: Finding 59 (pg 60) indicates that the companies participating in DI’s survey were not sure about the purpose of the survey. The intended purpose and objectives of the survey were clearly stated in the introduction to the online survey conducted during December 2018 and January 2019. The Evaluation Team Response #17: It should be noted that the evaluation team did not say there was no explanation but that respondents were unaware of it. IP’s Comment #18: EQ 2 recommendations (pg 49, paragraph 2) note that “23 percent of beneficiary companies would have invested in BiH even without DI grants”. We suggest rephrasing this to state that 77% of companies would have not invested in BiH or would have invested a smaller amount or at a later date had there not been DI support. Moreover, if the evaluators had considered the effect on employment (in addition to investment) under this question, we are confident that this rate would have been even higher. In addition, we feel that the current selection criteria for both grant and TA beneficiaries are already very restrictive (ownership, size, age, business sector), and that further constraints would seriously hinder implementation of the Activity. As noted above, 72% of companies served by Diaspora Invest so far are micro-level firms, while the remaining 28% fit in the small sized company category. As reflected in the evaluation report, the current beneficiaries are a very diverse and heterogeneous group when it comes to the business sector, location, investment origination and other characteristics, so we feel very confident that the development objectives of the Activity are best served through this varied pool of beneficiaries. Finally, any further restriction in terms of criteria without a clearly defined objective in terms of grant additionality would, in our opinion, not bring the desired result. Thus, we do not agree with the recommendation that USAID should adjust the beneficiary criteria at this time. 143 The Evaluation Team Response #18: Please see our response to comment #3. The evaluation team agrees with the IP’s comment that the existing selection criteria are already very restrictive, especially in terms of business sectors, and the proposed constraints are related only to a rather small share of beneficiaries assisted so far where the added value of disbursed grants is questionable. IP’s Comment #19: EQ 2 recommendations (pg 50, paragraph 2) indicate that “The latest contract modification related to Component 2 set the new cost-share investment target that recognizes working capital expenditures, workforce development expenditures and in-kind contributions as investments, rather than instead of used fixed assets. However, the evaluation team is of the opinion that for Component 2 and the small grants fund, USAID/BiH should consider keeping the same grant purpose as in the second and third requests for grant applications (investment into fixed assets).” To clarify, the contract modification allowed for the said expenditures to be counted as grantee cost-share, but still cannot be the purpose of grant funding. As illustrated by RFA 4 and 5 (as well as the two preceding calls) the procurement of fixed assets remains the sole purpose of the small grant. Thus, this recommendation should be withdrawn. The Evaluation Team Response #19: The evaluation team concurs with the IP’s comment and recommendation. The recommendation was deleted. IP’s Comment #20: EQ 3 recommendations (pg 59) discussing the allowable uses of grants and revisions to the Grant Manual should take into account that, as noted above, the sole purpose of grant funding, as indicated in RFAs 2- 5, is the procurement of fixed assets. Furthermore, the modifications of the Grant Manual have already been implemented to ensure full harmonization with the implemented evaluation and selection process. The Evaluation Team Response #20: Thank you for this input. This is noted by the evaluation team. MONITORING AND EVALUATION SUPPORT ACTIVITY (MEASURE-BiH) Fra. Anđela Zvizdovića 1 UNITIC Tower B, Floor 21 71000 Sarajevo Bosnia and Herzegovina Phone: + (387) 033 941 676 contacts@measurebih.com www.measurebih.com