December 2017 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by International Development Group LLC. Mid-term Performance Evaluation of the BIZ+ Project 1 TABLE OF CONTENTS Executive Summary ..................................................................................................................................... i 1.Introduction .............................................................................................................................................. 1 Evaluation Purpose ...........................................................................................................................................................................1 Evaluation Questions .......................................................................................................................................................................1 2.Project Background.................................................................................................................................. 1 Project Objectives and Indicators .............................................................................................................................................1 Brief History of Project and Scope of Activities ................................................................................................................2 3.Evaluation Methodology and Limitations ............................................................................................... 3 Methodology.........................................................................................................................................................................................3 Secondary Data ..............................................................................................................................................................................3 Online Survey .................................................................................................................................................................................3 KIIs.....................................................................................................................................................................................................3 Limitations.............................................................................................................................................................................................4 Design Limitations.........................................................................................................................................................................4 Implementation Limitations.........................................................................................................................................................4 4. Findings and Conclusions ........................................................................................................................ 4 4.1: Project Results vs. Objectives ....................................................................................................................................................4 EQ#1: To What Extent Were the BIZ+ Project Objectives Achieved?.........................................................................4 EQ#2: Did the Businesses Continue to Grow After the Completion of the BIZ+ Grant? .....................................10 EQ#2B: What Was the Nature of the Business Expansion for Firms after the Completion of the Grant?........10 4.2: Factors Influencing the Achievment or Non-achievment of the Project Objectives...................................................10 1. Political Environment.............................................................................................................................................11 2. Sector Business Enabling Environment .............................................................................................................12 3. Accessing Finance ...................................................................................................................................................12 4. Gender......................................................................................................................................................................13 5. Cultural Context....................................................................................................................................................13 6a. Type of Business (Sector).....................................................................................................................................14 6b. Type of Business (Legal Form of Ownership).................................................................................................15 7. Location ....................................................................................................................................................................15 8. Management Capacity and Market Orientation .............................................................................................17 9. Business Size ............................................................................................................................................................18 EQ#1B: What are Business Owners’ Perceptions of the Value of the Technical Assistance Provided by BIZ+ in Achieving or Not Achieving Objectives?...............................................................................................................................19 EQ#2C: Is There Any Correlation Between the Success of the Organization and the Percentage of its Matching Contribution to the BIZ+ Investment? ................................................................................................................20 4.3: Has There Been a Spill-over Effect of BIZ+-supported Businesses in the Primary BIZ+ Sectors, Such as Small￾scale Manufacturing and Food Production?....................................................................................................................................21 EQ#3A: How Have BIZ+ Activities Affected Suppliers, Customers and Workers Trained?..................................21 EQ#3B: What Effect Has BIZ+ Project Had on the Business Environment, Processes, and Networks?.............23 4.4: What Are The Recommendations for the Extension Period of the BIZ+ Project.....................................................25 EQ#4A: What Are the Successful Aspects of the Project that Should be Continued in the Next Phase?..........25 EQ#4B: Are There any Potential Collaborations to Leverage BIZ+ Resources, including Donor Programs, Private Sector and Financial Sector?.......................................................................................................................................26 EQ#4C: In What Ways Can the Participation of Women be Improved in the Expansion of the BIZ+ Project? ..........................................................................................................................................................................................................27 5.Recommendations.................................................................................................................................. 28 5.1 Consider What is The Best Project Model to Support Target SMEs ...........................................................28 5.2 Consider Alternative Models For Supporting Access To Finance..................................................................29 5.3 Select Beneficiary Firms Based On Market Orientation, Management, And Financial And Economic Viability...........................................................................................................................................................................29 5.4 Strengthen The Current Emphasis On Occupational Safety, Health, And Environment .................30 2 5.5 Increase Consultation With Grantees On Equipment Procurement Decisions ............................ 30 5.6 Continue to Promote the Inclusion of Vulnerable Groups as Project Partners .....................................30 5.7 Improve Performance Management Systems...........................................................................................................31 Annex I: Infographic ....................................................................................................................................... 32 Annex II: Evaluation SOW..................................................................................................................................................................34 Annex III: Sample Selection ................................................................................................................................................................46 Annex III-A: Selection Process for Sample of 20 Grantees For KIIs..............................................................................46 Annex III-B: KII Sample Selection (Continued)....................................................................................................................47 Annex III-C: KII Sample Selection (Continued) ...................................................................................................................48 Annex IV: Data Collection Instruments..........................................................................................................................................49 Business Online Survey Questionnaire ..................................................................................................................................49 Guidelines For KIIs With Businesses And Business Support Organizations................................................................55 Guidelines For KIIs With Businesses.................................................................................................................................55 Guidelines For KIIs With Business Support Organizations..........................................................................................57 Annex V: BIZ+ Annual Performance Data .....................................................................................................................................59 Annex VI: Case Studies........................................................................................................................................................................60 Annex VII: Sources of Information ...................................................................................................................................................60 Annex VIII: List of Grantees Interviewed .......................................................................................................................................64 Annex IX: Access to Finance Programming ...................................................................................................................................66 Annex X: Scopes of Work For Competitiveness Projects........................................................................................................69 Annex XI: DCA Loan Guarantee Programs ..................................................................................................................................75 Annex XII: Review of Documents and Forms Used by BIZ+ in the Solicitation, Review, And Approval Of Grant Applications............................................................................................................................................................................................78 Annex XIII: Disclosure of Conflict of Interest Form...................................................................................................................84 TABLE OF FIGURES Figure 2.1.1: Grants Disbursed by Region and Phase……………………………………………….…………….. 3 Figure 4.1.1: Grantees Achieving Job Creation Targets by Sector and Subsector………………………………... 6 Figure 4.1.2: Direct Jobs in Grantee Businesses by Location……………………………………………………… 6 Figure 4.1.3: Jobs in Grantee Businesses by Sector………………………………………………………………... .7 Figure 4.1.4: BIZ+ Grantees’ Growth in KPIs over Past Three Years per Survey Respondents…………………. 8 Figure 4.1.5: Impact of BIZ+ on KPIs per Survey Respondents…………………………………………………… 8 Figure 4.1.6: Summary Analysis of 20 KIIs…………………………………………………………………………. 9 Figure 4.1.7: Growth in KPIs of Completed Phase 1 Grantees Last Three Years…………………………………10 Figure 4.2.1: Impact of External Factors on Achievement of BIZ+ Objectives per Survey Respondents…………11 Figure 4.2.2: Survey Comments………….…………………………………………………………………………13 Figure 4.2.3: % of Grantees That Have Achieved Direct Jobs by Grantee Language……………………………... 14 Figure 4.2.4: Grantees that achieved 100% or more of Job Creation Targets by Sector, Subsector, and Phase……….……………………………………………………………………………………… 15 Figure 4.2.5: Grantees that achieved 100% or more of Job Creation Targets by Legal Form of Business……….………………………………………………………………………………………….. 15 Figure 4.2.6: Grantees that achieved Job Creation Targets by Location and Phase………………………………. 16 Figure 4.2.7: Relationship between Job Target Achievement and BCA First Round Score………………………. 18 Figure 4.2.8: Relationship between Job Target Achievement and BCA Second Round Score…………………….18 Figure 4.2.9: Relationship between Percent of Total Job Target Achieved and Total Direct Employment ………19 Figure 4.2.10: Contribution of BIZ+ Business Advice and Training to Growth per Survey Respondents……….. 19 Figure 4.2.11: Subject areas and ratings in as expressed in KIIs……………………………………………………20 Figure 4.2.12: Relationship between Job Target Achievement and Grantee Contribution as % of Total Contributions ……….………………………………………………………………………………………. 21 Figure 4.3.1: Increase in BIZ+ Customers’ Sales per Survey Respondents ………………………………………. 22 Figure 4.3.2: Impact of BIZ+ on Local Suppliers per Survey Respondents………………………………………... 22 Figure 4.3.3: Workers Trained by BIZ+ Grantees per Survey Respondents……………………………………....23 Figure 4.3.4: Survey Responses ……………………………………...................................................................................24 3 Figure 4.4.1: Leverage Options and Frequency of Choice………………………………………………………… 26 4 ACRONYMS APR Annual Performance Report BCA Business Capacity Assessments BDS Business Development Services CEA Central Environment Authority COP Chief of Party DCA USAID Development Credit Authority EIRR Economic Internal Rates of Return EQ Evaluation Question ET Evaluation Team FIRR Financial Return Rates of Return FT Full Time FTE Full-Time Equivalent FY Fiscal Year ICEI Instituto Cooperazione Economica Internazionale ICT Information and Communications Technology IDB Industrial Development Board IFC International Finance Corporation GTZ-CEFE German Technical Corporation Competency-based Economies through Formation of Enterprise GOSL Government of Sri Lanka KII Key Informant Interview KPI Key Performance Indicators LKR Sri Lankan Rupee LOP Life of Project MD Managing Director MDF Medium Density Fiberboard N Number of Survey Responses NGO Non-Governmental Organization PMP Performance Management Plan PPA Public Private Alliance PVC Polyvinyl Chloride RFA Request for Applications ROI Return on Investment SAIL USAID Supporting Accelerated Investment in Sri Lanka SMEs Small and Medium Enterprises SOW Scope of Work TSO Technical Support Office UPS Uninterrupted Power Supply USAID United States Agency for International Development USD United States Dollar 5 USG United States Government WA Weighted Average WHH Women Headed Households i EXECUTIVE SUMMARY 1. INTRODUCTION 1.1 EVALUATION PURPOSE The purpose of the mid-term performance evaluation of the USAID VEGA/BIZ+ project is to assess the effectiveness and sustainability of the project in stimulating economic growth and job creation in targeted regions in Sri Lanka. To inform the expansion of the BIZ+ project to other geographic areas, the evaluation provides recommendations and lessons learned to help USAID, implementing partners, the Government of Sri Lanka (GOSL), and other donors implement similar type of projects in economically lagging areas. 1.2 EVALUATION QUESTIONS 1) To what extent were the BIZ+ project objectives achieved? 2) Did the businesses continue to grow after the completion of the BIZ+ grant? 3) Has there been a spill-over effect of BIZ+-supported businesses in the primary Biz+ sectors, such as small-scale manufacturing and food production? 4) What are the recommendations for the extension period of the BIZ+ project? 2. PROJECT BACKGROUND The BIZ+ cooperative agreement between USAID and VEGA/Land O’Lakes was signed in October 2011, two years after the end of a 26-year civil conflict in 2009 between Tamil rebels and the Government. The project had a budget of USD 22,673,259 for the period October 1, 2011 to September 30, 2015. In May 2017, the budget was increased to USD 26.8 million and the period of performance was extended to July 31, 2019. The Program Description in the original agreement described the project ‘Goals and Objectives’ as: Increased Jobs and Income Generation in Northern and Eastern Sri Lanka through Enterprise Development. The original targets included the creation of 10,000 jobs and $30 million of annual income for vulnerable households. The program utilizes a public private alliance (PPA) methodology issuing approximately 2-year matching investment grant agreements with SMEs. Businesses and BIZ+ implement an agreed expansion plan. Typically, BIZ+ provides equipment and capacity building technical assistance. Grantee businesses submit documentation on employees and suppliers and financial reports. 3. EVALUATION METHODS AND LIMITATIONS 3.1 METHODOLOGY The evaluation employed a mixed methods approach that included analyses of secondary data from BIZ+ project documents, an on-line survey in three different languages (English, Sinhala, and Tamil), and face-to-face key informant interviews (KIIs) with grantees, BIZ+ management and staff, and USAID. 3.2 LIMITATIONS Although the evaluation covers all 50 grants made by BIZ+, funding was available for visits to only 20 grantees. To mitigate the effects, the Evaluation Team carefully selected a sample in coordination with USAID that would be as representative as possible of the different characteristics of grantees (e.g. regions, sectors, gender, legal form of ownership, language, and size). The primary data collected for this evaluation (online survey and KIIs) are subject to recall bias and behavioral bias (Hawthorne effect). To minimize bias, the Evaluation Team used triangulation. An additional challenge concerned language. To minimize information lost during translation, the Evaluation Team Leader briefed the translator on the background of the project, the purpose of the evaluation, and the goal of the KIIs. ii 4. FINDINGS AND CONCLUSIONS 4.1: PROJECT RESULTS VS. OBJECTIVES EQ#1: To what extent were the BIZ+ project objectives achieved? EQ#2: Did the businesses continue to grow after the completion of the BIZ+ grant? EQ#2B: What was the nature of the business expansion for firms after the completion of the grant? Achievement of Project Objectives. BIZ+ has created employment and increased incomes for the vulnerable populations in the North, East, North Central, and Uva provinces of Sri Lanka. As of June 30, 2017, BIZ+ reported that the “Number of jobs created through USG assistance”1 reached 6,112, or 94% of the target of 6,500 (through July 31, 2017, the previous project end date). 18 of the 50 BIZ+ grantees, i.e. 36%, have met or exceeded their total job creation targets. BIZ+ has enhanced income levels by creating, rebuilding, and expanding enterprises in the four target provinces. It has been successful in uplifting the lives of the vulnerable populations, nullifying the feeling of powerlessness and rejection felt by the disadvantaged, due to the lack of earning opportunities in the post-crisis transition villages and low-income areas. Thus, during its six years of operation, it has contributed to reaching the vulnerable populations in the North, East, North Central, and Uva Provinces in Sri Lanka and addressing the challenges of rebuilding the economy. Over a third of the jobs have been created in Vavuniya (Northern Province) and Trincomalee (Eastern Province) Districts. Survey data show that more than 90% of respondents have increased employment in the past three years. While only four of the 20 enterprises visited by the Evaluation Team attained their targets for total job creation, it is, nevertheless noteworthy that over 90% of surveyed businesses reported increased total wages. BIZ+ grantees have had important multiplier effects on customers, suppliers, and on the local economies where increased wages and profits are spent. The two ice factories have been very successful in supporting additional income earning opportunities for their business customers in the fishing sector. Although there is a Government mechanism in place to promote employment, large implementation gaps exist due to constraints in funds and human resources. It would have been difficult for BIZ+ grantees in the former conflict affected villages to recover to the levels they have reached without BIZ+ support. Growth after Grant Completion: Grants to 32 of the 37 Phase 1 grantees have been completed. Of the remaining five, three had to be suspended or terminated2 for failure to meet the terms and conditions of their agreements, while two are still active due to performance issues that BIZ+ is attempting to resolve. Over 90% of the 32 Phase 1 grantees who have completed their grants have experienced growth in key performance indicators. Phase 2 grants will not be closed out until November 2017. Nature of Growth after Completion of Grant: Growth ranging from 1% to over 26% has been achieved by over 90% of BIZ+ graduates over the last three years in investment, sales, employment, wages, and profits. 1 This measure includes both direct job creation within assisted enterprises as well as indirect employment created in businesses that either buy from or sell directly to assisted enterprises. 2 The Project Implementer noted that all three terminations were made with minimal costs to the project. iii 4.2: FACTORS INFLUENCING THE ACHIEVMENT OR NON-ACHIEVMENT OF THE PROJECT OBJECTIVES EQ#1A: What role did factors such as political environment, sector business enabling environment, access to finance, gender and cultural context play in influencing the achievement or non-achievement of the project objectives? EQ#2A: How did factors such as access to finance, gender of owner, type of business and location affect the growth of businesses after completion of the grant? Political Environment: While the political environment during BIZ+ has been a contentious one, has had an impact on USAID funding decisions, and at times has made it difficult for the project implementer to operate, it has not been a strong factor that affected the business results of BIZ+ grantees. Grantees rated the political environment as the least important factor in their achievement of project objectives. Sector Business Enabling Environment: Government sectoral and trade policies play a role in the success or failure of several BIZ+ grantees, especially for those who depend on import tariffs to protect them from strong foreign competition from larger producers in India and China. Accessing Finance: On average BIZ+ enterprises considered access to finance as only a moderate constraint to their businesses. However, the Evaluation Team noted that existing financial regulations and financial instruments are generally not conducive to SME growth, and existing financial institutions are not responsive to SMEs’ needs for short and long-term debt. Gender: While respondents to the survey and the KIIs minimized the role of gender in determining project outcomes, female-owned businesses lagged male-owned businesses in achieving employment targets. This does not appear to be due in any way to the business acumen of female owners and managers, many of whom are at least as motivated and qualified as their male counterparts. This suggests that during Phase 3 of BIZ+, additional attention needs to be given to promoting qualified and motivated female owners and managers. Cultural context: There appears to be a performance gap between businesses that are owned and operated by different ethnic groups. This also suggests that during Phase 3 of BIZ+, additional attention needs to be given to promoting qualified and motivated owners and managers from all ethnic groups. Location: While not definitive, there may also be a performance lag between different project areas, with the original beneficiary provinces of North and East falling somewhat behind the North Central and Uva provinces in terms of performance. Type of Business (Sector): Food processing firms had the most success in achieving total job targets in Phase 1, while manufacturing firms have been in the lead in Phase 2. Type of Business (Legal Ownership): Businesses organized for profit have a higher rate of success in reaching job targets than non-profits. Conclusions regarding factors that were not specifically mentioned in the SOW but were identified by the Evaluation Team as playing an important role in the achievement or non-achievement of project objectives: iv Management Capacity and Marketing Orientation: The most important determinants of project success are strong management and a focus on market needs and requirements. EQ#1B: What are business owners’ perceptions of the value of the technical assistance provided by Biz+ in achieving or not achieving objectives? Grantees’ perceptions were generally positive about the value of BIZ+ technical assistance in helping them to achieve their objectives. Accounting/Finance and Office Management was found to be the most appreciated subject area of the technical assistance offered. Most of the businesses have made improvements in their management and operations with the help of BIZ+ technical assistance. Several grantees expressed disappointment with technical assistance related to specifications development and procurement for equipment. In at least one case, this has caused a serious shortfall in the quantity and quality of production, which in turn has resulted in poor financial returns. EQ#2C: Is there any correlation between the success of the organization and the percentage of its matching contribution to the BIZ+ investment? In principal, higher contributions demonstrate higher commitment and greater capacity on the part of grantees, but the calculated correlation between achievement of job targets and the percentage of contribution provides no evidence of such a relationship. 4.3: HAS THERE BEEN A SPILL-OVER EFFECT OF BIZ+-SUPPORTED BUSINESSES IN THE PRIMARY BIZ+ SECTORS, SUCH AS SMALL-SCALE MANUFACTURING AND FOOD PRODUCTION? EQ#3A: How have BIZ+ activities affected suppliers, customers and workers trained? BIZ+ investments have had some positive spillover effects, although they tend to be concentrated among a few grantees, especially businesses that service the agriculture and fisheries sector, and larger firms, like hotels and garment manufacturers, which train workers up to international standards. EQ#3B: What effect has BIZ+ project had on the business environment, processes, and networks? BIZ+ appears to have had positive effects on the business environment, processes, and networks that extend beyond the direct effects on grantees, customers, suppliers. This is principally in the realm of the new, more disciplined approach to business investing introduced by BIZ+ to Sri Lankan businesses. 4.4: WHAT ARE THE RECOMMENDATIONS FOR THE EXTENSION PERIOD OF THE BIZ+ PROJECT? EQ#4A: What are the successful aspects of the project that should be continued in the next phase? Transparent Selection Process: This should continue through well-publicized Requests for Applications. Ownership Issues: Privately owned and operated grantees outperform cooperatives. BIZ+ should continue to seek strong entrepreneurs whose businesses have strong forward and backward linkages in rural communities, such as companies that process agricultural products or provided inputs to agriculture and fisheries. v Occupational Safety, Health, and Environment: BIZ+ has exposed its grantees to higher standards in all three areas. These efforts should be continued and reinforced in Phase 3. Accounting and Financial Management: This rated highest among grantees. It should be continued through: 1) regular reporting; and 2) support for accounting and financial management systems. Focus on Vulnerable Populations: The strong commitment of BIZ+ to vulnerable populations in all regions should continue. Such populations included persons who have been affected by the civil war, low-income groups, persons with physical or mental challenges, and groups that have suffered discrimination due to ethnic, religious, gender, or other reasons. There is no conflict between serving the interests of such groups and financial sustainability. EQ#4B: Are there any potential collaborations to leverage BIZ+ resources, including donor programs, private sector and financial sector? There are several potential collaborations to leverage BIZ+ resources: Donors: The World Bank Agriculture Modernization Project under the Ministry of Primary Industries and Ministry of Agriculture is a good opportunity for BIZ+ supported SMEs to finance expansion. Financial Sector: The SME financial services of the commercial banks could be useful in a tripartite cooperation (BIZ+, grantee, and bank) to buy down the risk of the commercial lender. This could possibly be supported through USAID’s Development Credit Authority. EQ#4C: In what ways can the participation of women be improved in the expansion of the BIZ+ project? The two successful women-owned businesses supported by BIZ+ are privately owned, limited liability companies, owned and managed by very dynamic female entrepreneurs. BIZ+ needs to replicate these successes in the next phase, perhaps by focusing more attention on privately owned and operated firms rather than cooperatives. 5. RECOMMENDATIONS The following recommendations are in addition to those that were specifically requested by the evaluation scope of work under Evaluation Question #4 above. 5.1 CONSIDER WHAT IS THE BEST PROJECT MODEL TO SUPPORT TARGET SMES USAID should consider alternative models for supporting target SMEs. The model used by BIZ+, i.e. grant funding of investment for selected SMEs, tends to be used as a stopgap measure in fragile states, for instance at the end of conflicts or right after natural disasters, when normal economic relations have been interrupted and until they can be put back into effect. While the grant model of BIZ+ might have made sense in the immediate post-war period, with the conflict now in the past, other models might be more appropriate. 5.2 CONSIDER ALTERNATIVE MODELS FOR SUPPORTING ACCESS TO FINANCE USAID should consider alternative approaches to promote access to finance for SMEs, including advisory and financial support for: 1) commercial banks; 2) non-bank financial institutions; 3) “embedded financing” within value chains between large firms (input suppliers or wholesale buyers) and SMEs; 4) vi equity funds, such as small business investment funds; and 5) the use of USAID’s Development Credit Authority to support any of the first four financing vehicles. 5.3 SELECT BENEFICIARY FIRMS BASED ON MARKET ORIENTATION, MANAGEMENT, AND FINANCIAL AND ECONOMIC VIABILITY. BIZ+ should strengthen its project selection process to select firms that are market-oriented, i.e. firms that clearly serve a defined market demand, demonstrate likely financial and economic viability, and have motivated and competent management. It is important to look at both financial internal rates of return and, to some extent, economic internal rates of return. 5.4 STRENGTHEN THE CURRENT EMPHASIS ON OCCUPATIONAL SAFETY, HEALTH, AND THE ENVIRONMENT BIZ+ has adopted sound policies for occupational safety, health, and environment issues affecting its grantees, and attempted to gain the compliance of grantees to relevant standards. During Phase 3, it should strengthen its emphasis on these issues among current grantees, whose agreements are still in force, and in the selection of new grantees. 5.5 INCREASE CONSULTATION WITH GRANTEES ON EQUIPMENT PROCUREMENT PROCUREMENTDECISIONS Selection and procurement of equipment/machinery should be done in closer consultation and coordination with the grantee and with outside technical experts as required to ensure that equipment purchased meets the assumptions made in project feasibility studies. A final sign-off on equipment specifications should be obtained from grantees. 5.6 CONTINUE TO ACTIVELY PROMOTE THE INCLUSION OF VULNERABLE GROUPS AS PROJECT PARTNERS BIZ+ should actively recruit and develop qualified and motivated female and minority owners and managers during the next phase of the project. BIZ+ should extend its outreach to these groups through organizations like higher educational institutions and regional chambers of commerce. This could include the establishment of ‘set-asides’ as well as incremental funding and active coaching for promising entrepreneurs. 5.7 IMPROVE PERFORMANCE MANAGEMENT SYSTEM BIZ+ should improve its performance management systems by more clearly distinguishing among the effect of its programs on grantee businesses and their employees, the ‘spillover’ effects on buyers and suppliers in the same value chains, and employment created through the ‘multiplier effect’ in other businesses. Separate targets should be established for direct employment within assisted businesses, ancillary employment created within customers and suppliers within the same value chain, and any other employment created through the ‘multiplier effect.’ The system should guard against any double counting of employment. 1 1. INTRODUCTION EVALUATION PURPOSE The purpose of the mid-term performance evaluation of the USAID VEGA/BIZ+ project is to assess the effectiveness and sustainability of the project in stimulating economic growth and job creation in targeted regions in Sri Lanka. The evaluation includes an assessment of the project’s progress against its Development Objective and intended results as per the BIZ+ cooperative agreement and modifications. To inform the expansion of the BIZ+ project to other geographic areas, the evaluation provides recommendations and lessons learned to help USAID, implementing partners, Government of Sri Lanka, and other donors implement similar type of projects in economically lagging areas. EVALUATION QUESTIONS Together with the USAID Sri Lanka Mission, the Evaluation Team developed evaluation questions and sub-questions that address four fundamental topics along the project’s theory of change. The overarching questions are: 1) To what extent were the BIZ+ project objectives achieved? a) What role did factors such as the political environment, sector business enabling environment, access to finance, gender and cultural context play in influencing the achievement or non￾achievement of the project objectives? b) What are business owners’ perceptions of the value of the technical assistance provided by BIZ+ in achieving or not achieving objectives? 2) Did the businesses continue to grow after the completion of the BIZ+ grant? a) How did factors such as access to finance, gender of owner, type of business and location affect the growth of businesses after completion of the grant? b) What was the nature of business expansion for firms after the completion of the grant? c) Is there any correlation between the success of the organization and the percentage of its matching contribution to the BIZ+ investment? 3) Has there been a spill-over effect of BIZ+-supported businesses in the primary Biz+ sectors, such as small-scale manufacturing and food production? a) How have BIZ+ activities affected suppliers, customers, and workers trained? b) What effect has BIZ+ project had on the business environment, processes, and networks? 4) What are the recommendations for the extension period of the BIZ+ project? a) What are the successful aspects of project that should be continued in the next phase? b) Are there any potential collaborations to leverage BIZ+ resources, including donor programs, private sector and financial sector? c) In what ways can the participation of women be improved in the expansion of the BIZ+ project? 2. PROJECT BACKGROUND PROJECT OBJECTIVES AND INDICATORS The Program Description in the original agreement between USAID and VEGA/Land O’Lakes signed in October 2011, with a budget of USD 23,999,645 for the period October 1, 2011 to September 30, 2015, described the project’s ‘Goals and Objectives’ as follows (italics and bolding added for emphasis): 2 “The Land O’Lakes team has designed a responsive technical approach that incorporates the three BIZ+ components to achieve the overarching program objective of: Increased Jobs and Income Generation in Northern and Eastern Sri Lanka through Enterprise Development. Through successful implementation of BIZ+, Land O’Lakes will create 10,000 jobs, resulting in USD 30 million of annual income for vulnerable households in Sri Lanka’s Northern and Eastern Provinces. Leveraging previous USAID investments in the North and East, BIZ+ will provide USD 14.4 million in grants, ranging from USD 100,000 to USD 500,000, to new and existing enterprises. In addition, Land O’Lakes will secure a minimum 12% cost-share (USD 2.86 million) and will leverage another USD 12.96 million from grant recipients and program partners.” The agreement has been modified several times since its inception. However, the project objectives have remained the same. The most important modifications were: • Modification 1, dated August 10, 2012, expanded “the geographic target areas to include ‘border’ areas of Northern and Eastern provinces, such as Uva, North Central, and North￾Western provinces.” • Modification 5, dated November 25, 2013, reduced the budget to USD 17.6 million, eliminated the budget for policy reform, reduced the target for jobs to 5,000, and reduced the target for income to ‘USD 4 million of annual income for vulnerable households in Sri Lanka’s North, East, and North Central provinces’ (…economically lagging regions)’. • Modification 8, dated May 25, 2015, increased the budget to USD 22.4 million and extended the end date to July 31, 2017. • Modification 10, dated May 24, 2017, increased the budget to USD 26.8 million and extended the end date to July 31, 2019. BRIEF HISTORY OF PROJECT AND SCOPE OF ACTIVITIES The project has been implemented in a period of economic and political transition in Sri Lanka. The Cooperative Agreement between USAID and VEGA/Land O’Lakes International Development for the project was signed in 2011, just two years after the end of a 26-year civil conflict in 2009 between Tamil rebels and the Government. The project was designed as a four-year program with a starting date of October 1, 2011 and an end date of September 30, 2015. Due to changes in the political environment, the project was modified, as mentioned above, several times to either take advantage of changes in government or modify components to reduce overlaps with programs funded by other donors. For instance, in November 2013, one of the three project components, Component 3: Improvement of enabling environment for enterprises, was deleted as this area was covered by many other programs implemented by other donors. The total cost of the project was reduced at that time and the remaining funds were shifted to the other two project components. The election of reform-minded President Maithripala Sirisena in 2015 represented an opportunity for the US Government to support economic growth in Sri Lanka. As such, BIZ+’s funding was increased by USD 4.8 million and the period of performance was extended to July 31, 2019. While these changes did not have an impact on the grantees and the success of their performance, it did influence the project’s operations and management. As of June 30, 2017, BIZ+ had signed 51 grant agreements and made disbursements under 50 grants over two rounds in four regions as shown in Figure 2.1.1 below. Three of the 51 grants were terminated early due to performance issues; one of the three had no disbursements and the other two had nominal disbursements. 3 Figure 2.1.1 Grants Disbursed by Region and Phase Region Phase 1 Grants Phase 2 Grantees Total East 13 1 14 North 18 4 22 North Central 6 4 10 Uva 0 4 4 Total 37 13 50 3. EVALUATION METHODOLOGY AND LIMITATIONS METHODOLOGY The evaluation employed a mixed methods approach to achieve a diverse and holistic evaluation. The methods included: 1) analyses of secondary data sources including BIZ+ project documents, 2) an on￾line survey through Survey Monkey in three different languages (English, Sinhala, and Tamil), and 3) face￾to-face key informant interviews (KIIs). Secondary Data The Evaluation Team reviewed several project documents to give relevant perspective and background to the evaluation. The main documents include the cooperative agreement between USAID and the BIZ+ implementers, and project documents including work plans, performance management plans, periodic progress reports, and technical reports. Online Survey The Evaluation Team conducted an on-line survey using Survey Monkey software to attempt to: 1) collect data from the 30 grantees who were not included in the KIIs; and 2) have comparable survey data on all 50 grantees. The survey was distributed to the 50 grantees by e-mail. None of the three terminated or suspended grantees responded to the survey. Survey questionnaires were initially drafted by the Evaluation Team and then shared with USAID and BIZ+ Project staff for comment. After the inclusion of comments, the survey was translated into Sinhala and Tamil. The survey questionnaire is included in Annex IV. The Evaluation Team relied on a project participant list provided by the BIZ+ Project implementing partner, Land O’Lakes, to identify participants. The response rate was 86% (43 out of 50 grantees). KIIs The Evaluation Team conducted KIIs with 20 grantees (13 from Phase 1 and seven from Phase 2), a business services organization (BSO) (Ceylon Chamber of Commerce), the BIZ+ Chief of Party and senior staff of the BIZ+ Project. The Evaluation Team selected a sample of 20 grantees to interview in coordination with USAID that was as representative as possible of the different characteristics of grantees (e.g. location, phase, sector, size, gender, and ethnicity). Annex III provides additional details on sample selection while Annex V lists the grantees interviewed. 4 LIMITATIONS While several limitations were encountered, they do not have a direct impact on the robustness of the mid-term performance evaluation. The section below presents the limitations in detail as well as the steps taken by the Evaluation Team to minimize their effect on the findings and results of the evaluation. Design Limitations Although the evaluation covers all 50 grants made by BIZ+, funding was available for visits to only 20 grantees. Therefore, the findings for grantees not visited are restricted to secondary data from the BIZ+ project, and the results of the survey. To mitigate the effects of not meeting with all grantees, the Evaluation Team carefully selected a sample in coordination with USAID that would be as representative as possible of the different characteristics of grantees as noted above. All four provinces (North, East, North Central, and Uva) and 10 of the 12 districts where the project operates were visited (only Anuradhapura and Mannar were not visited due to time and budget constraints). Furthermore, out of the 11 sector classifications used by BIZ+, representatives from six of the most important (in terms of the numbers of grantees) were interviewed (manufacturing, food processing, handicraft, apparel, tourism, and energy). Implementation Limitations The primary data collected for this evaluation (online survey and KIIs) are subject to recall bias and behavioral bias (Hawthorne effect) that may make the program seem more or less successful than it actually is. Since a majority of the grantees have already graduated from the BIZ+ program (64%; 32 of 50), their recollection of past experiences might not be accurate. In addition, respondents might change their responses since they are aware they are being studied. To minimize bias, the Evaluation Team used triangulation to collect the same type of information from different sources. During the desk review, the Evaluation Team noted relevant data and information and then used these data to triangulate and correlate key phrases from the desk review to data collected during interviews and online survey data. An additional challenge concerned language. While the Evaluation Team Leader speaks neither Sinhala nor Tamil, the two Sri Lankan team members both speak Sinhala. They provided interpretation during interviews conducted in Sinhala. For Tamil-speaking grantees the Evaluation Team relied on a locally contracted interpreter. In many cases, though, Sinhala-speaking grantees spoke English and Tamil￾speaking grantees spoke some English and/or Sinhala. To minimize information lost during translation, the Evaluation Team Leader briefed the translator on the background of the project, the purpose of the evaluation, and the goal of the KIIs. 4. FINDINGS AND CONCLUSIONS Findings and conclusions are organized by the four principal evaluation areas of inquiry: 1. Project Results vs. Objectives (evaluation questions (EQs) 1, 2, and 2B); 2. Factors Affecting the Achievement of Project Objectives (EQs 1A, 1B, 2A, and 2C); 3. Spill-over Effects (EQ 3, 3A, 3B); and 4. Recommendations (EQ 4, 4A, 4B, and 4C). 4.1: PROJECT RESULTS VS. OBJECTIVES EQ#1: To what extent were the BIZ+ project objectives achieved? As discussed in the Background Section (Section 2), the BIZ+ project has two main objectives: Increased Jobs and Income Generation in Northern and Eastern Sri Lanka through Enterprise 5 Development. Although the geographic scope of the project was expanded to the North Central and Uva provinces, the objectives of job creation and income generation have remained unchanged. Conclusions: 1. BIZ+ has created employment for the vulnerable populations in the North, East, North Central, and Uva provinces of Sri Lanka, reaching close to 90% of its overall target of 6,500 jobs created by July 31, 2017. However, progress in creating jobs in either grantee businesses or in related buyers and suppliers may be lower than reported. This is because the current practice of combining employment targets for these two categories of employment prevents an analysis of target achievement by employment type. 2. Four of the 20 enterprises interviewed are highly successful and sustainable, 11 are moderately successful and sustainable, four are categorized as least successful and sustainable. It is too early to gauge the success of the remaining enterprises interviewed. 3. BIZ+ has enhanced income levels by creating, rebuilding, and expanding enterprises in the four target provinces. It has successfully uplifted the lives of vulnerable populations, nullifying the feeling of powerlessness and rejection felt by the disadvantaged due to the lack of earning opportunities in the post-crisis transition villages and low-income areas. It has helped address the challenges of rebuilding the economy in areas affected by the civil war. Findings/Evidence: To determine the extent that BIZ+ project objectives were achieved, the Evaluation Team analyzed data from three sources: • BIZ+ project. The latest BIZ+ Quarterly Report for June 30, 2017 provides the most recent data on performance versus the targets set in the last Annual Performance Report (APR) for the project through July 31, 2017. • Online survey. The business survey asked BIZ+ grantees to estimate the growth in several key performance indicators (KPIs) including employment, wages, profits, sales, and investment over the past three years. • KIIs. The Evaluation Team conducted 20 interviews with grantees, thirteen from Phase 1 and seven from Phase 2. All but two of the companies interviewed from Phase I have been closed out or completed. All seven of the interviewed Phase 2 BIZ+ grants are still in the ‘Active’ mode, with completion scheduled for November 30, 2017. According to data from the BIZ+ June 30, 2017 Quarterly Report, the “Number of jobs created through USG assistance” reached 6,1123, 94% of the target of 6,500 for July 31, 2017, the previous project end date. However, BIZ+ fell well short of the 50% target for female employment: 1,365 jobs created for females through June 30, 2017 versus a life of project (LOP) target of 3,250. 4 According to BIZ+, an additional 4,241 indirect jobs were created, versus a target of 10,000, due to increased expenditures by grantees and their employees.5 Calculations made by the Evaluation Team using results from the June 30, 2017 Quarterly Report, adjusted by data gathered in the KIIs, show that BIZ+ has achieved 89% of the overall job creation targets in the Project’s Performance Management Plan (PMP). 6 3 "The BIZ+ June 30, 2017 reports a total of 6,111 total jobs supported by the project through that date. According to calculations made by the IDG ET, the total is 6,112." 4 According to USAID, BIZ+ has suggested that the 50% project target for female employment be revised downward. The Evaluation Team recommends that USAID and BIZ+ establish ambitious and achievable targets for female employment. One possible starting point for the establishment of a target is the percentage of females in the Sri Lankan labor force, which was 30.9% in 2016, according to the World Bank and ILO. (https://data.worldbank.org/indicator/SL.TLF.TOTL.FE.ZS). 22.3% (1,365 of 6,112) of BIZ+ supported jobs to date have been for females. 5 Indirect employment is calculated using multipliers developed by the International Finance Corporation. 6 The Evaluation Team initially tried to disaggregate BIZ+ targets for ‘Employment and Income Earning Opportunities’ between: 1. ‘Direct (full-time (FT) and full-time equivalent (FTE) at businesses’ and 2. ‘Micro Enterprises/ Entrepreneurs (along the value 6 BIZ+’s LOP direct job target of 6,500 includes both full-time (FT) and full-time equivalent (FTE) jobs created within grantee businesses as well as ‘micro-enterprises/entrepreneurs (along the value chain)’. While BIZ+ reports the actual numbers achieved for each of the two categories, it does not break down the overall target by category. The June 30, 2017 Quarterly Report also provides performance data by grantee. Figure 4.1.1 below shows that 18 of 50 grantees, or 36%, have met or exceeded their overall job creation targets, including FT and FTE employment within their businesses, and ‘micro enterprises/entrepreneurs along the value chain.’ Figure 4.1.1 Grantees Achieving Job Creation Targets by Sector and Subsector Sector Sub-Sector % That Achieved 100% or More # That Achieved 100% or More Total Grantees Industry Manufacturing (N=24) 21% 5 24 Food Processing (N=15) 40% 6 15 Construction (N=1) 100% 1 1 Handicraft (N=2) 50% 1 2 Services Hospitality (N=5) 60% 3 5 Transport (N=2) 100% 2 2 Equipment Rental (N=1) 0% 0 1 Total N=50 36% 18 50 As shown in Figure 4.1.2 below, BIZ+ detailed data indicate that 1,677 (892 in the East and 785 in the North) of the total 2,6007 jobs created in grantee businesses have benefitted residents of the Northern and Eastern Provinces, the original project target regions. Figure 4.1.2 Direct Jobs in Grantee Businesses by Location Province # of Jobs District # of Jobs # of Businesses East 892 Trincomalee 284 3 Batticaloa 388 5 Ampara 220 6 North 785 Vavuniya 225 5 Mullaitivu 113 4 Jaffna 229 7 Kilinochchi 211 5 Mannar 7 1 North Central 642 Anuradhapura 327 5 Polonnaruwa 315 5 chain) using the BIZ+ June 30, 2017 Quarterly Report, Table 3. BIZ+ Investment Grants - Report on Expenditures, Cost Share and Jobs as of June 30, 2017. This was attempted because some BIZ+ job data targets show separate numbers for jobs in the businesses and jobs outside. However, in an e-mail dated September 21, 2017, the BIZ+ COP explained that: "the program does not have two targets; the definition in our performance results indicator combines, not separates, Direct FT and FTE from Micro Enterprises/Entrepreneurs (along the value chain)." In the future, the Evaluation Team strongly recommends that 1) job targets be disaggregated between those created in grantee businesses and those created in related buyers and suppliers in the same value chain, and 2) results for each target be reported separately. 7 The Evaluation Team’s detailed list of grantees and jobs created, which is from the detailed breakdown in the BIZ+ June 30, 2017 Quarterly Report, adds up to 2,600 versus the 2,599 given by BIZ+ in its June 30, 2017 Quarterly Report. 7 Figure 4.1.2 Direct Jobs in Grantee Businesses by Location Province # of Jobs District # of Jobs # of Businesses Uva8 281 Moneragala 267 3 Badula 14 1 Total 2,600 50 In terms of sectors, the BIZ+ Quarterly Report dated June 30, 2017 shows that most of the 2,600 direct jobs in BIZ+-supported businesses have been created in manufacturing and in food processing, as shown below in Figure 4.1.3. Figure 4.1.3 Direct Jobs in Grantee Businesses by Sector Sub-Sector Jobs in Grantee Businesses Jobs in Related Customers and Suppliers Total Jobs # of Businesses Manufacturing 1,452 1,095 2,547 24 Food Processing 470 2,090 2,560 15 Hospitality 350 181 531 5 Transport 209 30 239 2 Handicraft 81 63 144 2 Construction 31 26 57 1 Equipment Rental 7 27 34 1 Grand Total 2,600 3,512 6,112 50 As presented in the summary Performance Data Table in Annex V, which was updated by the Evaluation Team using data from the BIZ+ June 30, 2017 Quarterly Report, the project reports that it has exceeded the LOP target for income: a USD 9 million increase versus a target of USD 6.5 million. The LOP target for the number of vulnerable households benefitting from the project was also surpassed: 8,134 versus a target of 8,000. In terms of the financial performance of the assisted businesses, the June 30, 2017 Quarterly Report shows that 58% of grantees have achieved positive operating cash flow exceeding the target of 50%. In addition, 90% of grantees have adopted new technical or management practices due to BIZ+ assistance, matching the target of 90%. The business survey provides additional supporting data for the conclusions reached by the Evaluation Team. The survey asked BIZ+ grantees to estimate the growth in several KPIs (investment, sales, employment, wages, and profits, over the past three years). Survey data show that more than 90% of respondents (N=37) have increased employment in the past three years; more than 40% of the respondents reported that employment in their firms increased by more than 26% over this period (see Figure 4.1.4 below which presents BIZ+ grantees growth in KPIs). While most of the enterprises visited by the Evaluation Team have not been able to generate direct employment to the expected levels, it is, nevertheless noteworthy that approximately 95% of surveyed businesses have also managed to increase total wages (discussed in detail below). 8 Uva Province was added to the project in Phase 2. 8 The business survey also asked BIZ+ grantees “To what extent have the following indicators increased as a result of USAID Biz+ support...?” (See Figure 4.1.5 below). The vast majority of respondents, (92%) said there was some impact on employment. Slightly more than one-third of the respondents (35%) said that BIZ+ had a ‘very high’ impact on employment. Although there is a Government mechanism in place to promote employment, large implementation gaps exist due to constraints in funds and human resources. It would have been difficult for most BIZ+ grantees in the former conflict affected villages to recover to the levels they have reached without BIZ+ support. In terms of increases in income, two measures are relevant: wages and profits. According to the survey data, 95% of the respondents indicate that wages have increased; almost 30% of the total say that wages have increased by more than 26%. Profits have also improved. 97% of respondents report increased profits; and 65% say that the increase was at 6% or greater. (See Figure 4.1.5 above). The survey asked BIZ+ grantees to assess their perceptions of the impacts of BIZ+ on wages and profits (see Figure 4.1.5 above). A third of the respondents (31%) indicated that the project had a ‘very high’ impact on wages. In terms of the other measure of income, (i.e. profits), 22% of respondents said that the impact of BIZ+ was ‘very high’. In all cases, the weighted average (WA) on a scale of 1 to 5 (where 1=very low, 2=low, 3=moderate, 4=high, and 5=very high) was between 3 and 4. According to the KIIs, BIZ+ grantees have had important multiplier effects on customers, suppliers, and on the local economies where increased wages and profits are spent. For example, a large agricultural 0% 2% 5% 5% 3% 18% 15% 11% 11% 32% 21% 10% 16% 24% 35% 21% 24% 24% 32% 19% 39% 49% 43% 29% 11% 0% 10% 20% 30% 40% 50% 60% % of Survey Respondents Investment N=38 Sales Revenues N=41 Employees N=37 Wages N=38 Profits N=37 % Growth in KPIs Figure 4.1.4: BIZ+ Grantees' Growth in KPIs over Past Three Years per Survey Respondents 0% 1-5% 6-10% 11-25% 26% or more 3% 0% 3% 0% 3% 16% 3% 5% 8% 19% 22% 42% 27% 36% 27% 24% 25% 30% 25% 27% 38% 31% 35% 31% 22% 0% 20% 40% 60% Investment (N=37, WA=3.8) Sales (N=36, WA=3.8) Employment (N=37, WA=3.9) Wages (N=36, WA=3.8) Profits (N=37, % of Respondents WA=3.4) Rating Scale Figure 4.1.5: Impact of BIZ+ on KPIs per Survey Respondents Very low=1 Low=2 Moderate=3 High=4 Very high=5 9 enterprise indicates that it is purchasing produce from 788 small agricultural producers, while an ice producer reports that it sells ice to between 500 and 755 fishing businesses. Information from key informant interviews presents additional evidence to support the conclusions that BIZ+ was successful in achieving its objectives. Figure 4.1.6 presents the interviewed grantees (20 total) organized first by phase and then by the level of success of the grantees in achieving project objectives of increasing jobs and income for vulnerable populations. For each phase, companies are grouped into three categories: 1) highly successful and sustainable, 2) moderately successful and sustainable, and 3) neither successful nor sustainable. Companies in the first category have met or exceeded job creation targets within their companies and have excellent prospects. Grantees in the second category have created jobs in their businesses at a lower rate than expected, but are still successful and will likely be sustainable. Companies in the third category are well below employment targets and are unlikely to be sustainable. Four of the 20 firms are judged by the Evaluation Team to be highly successful and sustainable. 11 are considered moderately successful and sustainable. Another four are considered to have low success and sustainability, while it is considered too early to assess the performance and prospects of the remaining grantee. Annex VI presents case studies for each of the interviewed businesses and provides more information on these conclusions. Level of Success and Sustainability % of Total Job Target Achieved Sector Subsector Principal Products or Services Legal form Province Phase Language Gender Grantee Contribution as % of Total Investment 278% Industry Manufacturing Ice LLC North 1 Tamil Male 53% 184% Industry Food Processing Coconut oil LLC North 2 Tamil Male 48% 162% Industry Manufacturing PVC pipe and plastic products LLC North Central 1 Sinhala Male 67% 141% Industry Manufacturing Garment and Textile LLC Uva 2 Sinhala Male 56% 95% Industry Manufacturing Ice LLC East 1 Tamil Male 63% 94% Industry Manufacturing Household and office furnishings LLC Uva 2 Tamil Male 64% 90% Industry Manufacturing Wood briquets LLC East 1 Tamil Male 58% 89% Industry Manufacturing Steel Hardware LLC North 1 Tamil Male 72% 67% Services Hospitality Hotel, restaurant, and tourism LLC East 1 Tamil Male 87% 64% Industry Manufacturing Garment and Textile LLC North 1 Tamil Female 43% 60% Industry Manufacturing Polyethylene bags LLC East 1 Tamil Male 69% 44% Industry Food Processing Baked goods LLC North 1 Tamil Female 50% 44% Industry Manufacturing Steel doors and frames LLC North Central 2 Sinhala Male 62% 29% Industry Manufacturing Retreaded tires LLC East 1 Tamil Male 49% 3% Industry Food Processing Rice Coop North 2 Tamil Male 45% 24% Industry Manufacturing Paper LLC North 1 Tamil Male 53% 19% Industry Manufacturing Natural broom fibers Coop North 1 Tamil Female 31% 16% Industry Handicraft Ceramics Coop Uva 2 Sinhala Female 40% 3% Industry Manufacturing Rope LLC East 1 Tamil Female 44% Too early 62% Industry Food Processing Spices Co. Ltd. by Gty. Uva 2 Sinhala Male 53% Low High Medium Figure 4.1.6 Summary Analysis of 20 KIIs 10 EQ#2: Did the businesses continue to grow after the completion of the BIZ+ Grant? EQ#2B: What was the nature of the business expansion for firms after the completion of the grant? Conclusion: Approximately 50% of the Phase I grantees who have completed their grants reported growth in the two key project objectives (jobs and income) since the completion of their grants. In addition, these same firms also reported growth in sales and investments. Phase 2 grants will not be completed until November 2017 and are therefore not included in the analysis. Findings/Evidence: According to BIZ+ reports, about 15 of the 32 grantees whose grant agreements have been either closed out or completed have either maintained or increased employment and income after the completion or close-out of their grants. Several have also made additional investments and have obtained independent certifications. KIIs conducted with eight of these same 15 firms confirmed that after grant completion, they have maintained the gains made during the grant period, and in several cases, have expanded sales, employment, and/or investment. Survey data provide additional information about the performance of graduated grantees over the last three years. Figure 4.1.79 below shows that more than 94% of grantees who have completed their grants have experienced growth in all KPIs over the past three years. Less than 10% of the grantees report no growth in employment or wages. KPIs include sales, employment, wages, investment, and profits. It should be noted that both BIZ+ management and the USAID COR report that it is difficult to track financial performance of grantees because of delays in the submission of audited financial reports. 4.2: FACTORS INFLUENCING THE ACHIEVMENT OR NON￾ACHIEVMENT OF THE PROJECT OBJECTIVES10 9 Figure 4.1.7 summarizes the responses of businesses that completed their grants, while Figure 4.1.4 summarizes the responses of all grantees to the same question. For four of five indicators, the percent of grantees reporting ‘26% or more’ among completed grants has slightly decreased. For profits, the percent reporting ‘26% or more’ remains the same for both groups. 10 Section 4.2 also examines evaluation questions concerning the effect of technical assistance on project success and the possible relationship between grantee contributions and project success. 0% 4% 7% 0% 0% 19% 12% 7% 22% 30% 11% 19% 26% 30% 41% 22% 31% 33% 26% 19% 48% 35% 26% 22% 11% 0% 10% 20% 30% 40% 50% 60% Sales N=27 Employment N=26 Wages N=27 Investment N=27 Profits N=27 % of Completed Phase 1 Grantees % Growth in KPIs Figure 4.1.7: Growth in KPIs of Completed Phase 1 Grantees Last Three Years 0% 1-5% 6-10% 11-25% 26% or more 11 EQ#1A: What role did factors such as political environment, sector business enabling environment, access to finance, gender, and cultural context play in influencing the achievement or non-achievement of the project objectives? EQ#2A: How did factors such as access to finance, gender of owner, type of business, and location affect the growth of businesses after completion of the grant? The evaluation examined all seven factors listed in EQ#1A and EQ#2A to determine the role they may have played in the achievement of project objectives, i.e. job creation and income growth, both before and after the formal completion of the BIZ+ grants. It also identified two additional factors that may have impacted both the achievement of project objectives and the growth of businesses after grant completion: management capacity and market orientation. The evaluation further examined how the size of the businesses and project phase may have affected performance. As mentioned in Section 4.1 above, 18 of the 50 grantees, or 36%, met or exceeded their overall job creation targets. 1. Political Environment Conclusion: While the political environment during the implementation of BIZ+ has been contentious, has had an impact on USAID funding decisions, and at times has made it difficult for the project implementer to operate, it has not been an important factor in affecting the achievement of project objectives either before or after the conclusion of individual grant agreements. Findings/Evidence: BIZ+ grantees were asked in the survey to rank the importance of six of the seven factors listed in the evaluation scope of work in the achievement or non-achievement of project objectives. The seventh factor, gender, was the subject of a separate question, as discussed below. As shown in Figure 4.2.1 below, the most important factors in terms of weighted averages were in descending order (from left to right in the figure): 1) location; 2) business sector; 3) cultural context; 4) access to finance; 5) business enabling environment; and 6) political environment. Grantees rated the political environment as the least important factor in their achievement of project objectives. Political environment was rated slightly less than low. In the KIIs, grantees also rated the political environment as the least important factor in their achievement of project objectives. Only one of the 20 interviewed grantees said that the political environment had an influence in achieving project objectives. This grantee mentioned political interference experienced during the planning of the investment. Although a powerful politician pressured them to change the location to an adjacent district, the company did not comply. This same 3% 11% 8% 32% 18% 55% 3% 11% 19% 13% 16% 16% 27% 35% 43% 8% 34% 18% 43% 27% 19% 18% 24% 5% 24% 16% 11% 29% 8% 5% 0% 10% 20% 30% 40% 50% 60% Location (N=37, WA=3.8) Business sector (N=37, WA=3.3) Cultural context (N=37, WA=3.1) Access to finance (N=38, WA=3.0) Business sector enabling environment (N=38, WA=2.9) Political environment (N=38, WA=1.9) % of Respondents Rating Scale Figure 4.2.1: Impact of External Factors on Achievement of BIZ+ Objectives per Survey Respondents Very low=1 Low=2 Moderate=3 High=4 Very high=5 12 company, however, mentioned that officials in the local government, where the company is located, had been very helpful in providing the approvals and licenses to construct and operate the factory without delays. 2. Sector Business Enabling Environment Conclusion: Government sectoral and trade policies have played a role in the success of several BIZ+ grantees, especially for those who depend on import tariffs to protect them from strong foreign competition from larger producers in India and China. Findings/Evidence: As shown in Figure 4.2.1 above, the business sector enabling environment was rated low to moderate in terms of importance to the achievement of BIZ+ objectives. In KIIs, seven out of 20 enterprises revealed that their businesses are affected by factors that impact the entire business sector. For example: • Tariffs: Several grantees discussed the impact of tariffs on their profitability and growth. A manufacturer of polyethylene bags found it extremely difficult to compete with imports when the Government removed a 25% tariff imposed on imported sacks. The number of shifts had to be reduced from three to two to scale down production. After the company formed an association of poly sack producers in the country, they successfully lobbied government to reinstate the tariff. A furniture manufacturer complained about high tariffs on its primary input, imported steel. • Energy: The intermittent power supply is a problem for some companies, especially those with long production runs. • Infrastructure: On the positive side, a manufacturer of PVC products and polyethylene sacks, benefits from its location in an industrial zone with the necessary high-tension power and access to good roads. It exceeded its employment target by 60%. 3. Accessing Finance Conclusion: While access to finance was critical for all grantees, the free capital from BIZ+ somewhat alleviated the problems that are normally faced by most Sri Lankan SMEs in starting up or expanding their businesses. In addition, these better capitalized BIZ+ graduates are now better positioned to access commercial finance. Findings/Evidence: Eleven out of the 20 interviewees indicated serious difficulties in accessing finance. Of the eleven businesses that expressed serious difficulties, two were rated by the evaluation as high in success and sustainability, five were medium, and four were low. The nine firms that did not express serious difficulties were either rate as high or medium in success and sustainability. These findings are consistent with general findings about the problems SMEs face in Sri Lanka in gaining access to finance. In general, existing financial institutions are not responsive to their needs for short and long-term debt. According to interviewees, the cost of capital was too high and BIZ+ grants were important factors in their investment decisions. For instance, an ice factory owner said that his decision to expand its operation would have taken a few more years if the BIZ+ grant had not been received. Similar views were expressed by a furniture manufacturer, a producer of coconut oil, and others, who needed finance to diversify production. In the business survey, access to finance was rated as the fourth most important factor out of six factors in terms of weighted averages with a rating of moderate to high (see Figure 4.2.1 above). It should be noted that BIZ+ supported several of its grantees to apply and negotiate with commercial banks, and also collaborated on an outreach activity in the North and Eastern Provinces with RDB Bank. 13 4. Gender Conclusion: While respondents to the survey and the KIIs minimized the role of gender in determining project outcomes, female-owned businesses lagged male owned businesses in achieving employment targets. This does not appear to be due in any way to the business acumen of female owners and managers, many of whom are at least as motivated and qualified as their male counterparts. This suggests that during Phase 3 of BIZ+, additional attention needs to be given to promoting qualified and motivated female owners and managers. Findings/Evidence: None of the five female-owned grantees achieved their overall job creation targets, while 40% (18 of 45) of male-owned businesses did so. However, none of the 20 grantees interviewed said that gender had played a significant role in the achievement or non-achievement of project objectives. Several did raise gender issues that affect their businesses. For example, the female owner of a garment factory shared her challenges as a mother and a wife in coping with the dual role of managing the business and household commitments. The managing director of an ice factory finds it difficult to hire female workers in his ice factory due to the nature of the work, which requires the lifting of heavy objects and night shifts. The owner of a coconut oil processor raised the disparity in wages between male and female workers. The owner/director of the enterprise was unaware of labor laws, policies, and international conventions regarding gender discrimination. Management of a resort hotel also faces difficulty in recruiting females for work, because of the negative view in Trincomalee of women who work in hotels. In the online survey, almost 80% of the survey respondents (34 of the 43) answered the following question regarding gender: “Please comment on any gender issues that have an impact on your business?” Most respondents simply mentioned that there are no issues. However, several elaborated, and their responses are captured in the Figure 4.2.2 below. Figure 4.2.2: Survey Comments Responses "All have equal opportunity and treated as same. No issues in gender at work place." "Many post-conflict women-headed households in our expanded base" "Men are at work sites. Female staff in office. No issues." "Mostly women working in the factory. Men working in the marketing and sales section. No issues." "Negative impact from stake holders from Insurance companies." "No changes or issues in gender in our coop society." "No gender issues but able to express their views and comments." "No issues in gender. Men are working with us at night for safety of women. We do open shop till 10pm." "No issues working together. Everybody is aware of gender balance, respect each other and safety." "No issues, but treated equal." "We all work as a team no gender issue. We are all sisters and brothers. We all same and equal chances. We respect females." "We don't have any gender problem. There are some girls also working in office and fuel station, except in vehicle service section." "We have never come across gender issues as we instruct or educate our staff at the time of interview and close follow up with them, as we concern more on women as they are 90% of our work force." "We plan to employ more women who lead single parent households. Of total recruitment, we plan 70% female." 5. Cultural Context 14 Conclusion: There appears to be a performance gap between businesses that are owned and operated by different ethnic groups: 46% (6 of 13) of Sinhala businesses met job targets, while 32% (12 of 37) of Tamil businesses met job targets. This suggests that during Phase 3 of BIZ+, additional attention needs to be given to promoting qualified and motivated owners and managers from ethnic groups that have had more difficulty in achieving project targets. Findings/Evidence: In KIIs, several enterprises indicated that cultural issues have played a role in the achievement of project objectives. According to the managers of a furniture manufacturer, workers in the Uva Province town where their factory is located are ‘by nature’ culturally reluctant or slow in adapting new methods and procedures. A factory owner in Jaffna said that there is a bias against shop floor work in private enterprises; people in the area are accustomed to government service sector jobs. He added that during the war, migration to foreign countries was high and this trend still prevails in the Northern province. A manufacturer of plastic bags mentioned that production is constrained by power interruptions. While purchase of an industrial uninterrupted power supply (UPS) would remedy the problem, the company is unwilling to invest possibly due to a religious stricture on the payment of interest. And, as already mentioned above under ‘Gender’, management of a resort hotel also faces difficulty in recruiting females for work, because of the negative view in Trincomalee of women who work in hotels. As shown in Figure 4.2.3 below, which is based on BIZ+ project data on the characteristics of its grantees, 46% (6 of 13) of Sinhala businesses met job targets, while 32% (12 of 37) of Tamil businesses met job targets. 6a. Type of Business (Sector) Conclusion: Food processing firms had the most success in percentage terms and numbers in achieving overall job targets, due largely to the ancillary employment created in agricultural suppliers. However, in Phase 2 manufacturing has outperformed food processing in percentage terms in achieving job targets. This may change after Phase 2 grantees complete their grants in November 2017. Findings/Evidence: Figure 4.2.4 summarizes the success of BIZ+ grantees in achieving job targets by sector, subsector, and phase. Using total job creation achievements in grantee businesses and related firms in the value chain, food processing outperforms other sectors in the number of businesses that reached their overall job creation targets; manufacturing follows close behind in terms of numbers, and has a lower rate of success (21% versus 40% food processing). Much of the success of food processing is attributable to ancillary employment that was created in food processing value chains, mainly agricultural suppliers. In KIIs, several food processors mentioned the negative impact that recent droughts have had on the agricultural sector and the resulting secondary impacts on their businesses. 46% 32% 36% 0% 20% 40% 60% % of Grantees Figure 4.2.3: % of Grantees That Have Achieved Direct Job Targets by Grantee Language Sinhala N=13 Tamil N=37 Total N=50 15 6b. Type of Business (Legal Form of Ownership) Conclusion: Businesses organized for profit have a higher rate of success in reaching job targets than non-profits. Findings/Evidence: Figure 4.2.5 below examines the relationship between the legal form of ownership of BIZ+ grantees and the achievement of job creation targets. 39% of for-profit businesses, which account for 88% of all BIZ+ grantees, have achieved total job creation targets, whereas only 17% of non-profits have done so to date. Figure 4.2.5 Grantees that achieved 100% of Total Job Creation Target by Legal Form of Business Legal Form % # Total # of Grantees For-Profit 39% 17 44 Limited Liability Company (LLC) 35% 14 40 Partnership 0% 0 1 Sole Proprietor 100% 3 3 Non-Profit 17% 1 6 Cooperative 25% 1 4 Company Limited by Guaranty 0% 0 2 Total 34% 19 56 7. Location Figure 4.2.4 Grantees That Achieved 100% or More of Job Creation Targets by Sector, Subsector, and Phase Sector Sub-Sector Phase 1 Phase 2 Total % # Total # of Grantees % # Total # of Grantees % # Total # of Grantees Industry Manufacturing 16% 3 19 40% 2 5 21% 5 24 Food Processing 56% 5 9 17% 1 6 40% 6 15 Construction 100% 1 1 100% 1 1 Handicraft 100% 1 1 0% 0 1 50% 1 2 Services Hospitality 60% 3 5 60% 3 5 Transport 100% 2 2 100% 2 2 Equipment Rental 0% 0 1 0% 0 1 Total 41% 15 37 23% 3 13 36% 18 50 16 Conclusion: Based on the data collected during the evaluation, there may also be a performance lag between different project locations, with the original beneficiary provinces of North and East falling somewhat behind the North Central and Uva provinces in terms of performance. This could be due to several factors, including the fact that BIZ+ started making investments in North Central and Uva later than it did in the North and East, and therefore may have benefitted from better project selection procedures. It may also be due to differences in management capacity, market opportunities in the different regions, and/or the business environment. Findings/Evidence: In KIIs, two enterprises indicated that location affected the achievement of project objectives. A metal furniture manufacturer said that the location of their new product line in the Uva Province has made it more difficult to reach its primary growth market, Colombo and the Western Province. The distance results in delays in after sales service, trouble shooting, installation, and fine tuning needed to meet customer needs. Even though the company was able on its own to establish a show room and a smaller scale workshop in Colombo, the company owners still feel that the situation would have been better for their business if there had been an option to locate the investment in the Western Province. A manufacturer of steel doors raised a similar concern. Location in an industrial zone in the Western Province would have afforded them easier access to raw materials, reduced transport cost, and better access to appropriate infrastructure and workshops. As shown in Figure 4.2.1 at the beginning of Section 4.2 above, on average the respondents to the online survey rated location as the most important factor in determining achievement or non￾achievement of project objectives. While their reasons are not entirely clear, the two grantees mentioned above noted the advantages of being in a major urban area like Colombo, where the market is much larger and where there is better access to the principal port and suppliers. Figure 4.2.6 below presents the relationship between location (province and district) and the achievement of job creation targets. A third of the grantees (31%) in the East and North Provinces, the original project target areas, achieved their job creation targets. This contrasts with the 50% success rate for grantees in the North Central and Uva provinces, which became eligible for BIZ+ support in 2012. There are several possible explanations for this. For example, it might be due to: the additional experience that BIZ+ had gained after about one year in operation; different business environments; or other factors, like differences in the quality of management of selected grantee firms. Figure 4.2.6 Grantees That Achieved Job Creation Targets by Location and Phase Phase 1 Phase 2 Total Province District % # Total Grantees % # Total Grantees % # Total Grantees East Ampara 0% 0 5 0% 0 1 0% 0 6 Batticaloa 40% 2 5 40% 2 5 Trincomalee 33% 1 3 33% 1 3 East 23% 3 13 0% 0 1 21% 3 14 North Jaffna 43% 3 7 43% 3 7 Kilinochchi 0% 0 3 50% 1 2 20% 1 5 Vavuniya 25% 1 4 0% 0 1 20% 1 5 Mullaitivu 67% 2 3 0% 0 1 50% 2 4 Mannar 100% 1 1 100% 1 1 North 39% 7 18 25% 1 4 36% 8 22 East and North Total 32% 10 31 20% 1 5 31% 11 36 17 Figure 4.2.6 Grantees That Achieved Job Creation Targets by Location and Phase Phase 1 Phase 2 Total Province District % # Total Grantees % # Total Grantees % # Total Grantees North Central Anuradhapura 50% 2 4 0% 0 1 40% 2 5 Polonnaruwa 100% 2 2 33% 1 3 60% 3 5 North Central 67% 4 6 25% 1 4 50% 5 10 Uva Moneragala 33% 1 3 33% 1 3 Badula 100% 1 1 100% 1 1 Uva 50% 2 4 50% 2 4 North Central and Uva Total 67% 4 6 38% 3 8 50% 7 14 Total 38% 14 37 31% 4 13 36% 18 50 The following three additional factors were not specifically mentioned in the scope of work of the evaluation but were identified by the Evaluation Team as playing an important role in the achievement or non-achievement of project objectives: 8. Management Capacity and Market Orientation Conclusion: Based on the KIIs conducted with 20 of the 50 BIZ+ grantees, it appears that the most important determinants of project success are strong management and a focus on market needs and requirements. Grantees, who in KIIs demonstrated strong management skills and expressed a strong interest and commitment to meeting market demand, also tended to be the businesses with better performance in profitability, sustainability, and job creation. However, no conclusions can be drawn from correlations between the Business Capacity Assessment (BCA) scores and the percentage of total job targets achieved, which range from ‘none to very weak’ using first round assessment scores to ‘weak’ based on second round assessment scores. Findings/Evidence: KIIs provided ample evidence of a strong relationship between successful firms and the ability of their management teams to identify market opportunities, raise capital, establish supplier networks, delegate responsibilities, and devote resources to human resources development. Annex VI provides details of these interviews. BIZ+ project data permit an examination of the success of all grantees in reaching total job targets (i.e. both within the grantee business and in related suppliers and commercial customers in the same value chain) based on their scores on BCA11 conducted by BIZ+ staff. The revision of initial estimates failed to show any strong relationship. Figure 4.2.7 shows this correlation using their initial BCA scores for 47 businesses prior to receiving any assistance from the project (first round BCA scores), while Figure 4.2.8 shows the correlation for 27 firms after they had received financial and technical support from the project (second round BCA scores). Figure 4.2.7 shows no, or very weak, correlation12, while Figure 4.2.8 shows a weak correlation after grantees had receive some BIZ+ support. 11 Business capacity assessments examine seven factors that are considered important for the continuing success of a firm: governance, management, human resources, financial resources, business operations and viability, gender considerations, environmental considerations. 12 Correlation is measured by the coefficient of correlation, r or R, which have the following meanings: -1.0 to -0.5 or 1.0 to 0.5 = strong relationship; -0.5 to -0.3 or 0.3 to 0.5 = moderate relationship; -0.3 to -0.1 or 0.1 to 0.3 = weak relationship; -0.1 to 0.1 = none or very weak relationship. 18 9. Business Size Conclusion: There is a weak, positive correlation between the size of a grantee business, as measured by employment within the business, and the achievement of total job creation targets. Findings/Evidence: The Evaluation Team used BIZ+ data to examine the relationships between the size of the businesses, as measured by the number of persons employed within the business, and the achievement of total job creation targets, both within and outside the grantee business. Figure 4.2.9 below shows a positive but weak correlation, between the business size of the 50 grantees, measured by employment, and their success in achieving total job targets. A more useful comparison would have used the percentage of the direct, in-company, employment target achieved, but these data were not available. 0% 100% 200% 300% 400% 500% 0 0.5 1 1.5 2 2.5 3 3.5 4 % of Job Targe Achieved Business Capacity Assessment Score Figure 4.2.7: Relationship between Job Target Achievement and BCA First Round Score (N=47, r= -0.00018) r= -0.00018 0% 100% 200% 300% 400% 500% 0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 % of Total Job Target BCA Second Round Score Figure 4.2.8: Relationship between Job Target Achievement and Second Round BCA Score (N=27, r=-0.19) r= -0.19 19 EQ#1B: What are business owners’ perceptions of the value of the technical assistance provided by Biz+ in achieving or not achieving objectives? Conclusion: Grantees’ perceptions were generally positive about the value of BIZ+ technical assistance in helping them to achieve their objectives, and most of the businesses have made improvements in their management and operations with the help of BIZ+ technical assistance. However, three of the 20 companies interviewed expressed dissatisfaction with the assistance received from BIZ+ for the procurement of machinery and equipment. Findings/Evidence: The BIZ+ June 30, 2017 Quarterly Report provides the latest results on several performance measures related to technical assistance. Their positive trends provide indications of the value of BIZ+ technical assistance to the 50 grantees. 13 According to that report: • 90% (46 of 50) of grantees had ‘adopted new technologies and management practices to improve productivity in their business’ (versus a 90% LOP target) • 57% (about 15 of 26 grantees for whom data is available) had decreased their ratios of operating expense to revenue ‘within two years of BIZ+ involvement’ (versus a 50% LOP target) • 83% (29 of 33 for whom second round assessments were made) had improved their business capacity scores (versus a 75% LOP target) Survey participants were asked: “How would you rate the contribution of the following types of business advice and/or training received or funded by BIZ+ to your company's growth?” All five types of assistance were rated on average from more than ‘moderate’ to more than ‘high’ in importance (i.e. weighted average ratings ranged from 3.3 for marketing/product development and supply chain management to 4.1 for accounting/financial assistance), as shown in Figure 4.2.10 below. 13 There are some minor discrepancies in the BIZ+ calculations of percentages. 0% 100% 200% 300% 400% 500% 0 50 100 150 200 250 300 350 % of Job Target Achieved Total Employment Figure 4.2.9: Relationship between Percent of Total Job Target Achieved and Total Direct Employment (r=0.25) r= 0.25 5% 8% 3% 8% 8% 0% 3% 3% 8% 5% 3% 5% 8% 28% 18% 18% 21% 23% 15% 23% 49% 36% 33% 15% 31% 26% 31% 31% 26% 13% 0% 20% 40% 60% Accounting and Financial Management (N=39, WA=4.0) Human Resource Management (N=39, WA=3.9) Operations (e.g. quality control, saftety, technology...) (N=39, WA=3.8) Marketing and/or Product Development (N=39, WA=3.3) Supply Chain Management (N=39, WA=3.3) % of Respondents Figure 4.2.10: Contribution of BIZ+ Business Advice & Training to Growth per Survey Respondents N/A Very low=1 Low=2 Moderate=3 High=4 Very high=5 20 Sixteen out of the 20 enterprises interviewed have received technical assistance in at least one of the following areas. The subject areas and ratings as expressed in KIIs are provided in Figure 4.2.11 below. Figure 4.2.11: Subject Areas and Ratings as Expressed in KIIs Subject Area Satisfactory Unsatisfactory No Remarks Accounting/ Finance and Office Management 4 0 2 Human Resource Development: Leadership, Team Work, Attitude Change/ Motivation 3 0 3 Technical: Industry Specific 3 0 1 Marketing 2 0 0 General Advice: Project Planning, Procurement of Goods & Services, General Compliances 2 4 2 Quality Control and Assurance 1 0 1 Technical: Environment Compliance Management 1 0 1 Occupational Health and Safety 1 0 2 Except for the “General Advice” subject area, most persons interviewed expressed satisfaction with BIZ+ technical assistance. Accounting/Finance and Office Management was found to be the most appreciated subject area of the technical assistance offered. Most of the grantees stated that the training of their staff was very useful, and grantees felt that the systematic procedures and documentation adapted during the cooperation with BIZ+ was a new experience for them and they have learned best practices that are being assimilated into their organizational processes. Procurement Advice: In KIIs, three of the 20 grantees complained about wrong or inappropriate specifications of the machinery and equipment, and discussed resulting complications in their production operations. Among these interviewees, one company had to scale down production due to the purchase of machinery that differs from the specifications that were developed by a BIZ+ consultant. EQ#2C: Is there any correlation between the success of the organization and the percentage of its matching contribution to the BIZ+ investment? Conclusion: In principal, higher contributions demonstrate higher commitment on the part of grantees, but the calculated correlation between achievement of job targets and the percentage of contribution provides no evidence of such a relationship. Findings/Evidence: Figure 4.2.12 which is based on data from BIZ+ reports for 47 of 50 grantees, shows a weak, negative correlation (r=-0.26) between the achievement of job targets and the percentage of a grantee’s contribution to total investment costs. The contribution rates for 17 of 18 grantees for whom data are available, that met 100% or more of their total job targets, ranged from 26% to 68%. For those 32 grantees who did not meet job targets, contribution rates ranged from 41% to 98%. The correlation decreased when only the 28 closed out grants were analyzed. 21 4.3: HAS THERE BEEN A SPILL-OVER EFFECT OF BIZ+-SUPPORTED BUSINESSES IN THE PRIMARY BIZ+ SECTORS, SUCH AS SMALL-SCALE MANUFACTURING AND FOOD PRODUCTION? EQ#3A: How have BIZ+ activities affected suppliers, customers and workers trained? Conclusion: BIZ+ investments have had some positive spillover effects, although they tend to be concentrated among a few grantees, especially businesses that service the agriculture and fisheries sector, and larger firms, like hotels and garment manufacturers, which train workers up to international standards. Survey respondents express a much more positive view of the effects their companies have had on buyers, suppliers, and trained workers, than the persons interviewed face to face in KIIs. Findings/Evidence: As discussed in detail below, several of the 20 businesses interviewed mentioned the positive effects that their businesses have had on their customers, suppliers, and/or workers. Customers: Of the 20 grantees interviewed, three are facilitating the growth and/or profitability of their customers, while a fourth has the potential for doing so. Most of the other grantees are either producing for the consumer end market or for business customers who would otherwise buy from importers (e.g. polyethylene bag manufacturers.) The most notable and frequently cited examples are two ice manufacturers that provide reliable sources of ice for fishing businesses of all sizes, from the single operator to the wholesalers who purchase fish from multiple sources. Together these two firms account for 16% of the total number of related customers and suppliers reported in BIZ+ Annual Performance Report for FY 2016. 14 A third grantee provides a low cost, environmentally friendly wood briquette as fuel for boilers in industries like apparel and agricultural plantations that need heat and steam to process products. A fourth company, a Phase 2 investee that is just starting up a new agriculture products processing plant, plans to provide organic fruits and vegetables to the tourism industry in nearby Aragambay. Suppliers: Four out of 20 grantees interviewed have a positive effect on employment and income in their suppliers; all are processing commodities. The briquette manufacturer purchases wood chips and saw dust from 15 suppliers in Ampara, Batticaloa, and Kurunegala. A farmers’ cooperative, which is upgrading its rice processing facility, has 1,000 farmers that will mill rice at its new facility. An organic food processor currently purchases organic products from 87 farmers and plans to expand that number to 283. And an enterprise that produces coconut oil for the local market, has over 50 suppliers of 14 16%, or 700 enterprises, represents the share of supported enterprises (i.e. buyers or suppliers) reported to the Evaluation Team during the two KIIs out of a total of 4,441 enterprises reported. 0% 100% 200% 300% 400% 500% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% % of Total Job Target Achieved % of Grantee Contribution Figure 4.2.12: Relationship between Job Target Achievement and Grantee Contribution as % of Total Contributions (N=47, r= -0.26) r= -0.26 22 coconuts. The latter’s agreement with BIZ+ requires that it purchase raw material from at least 44 growers. Trained Workers: While almost all grantees provide some job training, three of the 20 interviewed grantees stand out as ones that are making important improvements in the skill levels of local workers. A resort hotel in Trincomalee has a well-structured training program for new employees. A foreign￾owned maker of women’s garments for foreign markets, is also committed to training new and current workers. A much smaller apparel maker reports having trained several workers who have left to work with larger apparel manufacturers. The BIZ+ Data Performance Management Plan tracks the following pertinent indicator: “Number of micro enterprises/micro entrepreneurs benefitting directly or indirectly by BIZ+ investment and service grants.” According to the BIZ+ June 30, 2017 Quarterly Report, 4,507 micro enterprises and micro entrepreneurs had benefitted from BIZ+ support through October 1, 2016 versus a life of project target through July 31, 2017 of 4,500. These figures include both customers, such as fishermen who purchase ice from one of the two grantees who produce ice, and suppliers, such as the coconut farmers who supply the raw material to produce coconut oil, brooms, and rope. Survey participants were asked several questions about their perceptions on their effects on customers, suppliers, and workers trained. With respect to customers, BIZ+ grantees were asked: “To what extent have your customers increased their sales because of the goods or services provided by your company?” 76% of respondents said that the increases were ‘high’ or ‘very high’, as shown in Figure 4.3.1 below. With respect to suppliers, BIZ+ grantees were asked: “To what extent has the support provided by Biz+ resulted in an increase in the following: number of local suppliers and the total value of local purchases?” In both cases, approximately 44% of respondents report that the impact was ‘high’ or ‘very high’; the weighted average rating for all respondents in each case was 3.4, i.e. between ‘moderate’ and ‘high’. (See Figure 4.3.2) 12% 3% 3% 18% 44% 32% 0% 10% 20% 30% 40% 50% N/A Very low=1 Low=2 Moderate=3 High=4 Very high=5 % of Respondents Ratings Scale Figure 4.3.1: Increase in BIZ+ Customers' Sales per Survey Respondents (N=26, WA=4.2) 16% 14% 16% 16% 30% 27% 22% 19% 22% 19% 0% 20% 40% Number of local suppliers (N=37, WA=3.1) Total value of local purchases (N=37, WA=3.1) % of Respondents Ratings Scale Figure 4.3.2: Impact of BIZ+ on Local Suppliers per Survey Respondents Very low=1 Low=2 Moderate=3 High=4 Very high=5 23 With respect to workers trained (Figure 4.3.3), BIZ+ grantees were asked: “How many workers have you trained?” All respondents said that they provided some level of training. According to their responses, at least 980 workers were trained by the 38 respondents. The BIZ+ Quarterly Report for June 30, 2017 indicates that the “Number of individuals benefitting directly from technical assistance and trainings” was 3,795 individuals versus an LOP target of 3,000. EQ#3B: What effect has BIZ+ project had on the business environment, processes, and networks? This section examines effects that BIZ+ may have had beyond the changes to customers, suppliers, and employees discussed under EQ#3A. Conclusion: BIZ+ appears to have had effects on the business environment, processes, and networks that extend beyond the direct effects on grantees, and the customers and suppliers within the grantees’ value chains that were discussed under the previous question (EQ#3A). This can be attributed in part to the new, more disciplined approach to business investing that has been introduced to the targeted regions by the project. Findings/Evidence: The BIZ+ PMP indicator, “Percentage of enterprises supported by BIZ+ that adopted new technologies and management practices to improve productivity in their business”, provides one answer to this question from the perspective of changes within individual grantees (emphasis added). As of June 30, 2017, BIZ+ reports that 90% of its grantees had either adopted such new technologies or management practices. In addition, BIZ+ management and several grantees report a positive effect on business management practices of grantees through tools like business capacity assessments and the use of local networks of business development services to improve management and operations. BIZ+ management and the Evaluation Team believe that such tools may instill a level of management discipline that would not have existed without the continued oversight from a third-party investor (i.e. BIZ+). Beyond changes in grantee management practices are the effects on the local network of business development services (BDS) providers. For example, the most recent BIZ+ Quarterly Report dated June 30, 2017 discusses the use of local BDS providers by BIZ+ grantees: “BIZ+’s model for BDS delivery encourages grantee businesses to purchase the services needed. The program facilitates the process to locate and contract with service providers. This process demonstrates the demand for BDS and aims to improve access and supply of BDS in the regions that are economically 8% 26% 18% 24% 11% 5% 8% 0 1-10 11-25 26-50 51-75 76-100 101 or more 0% 10% 20% 30% # of Workers Trained % of Respondents Figure 4.3.3: Workers Trained by BIZ+ Grantees per Survey Respondents (N=38) 24 lagging. Depending on the urgency and ability to finance, the services are paid for in a number of ways—fully by the business, cost-shared, or by the program directly.”15 It should be noted that the transparent, competitive selection process introduced by BIZ+ for grantees in Phase I and Phase 2 has also represented an important shift in the way investment proposals are submitted and reviewed to funding agencies in Sri Lanka. The BIZ+ Requests for Applications (RFAs) have resulted in the development and submission of hundreds of concept papers interested in obtaining an infusion of equity capital. Interviews with grantees surfaced several additional cases where BIZ+ investments had an effect beyond the company, and its suppliers, customers, and workers. For example, a manufacturer of polyethylene bags for the agriculture sector saw a serious drop in its sales when the government removed a 25% tariff on imported bags. The company responded by forming an association of local polyethylene bag manufacturers and convincing the government to reinstate the tariff. 16 In terms of business processes, it appears that BIZ+ has had an important positive effect in two areas: procurement and finance. Several grantees commented on the disciplined and orderly procurement procedures followed by BIZ+, even if, as detailed elsewhere in the report, they were not always satisfied with the specifications developed by BIZ+ consultants. In finance, BIZ+ has introduced rigorous oversight and follow-up over investees, something that the latter may not have experienced if they had obtained finance through family or friends. BIZ+ also contributed to the development of new business networks by encouraging investment by businesses based outside of the targeted regions. This was the case with the Colombo-based company that established the wood briquette factory in the Eastern Province, and the apparel maker17 that is owned by Sri Lankan investors and a Hong-Kong based investor. In the online survey, 24 grantees answered the survey question “What other effects has the BIZ+ investment in your company had on other businesses in your region?” Most of the responses repeated information about the effects of their businesses on their suppliers and business customers. However, several offered new insights into the other effects on the business environment, processes, and networks, as shown in the comments below: FIGURE 4.3.4: Survey Responses Responses "It drops customers at other restaurants in the area. We are expanding our business and keep hygiene." "They are trying to change the way of productions and looking for new technology latest machines." "Yes, it is very effectively support[ing] in this area other businesses. It helps the agriculture and transport sector. Biz+ helped to increase purchases from local suppliers." "We hope when our project is in full operation it will further drive SME'S in the sector. They will also have firsthand exposure to best practices which we bring to the region. We will connect the farming communities with our export customers around the world." "Women empowerment" 15 The BIZ+ PMP notes that 4,554 days of consulting services were provided through June 30, 2017 versus a target of 2,800 days, although it does not provide a breakdown of local versus expatriate consultant days. 16 While this is an interesting example of the power of working in association with other businesses, it is arguable whether higher tariffs on imported products is good economic policy. It saves jobs in the protected industry, but it also increases prices for downstream manufacturers and consumers. 17 The factory is in Moneragala, so not in Eastern Province, though it is a bordering region to Ampara, which is in the Eastern Province. 25 Responses "Change the negative attitude about the thought of disability in the community to positively. It is open opportunities for disabled people to contribute something in their community." Most comments discuss the positive impacts of BIZ+ on the local business environment, such as the empowerment of women, the opportunities created for disabled persons, and the exposure of other businesses to best practices introduced at the grantee’s business. However, in one case the respondent provides a candid assessment of a negative effect on competitors: “It drops customers at other restaurants in the area”, while adding a point about the new hygiene practice introduced at his/her own business. 4.4: WHAT ARE THE RECOMMENDATIONS FOR THE EXTENSION PERIOD OF THE BIZ+ PROJECT? EQ#4A: What are the successful aspects of the project that should be continued in the next phase? The BIZ+ project is based on a model of development that calls for grant funding of equity capital and expertise to establish or re-establish businesses in geographic areas that have been seriously affected by civil war. Section 5 of this report provides additional analysis of why this model may not be applicable to Phase 3 of the project. Whether such a model is continued in Phase 3, the following aspects of the project were noted to be particularly successful and should be continued for any new investments: Transparent Selection Process: BIZ+ should continue its transparent process of grantee selection through RFAs that are publicized through local print and electronic media, social media, business chambers and associations, including those devoted to women. The openness and transparency of the selection process helped build a reputation among the population of the objectivity of the selection process. In addition to advertising publicly, the project should proactively share RFAs to associations whose members are targeted beneficiaries. Ownership Issues: Phase 1 and 2 demonstrate that businesses that are privately owned and operated outperform those that are organized as cooperatives or non-profits. BIZ+ should continue to seek out strong entrepreneurs whose businesses have strong forward and backward linkages in rural communities. For example, a processor of coconut oil is a well-run private limited liability company that is purchasing and processing significant amounts of agriculture produce, and thus stimulating the rural economy. It is achieving better results than other BIZ+ grantees that buy and process agricultural products that are organized as cooperatives. Occupational Safety, Health, and Environment: BIZ+ has exposed its grantees to higher standards in all three areas, although grantees do not always follow best practices in these areas. The project should continue to emphasize such standards in Phase 3 through the sub-grant agreements, technical support and training, and periodic follow-up inspection visits. Businesses should be educated on the benefits of adhering to these standards – higher standards do not only protect workers and the environment but also can reduce the probability of potentially high-cost ruinous events. Follow-on activities should support adherence to these standards and educate managers. Accounting and Financial Management: This is the area of technical assistance and support that was most appreciated by grantees. The following aspects should be continued: 1) BIZ+ requirements that grantees report on a consistent and regular basis; and 2) support for upgrading accounting and financial management systems. 26 Focus on Vulnerable Populations: The strong commitment of BIZ+ and its staff to vulnerable populations in all regions should be continued. There should be no conflict between reaching such groups while achieving financial sustainability. EQ#4B: Are there any potential collaborations to leverage BIZ+ resources, including donor programs, private sector and financial sector? There are several potential collaborations to leverage BIZ+ resources: Agriculture Modernization Project: The on-going World Bank sponsored development scheme under the Ministry of Primary Industries and Ministry of Agriculture is a good opportunity for BIZ+ supported SMEs to finance expansion. Two BIZ+ grantees have applied and pre-qualified for submitting proposals. This project provides matching funds up to 50% at three levels, i.e. small (minimum LKR 1.5 million total investment cost), medium (up to LKR 22 million total investment cost), and large (above LKR 22 million total investment cost.) Supporting Accelerated Investment in Sri Lanka (SAIL): The USAID SAIL Project is keen to be informed of the policy constraints that project beneficiary businesses are facing, and businesses may be interested to know what policy issues SAIL is working on. Within the work focused on reforming the labor code, SAIL would like to introduce provisions preventing discrimination, which could be helpful to project beneficiaries. At present, there are no labor code provisions that prohibit discrimination against women, ethnic minorities or other groups in the workplace—something SAIL hopes to be able to change. Financial Sector: The SME financial services of the commercial banks in the country provide potential financing opportunities for BIZ+ supported enterprises. However, anecdotal evidence suggests that these schemes are not very SME friendly, and that banks are competing for the best customers. Nevertheless, such schemes could be useful in leveraging investments through tripartite cooperation (BIZ+, grantee, and bank) to buy down the risk of the commercial lender. Venture capital companies, social investment funds, and social enterprise support networks are other options for leveraging BIZ+ investments, particularly in economically lagging areas of the country. 18 One grantee discussed its intention to apply for a special rate (5%) loan scheme under a development bank. When questioned in the KIIs about leveraging BIZ+ supported investments, 12 companies that expressed an interest in expansion mentioned borrowing at commercial rates (at 9% to 16% per annum) from banks (9 times) followed by private equity (8 times). Other options were mentioned one or two times. See Figure 4.4.1 below. Additional recommendations concerning finance are included in Section 5. Recommendations. 18 Source: VEGA, (2016), Quarterly Performance Report: USAID BIZ+ July – September 2016, Colombo, Sri Lanka: USAID. 1 9 2 2 8 1 0 1 2 3 4 5 6 7 8 9 10 Frequency Figure 4.4.1: Leverage Options & Frequency of Choice Borrowing from Relatives/ Friends Private Equity SL Govt. support Borrowing at Special Rate Borrowing at Commercial Rate Foreign Donor 27 EQ#4C: In what ways can the participation of women be improved in the expansion of the BIZ+ project? As mentioned previously, the two successful women-owned businesses supported by BIZ+ are privately owned, limited liability companies, owned and managed by very dynamic female entrepreneurs. BIZ+ needs to try to replicate its success in these two cases. The lesson learned from Phase 1 and 2 is that such private, women-owned and managed LLCs clearly outperform women-owned cooperatives. While there are several ways in which the participation of women can be improved in the expansion of the BIZ+ project, some of these may be difficult to undertake in the relatively short period, i.e. two years, planned for Phase 3. The first three recommendations relate to selection, while the last two relate to coaching and training. First, BIZ+ should first focus on finding women-owned enterprises whose owners and managers have strong commitment and entrepreneurial skills. BIZ+ should reach out to public and private institutions, including financial intermediaries, to identify dynamic women with innovative ideas, but who lack funds for expansion, and encourage them to apply for BIZ+ funding. Local organizations that can help with outreach to women entrepreneurs include: • Asian Development Bank and Export Development Board of Sri Lanka support female entrepreneurs engaged in the information and communication technology (ICT) sector and the fruit, vegetable, and food processing sectors. • Women’s Chamber of Industry and Commerce focuses on advocacy, trade, networking, corporate social responsibility, and skills development. The objective of the chamber is to support women led businesses to be part of the mainstream business society. Presently, the organization has 200 members. • International Labor Organizations’ Start and Improve Your Business and GTZ-CEFE provide business start-up and management training for rural entrepreneurs including women-led businesses. The Industrial Development Board (IDB) supports rural enterprises by providing technical training and advisory to adopt or adapt technology for start-up or improve a business. The agency also provides development consultancy, entrepreneurship development training, productivity enhancement advisory, and information dissemination services to support SMEs. These services are accessible at their district level branches and at the central headquarter in Colombo free of charge or at a nominal fee. Particularly, since mid-2016, IDB works in collaboration with the International Finance Corporation (IFC) in implementing a scheme to support entrepreneurship development for rural women. • The on-going IFC women entrepreneurship development program. • The newly started USAID Youth Employment and Business Start-up Program (YouLead!). Government counterparts that may help in reaching female (as well as male) entrepreneurs include: National Enterprise Development Authority, Sri Lanka Chamber of Small and Medium Industries, Ministry of Rural Economic Affairs, and National Vocational Training Institute. Second, USAID and BIZ+ should consider reserving a certain percentage of Phase 3 funds for women entrepreneurs, much like the ‘set-asides’ USAID establishes for small and disadvantaged businesses. Third, as BIZ+ did during Phases 1 and 2, prospective grantees should be trained in proposal writing and business plan development, as many of them lack this skill and may not be able to afford costly consultants. 28 Fourth, time permitting, USAID and BIZ+ may also want to take an incremental approach with less experienced female entrepreneurs, once they are selected for grants. This would involve making several small-scale investments, progressing step by step depending on the results achieved. The first step would be to assess the capacity of the prospective grantees, strengthen their managerial skills, and then invest in stages based on performance. Fifth, BIZ+ should also examine ways of increasing female participation in all its training activities, including those that focus on improving management, leadership, and finance skills. Promoting Female Workers: The above points aim to increase participation by female owners and managers. A second concern is to increase participation by female workers at all levels within a firm. Here we note that there is nothing in the labor code that prohibits explicit discrimination against women (a topic the SAIL project is working on), however, in practice explicit discrimination against women and harassment of women are widespread. We therefore recommend that BIZ+ or its successor create a set of mandatory policies that beneficiary firms are required to formally adopt and implement. These policies should be focused on prohibiting and preventing discrimination and harassment. In addition, the project should provide training and education on the policies, and the implementation of the policies and incremental changes in participation by female workers should be monitored and reported. 5. RECOMMENDATIONS The following recommendations are in addition to those that were specifically requested by the evaluation scope of work in Evaluation Question #4 above. 5.1 CONSIDER WHAT IS THE BEST PROJECT MODEL TO SUPPORT TARGET SMES BIZ+ uses a model of supporting target SMEs mostly through providing grants, plus some technical assistance, to SMEs chosen through a competitive application and selection process. There are many alternative models for supporting SMEs that could be considered. The model used by BIZ+, of giving grants to selected SMEs, tends to be used as a stopgap measure in fragile states, for instance at the end of conflicts or right after natural disasters, when normal economic relations have been interrupted and until they can be put back into effect; however, other models tend to be used in more normal circumstances. In this regard, the grant model of BIZ+ might have made sense in the immediate post￾war period, but with the conflict now in the past, other models might be considered. More important than providing capital in developing countries, USAID’s primary comparative advantage is in providing knowledge and skills transfer to the private sector in all the areas needed for the private sector to grow, create jobs, and build resilience. Areas and examples where a project can provide technical assistance and training include the following: • Strategic/business planning technical assistance. • Market development through a combination of technical assistance and helping companies participate in international trade shows and other means to build relationships with new potential partners. • Technical assistance in production methods tailored for each industry – agro-processing, light manufacturing, ICT, retail, etc. • Quality control and meeting international and national standards and certifications requirements. • Financial management. 29 • Human Resources management. • Workforce training for existing or new workers, both to teach “hard” skills needed for each firm’s production methodologies and “soft” skills to create effective, collaborative, modern teams. A general lesson learned from USAID competitiveness and private-sector development projects is that programs are most successful when several types of assistance are provided to firms (for instance including technical assistance and access to finance) instead of just providing one kind of assistance. Several examples of scopes of work for some USAID illustrative private-sector development projects are given in Annex X. 5.2 CONSIDER ALTERNATIVE MODELS FOR SUPPORTING ACCESS TO FINANCE USAID should consider alternatives to the current BIZ+ approach of grant financing for SME capital requirements, as it is not clear that this approach brings the most impact per dollar spent or is the least distortionary. There are a range of other approaches USAID might consider: • Assist commercial banks to increase lending to SMEs. Commercial banks can be assisted to better analyze credit-worthiness of potential borrowers, to move away from asset-based criteria to credit-risk-based criteria for deciding lending, and to streamline administrative processes so they are better able to consider loan requests from SMEs at a reasonable cost. • Assist a variety of non-banks to increase lending and equity availability for SMEs. In addition to banks, other sources can be developed for finance, including: “embedded financing” within value chains between large firms (input suppliers or wholesale buyers) and SMEs, and franchising. • Create equity funds that can invest in SMEs. The enterprise funds and small business investment funds that operated in Eastern Europe and the former Soviet Union, which have either extended loans or taken equity interests in small businesses, could serve as a useful model. • Use USAID Development Credit Authority (DCA)19 to establish a loan guaranty program for business; many models exist. See Annex IX for a more detailed discussion of how USAID could support access to finance for SMEs. See Annex XI specifically for examples of DCA programs. Any of the above methods could be tailored to focus on BIZ+ intended beneficiaries. For instance, eligibility could be based on having owners or a threshold of workers who are women, or ethnic minorities, or live in target areas, or meet the criteria of vulnerable populations. However, experience has shown that defining eligible borrowers too narrowly often leads to poor utilization. 5.3 SELECT BENEFICIARY FIRMS BASED ON MARKET ORIENTATION, MANAGEMENT, AND FINANCIAL AND ECONOMIC VIABILITY The Evaluation Team carefully reviewed BIZ+’s selection procedures, including public announcements, solicitation documents, and selection criteria, that were used by the project to recruit and select grantees during Phase 1 and Phase 2. The evaluation also examined business capacity assessments of grantee firms that were conducted by BIZ+. However, the Evaluation Team believes BIZ+ should strengthen its project selection process to select those firms that are market-oriented — clearly serving 19 According to USAID’s Development Credit Authority website page, USAID Sri Lanka has had two DCA agreements, both executed in FY 2008: 1) 383-DCA-08-032 with MicroVest (formerly Minlam) in the amount of $5 million for a loan guaranty program for the benefit of microenterprises; and 2) 383-DCA-08-002 with Lanka Orix Leasing Company for a portable guaranty that expired without being utilized. 30 a market demand, that demonstrate likely financial and economic viability, and have motivated and competent management. 20 It is important to look at financial internal rates of return (FIRR) and, to some extent, economic internal rates of return (EIRR). Assessing market demand means identifying some evidence that the market demands the goods or services to be produced by the firm, and that the firm can offer these at prices that are competitive. An Excel template should be developed into which information for each candidate firm could be placed. The FIRR should take into consideration: expected gross revenues from current and new sales; expected expenditures on current and any new costs arising from new production processes, expanded production volumes, higher quality standards, etc.; any new investment; and the time value of money. Conducting an economic analysis would correct for any price distortions in the financial analysis. The most significant instance of this would be to try to correct for any instances where firms are producing goods protected by Sri Lanka’s high tariffs, in which case the firm likely would not be profitable if selling at market prices. It does not make sense for USAID to support the increased production of goods and services that would not find markets in the absence of extensive protectionist policies. Additionally, a more thorough EIRR would also include any or all of the following factors: USAID program costs estimated to be spent on a candidate firm; additional tax revenues that the government would accrue as a result of project assistance; additional job creation and/or increase in wages; and environmental impact. We believe that Excel templates could be created to facilitate the calculation of EIRR and training provided to project staff. 5.4 STRENGTHEN THE CURRENT EMPHASIS ON OCCUPATIONAL SAFETY, HEALTH, AND ENVIRONMENT BIZ+ has adopted sound policies for occupational safety, health, and environment issues affecting its grantees, and attempted to gain the compliance of grantees to relevant standards. During Phase 3 it should strengthen its emphasis on these issues among current grantees, whose agreements are still in force, and in the selection of new grantees. 5.5INCREASE CONSULTATION WITH GRANTEES ON EQUIPMENT PROCUREMENT DECISIONS Selection and procurement of equipment/machinery should be done in closer consultation and coordination with the grantee and with outside technical experts as required, to ensure that the equipment purchased meets the assumptions made in project feasibility studies. A final sign-off on equipment specifications should be obtained from grantees in order to help avoid problems such as the ones experienced with the three firms cited previously. 5.6 CONTINUE TO PROMOTE THE INCLUSION OF VULNERABLE GROUPS AS PROJECT PARTNERS Findings and conclusions with respect to women and ethnic groups suggest that BIZ+ should actively recruit and develop qualified and motivated female and minority owners and managers during the next phase of the project. BIZ+ should extend its outreach to these groups through organizations like higher educational institutions and regional chambers of commerce. As mentioned in Section 4.4, this could 20 Annex XIII: “Review of Documents and Forms used by BIZ+ in the Solicitation, Review, and Approval of Grant Applications” provides additional details concerning existing BIZ+ documents and forms that are used in the solicitation, review, and selection of grant applications, as well as suggestions for changes based on the evaluation recommendations. 31 include the establishment of ‘set-asides’ as well as incremental funding and active coaching for promising entrepreneurs. 5.7 IMPROVE PERFORMANCE MANAGEMENT SYSTEMS BIZ+ should improve its performance management systems by more clearly distinguishing among the effect of its programs on grantee businesses and their employees, the ‘spillover’ effects on buyers and suppliers in the same value chains, and employment created through the ‘multiplier effect’ in other businesses. Separate targets should be established for direct employment within assisted businesses, ancillary employment created within customers and suppliers within the same value chain, and any other employment created through the ‘multiplier effect.’ The system should guard against any double counting of employment. 32 ANNEX I: INFOGRAPHIC 33 34 ANNEX II: EVALUATION SOW Statement of Work Biz+ Mid-Term Performance Evaluation I. Purpose of the Evaluation USAID/Sri Lanka is soliciting the services of a contractor to conduct the mid-term performance evaluation of its Biz+ activity. The purpose of this evaluation is to evaluate the mid-term achievements and implementation performance of the Biz+ activity in order to inform an expansion of the activity to other geographic areas. The period covered by the evaluation is October 2011-December 2016. USAID/Sri Lanka intends that the evaluation be an effective learning tool that can: 1. be used by USAID/Sri Lanka to assess the effectiveness of the Biz+ activity and in particular help in identifying areas that can be applied in other geographic areas or gaps that need to be addressed during the planned extension; and 2. be used by USAID, the implementing partner, VEGA and Land O’Lakes International Development (Land O’Lakes), Government of Sri Lanka (GOSL) entities, and other donor organizations in their future planning of similar enterprise development programs aimed at accelerating economic growth and reducing poverty in economically lagging areas. The principle deliverables under this Statement of Work (SOW) are (1) a comprehensive Mid-Term Performance Evaluation Report that addresses the evaluation questions and provides recommendations to be included in the scope of the Biz+ activity extension; and (2) a one-page infographic reflecting key findings. Biz+ activities stimulate economic growth, job creation, and increased household incomes in the under￾developed areas of Sri Lanka by providing small and medium-sized businesses with the technical and managerial skills and financial resources they need to succeed. The activity provides investment grants to new and existing enterprises, requiring the recipient enterprise’s 1:1 match. Each enterprise also receives strategic technical assistance to strengthen institutional capacity in addition to financial support. Enterprises cover a wide range of industries including manufacturing, dairy, construction, food processing, tourism, service, energy, garment and textile, fisheries and handicrafts. This assistance builds successful enterprises, which grows local and national economies. The main objective of Biz+ is to generate jobs and enhance income generation through economic growth in economically lagging regions of Sri Lanka through public-private alliances. II. Audience and Intended Users The audience of the evaluation report is the USAID/Sri Lanka Mission, specifically the Mission Director, Program Office, and the Economic Growth (EG) team; the EG team of the Asia Bureau; and the implementing partner. USAID will use the report to inform future strategies of the Biz+ activity, inform future EG programming, and share lessons learned with other stakeholders. 35 III. Background a. Project Identification Data Project: Biz+ Activity Award No: Cooperative Agreement Number AID-383-LA-11-00002 Award Dates: October 1, 2011 to July 31, 2017 Project Funding: $22.4 million Implementer: VEGA/ Land O’Lakes International Development AOR: Ms. Salma Peiris b. Development Context Sri Lanka has transitioned from a country requiring a constant stream of international humanitarian assistance, to a nation experiencing rapid political and economic changes that improve the country’s own longer-term development capabilities. With the end of the civil war, the recent democratic transition, and the good governance reforms now underway, the country has an opportunity to take advantage of its peacetime stability, geography, and educated workforce and capitalize on the ability of Sri Lankans to drive their own economic advancement. Sri Lanka has experienced rapid economic growth, achieving an average of 7-8 percent per year since the end of the 26-year civil war in 2009. However, that growth has not been equitable island-wide, with most of the economic growth experienced in the Western Province, which generates nearly 44 percent of the Gross Domestic Product (GDP). In regions of the country most directly affected by the long conflict, economic growth has been much slower, with the Eastern and Northern Provinces contributing only 6.3 percent and 4 percent to the GDP respectively. The Western Province is represented by the Sinhalese majority, while the Eastern and Northern Provinces are primarily comprised of the minority ethnic groups: Tamils and Muslims. This economic disparity is stark and has been, and will continue to be, a motivator of dissonance, which could threaten lasting peace in the country. The Government of Sri Lanka (GSL) has provided relatively rapid and large-scale infrastructure development to the North and East in the areas of electricity, roads, repair of bridges, and water supply. However, while these investments are extremely important, infrastructure development alone has not resulted in reconciliation or rapid economic growth in these regions. Equitable and comprehensive economic growth throughout the island, and especially in conflict affected regions, is an essential ingredient to reconciliation and sustainable peace. Accelerated economic growth in economically lagging regions such as the North and the East, and surrounding regions, will create employment, support productive local enterprises, and provide vulnerable groups new and expanded income earning opportunities. Growth in these economically depressed areas will also open and expand markets for all regions of the country, creating a network of business linkages from north to south and east to west. Increased employment opportunities, especially for vulnerable groups, combined with a growing 36 network of business linkages throughout the country will go a long way to rebuilding relationships between all peoples and regions and especially between the Tamil and Muslim minority groups with the majority Sinhalese. c. Program Description In October 2011, USAID awarded a $24 million, four-year cooperative agreement to VEGA/Land O’Lakes to implement Biz+ in Northern and Eastern Sri Lanka with a four-year period of performance (October 1, 2011 to September 30, 2015). According to the original program description, Biz+ was to adopt three integrated components aimed at its overall objective: Increased Jobs and Income Generation in Northern and Eastern Sri Lanka through Enterprise Development. • Grants to new and existing enterprises with good prospects for growth and increased employment: The heart of Biz+ is grants to help promising small and medium enterprises (SME) boost capacity, expand productivity, and create jobs. • Technical assistance to grant recipients and other associated institutions: Most grant-receiving SMEs also receive Biz+ technical and management assistance to develop competencies to ensure sustainable success. Biz+ also supports chambers of commerce and other institutions through a service grant to provide enterprise development support. • Improvement of enabling environment for enterprises*: To support the Sri Lanka Government’s efforts to establish a more business-friendly environment, Biz+ facilitates activities helping business and community leaders – including women and diverse ethnic populations – understand legal and regulatory issues, pursue advocacy, and strengthen ties between public and private sectors. *When the project was modified in November 2013, component three was deleted in its entirety. The total estimated cost (TEC) was also reduced and remaining funds were re-distributed among components one and two. More details can be found in the next section. Biz+ investments are disseminated via grants, with recipients evaluated through sound business and economic metrics to foster maximum sustainability and growth. Through Public Private Alliances (PPAs) with Sri Lankan SMEs, Biz+ fosters investments that yield job creation and economic growth. Priority is given to local enterprises and to those creating jobs for women, special-needs individuals and diverse ethnic populations. Over the course of the activity, Biz+ aimed to provide 60 grants to new and existing enterprises, secure a cost-share of $0.614 million, and leverage a further $17.5 million from grant recipients and program partners. Successful implementation of Biz+ meant creating 6,500 jobs, producing $6.5 million in annual income for vulnerable households in Sri Lanka’s Northern, Eastern, North Central and Uva Provinces. Grants and technical assistance would decrease risk for enterprises willing to invest in post-conflict regions. Biz+ grants coupled with recipient cost-shares and leveraged matches, would strengthen institutional capacity, increase productivity, raise employment, and enhance incomes for target firms and 37 vulnerable populations. Biz+ technical assistance focuses on business management, finance, and operations. Having identified specific sectors with high potential for sustainable business development, job growth, and income augmentation, Biz+ looked for beneficiaries within these high-potential sectors: 1) construction; 2) textiles; 3) services (motor vehicle repair, transport of products, catering/canteens for businesses, cleaning/maintenance for businesses, biomass production/processing, food packaging, cooling unit installation/maintenance, hostels, solid waste composting, borehole blasting, printing and photo processing, laundries); 4) food processing; 5) handicrafts; 6) tourism; 7) agriculture; 8) livestock; 9) dairy; and 10) fisheries and aquaculture. To date, Biz+ has assisted 47 businesses with investment grants and nine beneficiary organizations with service grants. Of the 47 business grants, 25 are closed and have no further contractual agreements with Biz+. However, the closed grantees continue to operate as businesses and many have been hosting events at the request of USAID. In April 2016, Biz+ started phase two of the activity and awarded 13 grants, which are currently under various stages of implementation. As of June 2016, Biz+ has: • Created 6,014 new employment and income earning opportunities; • Benefitted 8,272 vulnerable households; • Leveraged $14,542,872 from grantee businesses; and • Produced $8,176,079 in annual household income. d. Program Modifications In July 2012, the budget was realigned; a portion of funds that were allocated for program activities (direct cost) was shifted to personnel costs to fund an increased level of short and long term human resources required to expedite implementation. The program was also modified to expand the geographical coverage to include border areas to the Northern and Eastern provinces such as Uva, and the North Central province. In November 2013, the TEC was reduced from $24 million to $17.6 million, the program end date was changed to June 2015, component budgets were realigned to reflect the reduction in TEC, and the program was de-scoped and revised with component three deleted in its entirety. In June 2014, the NICRA of both VEGA and Land O’Lakes was reduced; excess funds under indirect costs were shifted to program activities (direct costs). Cost share and leverage were further defined to clarify partner contribution and the end date of the program was changed from June 2015 to July 2015. In January 2015, a new government was elected in Sri Lanka, whose development objectives aligned with those of the USG. USAID extended Biz+ to support President Sirisena’s reform agenda which includes the creation of 10,000 jobs. In commitment to Sri Lanka’s new Government and its priorities, the award was modified to increase the TEC to $22.4 million and extended the completion date from July 31, 2015 to July 31, 2017. 38 e. Existing Data During the evaluation, the following documents will be available to the team: • Work plans • M&E Plans • Baseline survey • Quarterly reports f. Development Hypothesis USAID/Sri Lanka has articulated its highest-level Development Goal as follows: Accelerate reconciliation and economic growth equitably and sustainably. One of USAID/Sri Lanka’s two Development Objectives (DOs) in support of this Development Goal is as follows: DO2: Increased and More Equitable Economic Growth in Former Conflict Areas. Biz+ serves DO2 under the following hypothesis: Increased and equitable growth in former conflict areas will be achieved by establishing public-private alliances with SMEs that create jobs and increase investment; by enhancing workforce skills and enterprise productivity for partner SMEs through technical assistance and training; and by supporting regulatory reforms to improve the business environment. g. Implementing Organizations VEGA is the primary implementing organization, with a sub-award to Land O’Lakes for active implementation, and further sub-awards from Land O’Lakes to PYXERA Global and SAVE Travels. h. Beneficiaries Biz+ targets two categories of grant recipients: primarily regional SMEs with investment grants under component one; and chambers of commerce and other business-relevant institutions with service grants, technical assistance and training under component two. The program emphasizes support to local enterprises in the North, East, North Central and Uva provinces that create jobs for women, special-needs individuals, and diverse ethnic vulnerable populations. Almost everyone residing in the former conflict regions of the north and east can be considered as vulnerable, with different sub-groups experiencing even greater vulnerability. Northern, Eastern, North Central and Uva Province households stand as indirect Biz+ beneficiaries to the extent that otherwise jobless members secure employment and to the extent that they benefit from increased general prosperity. Biz+ has a range of beneficiaries including business, business community, employees, micro-entrepreneurs, and microenterprises along the value chain. i. Previous Evaluations 39 There have been no internal or external evaluations to date. IV. Evaluation Questions The purpose of this evaluation is to evaluate the mid-term achievements and implementation performance of the Biz+ activity in order to inform an expansion of the project to other geographic areas. The evaluation questions are as follows: 1. To what extent were the Biz+ activity objectives achieved? Responses shall address: • Factors influencing the achievement or non-achievement of the objectives, such as political environment, sector, business enabling environment, cultural context, gender, and access to finance. • Business owners’ perceptions of the value of the technical assistance provided by Biz+ in achieving or not achieving objectives. 2. Did the businesses continue to grow after the completion of their Biz+ grants? Responses shall address: • Factors influencing growth and bottlenecks impeding growth, such as access to finance, gender of owner, type of business, and location. • Nature of business expansion, such as number of employees, amount of profit, increase in number of customers, increase in number of partnerships, and/or linkages. • Any correlation between the success of the organization and the percentage of its matching contribution to the Biz+ investment. 3. Has there been a spill-over effect of Biz+-supported businesses in the three primary Biz+ sectors? Responses shall address: • Effect on suppliers, customers, and workers trained. • The business environment, such as regulations, processes, and networks. 4. What are the recommendations for the extension period of the Biz+ activity? Responses shall address: • Successful aspects of the activity that should be continued in the next phase. • Potential collaborations to leverage Biz+ resources, including with donor programs, government initiatives, public private partnerships (PPP), and financial sector actors. • Opportunities to improve the participation of women at all levels. V. Evaluation Design and Methodology This final performance evaluation is intended to answer the evaluation questions presented above. The conceptual approach to answer these questions should focus on qualitative and quantitative data collection methods. This approach should include, but not limited to the following: • Desk study 40 • Quantitative analyses (e.g., return on investment analysis, as appropriate) • Individual interviews and/or group discussions with grantees, indirect beneficiaries and stakeholders (chambers, service providers) • Case studies of characteristics and common factors contributing to selected successful and unsuccessful enterprises The evaluators’ expertise and input in the proposal and during the evaluation design phase is anticipated. The team should have a mix of Sri Lankan and international consultants and collectively have fluent tri￾lingual skills in English, Tamil, and Sinhala. The independent external evaluation team is expected to work in conjunction with USAID/Sri Lanka’s Program Office and the Office of Economic Growth to plan and implement the proposed evaluation. The consultants are expected to provide significant overall leadership and direction, exercise a degree of autonomy, and have the final responsibility for conducting the evaluation and completing evaluation deliverables. The evaluation team will be required to evaluate this project in a timely manner. Data requirements, collection methods, and required analyses will be determined collaboratively with USAID/Sri Lanka, under the direction of an independent evaluation team leader. Details on final data collection methods (including evaluation instruments and key informants or respondents), and analytical framework(s) will be approved by USAID/Sri Lanka as part of the initial work plan approval. At minimum, the methodological approach should include a representative sample of between eight and ten of the “graduated” grantees and eight to ten on-going grantees for in-depth study, taking into consideration appropriate representation from (1) round one and two of the investment grants; (2) different sizes of organizations; (3) various business sectors; and (4) geographic location. USAID will approve the evaluation team’s proposed list; however the Mission expects the majority of grantees to be interviewed. Data, where applicable, must be disaggregated by sex, ethnicity, and location. The Mission is interested in gender issues, achievements or observations that the evaluation team encounters in the course of the evaluation, in addition to those specified in the evaluation questions. The Mission expects the evaluation team to present strong quantitative and qualitative analysis, within data limitations, that clearly addresses the evaluation questions. The proposal will address methodological limitations and plans to address them, e.g., language barriers and translation. 41 VI. Deliverables a. Timeline of Deliverables Due Date Deliverable 1 week after award Conference call: The team will hold a conference call with the USAID/Sri Lanka evaluation point of contact and PPS Office Director to review the SOW, answer questions, etc. 2 weeks after award Work Plan: The team will submit a detailed work plan, including the methodologies (evaluation instruments, methodology and design; logistics plan; and work schedule) to be used in the evaluation. The work plan will be submitted to USAID for approval and discussed with USAID prior to implementation. First work day in country (Week 3 of award) In-Brief with USAID: An in-country kick-off/in-brief with all team members will be held to present the evaluation plan, methodology and process to the Mission, and reach agreement on the evaluation work plan. Second week in country (Week 4 of award) Mid-Point Oral Briefing: The team will update the mission on the progress of the evaluation activities, initial findings, issues, etc. This could be conducted through a teleconference call, if the evaluation team is in the field. 48 hours prior to out￾brief Preliminary Draft Evaluation Presentation: A PowerPoint presentation of the preliminary findings will be submitted to USAID, who will provide comments during the mission out-brief. 1-2 days prior to leaving Sri Lanka (End of week 6 of award) Out-Brief with USAID: The team will present the key preliminary findings of the evaluation to USAID. The out-brief will include a discussion of preliminary findings and recommendations. The team will consider USAID comments and incorporate them into the evaluation report as appropriate. Out-brief with Partners: The team will present the key preliminary findings of the evaluation to USAID’s implementing partners (as appropriate and as defined by USAID) prior to the team’s departure from country. The out-brief will include a discussion of preliminary findings and recommendations. The team will consider comments and incorporate them into the evaluation report as appropriate. 10 business days after departure from Sri Lanka (Week 8 of award) Draft Evaluation Report: A draft report that follows USAID’s Evaluation Policy and How-To-Note and incorporates comments and inputs from the out-brief(s) will be submitted to USAID. USAID will provide comments on the draft evaluation within ten business days. 42 10 business days after receiving USAID comments on the draft report (Week 12 of award) Final Evaluation Report: The team will submit a final report in electronic MS Word and PDF formats within two weeks after USAID provides written comments on the draft report. Draft Translations of Executive Summary: Sinhala and Tamil draft translations of the Executive Summary of the final report will be submitted to USAID for approval. Within one week of report finalization (Week 13 of award, and no later than the award end date) Delivery of Final Report: Two hard copies of the final evaluation report will be delivered to the mission. The report will follow USAID branding procedures. Delivery of Final Translations of Executive Summary: Two hard copies of the Tamil and Sinhala executive summaries will be delivered to the mission. The summaries will follow USAID branding procedures. Within one month of report approval by the mission. Submission of Final Report to DEC: The final report will be uploaded by the Contractor to the public area of the Development Experience Clearinghouse (DEC) following final approval by USAID. b. Final Report Per the USAID Evaluation Policy and USAID ADS 201, draft and final evaluation reports will be evaluated against the following criteria to ensure the quality of the evaluation report.21 ● The evaluation report should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate what worked in the project, what did not, and why. ● Evaluation reports should be readily understood and should identify key points clearly, distinctly, and succinctly. ● Evaluation reports shall address all evaluation questions included in the SOW. ● The Executive Summary of an evaluation report should present a concise and accurate statement of the most critical elements of the report. ● Evaluation reports should adequately address all evaluation questions included in the statement of work, or the evaluation questions subsequently revised and documented in consultation and agreement with USAID. ● The evaluation methodology shall be explained in detail and all sources of information properly identified. If evaluation findings assess person-level outcomes or impact, they should also be separately assessed for both males and females. ● Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). ● Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or the compilation of people’s opinions. ● Findings should be specific, concise, and supported by strong quantitative and/or qualitative evidence. ● Recommendations need to be supported by a specific set of findings. 21 See Appendix I of the Evaluation Policy and the Evaluation Report Review Checklist from the Evaluation Toolkit for additional guidance. 43 ● Recommendations should be action-oriented, practical, and specific, with defined responsibility for the action. The executive summary should be 3–5 pages in length and summarize the purpose, background of the project being evaluated, main evaluation questions, methods, findings, conclusions, and recommendations and lessons learned (if applicable). The executive summary shall shadow the maid report, section by section, serving as a stand-alone document and containing key findings and recommendations, permitting the reader to refer to the main report for additional detail about each section. The executive summary must be translated into Tamil and Sinhala. The evaluation methodology shall be explained in the report in detail. Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (e.g., selection bias, recall bias, unobservable differences between comparator groups, etc.) The format of the evaluation report shall be as follows, modified as necessary: 1. Executive summary 2. Table of contents 3. Introduction 4. Background 5. Methodology (including description of methods and limitations) 6. Findings/conclusions/recommendations (as separate sections of each evaluation question) 7. References 8. Annexes The annexes to the report shall include: 1. A one page infographic (double-sided if necessary) that highlights key findings; 2. The Evaluation SOW; 3. Any statements of difference regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team; 4. All tools used in conducting the evaluation, such as questionnaires, checklists, and discussion guides; 5. Sources of information, properly identified and listed; and 6. Disclosure of conflict of interest forms for all evaluation team members, either attesting to a lack of conflicts of interest or describing existing conflicts of. In accordance with AIDAR 752.7005, the contractor will make the final evaluation reports publicly available through the Development Experience Clearinghouse within 30 calendar days of final approval of the formatted report. c. Other Deliverables The Executive Summary of the final report will be translated into Sinhala and Tamil by the Evaluation Team. The draft document will be submitted to USAID/Sri Lanka with the final evaluation report for approval. 44 All quantitative data collected by the evaluation team must be provided in machine-readable, non￾proprietary formats as required by USAID’s Open Data policy (see ADS 579). The data should be organized and fully documented for use by those not fully familiar with the project or the evaluation. USAID will retain ownership of the survey and all datasets developed. All modifications to the required elements of the SOW of the contract/agreement, whether in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline, need to be agreed upon in writing by the COR. VII. Period of Performance The period of performance is anticipated to be February 2017 to April 2017. The contract will be approximately three months from the date the contract is signed. Proposals should include a Gantt chart for the schedule of work over this period. VIII. Key Personnel The Evaluation Team should be composed of team members who possess a mix of skills and knowledge including SME development; monitoring and evaluation; fluent oral and written communication in English, Sinhala and Tamil; and the local business/economic context. Gender should be a factor in determining team composition. The Evaluation Team will be available to travel across Sri Lanka to conduct the evaluation as necessary. • A senior, external expatriate consultant as team leader with experience in Monitoring and Evaluation (M&E) of Economic Growth projects, especially small and medium enterprises (SME) sector, with a strong background of conducting evaluations of similar enterprise development projects (including experience as the team leader). Minimum ten years’ experience in business and finance, economic growth, or the international development sector, and a MBA or related degree. • A mid-level local consultant that specializes in SMEs and has strong M&E capabilities. Minimum five years’ experience in the economic growth, international development sector, with experience participating in evaluations. He/she must have minimum of a BA/BS degree in a related field. • A mid-level local consultant with a minimum of five years of experience possessing a strong economic and business management background, an in-depth knowledge of the local context, and experience in conducting evaluations. He/she must have minimum of a BA/BS degree in a related field. IX. Evaluation Management The contractor shall arrange international travel and visas. The evaluation team is responsible for arranging meetings as identified during the course of this evaluation and advising USAID/Sri Lanka prior to each of those meetings. The evaluation team is also responsible for arranging vehicle rental and drivers as needed for site visits in Sri Lanka. The evaluation team will be responsible for procuring its own work/office space, computers, internet access, printing, and photocopying. Evaluation team members will be required to make their own payments. 45 X. Level of Effort (LOE) An illustrative level of effort (LOE) is given below: Task/Deliverables Estimated Duration/LOE in Days Expat Team Leader Local Consultant -1 Local Consultant - 2 1. Review background documents & offshore preparation work 5 4 4 2. Develop detailed work plan, methodologies, data collection tools, field work schedule 3 3 3 2. Travel to Sri Lanka 2 3. Team Planning Meeting and meeting with USAID/Sri Lanka 2 2 2 4. Information and data collection: includes interviews and site visits in north, north-central and eastern provinces with key informants, stakeholders and donors in Colombo 30 30 30 5. Discussion, analysis, and drafting evaluation report in country; debrief & out-brief with USAID and key stakeholders in Colombo. 5 5 5 8. Depart Sri Lanka/Travel to US 2 9. USAID & partners provide comments on draft report (out of country) 10. Team revises draft report and submits final to USAID (out of country) 5 3 3 11. USAID completes final review Total Estimated LOE 54 47 47 46 ANNEX III: SAMPLE SELECTION Annex III-A: Selection Process for Sample of 20 Grantees for KIIs The following is a summary of the selection process for the 20 grantees selected for key informant interviews (KIIs): 1. The Evaluation SOW provides the following guidance on sample selection: “At minimum, the methodological approach should include a representative sample of between eight and ten of the “graduated” grantees and eight to ten on-going grantees for in-depth study, taking into consideration appropriate representation from (1) round one and two of the investment grants; (2) different sizes of organizations; (3) various business sectors; and (4) geographic location. USAID will approve the evaluation team’s proposed list; however, the Mission expects the majority of grantees to be interviewed”. 2. The approved Evaluation Work Plan including the following language on sample selection: “KIIs will be conducted with the primary stakeholders, including a sample of 20 grant recipients (13 from round 1 and 7 from round 2), representatives from USAID/Sri Lanka, the Biz+ implementing team, and chambers of commerce and BSOs that have been involved in the project. KIIs will be conducted with secondary stakeholders if time permits. The interviews will be semi-structured in nature to allow for an open framework, conversational communication, and detailed follow up questions. Annex 1 includes a proposed list of 20 grantees to be interviewed. Grantees were selected based on: (1) round, (2) size of enterprise (number of employees), (3) business sector, and (4) location. Annexes 3 includes questionnaires for KIIs with Biz+ grantees and chambers of commerce/BSOs. The questionnaires will be translated into Sinhalese and/or Tamil.” 3. Based on the above requirements of the SOW and Work Plan, and making use of the BIZ+ March 2017 portfolio list, the Evaluation Team prorated the sample size of 20 grantees by round (also referred to as phase) and location. 4. After prorating by round (phase) and location, the sample was checked for reasonableness against the other two criteria mentioned in the SOW and the work plan (size and sector) and an additional two factors (gender and ethnicity) (see Annex III-C). 5. Adjustments were made to the calculated prorated numbers to respond to USAID guidance and to obtain as broadly representative a sample of grantees, given limitations on travel time and budget. 47 Annex III-B: KII Sample Selection (Continued) This annex was removed as it contained direct identifiers of the grantees. 48 Annex III-C: KII Sample Selection (Continued) This annex was removed as it contained direct identifiers of the grantees. 49 ANNEX IV: DATA COLLECTION INSTRUMENTS Business Online Survey Questionnaire Sect#/ Quest# Section/Question/Answer Link to Evaluation Question I Your Business All 1 Name of business All 2 Legal Status All - Sole proprietor - Partnership - Limited liability company - Corporation - Non-profit organization - Cooperative 3 Years in operation All - 0-1 - 1-5 - 5-10 - 10 plus 4 Gender of principal owner All - Female - Male 5 Province All - North - North Central - East - Uva 6 District All 50 Sect#/ Quest# Section/Question/Answer Link to Evaluation Question 7 Divisional Secretariat Division (DSD) All 8 Grama Niladhari Division All 9 Sector All - Agriculture - Livestock - Manufacturing - Dairy - Construction - Food processing - Tourism - Energy - Garment and textile - Fisheries - Handicrafts - Services (motor vehicle repair, transport of products, etc.) 10 Other sector (specify) All 11 Sales (all figures in millions of Sri Lankan Rupees LKR) All - 0-5 - 5-10 - 10-15 - 15-20 - 20-50 - 50-100 - 100 or more 13 # of Employees All 51 Sect#/ Quest# Section/Question/Answer Link to Evaluation Question - 1-10 - 11-50 - 51-250 - 251 or more II Recent Performance 1, 2 14 Please estimate the growth that your company has experienced over the last three years in the indicators listed below. (0, 1-5%, 6-10%, 11-25%, 26% plus) 1, 2 - Sales - Employment - Wages - Investment III Impact of Biz+ on your business 1, 2, 3 15 To what extent has Biz+ helped your business increase the following? (Rate on a scale of 1-5, where 1=Very low, 2=Low, 3=Moderate, 4=High, 5=Very High) 1, 2 - Sales - Number of customers - Employment - Wages - Investment 16 How would you rate the contribution of technical assistance provided by Biz+ to your company's growth? (Rate on a scale of 1-5, where 1=Very low, 2=Low, 3=Moderate, 4=High, 5=Very High, NA=Not applicable) 1,2 17 In your opinion, what role did the following factors play in facilitating the growth in your business? (Rate on a scale of 1-5, where 1=Very low, 2=Low, 3=Moderate, 4=High, 5=Very High) 2 - Political environment 52 Sect#/ Quest# Section/Question/Answer Link to Evaluation Question - Business sector enabling environment - Access to finance - Business sector - Location - Cultural context/customs 17.5 Gender: Gender: Please comment on any gender issues that have an impact on your business? 18 Your Suppliers: To what extent has the support provided by Biz+ resulted in an increase in: (Rate on a scale of 1-5, where 1=Very low, 2=Low, 3=Moderate, 4=High, 5=Very High, NA=Not applicable) 3 - The number of your local suppliers - The total value of your purchases from local suppliers 19 Comments on Question 18 (open-ended) 3 20 Your Customers: To what extent have your business customers increased their sales as a result of the goods or services provided by your company? (Rate on a scale of 1-5, where 1=Very low, 2=Low, 3=Moderate, 4=High, 5=Very High, N/A) 3 21 Comments on Question 20 (open-ended) 3 22 Training: Have you trained workers in connection with Biz+ activities? 3 - Yes - No 23 How many workers have you trained? 3 - 1-10 - 11-25 - 26-50 - 51-75 - 76-100 53 Sect#/ Quest# Section/Question/Answer Link to Evaluation Question - 101 or more 24 What percentage of workers trained have remained with your company? 3 - 0-25% - 26-50% - 51-75% - 76-100% 25 Comments on Question 22-25 (open-ended) 3 26 What other effects has the Biz+ investment in your company had on other businesses in your region? 3 IV Future Needs 4 27 What is the importance of the following for the future success of your business? (Rate on a scale of 1-5, where 1=Very low, 2=Low, 3=Moderate, 4=High, 5=Very High) 4 - Access to markets and customers - New product development - Consistency and quality of suppliers - Labor (cost and quality) - Technology - Information technology - Loans - Equity - Energy - Telecommunications - Transportation - Waste management - Security 54 Sect#/ Quest# Section/Question/Answer Link to Evaluation Question - Taxation - Government policies, laws, and regulations - Other (specify) V Background on person completing survey (optional) NA 28 Name of person completing survey NA 29 Position in organization NA - Owner - President/CEO/Managing Director - Other (please specify) 30 E-mail NA 31 May we contact you for additional information? NA - Yes - No 55 Guidelines for KIIs with Businesses and Business Support Organizations Guidelines for KIIs with Businesses This section presents the guiding questions for KIIs with selected businesses (grantees) by evaluation question (EQ). EQ 1. To what extent were the Biz+ project objectives achieved? 1.1) What role did factors such as the political environment, sector business enabling environment, access to finance, gender and cultural context play in influencing the achievement or non-achievement of the project objectives? 1.2) What are Business owners’ perceptions of the value of the technical assistance provided by Biz+ in achieving or not achieving objectives? 1. What kind of support did you receive from Biz+? 1.1 2. Did Biz+ meet your expectations? 1.1 3. What are the key changes you have observed in your business after receiving support from Biz+? 1.1 4. What factors, such as the political situation, the business environment, finance, gender, or cultural context, have influenced the success of your business? 1.1 5. Did you receive technical assistance from Biz+? If so, please describe. What impact has it had on your business? 1.2 EQ 2. Did the businesses continue to grow after the completion of the Biz+ grant? How did factors such as access to finance, gender of owner, type of business and location affect the growth of businesses after completion of the grant? 2.1) What was the nature of business expansion for firms after the completion of the grant? 2.2) Is there any correlation between the success of the organization and the percentage of its matching contribution to the Biz+ investment? 2.3) What is the status of the equipment provided by Biz+? Are they operational? 6. After the Biz+ grant was formally closed out, what happened to your business? Has it grown? 2.1 7. What are the main factors, such as access to finance, gender, type of business, and location, affecting the growth and success of your business? 2.1 8. What is the current condition of the equipment received from Biz+? May we see it? 2.3 EQ 3. Has there been a spill-over effect of Biz+-supported businesses in the primary Biz+ sectors, such as small-scale manufacturing and food production? 56 3.1) How have Biz+ activities affected suppliers, customers, and workers trained? 3.2) What effect has Biz+ project had on the business environment, processes, and networks? 9. How have your customers been affected by your recent investment? 3.1 10. How have your suppliers been affected by your recent investment? 11. Were any of your workers trained with the help of Biz+? If so, what has been the impact on your business? 3.1 12. In your opinion, has Biz+ made any difference in the business environment? In processes? In business networks? 3.2 13. What factors might have caused those changes? 3.2 3. What are the recommendations for the extension period of the Biz+ project? 4.1) What are the successful aspects of project that should be continued in the next phase? 4.2) Are there any potential collaborations to leverage Biz+ resources, including donor programs, private sector and financial sector? 4.3) In what ways can the participation of women be improved in the expansion of the Biz+ project? 14. What have you liked about your experience with Biz+? What do you dislike? 4.3 15. What are priorities concerning your business that needed to be addressed? 4.1 16. Are there any successful models, tools, practices, etc. which you have observed during Biz+ that should be continued/replicated? 4.1 17. What financial resources are available for expanding/scaling up your business? E.g. donors, investors, financial institutions? 4.3 18. Do you think women should play a greater role in business? If so, how can a project like Biz+ support women? 4.2 19. In what ways have vulnerable persons such as physically challenged persons been accommodated in businesses? 20. Is there any child labor connected to the business or suppliers? 4.2 57 Guidelines for KIIs with Business Support Organizations This section presents the guiding questions for KIIs with selected business development services (e.g. Chambers) by evaluation question. 1. To what extent were the Biz+ project objectives achieved? 1.1) What role did factors such as the political environment, sector business enabling environment, access to finance, gender and cultural context play in influencing the achievement or non-achievement of the project objectives? 1.2) What are Business owners’ perceptions of the value of the technical assistance provided by Biz+ in achieving or not achieving objectives? 1. Did you receive any support from Biz+? What kind of support did you receive? What was the impact of the support on your organization? 1.2 2. Did you increase or improve your support to businesses as a result of your collaboration with Biz+? 3. What are your observations about the changes among the enterprises, which were supported by Biz+? 1.1 4. How has the business environment affected the success of the Biz+ project? 1.1 2. Did the businesses continue to grow after the completion of the Biz+ grant? How did factors such as access to finance, gender of owner, type of business and location affect the growth of businesses after completion of the grant? 2.1) What was the nature of business expansion for firms after the completion of the grant? 2.2) Is there any correlation between the success of the organization and the percentage of its matching contribution to the Biz+ investment? 2.3) What is the status of the equipment provided by Biz+? Are they operational? 5. Do you know if the businesses supported by Biz+ have continued to grow and prosper? 2.1 3.Has there been a spill-over effect of Biz+-supported businesses in the primary Biz+ sectors, such as small-scale manufacturing and food production? 3.1) How have Biz+ activities affected suppliers, customers, and workers trained? 3.2) What effect has Biz+ project had on the business environment, processes, and networks? 6. How has Biz+ affected the customers of the businesses supported by Biz+? 3.1 7. How has Biz+ affected the suppliers of the businesses supported by Biz+? 8. How has Biz+ affected the workers at the businesses supported by Biz+? 3.1 58 9. Have you observed any differences in the business environment, processes, and networks, as a result of Biz+? 3.2 4. What are the recommendations for the extension period of the Biz+ project? (4.1) What are the successful aspects of project that should be continued in the next phase? 4.2) Are there any potential collaborations to leverage Biz+ resources, including donor programs, private sector and financial sector? 4.3) In what ways can the participation of women be improved in the expansion of the Biz+ project? 10. What are the positive aspects of Biz+? Negative aspects? 4.3 11. What should Biz+ focus on in the future? 4.1 12. Are there any successful models, tools, practices, etc. which you have observed during Biz+ that should be continued/replicated? 4.1 13. What are your observations/opinions about women participation in businesses (as owners, managers, workers, etc.)? 4.3 14. What are your opinions about the potential or available external financial and other capital sources for expanding/scaling the business? 4.2 15. Do you think women should play a greater role in business? If so, how can a project like Biz+ support women? 59 ANNEX V: BIZ+ ANNUAL PERFORMANCE DATA22 22 Source: VEGA, (2017), Quarterly Performance Report: USAID BIZ+ April 1, 2017-June 30, 2017, Colombo, Sri Lanka: USAID. "The BIZ+ June 30, 2017 reports a total of 6,111 total jobs supported by the project through that date. According to calculations made by the IDG ET, the total is 6,112." BIZ+ Annual Performance Data Table Program Objective/ Performance Indicator Unit Disaggregation Year Value Target (10/1/2011 - 7/31/2017) Actual report by BIZ+ (as of 6/30/2017) 2011 0 6,500 6,111 Male 2011 0 3,250 4,713 Female 2011 0 3,250 1,365 Other 2011 0 33 Indirect Jobs (calculated through multipliers, optimistic scenarios) # 2013 0 10,000 4,241 B Increase in gross household income by those households benefitting from USG assistance $ 2011 0 $6,500,000 $9,021,589 C Number of vulnerable households benefitting directly from USG assistance # 2011 0 8,000 8,134 D Number of business firms and other organizations receiving grants or directly benefitting from USG assistance # 2011 0 60 60 1.2 Number of micro enterprises/ micro entrepreneurs benefitting directly or indirectly by BIZ+ investment and service grants # 2011 0 4,500 4,507 1.3 Value of resources leveraged from grant recipients $ 2011 0 $17,500,000 $17,714,328 1.4 Value of cost share resources from grant recipients, volunteers and other partners $ 2011 0 $714,000 $574,392 1.5 Amount of business revenue attributed to grant assistance $ 2013 0 $12,000,000 $6,190,464 1.6 Percent of assisted businesses reporting positive operating cash flow % 2013 0 50% 58% 2.1 Number of consultants and volunteers person-days mobilized to provide enterprise development/technical assistance to businesses # 2011 0 2,800 4,554 2.2 Number of new technologies and management practices made available for transfer # 2011 0 600 877 2.3 Percentage of enterprises supported by BIZ+ that adopted new technologies and management practices to improve productivity in their business. % 2011 0 90% 90% 2.4 Number of trainings/ technical assistance provided # 2011 0 187 189 2.5 Number of individuals benefitting directly from technical assistance & trainings # 2011 0 3,000 3,795 2.6 Percent of assisted businesses demonstrating improved operational efficiency within two years of BIZ+ investment % 2013 0 50% 57% 2.7 Percent of assisted businesses who improve their composite business capacity score % 2013 75% 83% A Number of jobs created through USG assistance Total Total Baseline Total Total Total Total Total No disaggregation Intermediate Results Component 1: Grants to new and existing enterprises Component 2: Technical assistance to grant recipients and other institutions related to enterprise development Total Total Program Total Total Gender # Total Total Total Total Total Total 60 ANNEX VI: CASE STUDIES This annex was removed as it contained direct identifiers of the grantees. 61 ANNEX VII: SOURCES OF INFORMATION Documents Reviewed Associate Cooperative Agreement No. AID-383-LA-11-00002, (2011), VEGA-LAND O’Lakes Sub-Agreement No. 001-BIZ+ Program. Volunteers for Economic Growth Alliance, (2015), Annual Performance Data Table: Results as of End of Year 4 (Revised), Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2015), Annual Performance Data Table: Results as of End of Year 4, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2016), Annual Performance Data Table: Results as of End of Year 5, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2016), Annual Performance Report: USAID BIZ + FY 2016, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2012), Baseline Survey of Household Incomes in North and East Sri Lanka Final Report, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2012), BIZ+ Implementation Timeline (Revised Year 2), Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2014), BIZ+ Implementation Timeline (Revised Year 4), Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2017), BIZ+ Investment Grant Status Report, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2012), Grants Manual, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2013), Performance Management Plan: USAID BIZ + Year 3, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2014), Performance Management Plan: USAID BIZ + Year 4: October 1, 2014-July 31, 2015, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2015), Performance Management Plan: USAID BIZ + Year 5: October 1, 2015-July 31, 2016, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2016), Performance Management Plan: USAID BIZ + Year 6: October 1, 2016-July 31, 2017, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2011), Performance Monitoring Plan: USAID BIZ + Year 2011 - 2014, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2011), Performance Monitoring Plan: USAID BIZ + Year 2011 - 2014, Colombo, Sri Lanka: USAID. 62 Volunteers for Economic Growth Alliance, (2012), Quarterly Performance Report: USAID BIZ + January – March 2012, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2012), Quarterly Performance Report: USAID BIZ + April – June 2012, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2012), Quarterly Performance Report: USAID BIZ + July – September 2012, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2012), Quarterly Performance Report: USAID BIZ + October – December 2012, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2013), Quarterly Performance Report: USAID BIZ + January – March 2013, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2013), Quarterly Performance Report: USAID BIZ + April – June 2013, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2013), Quarterly Performance Report: USAID BIZ + July – September 2013, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2013), Quarterly Performance Report: USAID BIZ + October – December 2013, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2014), Quarterly Performance Report: USAID BIZ + January – March 2014, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2014), Quarterly Performance Report: USAID BIZ + April – June 2014, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2014), Quarterly Performance Report: USAID BIZ + July – September 2014, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2014), Quarterly Performance Report: USAID BIZ + October – December 2014, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2015), Quarterly Performance Report: USAID BIZ + January – March 2015, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2015), Quarterly Performance Report: USAID BIZ + April – June 2015, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2015), Quarterly Performance Report: USAID BIZ + July – September 2015, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2015), Quarterly Performance Report: USAID BIZ + October – December 2015, Colombo, Sri Lanka: USAID. 63 Volunteers for Economic Growth Alliance, (2016), Quarterly Performance Report: USAID BIZ + January – March 2016, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2016), Quarterly Performance Report: USAID BIZ + April – June 2016, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2016), Quarterly Performance Report: USAID BIZ + July – September 2016, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2016), Quarterly Performance Report: USAID BIZ + October – December 2016, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2017), Quarterly Performance Report: USAID BIZ + April -June 2017, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2013), VEGA/BIZ + PMP Summary Table, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2014), VEGA/BIZ + PMP Summary Table, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2015), VEGA/BIZ + PMP Summary Table, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2016), VEGA/BIZ + PMP Summary Table, Colombo, Sri Lanka: USAID. Volunteers for Economic Growth Alliance, (2012), Year Two Work Plan Revision, Colombo, Sri Lanka: USAID. 64 ANNEX VIII: LIST OF GRANTEES INTERVIEWED Grant round Region Location First Eastern Ampara First Eastern Batticaloa First Eastern Batticaloa First Eastern Ampara First Eastern Trincomalee First Eastern Trincomalee First North Central Pollonaruwa Second North Central Pollonaruwa Second Uva Moneragala First Northern Jaffna 65 Grant round Region Location First Northern Vavuniya First Northern Mulathivu First Northern Jaffna Second Northern Mulathivu First Northern Jaffna First Northern Kilinochchi Second Northern Kilinochchi Second Uva Badulla Second Uva Moneragala Second Uva Moneragala 66 ANNEX IX: ACCESS TO FINANCE PROGRAMMING We recommend considering providing broader access-to-finance assistance, and for placing access-to￾finance programming within broader private-sector development competitiveness programming so that assisted SMEs are being assisted not only with access-to-finance but also in a range of areas where help is needed. Commercial Banks Comprehensive assistance could be given to commercial banks to increase their capacity for successfully lending to SMEs and to utilize more flexible collateral instruments for loan guarantee purposes. Suggested activities include: • Identify commercial banks or finance companies with strong SME client outreach potential and strong evidenced interest in accessing SMEs as a major new clientele; • Develop MOUs with each of these institutions detailing technical/training support activities and corresponding expected changes in risk analysis and management-related policies/procedures, along with up-scaled SME loan origination and loan administration-related (e.g., non-performing loans (NPL) rate, SME loans/officer) and product development targets; • Depending on bank interest, encourage at least one MOU to include a focus on outreach to women-owned businesses, as discussed in this report; • Provide technical assistance and training to banks to meet goals outlined in the MOUs. Experts ideally should be located with banks to provide assistance. Activities could include: o Review institutional and organizational structures; o Review existing SME and client identification/outreach and credit o Analyze and map the decision making process and procedures for appraisal, o Identify, review and analyze current sector-specific SME product offerings o Define clear strategies /policies on client eligibility criteria, loan destination, amounts, terms, grace periods, interest rates, repayment frequency and other loan terms and conditions. o Define clear step by step procedures for client outreach, loan application, analysis, approval, closing and disbursement for SME loans including to the agriculture sector (or other sectors). o Define clear step by step policies and procedures for loan monitoring, collection, structuring, sanctioning, provisioning and write offs. o Redesign loan documentation and formats including loan application (new and repeat borrowers), contract/closing and disbursement. o Select pilot branches for the testing of loan procedures and formats o In coordination with the marketing departments of each bank, propose marketing strategies for the new ag product including branding and marketing materials. o Develop and implement (how-to-do) guides for loan officers on loan promotion/ marketing, portfolio transfers among other. o Make adjustments to the job description and profile of loan officers and supervisors to reflect the new responsibilities as defined by the loan manual. o Redesign current staff incentive system to reflect the new approach to SME lending and client outreach. o Review current/planned credit staff training programs and make recommendations for improvement/design and implement new programs as required. Given the need for longer term finance by SMEs but the very short-term asset-liability mix of many banks, DCA facilities to guarantee loans for longer term on-lending could be appropriate. In these 67 cases, the DCA would enhance bank liabilities (loans for funding credit to clients) rather than bank assets (credit to clients). USAID should take a broad and long-term view of the question of finance for SME development. Efforts to increase the utilization of existing DCAs should be subsumed under a broader effort to make financial markets work without long term donor intervention. The work initiated by the GOL in collaboration with the IFC to expand the types of collateral and improve information flows should be at the heart of USAID assistance on the supply side of the equation. Non-bank financing It is important to keep in mind the relevance of non-bank sources of lending, which even in developed economies are a significant source of financing. • “Embedded financing”. Financing can be provided by partners in a value chain rather than by commercial banks, and these partners can be assisted in finance techniques by USAID. For instance, USAID could teach credit assessment skills and policies, and financing perspectives to agro-processes and agro input sellers for farmers, or to wholesalers that sell to retailers to provide financing to those retailers. These partners can be effective suppliers of credit to small producers and traders as they already have the market linkages and market knowledge. Trader credit and contract farming are “embedded financing” mechanisms that can expand access to finance in rural areas. Trader credit refers to short-term or seasonal loans between buyers and sellers. It is typically provided in commodity-based value chains such as rice or other grains. In contract farming, loans are tied to purchase agreements where farmers agree to sell to a given buyer, who in turn often commits to providing additional services such as technical assistance. • Franchising. USAID can consider providing technical assistance to support the establishment of franchising and the capacity building of franchisors. Franchisors can be an important source of financing for franchisees, in addition to other benefits the provide. Equity A constructive approach can be to combine together access-to-finance assistance for lending and for equity investment. For instance, in a USAID project in Tunisia that IDG implemented, IDG created a consortium of banks lending to small businesses, small business equity lenders, and small business capacity building organizations, for joint programs to select promising small businesses, mentor them, and help them gain access to bank and equity financing. 68 BiD Network: SME Web Platform On the USAID Tunisia ICT Competitiveness Project, IDG launched a Tunisian SME Portal to support SME financing, development and entrepreneurship through BiD Network. BiD Network is an online platform and service that prepares emerging market entrepreneurs for investors. The core goal of BiD Network is to facilitate the start, growth and finance of businesses in emerging markets, serving entrepreneurs, coaches, investors, business angels and business incubators with an international network of partners in Africa, Eastern Europe, Latin America, and the Middle. The BiD Network web platform (www.bidnetwork.org) – currently with more than 46,000 members – facilitates business plan competitions, business plan tools, online community building, coaching programs and the assessment of business plans. Upon registration on the platform, Tunisian SMEs were guided through the process of developing their project descriptions and business plans that will bring them to “fundable” status and served to invite investors. Tunisian partners on the initiative included BIAT, the country’s largest private bank, and the government￾owned Bank for the Financing of SMEs, Tuninvest-Africinvest Group (a leading private equity house in North and sub-Saharan Africa), Carthage Business Angels (an association that brings together individual investors in business ventures, known as angel investors, with high-impact entrepreneurs), and IM Bank (a merchant bank with a strong capacity to provide mentoring and expertise to start-ups and early-stage companies). Two other members are SOTUGAR, a government credit guarantee institution, and ESPRIT, a leading private university focused on engineering and technology. These partners collaborated to collectively identify, evaluate and vet SMEs for financing and support, using common criteria, on a pilot basis introducing improved lending and investment practices that later can be rolled out nationwide, organized and supported by the BiD Network platform. SMEs upload business plans and financial plans to the platform. The partners evaluated the plans and determine financial and technical assistance needs, and coordinated investment and lending decision-making, complemented by mentoring of local entrepreneurs. The platform was launched in the final months of the project. At project completion, the core group of Tunisian counterparts, with organizational and technical support from IDG, had reviewed the first batch of applications for financing from 15 start-up firms and early-stage companies. 69 ANNEX X: SCOPES OF WORK FOR COMPETITIVENESS PROJECTS This annex presents scopes of work for three USAID projects that successfully combined access to finance activities (including DCA loan guarantees) with other private sector development activities to yield a greater impact. Financial Inclusion for Rural Microenterprises (FIRM) FIRM is an example of an access to finance project that combines firm-level assistance and policy reform. FIRM is an $18 million, five-year financial services project supporting three U.S. government initiatives in Kenya – Feed the Future, Global Climate Change and Power Africa – in three key areas: agriculture, clean/renewable energy and policy reform. FIRM’s goal is to increase the productivity and growth of on and off-farm agriculture value chains by increasing financial services to underserved groups throughout Kenya, particularly in rural areas. FIRM utilizes DCA loan guarantees, technical assistance and consulting to de-risk credit processes used by financial and non-financial institutions across current market segments and new business opportunities. FIRM also supports key policy reforms critical to the ongoing safe and sound evolution of the financial sector. FIRM’s technical assistance to partners includes development of agriculture finance strategies, financial product development, financial modeling for producer groups, strengthening agriculture value chains through financial and market linkages, institutional strengthening which includes development of operational manuals and capacity building. Activity areas include: IR 1: Increase access to financial services for rural and agriculture enterprises - Establish and launch the value chain finance center with USAID Financial Sector Deepening Project (FSD) • Partner with FSD and combine financial resources to deliver agriculture value chain finance throughout Kenya with an emphasis on Feed the Future priorities and institutionalize the methodologies providing by FIRM inside Center and throughout the Kenya • Build the capacity and long-term viability (financial and managerial) to function as a permanent fixture in the Kenyan financial services marketplace necessary to outlive FIRM’s project life • Create financing opportunities for financial and non-financial value chain actors and respond to new and emerging opportunities IR 2: Expand access to and the use of clean/renewable energy - Establish Partnerships with Clean and Renewable Energy Firms and Providers of Specialized Business Development Services to Promote the Uptake of Innovation • Develop system to evaluate clean and renewable energy transactions, especially those deals with linkages to agriculture and energy • Partner with appropriate business service providers to deliver results • Undertake a diagnostic of legal, policy of regulatory barriers that stand in the way of developing the sector IR 3: Incorporate innovative ICT solutions to enhance inclusion - Establish linkages between ICT solutions providers and other components (agriculture, energy, gender, youth and policy reform), including financial and non-financial sector partners • Develop system to evaluate ICT transactions and development interventions, especially deals with linkages to agriculture and energy • Partner with appropriate business service providers to deliver results 70 IR 4: Promote new financial models for youth, women and very poor groups - Establish linkages between ICT solutions providers and other components (agriculture, energy, gender, youth and policy reform), including financial and non-financial sector partners • Develop system to evaluate ICT transactions and development interventions, especially deals with linkages to agriculture and energy • Partner with appropriate business service providers to deliver results IR 5: Maximize the use of DCA loan guarantee facilities • Manage existing portfolio of guarantees for performance • Generate and respond to emerging opportunities: o Feed the Future – agriculture o Clean and renewable energy o Water, health insurance IR 6: Enhance financial sector reforms - Partner with public and private sector institutions to affect necessary policy change and build capacity to improve stability, efficiency and inclusion • Branchless/agency banking • Credit reference bureaus • Risk-based microfinance supervision • Crisis management/stability enhancement • Anti-money laundering • Consumer protection • Agriculture commodity surveillance and response mechanisms • Collateral registry system For the private sector, FIRM enters into a partnership agreement (a contract) non-disclosures, confidentiality agreements and specific deliverables tied to measureable outcomes. From there, the project jointly works on a strategy with the partner enterprise and then supports the operationalization of the strategies which often leads to specific product development. As strategies produce quantifiable results for the partner, FIRM is often called upon to help upgrade internal systems and business functions (governance, human resource management, MIS/ICT, accounting/finance, treasury, etc.), to raise additional capital or to source US government loan guarantees to keep pace with growth in new clientele, income and profitability. With the Government of Kenya, FIRM responds to specific needs upon their request, drawing upon a deep pool of highly-qualified Kenyan and international experts. In support of devolution, FIRM encourages economic investment and development of Kenya’s newly-formed county governments to create the conditions for increased investment in Kenya’s critical agriculture and finance value chains. In partnership with the Government of Kenya, FIRM is working with National Economic and Social Council (NESC) to develop a national credit guarantee scheme. FIRM’s key accomplishments include establishing over 125 partnerships with local institutions in agriculture, clean/renewable energy and water finance, and helping to facilitate and unlock approximately $500 million in local commercial capital for Kenyans. Azerbaijan Competitiveness and Trade Project (ACT) 71 ACT was a three-year $22 million project that supported an improved business environment, a more transparent trade environment, and increased competitiveness of agriculture products in targeted value chains. Component 1 focused on several areas under the rubric of business enabling environment: commercial-oriented legal reforms; administrative barriers reduction, particularly in food safety; and support for the Central Bank. It saw some success with the inclusion of recommendations in several pieces of legislation still in the adoption process at the end of the project, the adoption of tax code amendments that the project had supported, and inspection checklists by the Veterinary Service to create consistency in inspections of food processing companies. It also supported capacity development of six different government departments to improve understanding and push for reforms in needed areas. The project saw its largest impact through its support of the different Central Bank departments, building capacity and helping adopt new policies, procedures, and methodologies, and it created a better understanding of best practices. Activities included: • Legislative and regulatory reform to improve the business environment over the long-term. These included competition, tax, property, secured transactions, e-commerce, licensing and permits, and food safety inspection regulations. • Support to the CBA. Strengthened central bank departments in several areas including: the research department to improve economic modeling capacity to inform policy decision-making; the supervision department to improve ability to monitor financial institutions; the financial management and strategic management departments to implement, maintain and fully utilize an activity-based management system; the risk department to establish and implement operational risk management policies and practices; the Azerbaijan Mortgage Fund to develop options for better management of the fund; and the financial monitoring service to oversee and enforce compliance with anti-money laundering /countering of the financing of terrorism legislation and practices. Supported the development of new Risk-Based Banking Methodology for the bank to transition from CAMELS rating system, which was formally adopted. Provided three-week exchange capacity development opportunity for onsite and offsite inspection personnel at the Bank Supervision Department with the Colorado and Texas state regulatory bodies. Established and provided hands-on training on the activity-based management system to support internal benchmarking, development of performance targets, and tracking of results. Component 2 focused on supporting improvements in the trade environment through the adoption of best practices in line with the WTO (World Trade Organization) agreements and EU requirements. Activities included legislative and regulatory reform, capacity development of involved government agencies and staff, the launching of the process of harmonizing local standards with international standards, and support directly to the government in the WTO accession process. Activities included: • Legislative and Regulatory Reform. ACT provided technical reviews, support, and advice to advance the legislative reform process and bring the legal framework in line with best practices and the WTO agreements. The work covered a wide array of areas including intellectual property, food safety, animal and plant life health, customs, trade remedies, licensing, foreign trade policy, standardization and accreditation, and technical regulations. 72 • Standards Harmonization. ACT helped the government begin the harmonization process with CODEX and IPPC, which was critical to enable WTO accession and improve compliance with international safety requirements and quality standards. Harmonization also helps improve compliance with import requirements in the Eastern European and EU markets and increases export potential. ACT also assisted the Standards Committee to develop a plan for converting its mandatory requirements to voluntary standards and technical regulations. • Institutional Reform and Capacity Building. ACT assisted in the planning, launching, and advancing of numerous institutional reforms and in building the capacity of a number of state bodies to properly and effectively implement and administer trade and WTO related laws and to benefit from being a WTO member and from integrating into the system. • Support for WTO Accession. ACT provided extensive support directly to the government of Azerbaijan in activities related to the accession process. • Trade Facilitation and Reduction in the Cost of Trade. ACT worked with a local partner to quantify the costs of trade by working on two pilot studies (import on animal feed and export of pomegranates) to map import and export processes and recommend elimination of bottlenecks for reducing the cost of trade. Component 3 focused on improving the competitiveness of targeted value chains including dairy, aquaculture, pomegranate, apple, and hazelnut. The project provided support by working with value chain players above the farm gate and with farmers directly on the farm. In addition, the project worked with financial institutions interested in lending more to rural farmers in the regions. Activities included: • Improved Competitiveness of Targeted Sub-Sectors. The project began with a selection process to identify four additional value chains to support in addition to the two already pre￾selected by USAID, dairy and aquaculture. Through a process of both qualitative and quantitative (through domestic resource costing developed under the previous project) studies, the project, in cooperation with USAID, selected value chains. Hazelnuts. The project focused on improving quality in order to improve processors’ ability to sell larger and better quantities. This meant helping processors adopt best international practices with a focus on both product and process standards. ACT supported the development of additional marketing avenues for some of the larger processors that were interested in diversifying their client base, producing new products and exploring new markets. This support provided opportunities for the companies to develop new products, such as blanched and sliced hazelnuts, leading to increased sales and profitability. The development of brand names and new packaging with branded logos also provide improved visibility and earning prospects for some of the players. ACT also supported the linking of local companies with potential buyers abroad such as in the US, Korea, China and Japan. Pomegranates. ACT supported a number of trade deals such as the export of pomegranate juice, concentrate, and sauce to Canada, Germany, Spain, and Poland in high quantities, and recommended participation is several trade fairs and events. ACT also focused on promoting the use of modern packaging materials to ensure the arrival of quality products at their destination. Apple/Cold Chain Development. ACT cooperated with over 25 cold-storage facilities providing regular training and technical assistance, along with accompanying written materials with storage conditions of most of the locally available fruits and vegetables. 73 Association formation. ACT assisted in the development of three associations, organizing general assemblies, and drafting charters, bylaws and all the other necessary legal documentation. Food Safety and HACCP Compliance. ACT designed a train-the-trainer program on HACCP to develop local capacity while at the same time support direct implementation in beneficiary companies. Access to Finance. ACT worked with four commercial banks and a non-banking financial institution in the implementation of ag-lending technologies. The project supported the streamlining of loan procedures, simplification of formats, reduction of documentation requirements and development of manuals and guidelines with the goal of improving access to credit for small farmers and rural entrepreneurs in the agriculture value chain. ACT focused on building the capacity of credit managers and loan officers with respect to SME ag-lending and the project carried out over 50 training sessions for branch managers, credit managers, trainers of trainers and loan officers from all of the partners. The project followed up with intensive field training to loan officers and supported evaluation of real loan applications. The project also conducted a benchmarking exercise to support the identification of potential non-performing clients. Tunisia ICT Competitiveness Project The USAID ICT Competitiveness Project was a $7.5m, 15-month project that aimed to provide assistance focused on ICT sector development to enhance its role as a catalyst for accelerated private sector growth and job creation. Component 1. Increasing and Developing ICT and ICT-Enabled Entrepreneurship The project assisted 83 enterprises in the following sectors: ICT, apparel and textiles, specialty foods, and manufacturing. Technical assistance included strategic/business planning, market development, production planning, quality control, and financial management. Training covered soft skills and technical skills, mainly for newly recruiting staff. Also worked on improving access to finance for SME with efforts focused on improving the ability of leading players in debt and equity markets to serve SMEs. The Project initiated the SME Finance Pilot Core Group whose objectives are to improve coordination among local financial institutions in the provision of debt and equity to SMEs, increase deal flow from local entrepreneurs, develop linkages between entrepreneurs and mentors, and ultimately increase the number of closed deals. The Project also placed short-term advisors in several financial institutions in an effort to assist them in improving their capacity to provide financial services to SMEs, including several banks and several private equity funds. Component 2. Growing and Developing Tunisia’s Workforce The Project’s workforce development strategy involved two major activities: 1) co-financed training for newly recruited employees, which was undertaken in close coordination with the enterprise development activities (described above); and 2) job matching and placement services that included job fairs and career placement centers. The project established six career centers in universities across Tunisia, and organized The Project organized two major job fairs in Tunis and smaller-scale fairs at campus career centers. The project worked with the National Agency for Employment and Self￾Employment on organizing the job fairs with a focus on building the agency’s capacity to organize job fairs in the future. 74 Component 3. Policy and Regulatory Reform Within the first months of implementation, Project staff developed a framework for policy technical support, outreach, and consensus-building. Meetings and focus group discussions were held with key institutional actors—ICT companies, ICT associations including the Tunisian Association for Communication and Technology (TACT), broader business associations including Tunisian American Chamber of Commerce (TACC), private policy think-tanks and research organizations such as IACE, commercial finance institutions, major donor agencies— and government policy-making institutions. The Project closely coordinated with USAID, TACT, and other major counterparts to formulate a prioritized enabling-environment reform agenda for ICT job growth. This involved evaluating key laws and regulations affecting the ICT sector; identifying and prioritizing the policy, legal, and regulatory constraints impeding ICT investment, growth, exports, and job creation; and developing a policy, legal and regulatory reform strategy to eliminate or reduce constraints. Initially, reform areas included: 1) finance; 2) telecom; 3) labor; 4) tax; 5) e-governance; 6) e-commerce; 7) public-private partnerships. The Project mapped the bottlenecks that restrain private sector growth and worked with counterparts to develop recommendations for streamlining administrative procedures and lowering entry barriers for domestic and foreign businesses. The project established in ICT taskforce to propose and agree amendments to the telecommunications code. This led to the formation of a public/private Working Group on Telecom Reform empowered to develop key amendments to the 2001 Telecom Code. The Project supported reforms to simplify taxes, broaden bases, and reduce high rates corporate income taxes, personal income taxes (PIT) and social contributions as well as to shift the tax burden to value-added tax (VAT) and indirect taxes. Using detailed simulation models for PIT, corporate income tax, and VAT that it developed with the Ministry of Finance, the Project conducted in-depth analysis of revenue and incidence implications of alternative policy scenarios for reducing tax rates and broadening tax bases, principally through removing exemptions and deductions. The project conducted an assessment of the labor code and made recommendations to bring it into compliance with international norms. Key results of the project include creating 2,409 jobs, trained 824 people, and helped produce three business environment laws, regulations, or administrative procedures. 75 ANNEX XI: DCA LOAN GUARANTEE PROGRAMS Sustainable Water and Sanitation in Africa (SUWASA) – Kenya The Sustainable Water and Sanitation in Africa (SUWASA) initiative in Kenya helped water service providers (WSPs) access approximately $3.5 million of loans to finance water infrastructure projects valued at a total of $4.6 million. These investments increased water supply with the upgrade of a water treatment plant; augmented distribution networks with pipeline extensions, household connections, and community tap stands with prepaid meters; and they reduced nonrevenue water through pipe rehabilitation projects. As a result, more than 48,000 low-income residents have first-time or improved access to clean water, with the associated benefits of convenience, quality, cost savings, and time savings. Furthermore, the investments helped improve the WSPs’ financial performance. WSPs reduced operational costs and boosted revenues through the reduction of water losses and an increase in customers and water supply. The enhanced financial performance, in turn, positioned WSPs to replicate viable investment proposals to continue increasing and improving services to unserved and underserved populations. Engaging multiple banks in water sector financing led to a more competitive commercial lending environment. Banks now compete on loan terms, grace periods, interest rates, and fees, resulting in more favorable lending terms to WSPs. Since 2012, banks have reduced annual interest rates from 21 percent to 15.5 percent. Three utilities secured Results Based Aid grants and contributed approximately $882,000 in equity towards project completion. SUWASA demonstrated that commercial financing is a solution for water companies with a certain minimum level of financial performance to increase and improve water infrastructure in low-income areas. Of equal importance, WSPs came to view the poor as a commercially viable market. Through specialized technical assistance, WSPs identified profitable investments to expand services, and they developed proposals that met banks’ financing criteria. Banks that previously viewed lending to WSPs as too risky now see this market as a profitable and viable opportunity if appropriate risk mitigation measures are applied. Banks provided millions of dollars in loans to improve water service delivery. SUWASA facilitated demand among WSPs for commercial loans and fostered bank interest in lending to WSPs. To encourage water sector lending, SUWASA facilitated credit enhancements, including access to USAID’s DCA guarantee and the WSPs’ attainment of Kenya’s Water Services Trust Fund (WSTF) Results Based Aid for commercially financed projects. Lastly, SUWASA increased the utilities’ awareness of the need for gender equality in project design, implementation, and M&E, as well as in their institutional policies, procedures, and work environment. Assistance to the Coffee Sector – Rwanda Rwanda embarked on a new coffee strategy with severely deteriorated production capacity, almost no processing capacity, limited access to and use of inputs (fertilizers, improved coffee varieties) by producers, and little understanding of how to produce high quality coffee. The coffee sector is highly diversified affecting an estimated 450,000 smallholder farmers who grow most of the country’s coffee in small plots of 150-300 trees. Yet, the industry suffered from difficulty financing crucial investments. The low quantity, low quality loop the coffee sector endured was mainly due to the state of coffee plantations after the war, difficulty of financial access for long-term coffee investments, the inability to renew old coffee trees, low yielding coffee varieties, and a virtually inexistent local consumption. To address these issues, it was necessary to develop a supporting financial network if the Rwandan coffee sector was to become competitive on international markets. Small credit lines at the grower level (e.g. 76 group lending model – micro credit schemes), skills upgrading of financial lending institution staff, and easier access to long term capital for coffee investors were some of the identified priorities. Rwanda had very limited capacity to fully wash coffee – a key element of the chosen strategy to increase coffee quality. To address this limitation, the strategy called for increasing the number of coffee washing stations (CWS) from six in 2002 to 107 by 2010 – largely through the efforts of private investors. However, most potential investors could not obtain the medium-term financing necessary to construct a CWS or the short-term seasonal financing necessary to operate the station. Banks were unwilling to lend to coffee sector investors because most investors lacked the capacity to demonstrate the creditworthiness of their projects with good business plans as well as sufficient or appropriate collateral to guarantee the size of loans required to build and operate a CWS. USAID/Rwanda supported the growth of the fully washed coffee sector at a number of levels. It developed technical assistance projects to enhance production, management, and processing capacity in the coffee sector. USAID/Rwanda’s assistance included: • The Partnership for Enhancing Agriculture through Linkages in Rwanda (PEARL) – PEARL focused on agribusiness enterprise development in several promising sectors including specialty coffee. The project was responsible for the first exports of specialty coffee from Rwanda to the US and UK. Even after the end of the PEARL project in 2006, USAID/Rwanda continued its support to these sectors through a new project – Sustaining Partnerships to Enhance Rural Enterprise and Agribusiness Development (SPREAD) which carries on PEARL activities. • Agribusiness Development Assistance in Rwanda (ADAR) – ADAR focused on improving the processing and marketing capacities of private sector agribusiness enterprises (and some associations and cooperatives) working in value-added export-oriented products. In 2004 it was supporting the coffee, pyrethrum, and passion fruit sub-sectors. • ACDI/VOCA Food Security Project – ACDI/VOCA provided grants to develop viable business entities and has funded agribusiness cooperatives in the coffee, tea, and rice sub-sectors. • ISAR Agricultural Technology Development and Transfer Project (ATDT) – The project focused on institutional development and dissemination of improved technologies in agriculture. The projects sought to enhance the capacity of farmers and processors to improve coffee quality and to develop links to high-value export markets. Most banks had little or no experience in agricultural lending, and they were not prepared to evaluate risks associated with lending to the agriculture sector in general or the coffee sector in particular or to effectively monitor loans. In 2004, USAID implemented a Development Credit Authority (DCA) loan guarantee with the Bank of Kigali (BK) with the objective of increasing access to credit for strategic export-oriented agricultural. The DCA guarantee covered 40 percent of the loss of principal on a maximum of $2 million in loans. In the 32 months that the guarantee was in effect, the Bank of Kigali issued over $1.7 million in investment and working capital loans to coffee washing station investors, thus utilizing 86 percent of the guarantee. The guarantee had a limited impact on the Bank of Kigali’s lending to the coffee sector outside of the protection of the guarantee. In spite of the bank’s demonstrated interest in continued lending to the sector and ample opportunity to lend, the bank has issued no investment loans to the sector since USAID suspended the guarantee. The bank has provided working capital loans outside of the guarantee to a handful of borrowers with which it gained experience during the guarantee but has not changed its usual lending practices when making the loans – that is, it requires 100 percent collateral. Some DCA borrowers were able to accumulate assets for use as collateral during the time that they made use of DCA loans which then gave them greater access to credit outside of the guarantee. Senegal Economic Growth Project Task Order 5 (PCE TO5) 77 As part of the inter-agency Feed the Future (FTF) strategic plan, led by USAID/Senegal’s Economic Growth Office (EGO), Economic Growth Project Task Order 5 supports the Government of Senegal (GOS) to increase food security and decrease poverty through the promotion of a productive and competitive private sector. PCE TO5 covers four core areas of Senegal’s FTF strategy closely tied to the Accelerated Growth Strategy and Country Investment Plan of the GOS. The four FTF core areas, the four components of concern to PCE TO5, are as follows: • Increased agricultural productivity and market linkages focused on rice (rain-fed and irrigated), maize, and millet value chains. • Enhanced agriculture policy environment, addressing key constraints with the broadest impact on agriculture by focusing on reforms to enhance the business environment—most specifically, the full implementation of the Agricultural Law of 2004 and continued discussions on land tenure. • Improved rural post-harvest infrastructure, including associated access to finance to ensure long-term agricultural productivity. USAID/Senegal provides entrepreneurship training and technical assistance to help investors succeed and develops loan mechanisms through Development Credit Authority (DCA) loan portfolio guarantees. • Increased institutional and human resource capacity to ensure access to, and the development of, next-generation agriculture in Senegal. This component is intended to lay a foundation for growth through a market-driven approach, including access to finance, training investors in business management, developing efficiencies in production and processing, promoting professionalization of cooperatives or producer organizations, and reinforcement of capacity within the associated civil and public institutions. Access to capital focuses on (i) development of financial models and financial coaching of value chain actors in support of the formal contract farming scheme for cereals; (ii) introduction of new financial instruments; (iii) provision of technical support to USAID’s DCA guarantees, especially for agriculture); and (iv) training of financial institution staff on agricultural lending. PCE has supported USAID/Senegal’s DCA program by working closely with banks. By involving the banks through its formal farming contract mechanism, the project facilitates producers’ access to farm inputs in the rice and maize value chains. The project also works with existing contract farming operators (in the SRV) to improve their operations, which results in improved access to inputs for rice farmers. The project financed two rice mills for a women’s group and took an extra step often missing similar grants or cost-sharing agreements: training the women (16 in this case) in their 25 operations. The project also introduced maize threshing equipment to producers. The project introduced innovative credit leasing to value-chain actors. PCE’s facilitation of a medium-term credit facility through the leasing company Locafrique constitutes an important result. Banks, when they do give credit, require collateral that many borrowers lack. This project activity, under which the equipment is the collateral, resulted in the acquisition of 15 new tractors by producer groups in 2011 and a rice mill by one GIE Naxari Deret in 2012. 78 ANNEX XII: REVIEW OF DOCUMENTS AND FORMS USED BY BIZ+ IN THE SOLICITATION, REVIEW, AND APPROVAL OF GRANT APPLICATIONS # BIZ+ Document/Form Title Date of Document Source Date Provided by E￾mail to IDG Description Comments and Suggestions 1 BIZ+ Process Fact Sheet - English - July2017 8/16/2017 USAID 12/4/2017 General description of the two step application and selection process, including elements of the applicant's required concept note, and selection criteria. None 2 Concept Note to Investment Grant Diagram 7/20/2017 USAID 12/4/2017 Flow chart describing application process. None 3 Request for Applications (RFA) 7/27/2015 BIZ+ 6/30/2017 The latest version provided to the evaluation team by BIZ+ was RFA 003, dated 7/27/2015, which solicited applications for Phase 2 of BIZ+. None 4 Concept Note Outline 9/2/2013 BIZ+ 6/30/2017 The latest version provided to the evaluation team by BIZ+ was attached to RFA 002, dated 9/2/2013, which solicited applications for Anuradhapura and Polannawura Districts. It consists of three sections: 1) Project Snapshot; 2) Organization Description; and 3) Description of Investment Opportunity. The latter section includes: 1) Business Overview; 2) Project Background; 3) Investment Purpose; 4) Key factors for success of business activity; and 5) Expected impacts, results, and sustainability BIZ+ should specifically ask applicants to list any government protection (e.g. tariffs) or subsidies that are essential for the ongoing success of the business. BIZ+ should also consider requesting additional information about key owners and managers such as education, experience in managing a private for-profit business, and experience in managing a business in the same line as the one being invested in by BIZ+. 79 # BIZ+ Document/Form Title Date of Document Source Date Provided by E￾mail to IDG Description Comments and Suggestions plan. 5 Full Application BIZ+ - Aug2017 8/22/2017 USAID 12/4/2017 Full application submission checklist and certification. BIZ+ should again request applicants to list any specific government protection or subsidies that are essential for ongoing success. 6 Planning and Financial Projections Tool 1 8/22/2017 USAID 12/4/2017 Financial information and projections model. While the existing templates are adequate for collecting data from applicants, additional templates should be developed for BIZ+ staff in order to calculate both the economic and financial internal rates of return for the proposed projects (EIRR and FIRR). (See discussion below for more information on EIRR and FIRR.) 7 Planning and Financial Projections Tool 2 8/22/2017 USAID 12/4/2017 Details on planned investments and employment (summary information transferred to Tool 1). Same as above 8 Template BCA 11/30/2017 USAID 12/4/2017 115 questions organized under seven different business subject areas: governance; management; human resources; financial resources; business operations and viability; gender considerations; and environmental considerations. Answers are rated against 57 criteria to arrive at an overall composite score. BIZ+ should review and update the BCA questionnaire and rating sheet. For example, under '5. Business Operations and Viability', applicants should be specifically queried about government protection (e.g. tariffs) or subsidies that are essential for ongoing success. Under '2. Management', more detailed information should be sought to determine the rating for 'g. Qualifications and abilities of firm’s management.' 80 # BIZ+ Document/Form Title Date of Document Source Date Provided by E￾mail to IDG Description Comments and Suggestions 9 Template Evaluation and Due Diligence Report 11/30/2017 USAID 12/4/2017 Internal memo from BIZ+ investment officer to BIZ+ management summarizing keys aspects of the investment and the due diligence process. The memo should highlight the opportunities and the risks involved in the investment. The discussion of risks should include market risks, systemic risks (e.g. increases in input costs due to currency devaluation), and especially note whether the success of the investment is dependent on government protection or subsidies. The memo should also highlight the financial and economic analysis conducted by BIZ+ staff and clearly discuss any differences between the projected financial and economic rates of return. 10 Template Pre￾award Assessment and Risk Determination 11/30/2017 USAID 12/4/2017 Internal memo that summarizes the seven subject areas in the BCA. May also be used in obtaining USAID approval of subgrants. This is another opportunity for BIZ+ investment staff to highlight the specific issues that may affect the success of the project. 81 Issues in How Best to Support Private-Sector Development: Focus on firms vs. business development providers? Grants vs. equity investments vs. lending? The role of cost-benefit analysis, and financial and economic rates of return. In planning the expansion of BIZ+, we propose to consider three fundamental questions regarding the best method to support a country’s private-sector development. Focus on firms vs. business development service providers? A first question to consider is whether it is better to focus private-sector development assistance directly on firms or on business service providers that serve firms. Some important lessons learned in this regard come from Europe and Eurasia, where USAID has considerable experience with such programs. 20 Years of USAID Economic Growth Assistance in Europe and Eurasia (2013) states that there was a: “move from what might be termed ‘retail’ intervention at the level of the firm to ‘wholesale’ assistance to groups of firms. Firm-level consulting gave way to facilitative approaches: training business development services (BDS) providers, fostering cluster development or value chain linkages, stimulating policy reform, and supporting trade certifications […] Due to expense and lack of sustainability of firm￾level assistance, by the second half of the 1990s, establishing “business development services” rose to prominence as an alternative approach. (page 63)” Since the mid-1990s, USAID has moved away from individual firm assistance toward assisting groups of firms through BDS providers, clusters, and value chains, although individual businesses still receive subsidized goods and services under these approaches. Grants vs. equity investments vs. lending? If financing is given directly to firms, it is important to consider whether the best approach to providing such financing is through grants, equity investments, or lending. Generally, USAID has favored equity and lending, and not grants. The 20 Years of USAID Economic Growth Assistance in Europe and Eurasia report (Appendices 12 and 13) does not support grants as a means for providing capital to firms but, rather, explains how firm-level finance has been extensively and successfully provided through credit guaranty programs like the Development Credit Authority (DCA) and through Enterprise Funds. DCA programs and Enterprise Funds have the proper institutions, processes and incentives for successfully providing capital to target firms; grants are not generally viewed as a proper instrument for providing capital to firms, particularly not in an environment like Sri Lanka with numerous traditional sources of capital available. Providing funds directly to firms for the purpose of their expansion is essentially a role for venture capital, with high returns if successful but high probability of failure. Enterprise funds are a good structure for providing such funds since they can retain high returns from the rare successes and also withstand a high failure rate. Grants can neither retain returns when investments are profitable nor can they tolerate a very high failure rate. The role of cost-benefit analysis, and financial and economic rates of return. In considering whether to provide funding to firms, it is important to use cost-benefit analysis to calculate the financial and economic rates of return from grants (or other methods of finance). BIZ+ should strengthen its project selection process to select firms that are market-oriented—clearly serving a market demand—and that demonstrate likely financial viability. It is important to look at the financial internal rates of return (FIRR) and, to some extent, economic internal rates of return (EIRR). Assessing market demand means identifying evidence that the market demands the goods or services to be produced by the firm, and that the firm can offer these at prices that are competitive. Information for 82 each candidate firm should be placed into an Excel template for calculating FIRRs and EIRRs. The current template collects useful information but does not adequately calculate FIRRs and EIRRs. Financial analysis examines whether a financial investment in a project can be recovered, given the expected incremental costs and benefits.23 A project is considered financially viable if the financial net present value (FNPV) is positive and the internal rate of return (IRR) on the investment exceeds the threshold financial rate of return. If the incremental FNPV is below zero, the project is not sustainable without support. The FIRR should take into consideration: expected gross revenues from current and new sales; expected expenditures on current and any new costs arising from new production processes, expanded production volumes, higher quality standards, etc.; any new investment; and the time value of money. Economic analysis determines the viability of the project based on its effect on the society as a whole. In order to express the benefits and costs at their true economic values to the society, the economic analysis adjusts the financial analysis to correct for a variety of economic distortions, such as taxes and subsidies. Similar to the financial analysis, a project is considered economically viable if 1) the economic net present value (ENPV) is positive, 2) the economic rate of return (ERR) on the investment exceeds the economic opportunity cost of capital and, 3) the ERR is above 12 percent (a relatively high rate of return threshold used as an unofficial USAID threshold).24 It is typically recommended that a project with a negative incremental ENPV to be rejected, discontinued, or take corrective measures to redesign the project. When conducting an economic analysis, even a limited methodology would at least correct for any price distortions in the financial analysis. This will allow to correct for any instances where firms are producing goods protected by Sri Lanka’s high tariffs, in which case the firm likely would not be profitable if selling at market prices. It does not make sense for USAID to support the increased production of goods and services that would not find markets in the absence of extensive protectionist policies. Additionally, a more thorough EIRR also would include any or all of the following factors: USAID program costs estimated to be spent on a candidate firm; additional tax revenues accrued by GOSL as a result of project assistance; additional job creation and/or increase in wages; and environmental impact. We believe that Excel templates could be created to facilitate the calculation of EIRR and training provided to project staff. The economic analysis should follow the logic in the graphic on the right. The following graphic suggests the appropriate decision USAID should take under different scenarios.25 23 Juan A. B. Belt and Clarence Zuvekas Jr., “Strengthening Cost-Benefit Analysis in USAID, 2011-2013: Lessons Learned and Future Directions” (U.S. Agency for International Development, March 11, 2014), http://pdf.usaid.gov/pdf_docs/pa00jqsp.pdf. 24 Traditionally, the World Bank uses 10% or 12% while the ADB uses 12% for the project rate of return threshold. USAID does not have an official threshold but uses 12% as a benchmark. 25 USAID Guidelines, Cost-Benefit Analysis, August 18, 2015, p39. 83 Source (USAID Guidelines, Cost-Benefit Analysis, August 18, 2015, P39.) For further reading, we recommend several useful documents to inform decision making, including: 20 Years of USAID Economic Growth Assistance in Europe and Eurasia (2013); USAID Guidelines, Cost-Benefit Analysis (2015); Strengthening Cost-Benefit Analysis in USAID, 2011-2013: Lessons Learned and Future Directions (2014); and Guide to Cost-Benefit Analysis of Investment Projects, European Commission (2014). 84 ANNEX XIII: DISCLOSURE OF CONFLICT OF INTEREST FORM 85 86