FINAL PERFORMANCE EVALUATION OF THE USAID/JAMAICA SOCIAL ENTERPRISE BOOST INITIATIVE December 2018 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by ME&A, Inc. FINAL PERFORMANCE EVALUATION OF THE USAID/JAMAICA SOCIAL ENTERPRISE BOOST INITIATIVE This publication was produced at the request of the United States Agency for International Development Developed under Request for Task Order No. SOL-72053218R00001 under PPL￾M&E IDIQ Contract It was prepared independently by: Colin Maclean, Team Leader Alexandre Monnard, Assistant Team Leader/Evaluation Specialist Collette Campbell, Evaluation Subject Matter Specialist Contractor: ME&A, Inc. 4350 East-West Highway, Suite 210 Bethesda, MD 20814 Tel: (301) 652-4334 www.meandahq.com DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ABSTRACT The Social Enterprise Boost Initiative (SEBI) is a six-year1 micro-enterprise development activity aimed at supporting and facilitating the growth of social enterprises (SEs) and their contribution to the social economy in Jamaica. SEBI’s activities are implemented under four specific, yet inter￾related technical objectives. SEBI achieves those objectives through four main initiatives focusing on strengthening the capacity of a select number of social enterprises/entrepreneurs to help them become viable businesses, creating an enabling environment for the SE sector by improving institutional capacity through information sharing, raising public awareness of the sector, and reviewing options for a permanent version of the program potentially through Social Enterprise Jamaica (SEJA2 ). This final performance evaluation was commissioned to determine whether the SEBI activity achieved its objectives. The evaluation team was tasked with answering three questions regarding whether program objectives were achieved, adopting a mixed-methods approach based on interviews with direct beneficiaries and key stakeholders and focus group discussions with indirect beneficiaries, both in Kingston and St. Andrew, as well as the rural parishes. Two surveys were also conducted—one with SEBI incubator and accelerator clients and a nationwide public awareness survey on SEs. The evaluation found that SEBI was instrumental in increasing public awareness of SEs although this varied according to geographic location. SEBI’s clients also benefited from capacity building measures and creation of a SE network. 1 SEBI was originally a three-year program subsequently extended via three modifications to the Cooperative Agreement. 2 To be established by the end of the program in December 2018. CONTENTS EXECUTIVE SUMMARY................................................................................................... I EVALUATION PURPOSE................................................................................................................................ I PROJECT BACKGROUND............................................................................................................................ I EVALUATION DESIGN, METHODS, AND LIMITATIONS ............................................................... I FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS............................................................................. II 1.0 EVALUATION BACKGROUND, PURPOSE AND QUESTIONS ........................ 1 1.1 EVALUATION BACKGROUND................................................................................................. 1 1.2 EVALUATION PURPOSE .................................................................................................................... 1 1.3 EVALUATION QUESTIONS ................................................................................................................... 1 2.0 PROJECT BACKGROUND....................................................................................... 3 3.0 EVALUATION METHODS AND LIMITATIONS.................................................. 4 3.1 EVALUATION METHODOLOGY .............................................................................................. 4 3.2 EVALUATION LIMITATIONS ...................................................................................................... 7 4.0 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS ............................... 8 4.1A EVALUATION QUESTION 1A: TO WHAT EXTENT WERE THE PROJECT OBJECTIVES ACHIEVED? ........................................................................................................................................................ 8 4.1B EVALUATION QUESTION 1B: WHAT FACTORS HAVE CONTRIBUTED TO THE SUCCESS/FAILURE OF THE DIFFERENT PROJECT COMPONENTS? ................................................................ 26 4.1C EVALUATION QUESTION 1C: DID THE ACTIVITY HAVE SIMILAR EFFECTS ON MEN AND WOMEN? ....................................................................................................................................................... 28 4.2 EVALUATION QUESTION 2: TO WHAT EXTENT HAS SEBI CONTRIBUTED TO THE LOCAL SOCIAL ENTERPRISE POLICY ENVIRONMENT? .............................................................................................. 29 4.3 EVALUATION QUESTION 3: BASED ON THE EVALUATION FINDINGS, WHAT ARE THE BEST PRACTICES AND LESSONS LEARNED FOR OTHER PARTNERS IMPLEMENTING SIMILAR ACTIVITIES SUCH AS LPD AND SOCIAL ENTERPRISE JAMAICA (SEJA)? ...................................................................................... 32 ANNEXES ........................................................................................................................36 ANNEX 1: STATEMENT OF WORK.............................................................................................................. 37 ANNEX 2: LIST OF MEETINGS AND INTERVIEWS........................................................................................ 48 ANNEX 3: LIST OF DOCUMENTS CONSULTED.......................................................................................... 53 ANNEX 4: OUT-BRIEF POWERPOINT PRESENTATION.............................................................................. 56 ANNEX 5: SEBI EVALUATION MATRIX....................................................................................................... 66 ANNEX 6: SURVEY REPORTS........................................................................................................................ 69 LIST OF TABLES Table 1: Breakdown of KIIs Conducted by Project Stakeholder Type and Site................................. 5 Table 2: Summary of Findings from FGDs and Direct Beneficiaries – Incubators and Accelerators – Related to Evaluation Question 1.................................................................................. 10 Table 3: Summary of Findings from Direct KIIs (Stakeholders) Related to Evaluation Question 1................................................................................................................................... 11 Table 4: Summary of Findings from FGDs and Direct Beneficiaries – Incubators and Accelerators – Related to Evaluation Question 2.................................................................................. 30 Table 5: Summary of Findings from Direct KIIs (Stakeholders) Related to Evaluation Question 2................................................................................................................................... 31 Table 6: Summary of Findings from FGDs and Direct Beneficiaries – Incubators and Accelerators – Related to Evaluation Question 3.................................................................................. 33 Table 7: Summary of Findings from Direct KIIs (Stakeholders) Related to Evaluation Question 3................................................................................................................................... 34 LIST OF FIGURES Figure 1: Demographic Characteristics of the 2013 and 2018 Awareness Surveys....................... 16 Figure 2: Perceived Importance of the Social Enterprises Listed........................................................ 17 Figure 3: Perceived Importance of the Social Enterprises Listed........................................................ 18 Figure 4: Perceived Value of a Trademark for Goods/Services of SEs.............................................. 18 ACRONYMS Acronym Description BDU Business Development Unit CA Cooperative Agreement CBO Community-Based Organization CBSI Caribbean Basin Security Initiative CDC Community Development Committee COMET Community Empowerment and Transformation Project CSO Civil Society Organization DAC Development Area Committee DBJ Development Bank of Jamaica EC European Commission ED Enumeration District ET Evaluation Team FGD Focus Group Discussion GOJ Government of Jamaica ISO Intermediate Service Organization JAJ Junior Achievement Jamaica JN Jamaica National JNF Jamaica National Foundation JSIF Jamaica Social Investment Fund KII Key Informant Interview LPD Local Partner Development MDA Ministry, Department & Agency M&E Monitoring and Evaluation MICAF Ministry of Industry, Commerce, Agriculture, and Fisheries MOFPS Ministry of Finance Public Service MLGCD Ministry of Local Government and Community Development MONA Mona School of Business and Management MRSL Market Research Services Limited MSME Micro, Small, and Medium Enterprise NGO Non-Governmental Organization PDC Parish Development Committee PIOJ Planning Institute of Jamaica SDC Social Development Commission SE Social Enterprise SEBI Social Enterprise Boost Initiative SEJA Social Enterprise Jamaica SME Small and Medium Enterprise SOW Statement of Work USAID United States Agency for International Development WB World Bank WBO Women Business Owners Jamaica YEA Young Entrepreneurs Association of Jamaica i EXECUTIVE SUMMARY EVALUATION PURPOSE The purpose of this evaluation of the Social Enterprise Boost Initiative (SEBI) is to determine whether the SEBI activity achieved its objectives and gain lessons for implementation of ongoing or future programs. The evaluation sought to identify: 1) implementation challenges, corrective actions to project management, and progress towards achieving expected results; 2) the factors that contributed to the success/failure of the different project components; 3) the effect of the activity on men and women; and 4) the extent to which SEBI contributed to the local social enterprise (SE) policy environment. The evaluation will serve to inform the United States Agency for International Development Jamaica Mission’s (USAID/Jamaica) Local Partner Development (LPD) program under the Caribbean Basin Security Initiative (CBSI) as well as the Social Enterprise Jamaica (SEJA), which is in the process of being registered as a non-governmental organization (NGO), with a view to continuing some of the work of SEBI once the project ends in December 2018. The exact nature of SEJA’s activities is unclear, although the Evaluation Team (ET) understands that the focus could primarily be on incubation services for SEs under the LPD program. PROJECT BACKGROUND SEBI was initiated by the Jamaica National Foundation (JNF) and jointly funded by JNF and USAID. The rationale for USAID’s decision to support SEBI in Jamaica, implemented by (JNF), was based on several important factors at the time, namely: 1) low levels of business competitiveness and productivity; 2) a stagnant economy; 3) a high debt burden; and 4) high levels of crime in the country. The genesis of SEBI was in JNF’s observation that not-for-profit and other organizations operating in the social sector were struggling to deliver their core services mainly due to a reliance on grants and donations. The underlying goal therefore was to introduce the social enterprise model to these organizations so that they could start generating their own income and become less reliant on grants and donations. SEBI began in July 2012 and is due for completion at the end of December 2018, following three extensions agreed in Modifications 2, 3, and 5 in the Cooperative Agreement (CA). EVALUATION DESIGN, METHODS, AND LIMITATIONS The ET pursued three separate, yet interlinked, data collection activities that formed the basis of its methodological approach to conducting the SEBI evaluation as follows: 1. Face-to-face meetings on site with SEBI’s incubator and accelerator beneficiary clients as well as representatives from key stakeholders, 3 including national and local government, the private sector, international community, civil society, NGOs, and business associations and agencies. In total, the ET met with 29 beneficiaries (20 incubators and 13 accelerators) and 30 stakeholders. Team members used semi-structured questionnaires to record answers for later comparative analysis and cross-reference purposes with findings from the other data collection methods, mentioned below. 2. The ET hosted four focus group discussions (FGDs) in the parishes (Trelawny, St. James, Westmoreland and Manchester) and one in Kingston with selected groups of SEBI indirect beneficiaries. In total, 31 participants attended the FGDs, providing views and responses 3 Annex 2 provides a complete list of all interviews conducted by the ET. ii to questions which were noted for later cross-referencing with findings from the direct beneficiary and public awareness surveys. 3. Two surveys were carried out by Market Research Services Limited (MRSL), based in Kingston, on behalf of the ET (see Annex 6 – Survey Reports for complete analysis of the findings from both surveys). The ET encountered no specific limitations that adversely affected its evaluation mission or ability to achieve its objectives. The only impediments to the evaluation that affected its outcome were associated with logistical issues in terms of locating and organizing KIIs and FGDs and the implementation of both surveys. Nonetheless, there are several important limitations inherent to the design selected for this evaluation, including: • Time and location constraints: Time in the field and accessibility of certain sites outside of Kingston limited the number of SEBI stakeholders that could be interviewed through KIIs and FGDs. The Kingston area was an important exception to this, as the ET had significantly more time to interact with beneficiaries and other stakeholders there. • Data availability and data quality: No project performance data, such as a baseline and detailed monitoring data spanning the duration of the CA, was made available by the implementer. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS Findings Conclusions Recommendations EVALUATION QUESTION 1a: To what extent were the intended project objectives achieved? Objective 1: Develop Solution-Based Social Enterprises Evidence suggests that most incubators and accelerators (26 of 29 interviewed) benefited from SEBI’s capacity building measures and networking activities. However, while acknowledging the relevance of improving business acumen, a significant number of SEs (12 of 29) indicated that they were experiencing difficulties in remaining solvent, and several (8 of 29) advised that without further support, including the ability to access finance, they would probably not survive. Note that these did not include 8 of 29 SEs that are part of charities and that 8 of 29 operated from their homes. This applied to SEs in Kingston and St. Andrew and the parishes. Although SEBI was largely successful in carrying out its capacity building mandate under its Project Objective 1 (Develop Solution-Based Social Enterprises), important questions remain about the viability and sustainability of a select number of SEs/entrepreneurs. This was principally due to the lack of available funding from microfinance institutions that the project was tasked with supporting through leveraging relationships with key partners. Furthermore, scaling up remains a concern for many other SEs mainly for the same reason—lack of development finance. For any current or future similar projects, the ET strongly suggests that a specific needs analysis should be conducted to identify the unique needs and challenges for each SE for targeted interventions to be developed and implemented at the pre-project design stage. In this way, proposed interventions will be demand driven and in line with beneficiary needs, which will serve to reduce the potential and/or likelihood of enterprises receiving project support to fail. In addition, a needs assessment could also be built into the first phase of a project implementation to identify appropriate interventions. iii Findings Conclusions Recommendations Objective 2: Creating an Enabling Environment The evaluation found that, at macro level, after six years of SEBI, there is still no legislative framework in place in Jamaica to support SEs and significant questions remain as to what that framework should look like and include. However, the government’s draft micro, small, and medium enterprise (MSME) policy does include a chapter on SEs, which can be attributed to SEBI’s advocacy work. At the micro level, SEBI’s information sharing activities (e.g., workshops, seminars, field visits) helped improve the enabling environment for SEs via mentoring and coaching initiatives for selected enterprises and ongoing discussions with the Development Bank of Jamaica (DBJ) to develop a product for MSMEs. SEBI was partially successful in achieving its mandate in fostering a more business-friendly environment for SEs at the macro level as defined in SEBI’s Project Objective 2. At the micro level, SEBI did help create a SE network to promote the collective work and needs of its members. However, this has to be tempered with the fact that there is little evidence to support SEBI’s wider aim of ensuring that such a network would lead to members operating profitably, as highlighted in Question 1a above, in light of the ET’s finding that just to remain solvent was a major issue for some of those members (direct and indirect beneficiaries). To build on the progress achieved so far, the ET recommends that any future similar projects incorporate SEBI’s enabling environment objectives to ensure that social entrepreneurship remains on the radar in Jamaica. However, there is a pressing need for project objectives to be far more defined and focused on what is possible (and measurable) with respect to issues related to creating (or helping create) a business-enabling environment. This is due to the inherent difficulties in attributing the extent to which a project can impact or influence such an environment where the success or otherwise of a business can depend on many interconnected factors, many of which will have little to do with the interventions of a single project. At the macro level, government policy or lack of it can influence enabling conditions, as can market and financial circumstances at the micro level. Objective 3: Raise Awareness of Social Entrepreneurship across Jamaica SEBI pursued a range of awareness raising activities (e.g., workshops, trade fairs, and media campaigns, such as the “Buy Social” social media campaign), which brought the concept of social entrepreneurship to a previously unreached audience (youth, women, government, and private sector players) in Kingston and St. Andrew. However, the level of awareness in the rural parishes was marginal among the same group because no similar outreach awareness raising activities were conducted by SEBI. SEBI partially fulfilled its mandate regarding social entrepreneurship because it concentrated its efforts in the Kingston and St. Andrew metropolitan area, to the virtual exclusion of the parishes, as was apparent from the responses from both key informant interviews (KIIs) and FGDs. While SEBI’s “Buy Social” campaign on social media (January through March 2018) contributed to spreading the word on SEs, it remained largely within Kingston and St. Andrew circles with almost no coverage in the parishes. Because the outreach campaign was so limited, this could not be viewed as “putting social entrepreneurship in Jamaica on the proverbial map,” as suggested in SEBI’s Project Objective 3. For future interventions supporting SEs, special effort must be made to include local stakeholders in the rural parishes, including hosting awareness events there. Schools should also be encouraged to continue promoting the SE concept in Jamaica, especially in the parishes. Attention should also be paid to the type of medium employed to disseminate messages to the wider public, particularly outside of Kingston and St. Andrew. Mobile networks may not always reach more remote areas. A more effective reach is likely to be through local events (e.g., fairs, church). iv Findings Conclusions Recommendations Objective 4: Transition to Social Enterprise Jamaica The evaluation found no broad￾based awareness of SEJA among project beneficiaries (direct or indirect) or stakeholders from KIIs and FGDs, in terms of its suggested purpose, structure, or business model. Instead, there was a more universal view that post-SEBI there is a need for a meaningful entity to offer support to SEs across Jamaica, with a focus on more practical, technical interventions, in order to increase the likelihood of viability leading to longer-term sustainability. SEJA may not necessarily be the correct vehicle to advance the cause of SEs in Jamaica if the goal is to support enterprises to become more efficient, resulting in their becoming profitable businesses. While the formation of an NGO, such as that proposed for SEJA, may offer general capacity building measures to support MSMEs in Jamaica (not dissimilar to SEBI’s mandate), such a body may not be equipped to offer real-time assistance to struggling enterprises (SEs or not). Before any such entity is contemplated all stakeholders must be consulted, including at local level in the parishes, in order to ensure that such a body is viable and sustainable and that it is aligned with the actual needs and requirements of intended beneficiaries. Potential support for business service providers may be an option for a follow-on entity to consider as a focus. This is lacking in Jamaica as a service aimed at improving private sector development including the business-enabling environment. EVALUATION QUESTION 1b: What factors have contributed to the success/failure of the different project components? Prior to the design and preparation of the SEBI project, there was no needs/gap analysis on the SE sector in Jamaica to identify organizations: 1) most likely to express interest in becoming SEs; and 2) with the greatest needs in line with the project objectives. This resulted in SEs being selected after the fact, i.e., interventions may not have been relevant. The ET noted that only the first 10 pilot SEs were selected prior to any sector analysis, although few went on to become profitable businesses, which was SEBI’s original mandate. SEBI’s efficiency and effectiveness in delivering Objectives 1 and 3, particularly, were adversely affected. Feedback from direct beneficiaries strongly suggests that assistance provided did not result in their becoming viable. For future similar programs it is imperative that the specific needs that are going to be addressed by project activities are clearly identified and mechanisms are put in place to ensure objectives are successfully achieved. EVALUATION QUESTION 1c: Did the activity have similar effects on men and women? Feedback from KIIs and FGDs indicate that there was an adequate representation of men, women, and youth among SEBI’s beneficiaries— direct and indirect. No group was excluded or marginalized. SEBI fulfilled its mandate to include SEs owned or operated by men, women, and youth. A similar approach to inclusiveness should be incorporated in any future program supporting social entrepreneurship. v Findings Conclusions Recommendations EVALUATION QUESTION 2: To what extent has SEBI contributed to the local social enterprise policy environment? At the local level, SEBI has engaged in many activities related to the creation of a SE network, which has resulted in building connectivity, education, and business alliances for SEs to constructively interact with one another. At the national level (as highlighted in Question1a above), SEBI was influential in gaining government approval to include a chapter on SEs in its MSME draft policy paper (not yet law). Evidence suggests that SEBI has been instrumental in fulfilling its mandate as indicated in its Statement of Work (SOW) to create an environment in which the social entrepreneurship sector can flourish and thrive. However, this is not yet reflected in any specific policies being adopted either at the local or national level. Thus far, SEBI has been a catalyst in bringing parties together from the public and private sectors to discuss SE inclusion in national legislation. Such a network needs to be formalized by ensuring that the necessary tools and frameworks to support such an endeavor are implemented fully, to ensure island￾wide access and reach in any future program or SE initiative. EVALUATION QUESTION 3: Based on the evaluation findings, what are the best practices and lessons learned for other partners implementing similar activities such as LPD and for SEs in Jamaica? Collaboration and sharing of information about SEs among direct and indirect beneficiaries (i.e., incubators, accelerators), private sector partners, government agencies, community-based organizations (CBOs), local authorities, and universities, among others, facilitated a more in-depth understanding and awareness of social entrepreneurship in Jamaica. The evaluation flagged several weaknesses in the design and delivery of the various program components (highlighted above) that should be addressed in any similar current or future project. SEBI met its overall obligations and objectives as per its original mandate and in line with USAID’s Sub Intermediate Result (IR) 2.3 under USAID/Jamaica 2010-2014 Country Assistance Strategy and USAID/ Jamaica’s Results Framework 2013- 2019, together with the Government of Jamaica’s (GOJ) National Development Vision 2030 Jamaica, to support enterprise development for micro-enterprises and creation of an enabling policy. It is evident that enterprise development is critical to Jamaica’s growth agenda, hence the design and delivery of project components must be designed and done right, and strategically located in as many collaborative arrangements as possible. Entities such as LPD and/or SEJA should ensure that such projects incorporate a knowledge sharing component and robust networking initiatives to sustainability and inclusion. Similarly to the strategic alliance between JNF and Digicel Foundation, where Digicel Foundation provided grants to SEBI beneficiaries, various development projects must seek to foster strategic alliances and sharing of resources to meet their mandate as well. 1 1.0 EVALUATION BACKGROUND, PURPOSE AND QUESTIONS 1.1 EVALUATION BACKGROUND The United States Agency for International Development Jamaica’s (USAID/Jamaica) decision to support and fund the Social Enterprise Boost Initiative (SEBI), initiated and implemented by the Jamaica National Foundation (JNF), was based on a number of important factors at the time, namely: 1) low levels of business competitiveness and productivity; 2) a stagnant economy; 3) a high debt burden; and 4) high levels of crime in the country. The genesis of SEBI was in JNF’s observation that not-for-profit and other organizations operating in the social sector were struggling to deliver their core services mainly due to a reliance on grants and donations. The underlying goal therefore was to introduce the social enterprise model (SE) to these organizations so that they could start generating their own income and become less reliant on grants and donations. The evaluation was conducted during the period August – December 2018, by a team assembled by ME&A, located in Bethesda, MD, and its subcontractor NORC. The team included a Team Leader, an Evaluation Expert, and a local subject matter expert. In addition, a local firm was hired to conduct two surveys. 1.2 EVALUATION PURPOSE According to the Statement of Work (SOW), the purpose of this evaluation is “to determine whether the SEBI activity achieved its objectives and gain lessons for implementation of ongoing or future programs.” The evaluation sought to identify: 1) implementation challenges, corrective actions to project management, and progress towards achieving expected results; 2) the factors that contributed to the success/failure of the different project components; 3) the effect of the activity on men and women; and 4) the extent to which SEBI contributed to the local SE policy environment. More specifically, the evaluation will serve to inform USAID/Jamaica’s Local Partner Development (LPD) program under the Caribbean Basin Security Initiative (CBSI) as well as Social Enterprise Jamaica (SEJA), which is in the process of being registered as a non-governmental organization (NGO), with a view to continuing the work of SEBI once the project ends in December 2018. 1.3 EVALUATION QUESTIONS The SOW provided a set of evaluation questions to be answered by the Evaluation Team (ET), which were subsequently elaborated upon in the ET’s Work Plan and Evaluation Design. They are as follows: 1. (a) To what extent were the intended program objectives achieved? (b) What factors have contributed to the success/failure of the different project components? (c) Did the activity have similar effects on men and women? 2. To what extent has SEBI contributed to the local SE policy environment? 3. Based on the evaluation findings, what were the best practices and lessons learned for other partners implementing similar activities such as LPD and for SEJA? 2 A detailed SEBI Evaluation Matrix highlighting the above questions and related sub-questions regarding the data collection methodology and analysis employed in this evaluation is provided in Annex 5 of the report. 3 2.0 PROJECT BACKGROUND Jamaica’s stagnant and underperforming economy, high debt burden, high levels of crime and violence, and low levels of competitiveness and productivity together conspired to adversely impact the living standards of Jamaican citizens and limit government spending on security, youth programs, and social services. One of the victims of this was Jamaica’s social sector, which relied heavily on donations and grants for its existence and, under the prevailing circumstances, found itself unable to bridge the gap and allow SEs to progress from non-sustainable entities to profit￾making businesses. USAID/Jamaica already had a strong history of working collaboratively with the Government of Jamaica (GOJ) to address these constraints, which were inhibiting Jamaica’s ability to become an economically viable, safe, and secure country in which individuals, businesses, and SEs could prosper and grow. SEBI was therefore a natural progression towards supporting Jamaica’s social and economic development aspirations and complementary to USAID’s Community Empowerment and Transformation Projects, COMET I (2006-2012) and COMET II (2013-2018). Since these economic and social factors inhibit the ability of SEs to become profitable business enterprises, USAID determined that without support for SEs, their ability to survive in the long term would be in doubt. SEBI began in July 2012 and is due for completion at the end of December 2018 following three extensions agreed in Modifications 2, 3, and 5 in the Cooperative Agreement (CA). SEBI has been running in Jamaica since 2012, implemented by the JNF, and is due for completion by the end of December 2018. 4 3.0 EVALUATION METHODS AND LIMITATIONS 3.1 EVALUATION METHODOLOGY The performance evaluation of SEBI utilized a mix of mutually reinforcing qualitative and quantitative methods, allowing the ET to triangulate findings. Data and information needed to respond meaningfully to USAID’s three main questions in the evaluation SOW were gathered from several qualitative and quantitative sources. Qualitative sources included review of project documents, and fieldwork consisting of key informant interviews (KIIs) and focus group discussions (FGDs) with project beneficiaries and stakeholders performed between October 9 and October 25, 2018. Quantitative sources included two separate surveys. The first targeted direct beneficiaries (SEs involved in the incubator or accelerator programs, or otherwise having received sustained SEBI assistance) to complement the KIIs being done with many of them, while the second targeted the Jamaican adult general population to measure any change in awareness and attitude towards SEs. 3.1.1 Qualitative Data Collection and Analysis The qualitative component of the performance evaluation drew on three data collection methods: • Structured desk review of SEBI’s available quarterly and annual reports and other implementation documentation (e.g., selection criteria for incubator and accelerator, SE capacity assessment), as well as relevant secondary information; • KIIs conducted with SEBI staff, direct project beneficiaries (SEs that were in the incubator or accelerator or otherwise received sustained assistance), and other project stakeholders, including national government officials, civil society organizations (CSOs), etc.; and • FGDs conducted with a mix of direct and indirect project participants and community members at five locations. Structured Desk Review The ET obtained project documents from SEBI and USAID/Jamaica, and reviewed and coded them in a structured manner to provide a shared understanding of the project; guide the development of the instruments and surveys; and enrich this report. Documents reviewed included: quarterly and annual progress reports from SEBI, SEBI’s original SOW and multiple modifications, SEBI’s available annual work plans, criteria used for selection in the incubator and accelerator, lists of equipment purchases made by SEBI for some of the direct beneficiaries, a report on the baseline public attitude and awareness of SE implemented by Market Research Services Limited (MRSL), and documents detailing the findings of various research commissioned by SEBI. In addition, the team reviewed various documents and reports about the local micro, small, and medium enterprise (MSME) and SE context, and government reports detailing plans for supporting the sector. These documents served to provide the context for SEBI and to better understand how the project filled the gaps identified by various experts. 5 Key Informant Interviews The ET conducted a total of 53 KIIs across six parishes of Jamaica (see Table 1) with a range of direct and indirect beneficiaries, government stakeholders, business support and CSOs, and others that have some relationship with SEBI, as well as SEBI staff and USAID staff. Many of the KIIs undertaken involved more than a single person from a given organization, but these are all counted only once in Table 1. All KIIs used semi-structured open-ended interview guides tailored to the type of project stakeholder interviewed, except for the guide for the interview with USAID, which used a series of ad hoc questions crafted by the team. Table 1: Breakdown of KIIs Conducted by Project Stakeholder Type and Site KII Type Parishes Total Kingston & St. Andrew Tre￾lawny St. Catherine West￾moreland St. James Man￾chester St. Thomas Direct beneficiary SEs 17 1 4 2* 2 2 1 29 Indirect beneficiary SEs 1 - - 1 - - - 2 SEBI Staff 2 - - - - - - 2 National government stakeholders 7 - - - - - - 7 Local government stakeholders - 2 - 1 - - - 3 Business support stakeholders 5 - - - - - - 5 Civil society stakeholders 1 - - - - - - 1 Academic researchers 3 - - - - - - 3 USAID staff 1 - - - - - - 1 Total 37 3 4 4 2 2 1 53 * Two SEs were part of the FGD in that location and are therefore counted here as well. Focus Group Discussions The ET conducted a total of five FGDs, one in each of the following parishes: Trelawny, St. James, Westmoreland, Manchester, and Kingston/St. Andrew. FGDs were organized using lists provided by SEBI, from which up to 31 individuals were selected at random and invited, keeping in mind refusals and no-shows. FGDs used semi-structured open-ended interview guides tailored to the type of project stakeholder interviewed and were animated by team members with experience facilitating this type of discussion. 6 Notes obtained from KIIs and FGDs were reviewed and cleaned before the ET coded their content systematically around themes relevant to the evaluation questions at hand. The coding also allowed for sorting by stakeholder type, location, and phase at which SEs became involved. 3.1.2 Qualitative Data Collection and Analysis Two surveys were commissioned by the ET: 1) the direct beneficiary survey; and 2) the general population awareness and attitude survey. Both surveys were conducted in October 2018 by MRSL, a local firm which had already done survey work for SEBI. MRSL used trained enumerators monitored by experienced fieldwork coordinators and targeted different groups of respondents. Direct Beneficiary Survey This survey was administered using Computer Aided Telephone Interviewing (CAPI) to all the direct beneficiary SEs of SEBI as a follow-up to the KIIs done with many of them. The instrument used was designed by the ET to collect information that was not easily collected during KIIs and that filled some of the gaps identified in the KII instruments. This survey supplied quantitative data based on multiple choice and Likert scale questions, analyzed using descriptive statistics with disaggregation by type of organization (charity-based or business), location, and services provided. Of the 35 SEs that were active beneficiaries and surveyed, 24 were available and accepted to participate (a 69 percent response rate.) General Population Awareness and Attitude Survey This survey was first implemented by MRSL and SEBI in 2013 to gather information regarding the level of awareness and the overall attitude of the Jamaican public. After a review and internal assessment by the ET, the same instrument and methodology were reused in 2018 to assess whether any evolution had occurred over the five-year period between rounds, during which SEBI was active. Both rounds of this survey were done in-person at the residence of randomly selected Jamaican adults (18 years and older) across all 13 parishes. Stratified sampling was used to select enumeration districts (EDs) based on geography (parishes) and demographic characteristics (socio-economic status, rural or urban, etc.). In each selected ED, a random walk was performed to selected households. In each household, only a single person was surveyed, and the selection was based on which adult had celebrated a birthday most recently (with no substitution allowed). In 2013, 1,000 Jamaican adults took part in the survey, although interactions occurred with a greater number of individuals/households due to refusals, etc. In 2018, due to inclement weather conditions during the data collection period and general time constraints to implement the survey, 700 Jamaican adults were successfully surveyed. A comparison of key respondents’ characteristics can be found in Figure 1. The two samples were generally similar despite re￾running the selection process for EDs (in other words, not reusing the list selected in 2013). Survey data was tabulated by question and broken down based on various respondent characteristics to allow sub-group analysis. To facilitate comparability with the 2013 round, the determination of urban or rural status was done based on whether a parish was predominantly urban (Kingston and St. Andrew, St. Catherine) or rural (the remainder of parishes) according to the Statistical Institute of Jamaica.4 4 Determination was made on the basis of the number of households in urban versus rural areas for each parish as of the most recent census, which took place in 2011. Please see 7 3.2 EVALUATION LIMITATIONS The only impediments to the evaluation that affected its outcome were associated with logistical issues in terms of locating and organizing KIIs and FGDs and the implementation of both surveys. Nonetheless, there are several important limitations inherent to the design selected for this evaluation: • Time and location constraints: Time in the field and accessibility of certain sites outside of Kingston limited the number of SEBI stakeholders that could be interviewed through KIIs and FGDs. The Kingston area was an important exception to this, as the ET had significantly more time to interact with beneficiaries and other stakeholders there. • Data availability and data quality: No project performance data, such as a baseline and detailed monitoring data spanning the duration of the CA, was made available by the implementer. • Selection bias: As some individuals declined to answer the beneficiary survey or the public awareness survey, there was a possibility of selection bias, i.e., those respondents who accepted to be interviewed may differ in some important dimensions from those who did not, for example in terms of their attitudes and perceptions, affiliation with government/non-government structures, socio-demographic characteristics, and experience. • Recall bias: KIIs, FGDs, and the beneficiary survey all relied on self-reports about events and perceptions that may date back several years, which could be incomplete due to lapses in memory and inaccurate or incomplete recall. • Response bias: KIIs, FGDs, and the beneficiary survey all relied on self-reports about events and perceptions that may be biased due to social desirability or to respondents wanting to provide the answers they thought interviewers “want to hear.” For example, some SEs that had received SEBI support may have been hesitant to criticize certain elements of the program out of concern it might affect the support they are still receiving or hope to receive in later iterations of similar programming. To mitigate this limitation, the ET outlined confidentiality and anonymity guarantees to all who participated in KIIs and FGDs. In addition, probing questions were used to understand the rationale behind certain answers and the resulting data was used in coding the data as well as to inform the analysis of the data. http://statinja.gov.jm/census/popcensus/completed/NumberofDwellingUnitsandHouseholdsbyUrbanRuralDistributio nbyParish.aspx for the complete data. 8 4.0 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 4.1A EVALUATION QUESTION 1A: TO WHAT EXTENT WERE THE PROJECT OBJECTIVES ACHIEVED? 4.1.1a Findings Objective 1: Develop Solution-Based Social Enterprises SEBI activities were expected to strengthen the capacity of a select number of SEs and entrepreneurs to make them more efficient and viable businesses within the context of Jamaica’s economic structure for SEs. This was to be achieved through the implementation of a SE virtual incubator and accelerator program. The project was expected to have island-wide coverage; however, beneficiaries were to be selected from specific targeted parishes where the intervention was needed the most, based on research findings. Under SEBI, 21 incubators and 14 accelerators were selected as project beneficiaries. From the evidence gathered from the KIIs, interviews, and desk review, the highest impact of the project on the beneficiaries came from SEBI’s capacity building activities, which included seminars, workshops, networks, field trips, a travel tour, marketing assistance, strategic planning and trainings in business management, marketing and promotions, branding, product development, financial literacy, etc. These activities were highly rated by all direct and indirect beneficiaries interviewed, who commented on the high quality of the program and the value they received from the engagements. Interventions improved the quality of the operating environment of the SEs as they were able to incorporate what they had learned from specific workshops into their everyday activities. However, evidence further suggests that this component was most impactful because of the stage of development of the incubator and accelerator beneficiaries when they were selected. More than 95 percent of the SEs were already fully operational, were struggling, and needed critical support for them to continue or remain in business. The availability of grant funding provided to direct beneficiaries went a long way to helping improve operations of businesses, at least in the short to medium term. Eighteen (18) of 29 beneficiaries that received critical items to improve their operations said they would not have been able to purchase these on their own. However, careful analysis of the data from the site visits revealed limited ability of most of the SEs to scale up even after the interventions, because the critical resources needed for growth and expansion are not within their reach. The ET acknowledges that such resources were not within SEBI’s mandate to provide. However, it was within SEBI’s brief to prepare SEs to successfully compete for and obtain these resources, much of which emanated from the Digicel Foundation, although there were few reported instances of SEs obtaining investment funding. Even though business plans were prepared, and they were taught how to properly run a business, the selected SEs were either not in a position where they could present a case for financing, lacked the confidence or did not see the need to grow. 9 From all indications, this was true for the SEs that were selected as part of the incubator program, the majority of which were charities and not-for-profit SEs who clearly stated that they would not be allowed to take loans but might be able to access grant funding from different sources. However, under the SEBI II accelerator program, there was a shift to mainly for-profit SEs that had business models that had potential to scale. Despite this shift in focus, the selected SEs all had a dominant dependence on donor/grant funding, with limited interest to transition to the market environment. The evaluation noted that the first three years of SEBI focused on capacity building activities, research, identification, and selection of beneficiaries for the project. This evidenced a lack of capacity in those three years towards project implementation to support the sequencing of activities to achieve the project objectives. Progress reports indicated several initial challenges SEBI faced during the first few years of implementation that impacted performance, including staffing issues; a lack of local expertise in the SE sector, resulting in SEBI having to train service providers; and much of the burden of technical assistance falling on the SEBI Project Manager. The ET found that the selection and identification of beneficiaries was not highly effective or appropriate. Those SEs selected to participate in the incubator program were not selected in a structured manner. Given that this was the creation of a new sector, there was not enough awareness about SEs to establish the required criteria for eligibility and identify enterprises that could meet the criteria, let alone those that could go on to become efficient and viable businesses. Consequently, only 7 of the 21 incubators were able to graduate to the accelerator component of the program. Tables 2 and 3 below provide a brief summary of the responses from FGDs and KIIs with respect to the first evaluation question. The main finding of significance from KIIs with stakeholders was that 12 of the 14 interviewed (11 of whom had direct contact with SEBI) were unsure whether SEBI’s interventions had supported SEs to become more efficient (and profitable) enterprises, which was SEBI’s primary objective when it started in 2012. 10 Table 2: Summary of Findings from FGDs and Direct Beneficiaries – Incubators and Accelerators – Related to Evaluation Question 1 Sources Location Evaluation Q1: To what extent were the intended program objectives achieved? FGD Q1: Consider your business an SE? FGD Q2: Contacted or aware of SEBI? SE Q1: Business more efficient and/or profitable after SEBI support? SE Q2: Is your business currently operating? Yes No Not Sure Yes No Not Sure Yes No Not Sure Yes No Not Sure FGD Bunkers Hill, Trelawny 1 11 - 1 - - - - - - - - Incubator and Accelerator - - - - - - - 1 - - 1 - FGD Montego Bay, St. James 5 - 1 - 6 - - - - - - - Incubator and Accelerator - - - - - - - 2 - 2 1 - FGD Savanah-la-Mar, Westmoreland 6 - - 1 5 - - - - - - - Incubator and Accelerator - - - - - - - 2 - 1 1 - FGD Mandeville, Manchester 5 - - 5 - - - - - - - - Incubator and Accelerator - - - - - - 1 - 2 3 - - FGD Kingston and St. Andrew - 2 - 2 - - - - - - - - Incubator and Accelerator - - - - - - 7 6 3 17 - - TOTAL 17 13 1 9 11 0 8 11 6 24 3 0 11 Table 3: Summary of Findings from Direct KIIs (Stakeholders) Related to Evaluation Question 1 KIIs Evaluation Q1: To what extent were the intended program objectives achieved? KII Q1: Have you now or any time in the past been actively involved with SEBI in any capacity? KII Q2: Do you believe that SEs are now more efficient/ profitable businesses following SEBI interventions? Group Yes No Not Sure Yes No Not Sure Government 6 - - - - 6 Local Authorities - 1 - - - 1 Women/Youth Associations 1 - - - 1 - NGOs 2 1 - - - 3 International Donors/ Organizations - 1 - - - 1 Education 2 - - - 1 1 TOTAL 11 3 0 0 2 12 Objective 2: Creating an Enabling Environment At the macro level, evidence suggests that after six years of SEBI there is still no legislative framework in place in Jamaica to support SEs and significant questions remain as to what that framework should look like and/or include. However, because of SEBI’s advocacy work, the GOJ has now included a chapter on SEs in its latest revised draft MSME policy document. 5 It states, “that the revised policy…is updated and a definition agreed for MSMEs and added the inclusion of social enterprises and social value creation.” However, the policy contains no elaboration regarding how this is to be implemented. It only states that the policy refocuses on the development of an enabling environment for MSMEs and fostering of a culture of entrepreneurship in the economy. There is no apparent separation of SEs and MSMEs in this policy revision, nor is there any distinction as to how either will benefit from the policy’s intentions, particularly in the absence of any legal framework. At the micro level, SEBI’s information sharing activities (e.g., workshops, seminars, networking) helped improve the enabling environment for SEs. This is supported by 22 of the 29 direct beneficiaries, who agreed that these initiatives were appreciated and led to either active or visible improvements in their operations—business plan preparation or revision and/or support in marketing and rebranding mostly commonly cited—or facilitated interaction with other similar organizations, the GOJ, or corporate clients. SEBI also developed a partnership with the Ministry of Industry, Commerce, Agriculture and Fisheries (MICAF) and the World Bank (WB) to promote capacity building activity among government business enablers, delivering a two-day SE workshop to 61 (40 female and 21 male) government policymakers. Several SEs made presentations at this training session, the result of which was a more informed audience on the merits of supporting and developing the sector. 5 GOJ Policy Development Program as of March 31, 2018. 12 Media coverage of SEBI’s activities has been relatively high profile in recent years, particularly in the Jamaica Observer, a large local newspaper. Articles have discussed and commented on SEs in Jamaica and on SEBI’s involvement in helping promote the sector.6 SEBI also staged nine networking events (6 in Kingston and 3 in Montego Bay) on social entrepreneurship to spread the SE message. The first event to qualify and be framed as networking was held in Q2 of 2014, “SEBI organized a corporate mingle event in Kingston designed primarily for corporate audiences. 140 people participated and a presentation of SE and SEBI was made. Afterwards, several individuals offered their support on a voluntary basis for Superior Crafts and Dress for Success. Media interviews also occurred and led to coverage by print press and TV.” A second was held in Q3 of 2014 in Montego Bay: “SEBI organized a business mixer in Montego Bay, focused on intro of SEBI, education on SE, networking with private and public sector. 80 people incl. several SEs were present. Key linkages were made between a Negril hotel and Grotto and The Source, particularly in terms of purchasing juice for a primary school supported by the hotel foundation. SEBI was also invited to meet with the Rastafarian Indigenous Village who was seeking involvement in SEBI, but the group was not found suitable for participation.” A further event was held in Kingston in November 2016 to key partners such as MICAF, Planning Institute of Jamaica (PIOJ), Small Business Association of Jamaica (SBAJ), and Digicel Foundation, among others. Another, in which SEBI partnered with PIOJ to introduce members of its Community Renewal Program to the SE model, took place in Montego Bay. SEBI was also instrumental in developing the Jamaica Buy Social Mark which was launched in December 2017. This trademark was intended as a signal to consumers and the public that products and services using this unique mark are made by a SE. However, it is too early to ascertain whether such an initiative will bear fruit or if surveys will be subsequently conducted to determine consumer and public attitudes towards the trademark and the extent to which it influences purchasing decisions. The evaluation, however, noted that, except for Montego Bay, nearly all business enabling environment activities mentioned above concentrated actions almost exclusively in the Kingston and St. Andrew metropolitan area, with minimal outreach to the parishes. The ET verified this, finding that none of the FGD participants in Bunkers Hill, for example, were aware of any initiatives aimed at supporting SEs. In addition, four of the six participants in the Montego Bay FGD had no knowledge of these activities, nor did five of the six participants in the Savanna-La￾Mar FGD session. This is significant, as all parishes were to be impacted through Objectives 2 and 3 via the SE network and public awareness activities and latterly via Objective 4. Only project beneficiaries were found to have knowledge of SEBI. Lack of any local media campaigning was cited as the primary reason for this anomaly, together with the universally held view that none of the participants in any of these locations really understood what a SE is or how it is defined. Furthermore, a prevailing view existed among both KIIs and FGD participants in the parishes that SEBI’s media and networking sessions were “preaching to the converted,” with little understanding of what is needed in the more rural parishes where communication channels with Kingston are limited, and where day-to-day survival 6 SEBI-related articles appeared chronologically in the Jamaica Observer on March 4, 2015, November 3, 2015, August 3, 2016, June 8, 2017, August 25, 2017, and August 19, 2018 (which specifically mentioned SEBI beneficiary EcoFarms). 13 for small businesses is deemed more important than reflecting on what constitutes an SE or how it should be defined. Objective 3: Raise awareness of social entrepreneurship across Jamaica One of SEBI’s central objectives is to raise the awareness of SEs and social entrepreneurship across Jamaica. Consumers, the GOJ, and financial providers were the three groups that JNF had identified as a priority for SEBI’s awareness-building campaign. To assess whether SEBI achieved this program objective, several data sources were combined to capture as broad and diverse of a perspective as possible. During fieldwork, the ET asked SEs that were directly assisted by SEBI in a sustained manner for their perspective regarding the level of awareness and understanding they have noticed when interacting with the public. Other project stakeholders ranging from government agencies to researchers were asked the same question. In addition, the ET asked indirect beneficiaries of SEBI that attended FGDs in parishes throughout Jamaica to speak about their own awareness and understanding of SEs, as well as that of their respective communities. Finally, the team commissioned an end-line survey conducted in October 2018, targeting randomly selected members of the general population (who had zero involvement with SEBI) throughout Jamaica. This survey used the same instrument used in 2013, when SEBI sought to obtain a systematic understanding of the context in Jamaica, which allows for a comparison over time of the public’s understanding of SEs and their attitudes towards these organizations. Perception among direct beneficiaries Among the SEs interviewed, there is broad agreement that awareness and understanding of SEs by the public has improved, from being virtually nonexistent to being generally low except for certain specific subgroups where greater progress was accomplished. A majority (18 out of 24 who expressed an opinion on the subject) indicated that awareness and understanding of SEs by the public remained quite low in 2018. To explain their answers, they often noted that they have to explain the concept frequently when interacting with others or regularly hear people using the term “social enterprise” in a way that suggests a lack of understanding of the underlying concept. For example, “social enterprise” is often equated with charities or nonprofits. Approximately one third of the direct beneficiaries interviewed also noted that they had encountered one or more individuals that are operating or working in something very close to a SE but did not know it and were intrigued and interested when they were provided more information on the concept. The remaining six SEs indicated that awareness and understanding of the concept had improved significantly, but nearly always qualified it by providing one or more specific sub-groups of the Jamaican population for which that has held true in their experience. The most often-cited sub-group was individuals involved in NGOs and charities, followed by upper-middle class (e.g., “professionals,” “startup scene”). The SEs that were most positive about an increase in awareness and understanding of the SE concept among the Jamaican population lived in Kingston. On the other hand, the nine SEs located in rural areas systematically noted that there was little to no awareness and understanding of the concept in their communities or at the grassroots level. 14 Perception and awareness among indirect beneficiaries FGDs, nearly all of which took place in rural areas of Jamaica, confirmed both uneven and often limited awareness and understanding on the SE concept as well as the important disparity in awareness and understanding between the Kingston area and the rest of the country. Participants in these discussions varied somewhat by location but were typically a mix of indirect beneficiaries of SEBI, local business owners that did not interact with SEBI, and some Community Development Committee (CDC) members. The level of exposure to SEBI among focus group participants also varied widely based on location. For example, the focus group in Bunkers Hill involved individuals who had never had contact with SEBI and only one participant out of 12 could articulate what a SE is and how to define it. However, awareness and understanding of the SE concept was significantly higher in a focus group in Montego Bay, where again none of the participants indicated being aware of SEBI. Yet the consensus among participants was that a SE is responsible for returning all profits back to the community through support of various community initiatives. They also noted that, in their view, the wider business and finance community is unaware of what SEs need to develop and grow— and remains largely uninterested in providing loans (and are even suspicious of them). The participants indicated that this suspicion likely stems from a lack of accountability and monitoring of organizations claiming to be SEs to ensure that profits are indeed returned to the community. In Savana-La-Mar and in Mandeville, the focus group participants were typically much more familiar with SEBI, having received some training and assistance from the project. However, at one of the two locations, they were unsure about the difference between a normal business and a SE and were confused about how to define a SE in such a way that does not contradict the need to make money to function profitably. At the other location, participants were able to articulate the SE concept but noted that they saw no real understanding of the concept at the local level by the public or private sector, largely due to the minimal awareness-raising efforts that had been done. Perception among other project stakeholders Two principal types of stakeholders were queried on the topic of awareness and understanding of SEs: 1) individuals who interacted with SEBI at the national government level; and 2) individuals and organizations that were involved with SEBI in some fashion. Discussions typically covered the perceived awareness and understanding of the public and of the government but also occasionally touched on specific subgroups of interest. On the topic of public awareness, nearly all stakeholders gave credit to SEBI for moving the needle on awareness and praised SEBI’s branding and visibility, although three noted that the magnitude and impact of SEBI’s role and the perception that SEBI introduced the SE concept to Jamaica were exaggerated. And while stakeholders were nearly all positive on what SEBI accomplished regarding public awareness of the SE concept, nearly all of them noted that much more needs to be done for this awareness to spread further and wider. Two noted a need to reach out to and focus on rural areas, which they noted had significant potential left mostly untapped. Furthermore, multiple stakeholders across both GOJ and project partners expressed concern that, while the term is becoming trendy in some circles and awareness has increased somewhat, there is much confusion and lack of understanding about the true meaning and structure that the 15 term is meant to carry. They noted that people have their own interpretations (e.g., make a profit and be tax-exempt, make money now and give back down the line) or that the public regularly confuses SEs with traditional charities or the corporate social responsibility programs of large corporations. Two recurring themes emerged to explain this situation. One was that the concept itself is somewhat counter-intuitive to most people and tends to be amalgamated with existing frameworks that are familiar to people. The other is that, while SEBI and others work together to craft a definition for what a SE is in the Jamaican context, that definition is incomplete and is lacking in specifics. Although a working group was established by MICAF to develop a definition for SEs in collaboration with key stakeholders, the MSME policy framework references two different definitions, neither of which indicates that one or the other is the ‘formal’ definition. Some of the thornier issues that also present a challenge from the policy perspective remain unaddressed, leaving room for people to fill the gaps in various ways with their own interpretations. On the other hand, awareness and understanding at the national government level were typically considered to be much improved. National government stakeholders acknowledged SEBI’s contribution in that regard and reported that SEBI met with public sector officials regularly, shared information, and provided fora for sensitization. Several non-government stakeholders also said that they saw a real change in awareness and priority within the government over the last three or so years. It was noted that most government departments and agencies have included social entrepreneurship in their strategic plan as a result. However, some gaps were also identified at specific key ministries and both non-government and government stakeholders acknowledged that some further awareness-raising work still needs to be done to reach a broader awareness across all levels of national government. The ET only obtained limited information regarding the level of awareness and understanding that exists at the local government level, therefore a clear picture cannot be provided. Perception and awareness among the general population The end-line population awareness survey sought to capture the perception and awareness of the SE concept among the general population across all parishes of Jamaica. In total, 700 respondents were interviewed (for more details on the design and selection process, please refer to the Methodology section). As previously noted, this survey was previously conducted in 2013 among 1,000 respondents picked in the same manner across Jamaica. The objective was to measure any change that may have taken place within the five years which SEBI was active. The population reached by each round of survey was broadly similar. Figure 1 below provides an overview of the key demographic characteristics of the respondents of each round. 16 Figure 1: Demographic Characteristics of the 2013 and 2018 Awareness Surveys 0% 10% 20% 30% 40% 50% 60% 70% Males Females 18-24 25-34 35-44 45-54 55-64 Lower & lower-middle… Middle class Middle-upper & upper… Urban Rural 2018 2013 The most notable, but nonetheless minor, differences are a somewhat larger proportion of respondents in rural areas (60 percent versus 55 percent in 2013); a smaller number of upper￾class individuals in 2018 (2 percent versus 5 percent in 2013); and fewer individuals falling within the 25-34 years old age group (19 percent versus 27 percent in 2013). Instead, there were slightly more respondents older than 45 (a 6 percent increase compared to 2013 for the 55-64 group and a 3 percent increase for the 45-54 age group). One of the first questions asked in the survey was for respondents to list what comes to mind when hearing the term “social enterprise.” In 2018, a significantly smaller number of respondents did not know or were unsure (13 percent vs. 29 percent in 2013) and more of them listed themes related to charities, nonprofits, and community development (27 percent vs. 12 percent in 2013) with a corresponding decline in themes related to business (23 percent vs. 32 percent in 2013). Respondents to the 2018 survey located in rural areas were particularly prone to list themes related to charities, nonprofits, and community development (32 percent vs. 18 percent in urban areas.) This information suggests that people might be more familiar with the term, but still could not pinpoint the nature of SE, seeing it more as a charity than a business with social goals, especially in rural areas. However, this is not strongly supported by answers to later questions. One of the questions provided the definition of SE that SEBI had promoted in Jamaica and asked respondents whether they had heard of this type of organization before. In 2013, 35 percent of respondents said they had heard of it, and this figure rose by only 3 percentage points in 2018, to 38 percent. There was no significant difference in response (over time or otherwise) between genders, but across both surveys older as well as wealthier respondents indicated greater knowledge of SEs. In 2018, slightly less than one third of respondents indicated having heard of at least one Jamaican SE when provided with a list of approximately 30. However, most of these respondents (80 percent) reacted to one organization, which is a well-known charity with several sites across 17 Jamaica. When this same subset of respondents was asked what type of work this organization does, they nearly universally listed direct assistance to various disadvantaged groups, and only a tiny minority noted activities related to their SEs. This reinforces the idea that SEs are associated (or perhaps even amalgamated) with charities in the mind of much of the public. That said, 42 percent of the respondents did know at least one other SE that is not so obviously tied to a charity or nonprofit. Regardless of how well respondents understood the nuances of the SE concept, most considered them as at least “fairly important” to the development of Jamaica. As shown in Figure 2, this was true both in 2013 and in 2018, as the percentage of respondents that provided a rating of at least “fairly important” remained constant over time (97 percent) despite changes to the distribution of the ratings between surveys. This also holds true across genders, age groups, socio-economic status, and whether the respondent lives in a rural or urban parish. The nearly universal reasoning provided by most of the respondents who considered SEs as at least “fairly important” systematically revolved around helping the community and making a difference in various ways (e.g., help the poor and marginalized, help children.) Figure 2: Perceived Importance of the Social Enterprises Listed 0% 10% 20% 30% 40% 50% 60% Not at all important Not really important Fairly important Very important Extremely important 2018 2013 Several questions focused on respondents’ attitudes toward supporting SEs, which provides a window into how they understand the workings of SEs. One such question hypothetically asked in what way(s) respondents are most likely to offer support and allowed several answer choices to be selected. The results are shown in Figure 3 below and outline again the strong connection between SE and charities. Most respondents selected volunteering and only 36 percent and 15 percent selected buying products and using services, respectively. More concerning, however, is the 13-point decline in the percentage of respondents who selected these two options, especially “buying their products,” between 2013 and 2018. The 2018 data also suggests that wealthier respondents were significantly more inclined to consider buying products (31 percent for lower class, 44 percent for middle class, and 53 percent for upper class) or using services (13 percent for lower class, 17 percent for middle class, and 35 percent for upper class). However, based on the lack of systematic differences between classes in other questions, this seems more likely to be related to disposable income than to a greater understanding of the SE concept. 18 Figure 3: Perceived Importance of the Social Enterprises Listed 0% 10% 20% 30% 40% 50% 60% 70% Volunteering my services Donating money Donating gifts Purchasing their products Using their services Would not support in the future 2018 2013 In late 2017, SEBI provided a subset of approximately 35 SEs with the right to use an official “Buy Social” trademark and logo, which was meant to assist prospective consumers in identifying products and service sold by SEs, thereby increasing demand. Among the SEs the ET spoke to, eight reported making use of it in their marketing material, labels, etc. and one was about to use it for the first time at a trade show. The consensus about the trademark (whether they use it already or not) among the SEs the team spoke to is that it remains relatively unknown (likely due to its recent roll-out) and that its meaning was possibly misunderstood by customers, but that it was nonetheless useful as a conversation starter and as an entry point to tell their respective stories to customers. The survey asked respondents how they felt about the idea of such a trademark, and their response was broadly positive as shown in Figure 4. Figure 4: Perceived Value of a Trademark for Goods/Services of SEs 0% 10% 20% 30% 40% 50% 60% Definitely not a good idea Not such a good idea A fair idea A very good idea An excellent idea 2018 2013 19 In both 2013 and 2018, more than 90 percent of respondents thought it was at least a fair idea. In 2018, this held broadly true across gender, age groups, socio-economic status, and location. However, their enthusiasm has declined over time. While 87 percent of respondents thought it was at least a very good idea in 2013, that figure dropped to 74 percent in 2018. In 2018, the most commonly provided reason to back up their reasoning was easy identification of the products/services (53 percent), good marketing (14 percent), and authenticating or legitimizing the product as SE-made (11 percent). This suggests that the trademark has the potential to be well accepted if it is further developed and promoted, but also highlights the need for a valid and trusted mechanism to authorize its use by SEs that meet certain standards. Finally, the survey asked respondents whether they had heard about SEBI. In 2013, a mere 3 percent reported that they had heard about SEBI, a figure that did not change in 2018. No notable difference emerged across gender, age groups, socio-economic status, or location in this regard. Most respondents who had heard of it did so through the media (e.g., radio, TV, newspaper), but there are unfortunately too few data points to draw meaningful conclusions regarding whether certain subgroups were more affected by a specific communication channel. Similarly, the level of interest in learning more about SEBI has stayed stable over time, and slightly over 75 percent of respondents indicated being at least interested in doing so. That said, 10 percent fewer respondents reported being very interested in 2018, with a corresponding increase in those reported being interested. Objective 4: Transition to Social Enterprise Jamaica One of the more revealing findings from this evaluation concerns the lack of knowledge by any of the parties interviewed, including the MICAF who at the time of the interview were awaiting feedback from SEBI on the status of SEJA, of what was being proposed for SEJA or what it would mean for potential beneficiaries and for the SE sector in Jamaica in general. While MICAF and PIOJ have led discussions via a Policy Working Group on MSMEs in Jamaica, including an Action List with options for SEJA, the ET found no evidence that GOJ knows what a proposed institutional framework for SEJA would encompass. In short, it was more than evident from feedback from KIIs and FGDs that there was no broad￾based awareness of SEJA among either direct or indirect beneficiaries or from stakeholders interviewed. Furthermore, none realized that SEBI was coming to an end in December 2018, which would account for the fact that none had knowledge of any transition of SEBI to a follow￾on entity offering similar, and mainly capacity building, services to the SE sector in Jamaica. When direct beneficiaries (incubators and accelerators) and indirect beneficiaries were asked to comment on what would be needed from any follow-on entity, all suggested that priority must be given to the provision of technical and financial services to SEs to overcome constraints that SEBI was unable to address, namely, to become productive, profitable, and viable business enterprises. When further asked whether they would be willing and/or able to pay for services in the future or similar services to those offered by SEBI, 10 of the 26 enterprises said they would, but only if they believed they had sufficient funds to afford it at the time and if services offered were tailored to the specific needs of their business, leading to tangible results or alleviation of a particular issue or problem. The remaining 16 enterprises said that they would not pay as they would not be able to afford it, felt that they had no need for external support, or suggested that they would search for other projects that offered their services for free (Branson Center was 20 suggested as an example, although it has gradually transitioned to offering paid services and has changed its strategy to focus on higher revenue earning organizations with sales over J$25 million. 4.1.2a Conclusions Objective 1: Develop Solution-Based Social Enterprises Feedback from SEBI’s beneficiaries on the impact of the capacity building activities on them as individuals and enterprises has been invaluable and has thus largely contributed to achieving its mandate in developing the capacity of SEs to become Solution-Based Social Enterprises. However, based on the current challenges facing the majority of SEs relating to sustainability, growth, and expansion, if the SEBI program implementation process was more streamlined and the same beneficiaries from the incubator program were transitioned to the accelerator program during the life of the project, the gains would have been more impactful as more time and resources could have been used more effectively to understand the specific needs of the SEs. Given that more than 95 percent of the SEs were already operational upon joining the SEBI program, more time and resources could have been spent on the accelerator component of the program. The findings also indicate that issues related to efficiency and viability of the SEs under the project are a concern. The ET found that most beneficiary SEs were highly dependent on grant funding for the growth and development of their enterprises. This included both those registered as not￾for-profit and for-profit. Information gathered from direct interviews suggests that only a small minority of beneficiaries (three of 29) said they had any interest in accessing loans or any type of financing to scale and grow their business. In fact, most said that they were looking at other projects to access resources to scale up. The evaluation findings allowed the ET to conclude that the grant funding7 accessed by the project beneficiaries kept a significant number of them in business. This highlights the dire need for innovative financing solutions to support the social sector and SEs particularly. Based on the statements made by SEs during the interviews, the perception is that they would be able to increase revenues and employment, and thus impact, if they had critical resources for growth and expansion. However, there is still a major challenge related to the risk appetite of the SEs to access loans and other financing solutions from financial institutions. The absence of a clear road map to guide the sequencing of activities had a significant impact on the timeline in which to identify, select, and develop solutions and impact the right SEs in a way that they would become solution-based and profitable SEs. Based on the number of modifications and time lost in trying to get up to speed with the project’s implementation activities, it is evident that there were variations between what was articulated in the project proposal and the capacity and resources available to implement the activities under the SEBI project. Based on the SEs that were selected under the first phase of the project, the evidence suggests that more effort was placed on selecting beneficiaries to meet project targets than establishing a process that could select and identify the right SEs (with regards to fit) to meet SEBI’s mandate. In hindsight and after various studies, scoping reports, market landscaping, capacity, and knowledge assessments were done, SEBI took a more informed approach towards the final phase of the project in selecting more suitable SEs, mainly through a call for proposals, which was highly 7 . In the Cooperative Agreement under ‘Project Resourcing’ (page 22) provision is made, “that SEBI will ensure that financial products are developed to support the capital and business growth of social enterprises and it is envisioned that for the 10 SEs that fall under Objective 1, that resourcing of $US10,000 can be found for 5 and $US20,000 for the other 5 to support their growth by the end of the project”. 21 subscribed. However, the ET acknowledges that both Phase 1 and 2 of the project issued a national call for proposals from which SEs were selected, although one of the issues related to Phase 1 (2013) was that the concept of an SE was new to most people and that the entities that applied and were selected were not practicing SEs. Following promotion of the concept, by Phase 2 (2016), applicants could identify a business venture to go along with their social mission. Only 7 of the 23 direct beneficiaries interviewed indicated an increase in operational efficiency of some shape or form due to the interventions. The remainder indicated that it was too early to say. However, the ET noted that several SEs received grants and formed new business partnerships with SEBI’s direct support (Deaf Can, Superior Craft, Inzpire, Educatours, among others). However, nearly all grants came from Digicel and those SEs that received them only attributed credit to SEBI in that they might have shared the call for proposals and in a few rare cases, grant application training was also credited. Objective 2: Creating an Enabling Environment Evidence from SEBI’s records and from media coverage of the project’s SE-related activities indicates that it has partially been successful in achieving its mandate in fostering a more business￾friendly environment for SEs and promoting the concept of social entrepreneurship in Jamaica. However, it is also evident that by concentrating its efforts in Kingston and St. Andrew, SEBI has, by default, not had the desired meaningful impact in the more rural parishes which were also target areas for the project under this objective. This has led to a general perception among indirect beneficiaries, stakeholders, and several direct project beneficiaries (incubators and accelerators) that SEBI’s activities are essentially directed at SEs in the Kingston and St. Andrew region to the exclusion of local community who feel that little has been done to reach them through the project, nor have their circumstances been acknowledged by the government at the national level. The ET acknowledges, however, that SEBI did make an effort to reach rural SEs, the result of which was that, unfortunately, few were able to meet the eligibility criteria to join the project and become direct beneficiaries. This has resulted in few meaningful local initiatives being implemented in the SE sector as the institutional structures involving, for example, community-based organization (CBOs), CDCs, Development Area Committees (DACs), Parish Development Committees (PDCs), the Social Development Commission (SDC), and Municipal Corporations, are not sufficiently well developed to plan, organize, and oversee SE initiatives. A case in point was highlighted in a comment from participants in the Montego Bay FGD that SEs or any small business had “no support whatsoever from the SDC in providing advice on access to funding or how to manage and run a social enterprise.” In the Bunkers Hill FGD, only one participant had heard of a SE. The ET acknowledges and accepts that this lack of local institutional infrastructure to support social sector initiatives was not part of SEBI’s mandate. During the evaluation session the participants recognized that, in the absence of a CBO to represent their interests, little progress would be made in developing the local economy. In the presence of the ET, a CBO was formed there and then and was subsequently acknowledged by the CDC in Falmouth, Trelawny. An opportunity therefore appears to have been missed by SEBI to reach out to rural parishes and particularly to understand local needs and implement initiatives aimed at alleviating some of the more pressing institutional anomalies identified by the evaluation field trip, which have to a large degree hindered progress in SE development. 22 Objective 3: Raise Awareness of Social Entrepreneurship across Jamaica Feedback from SEBI beneficiaries and project stakeholders on the level of awareness and understanding of the SE concept suggests some limited improvement but much room for improvement. While certain subgroups (e.g., the national government, the social sector, entrepreneurs) may have seen somewhat more important improvements, the surveys of Jamaica’s general population in 2013 and 2018 do not indicate widespread and significant change in awareness or understanding, show no change in awareness about SEBI, and point to some confusion and amalgamation between SEs and more commonly known types of organizations. Concerns were also raised regarding the level of awareness and understanding that exist in rural areas, which was generally confirmed during FGDs. Thus, SEBI only partly met this objective. From reviewing SEBI’s progress reports and interviews, SEBI spent a significant amount of effort and resources on marketing and promoting the SE concept in Jamaica. To do this, SEBI used channels such as the traditional media (e.g., TV, radio, newspapers), social media, and events targeting national government, SEs, and various actors that are part of the entrepreneurship and social development sector. Most project beneficiaries and stakeholders praised these efforts as a positive and necessary step towards creating broader awareness and understanding of the SE concept in Jamaica. Project beneficiaries and stakeholders generally felt that public awareness had at least somewhat improved during the period SEBI was active (especially during the second part of the project) but that it did so from a very low level and in an uneven fashion. Most individuals outside of the Kingston area noted very little change in terms of awareness and understanding in their communities (and in some cases were themselves unable to articulate the concept). On the other hand, specific subgroups more heavily targeted by SEBI and more likely to have been reached by its marketing and awareness campaigns were often cited as being better informed. These subgroups include those involved in the social sector (e.g., charities, nonprofits, benevolent societies, etc.), professionals and white-collar workers, and entrepreneurs. Both government and non-government stakeholders also broadly acknowledged that the level of awareness and understanding of the SE concept among national government ministries and agencies has improved significantly in recent years, in part due to SEBI’s efforts. But those same stakeholders noted that gaps remain across these government organizations and within them. Furthermore, the broad consensus among project beneficiaries and stakeholders was that awareness and understanding remains low among the general population and that much more will be needed to reach a point where there is broad awareness and understanding of the SE concept. This is confirmed by survey data, which points to a very small improvement between 2013 and 2018 in the percentage of general population respondents who reported having heard of SEs after being provided with a definition, and no change in awareness about SEBI during that same period. Some SEs and stakeholders also noted still running into organizations that operate in a form close to or even matching a SE but being unaware of the concept, suggesting that even among those most important to target there remains a gap in awareness and understanding. Another important concern raised by both SEs themselves and by various government and non￾government stakeholders was that, while the term “social enterprise” may be slightly better known among the general population and has even become trendy in some circles, the level of understanding of the concept itself remains lackluster. This was partly blamed on the somewhat counterintuitive nature of the concept and its proximity to well-known types of organizations, 23 such as charities, which may have been compounded by the fact that many of the organizations SEBI worked with were charities which in fact demonstrated no interest in becoming profitable business enterprises. But it was also attributed to the lack of a clear, concise, and understandable definition of the concept that left room for interpretation (e.g., make profits now and give back later, being able to make a profit while being tax exempt). Indeed, the survey data also supports this, as respondents often seem to merge or amalgamate SEs with charities. For example, they associated the term much more with charities and community development and much less with business in 2018 than in 2013 and would support SE primarily through volunteering and donations rather than through the purchase of their products and services. This last element raises concerns because demand based on a quality product or service that is sold to be a SE needs to become stronger in order for the existing and emerging SEs to grow and remain in business. Objective 4: Transition to Social Enterprise Jamaica SEBI has not been successful in disseminating information to a wide audience in Jamaica regarding SEJA or what it proposes as a follow-on entity to SEBI’s activities once the project ends in December 2018. Furthermore, SEJA had not yet been formally established as a legal entity at the time of writing and the ET were unaware of when this would be accomplished. However, there is compelling evidence to suggest that SEJA may not necessarily be the most appropriate or relevant vehicle to advance the cause of SEs in Jamaica. Instead, an alternative approach may be necessary with a focus more on practical, technical assistance being made available to client enterprises, based on demand and on their individual specific needs and requirements. 4.1.3a Recommendations Objective 1: Develop Solution-based Social Enterprises To achieve the highest impact on the SE sector and ensure that SEs are viewed as viable partners in the business development space, access to capacity building activities such as those implemented under the SEBI program must be in place. While there may be a challenge with SEs making time available to attend some activities, the availability of information, communication, and technology (ICT) tools such as online learning and community fora could go a long way in addressing some of these needs. For example, access to regional workshop spaces could be a preferable option, thus alleviating the need to travel to Kingston and St. Andrew. Furthermore, the accelerator batch was quite enthused with the reading lists that were shared by SEBI, which could be handled remotely and isn’t high-tech. All the beneficiaries of the project reported positive feedback on the high quality of the presentations and interactions with the SEBI team, which demonstrates that there are some things that SEBI has the capacity to do well. For any further interventions that JNF should undertake, their focus should be on these areas. Given the significant impact the grant funding component of the project had on the direct beneficiaries in helping them to improve their operations in the medium to short term, access to financing will be critical to the sustainability of the SEs. To develop solution-based SEs using an incubator and accelerator model requires that careful analysis of the risk appetite of the targeted SEs is assessed at the needs/gap analysis of the program. This will determine if the entrepreneurs that have been identified with scalable business models have the mindset to accept the various financing options that could be made available to them. This would also allow for a more 24 comprehensive assessment of the needs of the SEs as it relates to building organizational capacity to operate a profitable enterprise. For future interventions in the SE sector, a clear distinction must be made in the development of components for interventions, between charity/not-for-profit and for-profit SEs. The various types of organizations will have needs and capacities and thus require some interventions which may not be appropriate for all types. This should result in the two main types of entities (not￾for-profit and for-profit) being placed on separate growth paths to achieve their sustainability goals. The proposition is that the not-for-profit would seek to expand its impact through the establishment of a for-profit entity that would plow most of its income into achieving its mission. The for-profit SEs would be placed on a path to increase its profits and increase impact, with clear mechanisms to ensure that the business model can attract the kind of financing and support in a typical market environment. For future interventions supporting SEs and MSMEs, there is a great opportunity for JNF to leverage Jamaica National (JN) group resources to demonstrate to all stakeholders how collaboration and the sharing of resources can be used to solve challenges in the context of limited resources. The CA included several ambitious activities to be undertaken under the various components of SEBI which, if they were done, would have significantly improved SEBI’s impact and effectiveness in creating solution-based SEs. Namely, on page 22 of the CA, under the heading “Project Resourcing,” it stated that “the relationship with JN Group holds the key” and that “SEBI will ensure that financial products are developed to support the capital and business growth of social enterprises” through partnerships with organizations such as JN Group. These include: access to JN Small Business Loans, a micro-finance agency set up to provide access to credit for the under-banked; JNBS, the business development unit within JN mandated to support the development of business opportunities for the Group and equipped with business development specialists; and I-Support Jamaica, a project of the JN Group, which is a conduit by which people anywhere in the world can loan or grant funds directly to micro-enterprises in Jamaica. Objective 2: Creating an Enabling Environment To build on progress made to date, the ET recommends that future similar projects incorporate SEBI’s enabling environment objectives to ensure that social entrepreneurship remains on the radar in Jamaica. However, this must be tempered by the need for a nationwide approach which is all inclusive and focused on defining exactly what a SE is, both in principle as well as in its legislative ramifications. Without a formally recognized, clearly understood definition for a SE in Jamaica, progress in spreading the word is destined to be limited, as the existing confusion and contradictions as to what should be priority for a SE demonstrate. SEs in Jamaica struggle over whether they should first become profitable to retain funds in the business and local community for a just cause or need or whether they should operate in a socially responsible manner where profit-making is a secondary priority in preference to conducting business as a non-profit enterprise for charitable purposes. Whatever the scenario, a business of any description must survive within a commercial context. Any existing or future project including SEs should have clear objectives as to its formal and legal structure, including its modus operandi and whether, in the long run, it can prosper and grow while contributing to the development and well-being of the community in which it is based and operates. 25 Objective 3: Raise Awareness of Social Entrepreneurship across Jamaica To reach a larger segment of the population, future interventions focused on the SE sector will need to continue and scale up efforts to raise awareness and understanding of SEs. In doing so, they should seek to place significantly more emphasis on awareness-raising campaigns in rural areas. These areas are both in great need of strong SEs to develop the community and provide avenues for employment and have much to offer in terms of potential products and services to be offered. This must also be done using methods of awareness-building that are better adapted to the context and habits of those living there, such as booths at community events and religious festivities, presentations to local government officials, and interactive sensitization events (e.g., theater, showcase of local SEs) in public spaces. In addition, more emphasis needs to be placed on communicating more clearly what a SE really is, the diversity of forms they may take (stemming from a charity, founded as a business but meeting specific criteria, etc.), and yet how they nonetheless differ from traditional charities and businesses. Having a clear and concise definition that covers the diverse legitimate SEs in existence in Jamaica, while differentiating them clearly from more commonly known organizational forms, will be important in that process. At the same time, a clear definition will also facilitate progress on the policy front and limit confusion among those interested in becoming a SE or who are already one but were not aware of it. There is also the need for a greater role by the GOJ, which has made SEs one of its priorities. Particularly, information needs to be effectively conveyed to the local level in an actionable way to promote both awareness and understanding, but also to support and develop the sector. This may be done as part of training of local government staff, via systematic dissemination of approachable documentation, and via GOJ’s own set of promotional campaigns. Local government officials also need to be given a clear mandate to promote the SE sector, which would likely help to systematize and scale their efforts. Finally, many stakeholders have pointed to schools as a tremendous opportunity to promote awareness of SEs, noting that youth are a powerful agent of change in the community. Some steps were already taken in that direction, such as the recent revisions to the school curriculum to add or enhance an entrepreneurship component for certain grades. Junior Achievement Jamaica (JAJ) is also in the process of scaling up their hands-on and interactive activities promoting entrepreneurship (with support from USAID.)8 This represents an opportunity to work with or through JAJ to promote and educate youth about SEs. Indeed, some of JAJ’s programs (e.g., in Armenia) include a component that encourages students to invest the proceeds of their businesses into community projects. This creates a valuable opening for discussions on SEs and could be combined with an emphasis on products and services that also have social goals. In complement or independent of this, SEs should also be systematically encouraged to regularly present what they do and what they are at their local schools. Again, this is highly compatible with the current direction in which the curriculum is moving and aligned with JAJ’s program. Objective 4: Transition to Social Enterprise Jamaica Before any such entity is contemplated, all stakeholders must be consulted, including at the local level in the parishes, to ensure that such a body is viable and sustainable. Serious consideration must be given as to whether such an entity, if as suggested is an NGO, can have the necessary 8 See, for example, https://jis.gov.jm/entrepreneurship-classes-in-all-high-schools-september/ 26 skills, experience, and expertise to offer SEs the services they need to help them improve efficiency and progress towards becoming profit-making businesses. Whatever the proposed structure of such a body—whether it is an NGO or private sector managed and run—it should essentially be demand driven where a need has been identified and services can be offered to meet those demands, with or without a fee-paying element. Suggestions from the evaluation include a public-private partnership format, a private company with a fee￾paying element for members, or an NGO-type structure (the least preferred option). 4.1B EVALUATION QUESTION 1B: WHAT FACTORS HAVE CONTRIBUTED TO THE SUCCESS/FAILURE OF THE DIFFERENT PROJECT COMPONENTS? 4.1.1b Findings This is a story of a project which, based on the geographic location of the respondents, produced two separate and distinct findings during the ET’s research and investigations. On the one hand, while SEs were not new to Jamaica prior to SEBI’s involvement, there was little understanding of what they really were, what they stood for, or what they were meant to achieve relative to a typical MSME. Recognizing this, SEBI set out several objectives aimed at increasing the public’s awareness of the SE sector and social entrepreneurship having in mind SEBI’s over-riding goal, “to show that social enterprises can be profitable businesses while providing solutions to social, environmental and economic problems.” Objective 2 (Creating an Enabling Environment) and Objective 3 (Raise Awareness of Social Entrepreneurship across Jamaica) were both successful in degrees as highlighted above. This was due mainly to the consistency in initiatives over a six-year period of media events, workshops, seminars, networking events, and study tours. Feedback from interviews across the board acknowledged the significance and relevance of these initiatives as well as the importance of gaining government acceptance of the need to have the sector recognized eventually in legislation. This was highlighted in the “Updated 2017 JAMAICA – MSME & Entrepreneurship Policy” prepared by MICAF. On the other hand, there was less enthusiasm in the more rural parishes. In these areas, communication from SEBI on the SE sector and what it should mean for local communities was found wanting in its delivery (if it existed at all) given that the intention was for the project to have a meaningful outreach strategy beyond a limited area. The major factor inhibiting a more successful outcome was SEBI’s concentration of effort in and around Kingston and St. Andrew. In these areas, supporting incubator and subsequent accelerator beneficiaries assumed greater importance and occupied a significant proportion of SEBI’s time, over supporting activities in the more rural parishes. Furthermore, creating awareness of SEs in the parishes was perceived as less relevant to struggling businesses, which were often family owned, where simply surviving was the main priority. However, with most direct beneficiaries based in Kingston and St. Andrew along with the remainder based in the other parishes, the coverage, in fact, was broadly in line with SEBI’s Objective 1 (Develop Solution-based Social Enterprises). SEBI also focused its activities for incubators and accelerators in St. Catherine, Trelawny, Westmoreland, Manchester, Clarendon, and St. James. Nevertheless, the fallout from this approach manifested itself in the rural parishes where FGD indirect beneficiaries, particularly, felt unserved by the project (whether they had heard of its existence or not). To an extent, this also had the effect of confirming locally held perceptions 27 that Kingston and St. Andrew would always have priority over other areas no matter the initiative or field of endeavour. To compound matters, the evaluation found that 16 of the 26 SEs interviewed were in urgent need of financing, ranging from working capital or cash flow needs, to funding for expanding or purchasing machinery or equipment. More worryingly, four of the enterprises were in imminent danger of going out of business, while three were not actually trading at all (two were involved as incubators and one was an indirect beneficiary who had participated in some of SEBI’s workshops). 4.1.2b Conclusions In retrospect, it is arguable that SEBI’s overall performance was not as effective as it might have been over its six-year time span. This is largely due to its predominant concentration of effort in Kingston and St. Andrew as well as the surrounding parishes, where Objectives 1, 2, and 3 could not be said to have been part of a nationwide mission to extol the virtues of SEs and what they could do for owners or local communities. This opportunity was clearly missed and could be considered a major fault-line in the project’s initial design. Creating and/or supporting the concept of a SE sector and social entrepreneurship with SEBI at the helm was further constrained by the fact that no universal definition of what an SE is exists in Jamaica. As a result, project activities and actions did little to ensure that SEs did become profitable businesses as specified in the project’s SOW. Without follow-up to verify the status of a SE (whether an incubator or accelerator) regarding its financial conditions, how would success be measured in achieving this aim? Given that 62 percent of the SEs were experiencing financial issues or difficulties, four said they would not be able to survive in the short term without additional funding or an increase in sales, and three others were not in fact operating at the time of the evaluation site visit, this must bring into question the project’s underlying priorities. It also raises the question of whether sufficient attention has been paid to the how SEs could be supported and/or if the status of an SE is the right structure for them in the first place. 4.1.3b Recommendations A needs/gap analysis on the SE sector in Jamaica prior to the design and preparation of the SEBI project components to identify the greatest needs of the SEs and to align the project activities with project objectives should have been done. Going forward, care must be taken to ensure that all project inputs and expected outputs are realistic and achievable with the context of the available resources. This resulted in SEs being selected after the fact, i.e., interventions may not have been relevant or appropriate.It is therefore strongly suggested that any future intervention(s) regarding the SE sector must be demand driven based on a needs/gap analysis of prevailing circumstances pertaining to SEs in Jamaica, and what actions would or could contribute to the development of the sector. While capacity building measures have, to an extent, helped SEBI’s incubator and accelerator beneficiaries comprehend the need to adopt certain business-related practices to become profitable, the clear lack of any planned follow-up activity to determine the extent to which those beneficiaries have, in fact, become profitable remains a serious weakness in SEBI’s operating structure. Any future projects must, therefore, be designed with clear targets and indicators for SEs, which themselves must be clearly defined. This is true not only on mission, purpose, and 28 intent, but also on their legal status and how it impacts their ability to function as profitable enterprises. 4.1C EVALUATION QUESTION 1C: DID THE ACTIVITY HAVE SIMILAR EFFECTS ON MEN AND WOMEN? 4.1.1c Findings There is consensus among all groups interviewed that there are no perceived or established barriers that would limit the full participation of men and women engaging in entrepreneurial activities in Jamaica. In fact, feedback from the direct beneficiaries, KIIs, and FGDs indicated that there was an adequate representation of men, women, and youth among SEBI’s beneficiaries. Based on the composition of the group of SEs, the owners are made up of an evenly distributed number of men as sole owners, men and women as co-owners, and women as sole owners. There is no evidence to suggest that implementation of the project activities had different effects on men and women. The responses about the impact of SEBI’s activities on them as individuals and on their SEs yielded similar responses from both men and women. On the question of the quality of service delivery of the capacity building activities, the feedback was largely uniform among the groups, indicating a high level of satisfaction across the board. Most of the beneficiaries commented on the positive impact the various capacity building activities had on making them more aware, self-confident, and organized as well as increasing their exposure and preparing their mindset to operate a successful business. As it relates to sustainability issues of efficiency and viability, the realities were the same for men and women. All groups expressed similar concerns related to the growth and expansion of their SEs, suggesting that—whatever shortcomings they might have had in this area—it was having the same impact on them regardless of gender considerations. Given the time limitations of the evaluation, the ET could not determine whether the beneficiary SEs under the SEBI program were required to incorporate gender integration as part of the conceptual model for their SEs. Consequently, the team could not assess whether gender consideration was integrated into their business operations, employment practices, and customer base as well as through the types of goods and services offered. The impact could also be greater if SEBI communicated a similar message to citizens in the rural parishes who are involved, are operating, or have the potential to establish SEs to include these gender integration considerations in their planning. 4.1.2c Conclusions SEBI fulfilled its mandate to include organizations/SEs owned or operated by men, women, and youth. Based on the ET’s findings, attempts were made to ensure that no group was adversely affected or could claim to be marginalized by the project activities. SEs reported that one of the strengths of SEBI were the networking activities. These were a highlight of the program in which SEs were able to support each other, visit each other’s locations, and purchase goods and services from each other. This would suggest a high level of engagement among the selected SEs regardless of whether they were men or women. 4.1.3c Recommendations A similar approach to inclusiveness should be incorporated in any future program supporting social entrepreneurship in Jamaica. The most outstanding features of the SEBI intervention, which 29 facilitates the sharing of expertise, knowledge, know how, mentoring, relationship building, networking, and exposure among gender groups, should be incorporated as a distinct component of future similar interventions. It was also evident from the feedback that the SEBI team got it “right” as it relates to the quality of the programs for both males and females. SEBI should share the institutional knowledge garnered as a product of the project and ensure that the practices are communicated to various stakeholders. For future SE interventions, the concept of gender integration in SEs should be a main feature of the SE business model. Recognizing that SEs are presenting a value proposition that is unique in a very competitive market environment, future interventions must be able to demonstrate how the SE model highlights how gender integration affects the production of their goods and services, thus impacting business outcomes for the SEs and social outcomes for the customers. 4.2 EVALUATION QUESTION 2: TO WHAT EXTENT HAS SEBI CONTRIBUTED TO THE LOCAL SOCIAL ENTERPRISE POLICY ENVIRONMENT? 4.2.1 Findings This question has already been partially addressed in Section 4.1.1a Objective 2 above under the enabling environment discussion. As previously discussed and shown in Table 4, both SEBI’s direct beneficiaries and FGD participants agreed that 1) local communities receive no meaningful assistance from local authorities and 2) local authorities are not involved in any discussions with them on how best to develop their communities, including SEs. As also previously stated, SEBI’s SE awareness activities eventually led to the GOJ being persuaded to include the SE sector in its most recent draft MSME policy paper prior to legislation. This legislation would clearly be a major achievement for the project if it ultimately becomes law. When that happens, it will set the stage for SEs to operate within a legal framework. This will, without doubt, set many minds at rest across the social, academic, business, and political spectrum, in terms of the respective roles all concerned parties will be expected to play in ensuring that SEs and social entrepreneurship remain at the forefront of economic development in Jamaica. However, until such legislation comes into force, many of the issues raised in this evaluation regarding constraints to growth for SEs will continue. In addition, these issues may potentially go unaddressed once SEBI comes to an end in December 2018. Although SEJA has been advanced as a natural successor to SEBI, with LPD as an influencing medium, this potential role remains to be clarified, including the extent to which such an entity has the funds, expertise, experience, and tools to continue with policy advancing initiatives (if so envisaged within its mandate). Although beyond the scope of both SEBI and this evaluation, the extent of the GOJ’s resolve to move towards passing SE legislation also remains to be seen. SEBI also advocated for adapted financing options for SEs, primarily through the Development Bank of Jamaica (DBJ). Progress in that area was hampered by limited nationwide research available on the topic. To proceed on solid grounds, the DBJ and SEBI decided to perform their own nationwide research from late 2016 through early 2017 using the definition of SE introduced in the draft MSME policy. The research found that there is some interest in loans among SEs. Based on this finding, the DBJ started to develop a business case for financial products better adapted to SEs. However, as of the writing of this report, the nature of these products remains undetermined and no commitment has been made to introduce new products onto the market. 30 The DBJ has, however, now included SEs in all its calls for proposals for MSMEs. Nevertheless, in the absence of a regulatory framework, definition of SEs, registration status / legal personality, or clarity regarding tax implications, these enterprises would for all intents and purposes essentially still be classified as ‘regular’ MSMEs, not-for-profit or charities. Table 4: Summary of Findings from FGDs and Direct Beneficiaries – Incubators and Accelerators – Related to Evaluation Question 2 Sources Location Evaluation Q2: To what extent has SEBI contributed to the local SE policy environment? FGD/SE Q1: Local authorities support SEs with policy or action? FGD/SE Q2: Local communities involved in SE policy discussions with local authorities? Yes No Not Sure Yes No Not Sure FGD Bunkers Hill, Trelawny - 10 2 - 12 - Incubator and Accelerator 1 - - - 1 - FGD Montego Bay, St. James - 6 - - 6 - Incubator and Accelerator - - 2 - 2 - FGD Savanah-la-Mar, Westmoreland - 5 1 - 6 - Incubator and Accelerator - 1 1 - 2 - FGD Mandeville, Manchester - 5 - - 4 1 Incubator and Accelerator - 2 1 - 2 1 FGD Kingston and St. Andrew - 2 - - 2 - Incubator and Accelerator - 10 7 - 10 7 TOTAL 1 41 14 0 47 9 Several SEBI stakeholders from the national government and from other sectors (e.g., academia, business support organizations) expressed concern that the definition of SE that was introduced in the draft MSME framework lacks specificity and enforceability. Indeed, the language in the MSME framework relies primarily on the presence of a stated social mission of solving social, economic, cultural, or environmental problems in the community or nationwide and the expectation of ethical business practices to differentiate them from other “traditional” MSMEs. While other definitions have emerged from both private and public sectors from various corners that include specific criteria such as a 51 percent threshold of profit reinvestment into the 31 community, even these are not immune to abuse (e.g., paying C-level9 staff high salaries and leaving little to reinvest, which may nonetheless meet the 51 percent threshold.) Table 5: Summary of Findings from Direct KIIs (Stakeholders) Related to Evaluation Question 2 KIIs Evaluation Q2: To what extent has SEBI contributed to local SE policy environment? KII Q1: Local authorities support SEs with policy or support? KII Q2: Local communities involved in SE policy discussions with local authorities? Group Yes No Not Sure Yes No Not Sure Government 1 3 2 2 4 - Local Authorities - 1 - - 1 - Women/Youth Associations - - 1 - - 1 NGOs - - 3 - - 3 International Donors/ Organizations - - 1 - - 1 Education - 1 1 - 1 1 TOTAL 1 5 8 2 6 6 This situation was cited as hindering policy progress when it comes to any change that would give SEs an advantage over “traditional” MSMEs, such as an exemption of the value-added tax or preferential lending terms. These types of advantages must be granted based on observable, objective, and auditable criteria to avoid creating an incentive for many MSMEs to simply claim they are also an SE. SEBI was aware of these concerns and set up a steering committee to develop a body for accountability; however, little progress was made, and the steering committee was eventually disbanded. Other avenues pursued by SEBI somewhat more successfully have included its involvement in the National Policy Working Group on Social Enterprises, where it played a role in crafting a code of ethics for SEs, as well as collaborating with Digicel Foundation to develop and test a toolkit to measure social return on investment. To complicate matters further, there already exists an uneven playing field among SEs, based on whether they are operated by actors of the social sector (i.e., charities, nonprofits, benevolent societies, etc.) or by actors in the private sector (LLCs, sole proprietorships, etc.). Indeed, it is common for actors of the social sector to structure their SE as another name of the organization (“doing business as”), meaning these SEs are already receiving the tax benefits of the duly registered social sector organization they belong to. That said, policy incentives may not be the best approach to promoting SEs and it might instead be wiser to focus on the value proposition of their products and services, embedded in which is 9 C-level also called the C-suite is used to describe high-level executive titles within an organization. 32 the social value these organizations deliver to their communities. As noted by Deloitte in its 2018 Human Capital Trends: “Engagement with other stakeholders on topics such as diversity, gender pay equity, income inequality, immigration, and climate change can lift financial performance and brand value, while failure to engage can destroy reputation and alienate key audiences.”10 That alone may be sufficient to encourage sales and create an economic incentive for new SEs to emerge and, regardless of meeting some definition or even upholding one, for MSMEs and large businesses to place greater emphasis on social impact and corporate citizenship. 4.2.2 Conclusions Influencing government policy was always going to be a major challenge for SEBI. Yet the project was able to ultimately ensure, through its various awareness-raising activities over a prolonged period, that SEs would be included in the government’s draft MSME policy paper. However, given that SEBI will end in December 2018, no further action is anticipated on this front. The ET understands that LPD has been engaged with MICAF and PIOJ on the development of the government’s draft policy and has held several discussions and workshops with other GOJ ministries, departments, and agencies (MDAs) over a recent 6-month period. However, at the time of writing there is still nothing in place and the ET has no knowledge as to when this might happen. 4.2.3 Recommendations There is nothing that SEBI can do now for the remainder of the project other than to offer its advice and progress to date to LPD (if this project is to pick up the baton on its own account or in collaboration with the proposed SEJA). However, if LPD can help pursue a clearer definition of the SE concept (generally and legally) this will be an important precursor to the GOJ’s policy work. This should be conducted in collaboration with GOJ stakeholders MICAF and DBJ with a focus on tax exemption issues and redistribution of profits to communities, among others. 4.3 EVALUATION QUESTION 3: BASED ON THE EVALUATION FINDINGS, WHAT ARE THE BEST PRACTICES AND LESSONS LEARNED FOR OTHER PARTNERS IMPLEMENTING SIMILAR ACTIVITIES SUCH AS LPD AND SOCIAL ENTERPRISE JAMAICA (SEJA)? 4.3.1 Findings Collaboration and sharing of information about SEs among direct and indirect beneficiaries (incubators and accelerators), private sector partners, government agencies, CBOs, local authorities, and universities, among others, facilitated a more in-depth understanding and awareness of social entrepreneurship in Jamaica through SEBI’s various campaigning and media activities. The LPD project is a five-year USAID-funded capacity-strengthening initiative implemented by FHI360, launched in May 2017. LPD is tasked with providing extensive capacity building services for Jamaica’s civil society sector, in partnership with key GOJ partners. The intervention is expected to improve the sustainability of these organisations, which will also include SEs, NGOs, intermediate service organizations (ISOs), and CBOs. It will also enhance their ability to advocate for improvements in their enabling environments, and support community-based crime and violence-prevention initiatives. LPD has four key components: 1) strengthening the capacity of local entities (CSOs and ISOs); 2) supporting SEs through a business 10 Access the report at https://hctrendsapp.deloitte.com/reports/2018/the-rise-of-the-social-enterprise.html 33 incubator model; 3) improving the enabling environment for CSOs and SEs; and 4) promoting collaboration among CSOs, the GOJ, and the private sector. LPD built on earlier programs that were supported by USAID/Jamaica Office of Citizen Security and it incorporates the insights of the stakeholders in the civil society and citizen security sectors. Table 6: Summary of Findings from FGDs and Direct Beneficiaries – Incubators and Accelerators – Related to Evaluation Question 3 Source Location Evaluation Q3: Best practices and lessons learned for other implementing partners and SE’s in Jamaica? FGD/SE Q1: Have you received practical advice or grants from anyone else to help your business become more efficient? FGD/SE Q2: Do you know about SEJA and /or that it is planned to follow on from SEBI re: SE business development? Yes No Not Sure Yes No Not Sure FGD Bunkers Hill, Trelawny - 12 - - 12 - Incubator and Accelerator - 1 - - 1 - FGD Montego Bay, St. James 5 1 - - 6 - Incubator and Accelerator 2 - - - 2 - FGD Savanah-la-Mar, Westmoreland 2 4 - - 6 - Incubator and Accelerator 2 - - - 2 - FGD Mandeville, Manchester 1 4 - 1 4 - Incubator and Accelerator 2 1 - 2 1 - FGD Kingston and St. Andrew - 2 - - 2 - Incubator and Accelerator 14 3 - - 17 - TOTAL 28 28 0 3 53 0 LPD offers grants to clients that have been in the incubator for six months and have prepared an enterprise development plan. LPD draws a distinction between its work and that of SEBI, namely, that it selects only SEs “that are deliberately civil society grounded,” unlike SEBI which works with SEs that are businesses with a civil society aspect. According to LPD, it will try to continue SEBI’s SE definition and enabling environment work; however, there is an issue with the term “social enterprise” now being within the MSME framework and thus private sector focused, which seems incompatible with the concept of a SE. LPD would further help strengthen the SE definition and policy-making processes through examining local laws and legal framework to identify loopholes constraining the definition of SEs in particular. 34 According to LPD, SEBI should transition to SEJA, with USAID/Jamaica viewing LPD as the facilitator of such a transition. How this was to be achieved was not clear given that LPD’s understanding of SEJA was that it planned to offer fee-paying services to SEs, which might prove problematic given their financial status. SEBI, however, advised that it had no expectations that SEs could pay for services. However, as discussed above in 4.1.1a Objective 4, 38 percent of direct beneficiaries said they would be prepared to pay for services if they led to tangible improvements including increased profits. Table 7: Summary of Findings from Direct KIIs (Stakeholders) Related to Evaluation Question 3 KIIs Evaluation Q3: Best practices and lessons learned for other implementing partners and SE’s in Jamaica? KII Q1: Do you believe stakeholders now share information and collaborate to facilitate awareness of SEs in Jamaica? KII Q2: Do you know about SEJA and/or that it is planned to follow on from SEBI re: SE business development? Group Yes No Not Sure Yes No Not Sure Government 4 - 2 - 3 3 Local Authorities - - 1 - 1 - Women/Youth Associations - - 1 - 1 - NGOs 2 - 1 2 - 1 International Donors/ Organizations 1 - - - - 2 Education - 1 1 1 - 1 TOTAL 7 1 6 3 5 7 In the absence of a clear structure and format for SEJA and how it proposes to operate, it is not possible to comment meaningfully on what best practices might be appropriate for an entity such as SEJA, which, at the time of writing, was not yet registered as a legal entity in Jamaica with a modus operandi. Furthermore, evidence gathered during the evaluation confirmed beyond doubt that those SEs interviewed were only interested in technical assistance support to address operating issues, not least of which was simply remaining solvent and in business. The SEs explained this should be the focus of any SEJA activities. Thus, further capacity building measures would be of no value and irrelevant to them, whether they viewed themselves as an SE or not. 4.3.2 Conclusions In summary, there appears to be an obvious contradiction between the perception of continuing SEBI’s work with LPD as facilitator and the need for a demand-driven entity that focuses on real needs as identified by SEs. While LPD in such a capacity will, by definition, want to focus on SEs with a clear civil society mandate (LPD’s interpretation), there is no compelling evidence that 35 suggests that such an approach will in fact support local economic development. In terms of lessons learned from SEBI’s activities, it is apparent that unless SEJA has clearly defined objectives that are based on a thorough needs analysis following dialogue with major stakeholders, such an entity will have difficulty in being sustainable. Nor is it clear how such an entity will be funded or from what source. 4.3.3 Recommendations All parties concerned with the creation of SEJA must reflect on what it proposes to achieve, when, and how over its lifetime. It could take the form of a follow-on from SEBI with a capacity building focus and possibly a fee-paying element for specific business-related services for SEs. Or it could be a fully functioning, demand-driven, business service provider that supports SEs and MSMEs that do not necessarily identify as a SE. Unless this matter is addressed, it is difficult to see how SEJA can operate effectively or even become viable. 36 ANNEXES 37 ANNEX 1: STATEMENT OF WORK 38 SECTION C - STATEMENT OF WORK C. 1 TITLE OF ACTIVITY The title of the activity is USAID/Jamaica – Final Performance Evaluation of the Social Enterprise Boost Initiative (SEBI) C.2 BACKGROUND and USAID’s Response Development Context The most significant challenges confronting Jamaica are related to high levels of crime and violence; a stagnant economy: a high debt burden; and low levels of competitiveness and productivity. Austerity measures have been the focus of fiscal controls. When combined with the adverse economic impact of recent weather events, these conditions have conspired to constrain government spending on security, youth programs, and social services, which has perpetuated a downward spiral in safety and security conditions for many Jamaicans. Against this backdrop United States Agency of International Development/Jamaica (USAID/Jamaica) has a strong history of working collaboratively and productively with the GOJ and the private sector to address these major constraints, which inhibit Jamaica becoming a strong, viable, safe and secure country. The Social Enterprise Boost Initiative (SEBI) being implemented by the Jamaica National (JN) Foundation targets the social sector,11 which relies heavily on donations and grants for its existence and has not been able to bridge the gap and move from non-sustainable entities to profitable business enterprises. Without support, the majority of social sector players are unable to survive, resulting in their disappearance from the landscape and unfulfilled mandates. SEBI’s primary goal is to show that social enterprises12 (SEs) can be profitable businesses while providing solutions to social, environmental and economic problems. JN Foundation was established in 1990 as the charitable arm of the Jamaica National Bank (formerly Jamaica National Building Society). Its mandate is to contribute to the developmental needs of Jamaica and does this by providing financial and technical support to projects and programs both at the community and national levels. The Foundation is the first organization to do any significant work in the SE sector locally and is seen as the leader and an influencer in the sector. It has developed strong relationships with key stakeholders locally and internationally, including the Government ministries and agencies including the Ministry of Industry, Commerce, Agriculture & Fisheries (MICAF) that has responsibility for the small, medium and micro business sector under which social entrepreneurship falls; the Planning Institute of Jamaica (PIOJ), which is the arm of government that deals with policy formulation on economic and social issues; and established social enterprise organizations such as Social Enterprise United Kingston and Social Enterprise USA. In 2011, USAID/Jamaica received funding from the USAID/Washington Development Grants Program to implement the SEBI, a micro-enterprise development activity aimed at supporting 11 Also called the Third Sector, the social sector is a voluntary or non-profit sector of an economy; “an intermediary space between business and government where private energy can be deployed for public good.” (2017 Micro, Small & Medium Enterprises (MSME) & Entrepreneurship Policy) 12 A “type of not-for-profit business that employs people and earns income along accepted business norms in order to help address perceived social or environmental issues.” [Updated 2017 Micro, Small & Medium Enterprises (MSME) & Entrepreneurship Policy]. 39 and facilitating the growth of social enterprises and their contribution to the social economy. Over the initial three years, SEBI made significant strides in formalizing the SE sector in Jamaica while developing the capacity of several non-governmental organizations (NGOs) and community-based organizations (CBOs) that showed interest in the SE model. SEBI started with three objectives: 1. Making social enterprises more efficient businesses; 2. Creating an enabling environment; and 3. Improving the public’s attitude toward social businesses. The initial three objectives were slightly modified in 2016 and a fourth objective - Transition to Social Enterprise Jamaica - was added. The initial phase of SEBI was implemented under USAID/Jamaica’s 2010-2014 Country Assistance Strategy and program objectives were in direct alignment with Priority Goals 2 and 3 - the Promotion of Economic Prosperity and Sustainable Development and Investing in People - with a focus on enterprise development for microenterprises and the creation/strengthening of an enabling policy/regulatory environment. The expanded program also complements the Mission’s current Country Development Cooperation Strategy’s three Intermediate Results – Resilience and Sustainability of Targeted Livelihoods and Ecosystems Increased; Violence in Community Renewal Program (CRP) Communities Reduced; and Prevalence of HIV/AIDS in Key Populations Reduced, all of which have sub-intermediate results on institutional capacity strengthening of local organizations. The Results Framework in Figure 1 represents the overall USAID /Jamaica strategy, within which the SEBI activity is mainly implemented under IR 2.3. Figure 1: USAID/Jamaica’s Results Framework (Source: Jamaica CDCS 2013-2019) 40 SEBI is also in line with the GOJ’s National Development Plan, Vision 2030 Jamaica: Planning for a Secure and Prosperous Future. Specifically, program impact outcomes under Goal 3: A Stable Macroeconomy, An Enabling Business Environment, Strong Economic Infrastructure, and Internationally Competitive Industry structures. • 2. Target Areas and Groups While the initiative is geared toward the entire country, under Objective 1, the social enterprises that have been part of the incubator and accelerator programs are predominantly from the parishes of Kingston, St. Andrew, St. Catherine, Trelawny, Westmoreland, Manchester, Clarendon, and St. James. All parishes are impacted through Objectives 2 and 3, via the social enterprise network and public awareness activities and eventually Objective 4 when Social Enterprise Jamaica (SEJA) is operational. Table 1: SEBI Incubator and Accelerator Participants, Listed by Parish Kingston & St. Andrew St. Catherine Clarendon Manchester 360 Recycle Mustard Seed Communities (MSC) Network of Women Bartley’s All in Wood Alpha Wear Napier Mouldings, Castings & Patterns - Ecofarms Limited Avant Academy of Music PSDO Mobility Center - Young Men & Women of Purpose Bresheh - - - Council for Voluntary Social Services - - - Deaf Can! Coffee - - - Dress for Success - - - Educatours JA - - - JAD Binders - - Multicare Foundation - - - Music Inc. - - - Petal ‘N’ Roots - - - Plant Jamaica - - - Rockfort Benevolent Society - - - Superior Craft & More - - - UCA Marketing - - - Trelawny St. James Westmoreland St. Thomas Bunkers Hill Cultural Xperience Live Wire Act Grotto Community Group Inzzpire Ulster Spring Women Group Montego Bay Marine Park The Source Savanna-la-Mar - The entities listed above are current participants of the accelerator program or have participated in the business incubator. Numerous other organizations that identify as social enterprises (see 41 list attached) have received business development training, access to training material provided to incubator participants, access to/participation in SEBI and non-SEBI networking events; and trade shows and accounting software through the SEBI Open Network. Approach and Intended Results Jamaica, the place of choice to live, work, raise families, and do business,13 national goals 3 and 4, a prosperous economy and healthy natural environment, respectively, we must find creative ways of building the capacity of Jamaicans to provide solutions to this end. As such, the development of a thriving third sector within the social economy is critical for growth. Research executed in 2013 by the University of the West Indies Office of Social Entrepreneurship and by Market Research Limited highlighted a need for stronger support to the Jamaican social enterprise sector. It suggested the formation of a Jamaican definition of a social enterprise, development of policy and regulatory frameworks on social entrepreneurship associated with a local government reform, a collaborative approach to the development of the sector, a dedicated legal status for SEs, and increased access to capacity development support and financial resources. It also pinpointed very low public awareness on the existence of the SE sector, the specifics of the sector, and the importance to economic development and social progress. These findings guided the first three years of SEBI. Since 2016, the activities of SEBI are being implemented under four specific, yet inter-related and mutually reinforcing technical objectives. A brief summary of each objective is provided below. OBJECTIVE 1: Develop Solution-Based Social Enterprises Under this objective, the main focus is on strengthening the capacity of a select number of social enterprises/entrepreneurs to make them more efficient and viable businesses within the country’s economic structure by leveraging relationships with key partners to get support on finding microfinance solutions for SEs, provide training and technical assistance, and development of a social enterprise toolkit to be used to guide new and existing SEs as well as other entities that want to adapt the social enterprise model. During the life of the activity and thereafter, a spread effect is expected within the sector at large via value chain development (market linkages) – linkages created with higher-value, higher￾volume, or more stable markets, so that SEs can earn more; strengthening business linkages and upgrading business performance. In order to properly support the pilot SEs with market linkages using a market-based approach; SEBI limits the industries that the SEs operate within to include: Education, Health, Sustainability (clean energy, clean water and sanitation, sustainable living and production, climate change, biodiversity, and ecologic systems), and rural development. Social enterprises may solve a need related to women and youth, food production, culture (including community tourism), poverty reduction, or economic development. The expanded SEBI activities make special effort to include organizations that are run by youth and women or whose core line of activities supports vulnerable youth and women. 13 Vision 2030 Jamaica; National Development Plan. 42 OBJECTIVE 2: Creating Enabling Environments The main focus under this objective is to create an enabling environment for the social enterprise sector by improving institutional capacity through communications and information sharing; formation of a social enterprise network; seminars and workshops; case studies and field visits; and the development and launch of a website. The framework for a policy paper on the social enterprise sector in Jamaica and establishment of a national governing body to look at standards, compliance, etc. are ongoing activities under this objective. Objective 2 seeks to lay the groundwork for the social enterprise sector. Through the development of a number of initiatives, SEBI creates and strengthens a social enterprise network capable of promoting the work and needs of its members and their ability to operate profitably, thereby facilitating the application of “Communities of Practice” where people will be engaged in a process of collective learning in a shared domain of human endeavor. 14 The network is open to all individuals and entities with an interest or a stake in the field of social entrepreneurship, including but not limited to commercial businesses, non-profit organizations, NGOs, existing and startup SEs, public bodies, academics, or politicians. The main objective of the network is to create an environment in and through which the Jamaican social entrepreneurship sector can connect, be educated, spread, and flourish. OBJECTIVE 3: Raise Awareness of Social Entrepreneurship Across Jamaica SEBI has put social entrepreneurship in Jamaica on the proverbial map. Prior to SEBI, awareness of the sector was extremely limited. The initiative tries to sensitize consumers, GOJ, financial providers, and businesses to the social and business benefits of SEs through awareness building, OBJECTIVE 4: Transition to Social Enterprise Jamaica The positive response to the initial phase of SEBI by all of the program’s stakeholders pinpointed the need for a permanent version of the program. The JN Foundation developed a blueprint for a new non-governmental organization, Social Enterprise Jamaica (SEJA), which is to be set up by the end the program in December 2018. SEJA will become a permanent provider of many of the services that are provided through SEBI. SEBI is currently examining ways to transition the most successful activities into sustainable business units of SEJA and identifying what lessons from SEBI should be applied to the constitution of SEJA’s activity portfolio. Implementation In the six years since award signing, there have been six modifications to incrementally fund the award, re-align and increase the budget, expand activities within the scope of the award, and extend completion date. The initial award was for three years. In 2015, the award was modified to extend the end date by six months and re-align the budget to provide additional mentorship to the first 10 pilot social enterprises. In the Jamaican context, social entrepreneurship was a novel approach which presented many opportunities and challenges. Several lessons were learned and interests in the activity and social entrepreneurship broadly increased. This led to the award being modified again in 2015 to build on the program through expanded activities that were within the scope of the program to include additional beneficiaries and increase the range of services 14 http://www.ewenger.com/theory/index.htm. 43 offered to the beneficiaries. The total estimated cost of the award was increased, the activity completion date extended by two years and the program description updated. The award was modified again to extend the activity end date to December 2018, increase and re-align the budget and allow the recipient to scale up 15 enterprises (Objective 1), while also still focusing on the other objectives. Generally, the activity has been implemented based on the objectives outlined in the program description. The award was modified over time to revise some activities based on lessons learned (for example, pilot enterprises requiring extended mentorship and resources) and new opportunities. Existing Data A variety of documents and datasets that are available will be shared with the evaluation team upon award, including but not limited to: • Social Enterprise Scoping Report 2013 • SE Landscape Market Study • SE Awareness Attitude Survey • SEBI (self-assessed) Mid-Year Review 2012-2014 • Social Enterprise Organizational Capacity Report 2015 • Social Enterprise Knowledge Assessment Report 2015 • Social Enterprise Study Tour Report 2017 • MSME & Entrepreneurship Policy (Draft) • SEBI Annual Work Plans, Quarterly Reports and Annual Reports • SEBI Cooperative Agreement and Award Modifications C.3 EVALUATION RATIONALE USAID’s Automated Directives System (ADS) Chapter 20115 establishes Agency Program Cycle guidance and procedures, including those for learning from performance monitoring, evaluations, and other relevant sources of information to make course corrections as needed and inform future programming.…According to the policy, “[e]valuation is the systematic collection and analysis of information about the characteristics and outcomes of strategies, projects, and activities conducted as a basis for judgments to improve effectiveness, and timed to inform decisions about current and future programming. Evaluation is distinct from assessment or an informal review of projects.” Further, “[t]he purpose of evaluations is twofold: to ensure accountability to stakeholders and to learn to improve development outcomes.” A) Final Performance Evaluation Purposes The purposes of the final performance evaluation are to determine whether the SEBI activity achieved its objectives (June 2012-December 2018) and gain lessons for implementation of ongoing or future programs. The evaluation will identify implementation challenges, corrective actions to project management and progress towards achieving expected results. Findings and recommendations from the evaluation will be informative for the Local Partner Development (LPD) program, a USAID/Jamaica activity that has a social enterprise component and Social Enterprise Jamaica (SEJA), which will carry on work in the social enterprise space after the SEBI 15 ADS 201 can be found at https://www.usaid.gov/ads/policy/200/201 44 activity ends. The primary aim of LPD is to build the capacity of Jamaican NGOs, social enterprises, Community Based Organizations (CBOs), other CSOs, and government entities that partner with USAID/Jamaica Office of Citizen Security. Specifically, LPD seeks to accomplish the following: • Strengthen the capacity of targeted local entities; • Support social enterprises (SEs) through a business incubator model; • Improve the enabling environment for civil society organizations and social enterprises; and • Improve collaboration between and across civil society, government, and the private sector. SEJA will be established as a non-governmental organization that will become the permanent provider of the services offered by SEBI. Audience and Intended Uses The primary evaluation stakeholders are: USAID/Jamaica, the Ministry of Industry, Commerce, Agriculture & Fisheries, Ministry of Local Government and Community Development (MLGCD) via the Social Development Commission (SDC), the Ministry of Finance and the Public Service (MOFPS) via the Planning Institute of Jamaica (PIOJ), JN Foundation, and each of the social enterprises targeted under the activity. SEBI is now an activity under the broader Caribbean Basin Security Initiative of the U.S. Department of State and USAID in Washington, and both USG agencies will take a particular interest in any lessons for CBSI success. USAID/Jamaica will disseminate evaluation findings to secondary audiences, such as LAC missions with similar programs and with USAID’s E3 and DCHA Bureaus for sharing in other regions. Multilateral organizations operating in this sphere, such as the British Council, World Bank, and the Inter-American Development Bank would also benefit from reviewing the evaluation results. Evaluation Questions In alignment with the USAID Jamaica Country Development and Cooperation Strategy (CDCS) for FY2013 – FY2019 and SEBI implementation objectives, the Mission asks the following questions for SEBI activities. Evaluation Questions 1. To what extent were the intended program objectives achieved? (b)What factors have contributed to the success/failure of the different project components? (c) Did the activity have similar effects on men and women? In answering these questions, the Evaluation Contractor shall explicitly discuss the unexpected conditions or challenges that were experienced by the activity during implementation and how the implementing partner adapted to or resolved those challenges. 2. To what extent has SEBI contributed to the local social enterprise policy environment? 3. Based on the evaluation findings, what are the best practices and lessons learned for other partners implementing similar activities such as LPD and for SE Jamaica? 45 C.4 EVALUATION DESIGN AND METHODOLOGY A) Evaluation Design The Contractor must answer the evaluation questions presented above in the final performance evaluation report. The Contractor’s conceptual approach to answering these questions will start with analysis of the SEBI activity monitoring data and will be complemented with analysis of original data collected during field work. Original data collection may involve key informant interviews, perception surveys, observations from site visits, focus group discussions, consultations with relevant stakeholders, and other data gathering methods proposed by the evaluation team in its inception report. It is expected that all updates and revisions to the evaluation design will be made in consultation and with the agreement of the evaluation COR at USAID/Jamaica. The evaluation contractor is responsible for documenting such revisions to the evaluation design. The evaluation contractor is required to read and comply with the guidance in ADS 201.16 As noted in the guidance, evaluations should use methods that generate the highest-quality and most credible evidence that corresponds to the questions being asked, taking into consideration time, budget, and other practical considerations. A combination of qualitative and quantitative methods applied in a systematic and structured way yields valuable findings and is often optimal regardless of evaluation design. In keeping with this guidance, the contractor is asked to take special note of the difference between anecdotes and empirically based findings. Evaluation findings must be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or simply the compilation of people’s opinions. Findings should be specific, concise, and supported by strong quantitative or qualitative evidence. This SOW sets out evaluation questions and recommended methods. It is expected that the precise evaluation methods will be further developed by the evaluation contractor in consultation and with the agreement of the evaluation COR at USAID/Jamaica, including sharing any draft data collection instruments. Such revisions to the evaluation design and USAID approval must be appropriately documented. For the evaluation, the Contractor’s independent external consultants must work in conjunction with other team members to plan and implement the proposed evaluation. USAID, SEBI staff and the full evaluation team will need to be involved with design, planning, and logistics, but the evaluation team must provide significant and overall leadership and direction, as well as full responsibility for the evaluation duties and deliverables. B) Data Collection and Analysis Methods The evaluation team will start by analyzing the monitoring information collected by SEBI and identifying information gaps in an inception report, to guide additional data collection required as part of the evaluation. Data requirements, collection methods, and required analyses will be explicitly defined in the inception report. Once existing information has been thoroughly reviewed and gaps in monitoring information for purposes of the evaluation are identified, the evaluation team will complete the “Inception Report Template for Pre-Field Visit Data Needs and Analytical Guide” in Figure 3. The template will be 16 For additional information and guidance, the evaluation contractor should refer to the Evaluation Toolkit, which is available at https://usaidlearninglab.org/evaluation. 46 accompanied by narrative describing its content and the rationale behind the intended approach. Written evaluation COR approval of the inception report is a prerequisite for field work, and as with all deliverables, the first drafts may receive comments and requests for revision before approval is provided. Inception reports will be prepared and discussed, and final versions approved by the evaluation COR before field work is approved. Interviews with SEBI staff cannot be the sole source of information collected by the evaluation team to complement monitoring data and answer the evaluation questions. Documentary evidence and discussions with community and other stakeholders will be important, among other sources the evaluation team may propose. Consistent with the Principles of Evaluation outlined in ADS 201.3.5.10, a combination of quantitative and qualitative methods in data collection and analysis should be employed by the evaluation team in the process. A triangulation method should be applied by the evaluation team to increase level of validity in data collection and analysis. By approaching the same question from more than one perspective or by using more than one technique, evaluators can then compare and contrast the results from these different methods. If the findings from the different methods are similar, or reinforce one another, then users can have greater confidence in the findings than if they are based on only one method. If, on the other hand, the findings from different sources do not coincide, that often signals the need for deeper probing and investigation before reliable conclusions and recommendations can be determined. Figure 3: Inception Report Template for Pre-Field Visit Data Needs and Analytical Guide Final Performance Evaluation Questions Relevant Existing Project Monitoring Data Recommended New Data to be Collected during Evaluation New Data Collection Method(s) Sampling or Selection Criteria of New Data Outstanding Data Needs that Cannot be Met (Data Limitations) 1. (a) To what extent were the intended program objectives achieved? (b)What factors have contributed to the success/failure of the different project components? (c) Did the activity have similar effects on men and women? 1. 2. 3. … n. n = number of existing project monitoring data indicators relevant to this question 1. 2. … m. m = number of proposed new data indicators, to be collected by the evaluation team during field work or from 3rd party sources, relevant to this question 1. 2. … m. m = number of proposed new data indicators, to be collected by the evaluation team during field work or from 3rd party sources, relevant to this question 1. 2. … m. m = number of proposed new data indicators, to be collected by the evaluation team during field work or from 3rd party sources, relevant to this question 1. 2. … p. p = number of indicators that would be valuable if they existed, but which cannot be obtained by the evaluation team during field work or from project data, relevant to this question 47 Final Performance Evaluation Questions Relevant Existing Project Monitoring Data Recommended New Data to be Collected during Evaluation New Data Collection Method(s) Sampling or Selection Criteria of New Data Outstanding Data Needs that Cannot be Met (Data Limitations) (2) To what extent has SEBI contributed to the local social enterprise policy environment? - - - - - (3) Based on the findings of the evaluation, what are the best practices and lessons learned for other partners implementing similar activities such as LPD and for SE Jamaica? - - - - - Communities to Visit for Site Visits and Data Collection The SEBI activity supports efforts in a number of communities across 7-8 parishes, and the evaluation team is anticipated to visit some of these. The number of sites to be visited, the criteria or considerations for their purposeful selection, and the advantages and limitations of said selection, needs to be determined in consultation with USAID/Jamaica and stated explicitly in the inception report. [END OF SECTION C] 48 ANNEX 2: LIST OF MEETINGS AND INTERVIEWS 49 List of KIIs and FGDs by SEBI Evaluation Team # Contact Person Position Organization Location Date 1 Jason D. Fraser Director USAID/JAMAICA Kingston 4/10/18 2 James Burrowes Director – Office of Program, Policy & Management USAID/JAMAICA Kingston 4/10/18 3 Dacia Samuels Development Program Specialist USAID/JAMAICA Kingston 4/10/18 4 Claudette Anglin Program Management Specialist USAID/JAMAICA Kingston 4/10/18 5 Margot O. Francis Senior Acquisition & Assistance Specialist USAID/JAMAICA Kingston 4/10/18 6 Rebecca Robinson Supervisory Program Manager USAID/JAMAICA Kingston 4/10/18 7 Opal Whyte Project Manager SEBI Kingston 4/10/18 8 Nora Blake Project Manager - Social Stock Exchange Jamaica Stock Exchange Kingston 8/10/18 9 Tafari Burns Senior Director Ministry of Culture, Gender, Entertainment and Sport Kingston 8/10/18 10 Damion Campbell Project Manager in Society British Council Kingston 8/10/18 11 Olayinka Jacobs￾Bonnick Country Director British Council Kingston 8/10/18 12 Vinty Davis President (Incubator)) Superior Craft & More Kingston 9/10/18 13 Patrice Smith-Sterling Grants Manager FHI360 – Local Partner Development Kingston 9/10/18 14 Rhian Holder Project Manager – Social Development Jamaica Social Investment Fund Kingston 9/10/18 15 Taneshia Stoney Social Officer – Poverty Reduction Program Jamaica Social Investment Fund Kingston 9/10/18 16 David Silvera Business Unit Manager (Incubator) Mustard Unit Communities Kingston 9/10/18 17 Dimitri Dawkins Entrepreneurship Director Branson Center of Entrepreneurship Caribbean Kingston 10/10/18 18 Serika Sterling Member Young Entrepreneurs Association of Jamaica (YEA) Kingston 11/10/18 19 Ruth-Ann Woolcock Business Development Officer The NVC Kingston 11/10/18 50 List of KIIs and FGDs by SEBI Evaluation Team # Contact Person Position Organization Location Date 20 Avril Ranger Director Social Development Commission Kingston 11/10/18 21 Charmaine Lewis Founder & Director (Incubator) Dress for Success Kingston 11/10/18 22 Tina-Marie Williams Manager, Promotions & Outreach Ministry of Industry, Commerce, Agriculture & Fisheries MSME Division Kingston 11/10/18 23 Clifford Spencer Director Projects & Programs Ministry of Industry, Commerce, Agriculture & Fisheries MSME Division Kingston 11/10/18 24 Carol Narcisse Owner (Incubator) Petals and Roots Kingston 11/10/18 25 K’Adamawe K’nife Head of OSE Mona School of Business – Office of Social Entrepreneurship (OSE) Kingston 12/10/18 26 Charmaine Brimm Technical Specialist – Socio-Economic Development Community Renewal Programme PIOJ Kingston 12/10/18 27 Clover/O’Brian Gordon Owner (Accelerator) Bunkers Hill Cultural Xperience & River Tour Bunkers Hill, Trelawny 15/10/18 28 FGD Indirect beneficiaries Various Bunkers Hill, Trelawny 16/10/18 29 Paula Barrett Parish Manager Social Development Commission Falmouth, Trelawny 16/10/18 30 Shahani-Javier Emmanuel Community Development Officer Social Development Commission Falmouth, Trelawny 16/10/18 31 Blake Widmer Founder and Project Manager (Accelerator) Deaf Can! Coffee Kingston 16/10/18 32 Deniese Badroe Director and Manager (Incubator) JAD Binders Kingston 16/10/18 51 List of KIIs and FGDs by SEBI Evaluation Team # Contact Person Position Organization Location Date 33 Prince Graham-Haynes Lecturer on SE University of Technology – Joan Duncan School of Entrepreneurship Kingston 16/10/18 34 Simier Landsend & Kadeem Petgrave Co-founders (Accelerator) EducaTours Kingston 16/10/18 35 Seretse Small Founder and Director (Accelerator) Avant Academy of Music Kingston 17/10/18 36 Kristofferson Nunes Founder (Accelerator) UCA Marketing Kingston 17/10/18 37 Opal Whyte PM JNF - SEBI Kingston 17/10/18 38 Brian Brown Founder and Executive Producer (Accelerator) LivewireAct St. James 17/10/18 39 Kesha Cole Founder and Director (Accelerator) 360 Recycle Kingston 18/10/18 40 Charles Arumaiselvam Development officer in charge of SE (Incubator) Alpha Wear Kingston 18/10/18 41 Claudette Thompson Founder (Incubator) Superior Craft and More Kingston 18/10/18 42 Hugh Shim Executive Director (Incubator) Montego Bay Marine Park St. James 18/10/18 43 FGD Indirect Beneficiaries Various Montego Bay, St. James 18/10/18 44 Jonathan Bartley Councillor - Wakefield Division Trelawny Municipal Corporation Trelawny 18/10/18 45 Julien Pringle Center Administrator Community Resource Center Savanna-la￾Mar 19/10/18 46 Heather & Turner Clarke Owners (Incubator) Grotto All Natural Juice Savanna-la￾Mar 19/10/18 47 FGD Indirect Beneficiaries Various Savanna-la￾Mar 19/10/18 48 Harold Buchanan Co-Founder and Technical Director (Incubator) PSDO Mobility Center / Portmore Self-help Disability Watson Groove, Gregory Park 19/10/18 52 List of KIIs and FGDs by SEBI Evaluation Team # Contact Person Position Organization Location Date 49 Randy McLaren Co-founder and Director (Accelerator) Bresheh Cumberland Portmore 19/10/18 50 Kevin Miller and Lashawnda Bailey Co-Founders and Directors (Accelerator) Inzzpire Kingston 19/10/18 51 Suzette Dixon Administrator (Incubator) Mustard Seed - Jerusalem Spanish Town 19/10/18 52 Grace Foster-Reid Managing Director (Accelerator) EcoFarms Mandeville 20/10/18 53 Lacey-Ann Bartley Managing Director (Accelerator) Bartley’s All In Wood Mandeville 20/10/18 54 FGD Indirect Beneficiaries Various Mandeville 22/10/18 55 Lilly Claire Bellamy Executive Director Jamaica Intellectual Property Office Kingston 22/10/18 56 Latoya Fry Director Forward Step Foundation Kingston 22/10/18 57 Andrew Bruce Founder and director (Incubator) Plant Jamaica Kingston 22/10/18 58 Rosalea Hamilton CEO LASCO Foundation Kingston 22/10/18 59 Joshua Chamberlain Founder and Director (Incubator) Music Inc. Kingston 23/10/18 60 Whitney Haughton/Tyrone Napier Founder/Co-Owner (Incubator) Napier Patterns Moulds & Castings Kitson, St. Catherine 23/10/18 61 Tishauna Mullings Founder (Accelerator) Nexx Stepp St. Thomas 23/10/18 62 FGD Indirect beneficiaries Various Kingston 24/10/18 63 Ann Astwood & Gina￾Anne Cameron-Turner Programs Manager & SE Specialist (respectively) – (Incubator) Multi-Care Foundation / Breezy Castle Kingston 24/10/18 64 Oswald Service Chairman of the Development Counsel and Chief of Operations of RODECO (Incubator) RODECO / Rockfort Development Benevolent Society Kingston 24/10/18 65 Natalie Wheatle Business Management Specialist FHI 360 Local Partner Development (LPD) Project Kingston 24/10/18 66 Dianna DaCosta Paul A. Chin Edison Galbraith Manager, Research & Business Services General Manager, Microfinance Services General Manager, Loan Origination & Portfolio Management Development Bank of Jamaica Kingston 25/10/18 53 ANNEX 3: LIST OF DOCUMENTS CONSULTED 54 Title of Document Date 1. Cooperative Agreement No. AID-532-A-12-00004 07/11/12 2. Modification 1: Approval of Key Personnel 03/06/13 3. Modification 2: Project extension to 31 December 2015 & incremental funding 01/16/15 4. Modification 3: Project extension to 29 December 2017, increase incremental funding, increase recipient’s cost share & addition of two Key Personnel positions 12/24/15 5. Modification 4: Increase total obligated amount from $969,992 to $1,480,992 03/13/17 6. Modification 5: Extend completion date of the Cooperative Agreement by 12 months to 28 December 2018 09/01/17 7. Modification 6: Increase total estimated amount from $1,480,992 to $1,969,992, fully fund the agreement with $489,000 and increase recipient cost share to $588,752 01/19/18 8. SEBI Annual Report – Oct15 – Sept 16 Final 10/13/16 9. Quarterly Reports (19) from period 2012-2018 Various 10. “Scoping Jamaica’s Social Enterprise Sector” – Prepared by Mona School of Business & Management May 2013 11. “Survey of Jamaica’s Social Enterprise Landscape” – Prepared by Mona School of Business & Management (Dr. K’nife) February 2016 12. “Attitude and |Awareness of Social Enterprises in Jamaica” – Conducted by Market Research Services Limited (MRSL) November 2013 13. SEBI Mid-Year Review 2012-2014 (self-assessed) October 2014 14. SEBI Organizational Capacity Report 2015 No Date 15. SEBI Knowledge Reports 2013-2015 No Date 16. SEBI – “Report on Study Tour of Social Enterprises in the United Kingdom” June 2017 17. SEBI – Year 1 – Detailed Implementation Work Plan (FY2012-2015) October 2012 18. SEBI – 3 Year Implementation Work Plan (FY 2012-2015) October 2012 19. SEBI – Year 4 Work Plan – January 2016 to December 2016 February 2016 20. SEBI – Year 5 Work Plan – January 2017 to December 2017 December 2016 21. SEBI – Year 5 Work Plan – January 2018 to December 2018 December 2017 22. SEBI – Year 3 – Detailed Implementation Plan (FY2012-2015) June 2014 23. SEBI – Monitoring and Evaluation Plan (FY2012-2015) October 2012 24. SEBI – Monitoring and Evaluation Learning Plan (MEL) September 2017 25. Government of Jamaica Policy Development Program as at 31 March 2018 April 2018 26. “JAMAICA - MSME & Entrepreneurship Policy” – Prepared by Ministry of Industry, Commerce, Agriculture & Fisheries Updated 2017 27. “Vision 2030 Jamaica – National Development Plan” - Prepared by the Planning Institute of Jamaica 2010 28. USAID – “Country Development Cooperation Strategy 2014-2018” August 2013 29. Government of Jamaica – Ministry of Industry, Commerce, Agriculture & Fisheries (MICAF) – Strategic Business Plan 2017-2021 02/16/18 30. Government of Jamaica – Ministry of Local Government & Community Development (MLGCD) – Sectoral Presentation 2018-2019 – “Developing Local Government Capacity and Performance Through Elevated Capital Investment” 05/16/18 31. Development Bank of Jamaica – Annual Report 2015-2016 No date 55 Title of Document Date 32. JN Foundation Annual Reports – Year in Review 2016 33. Digicel Foundation – Annual Report 2017 56 ANNEX 4: OUT-BRIEF POWERPOINT PRESENTATION 57 58 59 60 61 62 63 64 65 66 ANNEX 5: SEBI EVALUATION MATRIX 67 Evaluation Matrix Evaluation Criteria # Evaluation Questions # Sub Questions Data Collection Method & Sources • Effectiveness • Impact • Sustainability • Relevance • Efficiency 1. To what extent were the intended program objectives achieved? 1.1a What have been unforeseen results of activity interventions to create an enabling environment for social enterprises? Why did they occur? • Document search & review sourced from USAID & SEBI • Secondary data search & analysis from publications in the public domain – government, donors, private organizations, NGOs, CSOs • Feedback from KIIs pre-identified as potential sources of relevant information re: SEBI and its activities (see Table 4 for proposed list mainly Kingston) • Feedback from KIIs in the parishes potentially with local authorities (to be discussed with SEBI) • FGDs with indirect beneficiaries – how many, where and when to be confirmed at the Inception Report stage • Site visits to selected SEs – subject to logistics • Perception survey of targeted households - Market Research Services Limited (MRSL) • SE beneficiary survey – MRSL • SEBI documents: annual and quarterly reports, monitoring and evaluation (M&E) data, assessments, studies, and surveys • Documents of SEBI’s direct SE beneficiaries’ • USAID/Jamaica staff • SEBI/JNF staff • SEBI direct and indirect SE beneficiaries • Pre-identified “stakeholders” – individuals/organizations with direct/indirect knowledge of SEBI including women and youth organizations • Government line ministries MICAF & MLGCD 1.1b What have been unforeseen results (if any) of activity interventions to assist social enterprises to become efficient? Why did they occur? 1.1c What have been unforeseen results (if any) of activity interventions to increase public awareness and generate support for social enterprises? Why did they occur? 1.1d What have been unforeseen results (if any) of activity interventions to transition the initiative to become an NGO: Social Enterprise Jamaica (SEJA)? Why did they occur? 1.2 What factors have contributed to the success/failure of the different project components? 1.2a -to create an enabling environment? 1.2b -to assist social enterprises in becoming efficient? 1.2c -to increase public awareness and generate support for SEs? 1.2d -to transition SEBI to SEJA? 1.3 Did the activity have similar effects on men and women (including youth)? 68 Evaluation Matrix Evaluation Criteria # Evaluation Questions # Sub Questions Data Collection Method & Sources • Effectiveness • Impact • Sustainability • Relevance 2. To what extent has SEBI contributed to the social enterprise policy environment? 2.1 How did activities contribute to the local SE policy environment? • Review sourced from USAID & SEBI • Secondary data search & analysis from publications in the public domain – government, donors, private organizations, NGOs, CSOs • Feedback from KIIs pre-identified as potential sources of relevant information re: SEBI and its activities (see Table 4 for proposed list • Feedback from KIIs in the parishes potentially with local authorities (to be discussed) • SE beneficiary survey • SEBI documents: annual and quarterly, quarterly reports, assessments, studies, and surveys • Documents of SEBI’s direct and indirect SE beneficiaries • USAID/Jamaica • SEBI/JNF staff • SEBI direct and indirect SE beneficiaries • Official reports and/or publications from GOJ line ministries related to social enterprise development and policy including MICAF, MLGCD via the SDC, and MOFPS via the PIOJ • Development Bank of Jamaica • Companies Office of Jamaica • University of West Indies School of Social Entrepreneurship • Parish local authorities – municipal corporations and councils (to be identified) • Members of the SEBI Open Network • Social Enterprise United Kingston and Social Enterprise USA • PSOJ • Young Entrepreneurs Association of Jamaica (YEA) • Women Business Owners of Jamaica • Donors: British Council, Inter-American Development Bank (IDB), WB, and European Union (EU) 2.2 Which internal and external factors and challenges affected implementation? 2.3 Were there alternative approaches that could have maximized improvements in SE policy environment? 2.4 Which activities can improve SE policy environment and results over the remaining period of performance and in the transition to SEJA? 69 ANNEX 6: SURVEY REPORTS 70 BENEFICIARIES’ SURVEY REPORT SEBI SOCIAL ENTERPRISES STUDY Prepared by: “Always Asking Questions, Always Providing Answers” 16 Cargill Avenue Kingston 5. Prepared for: Monitoring & Evaluation Date: November 5, 2018 71 Table of Contents PAGE # BACKGROUND & METHODOLOGY............................................................................................................... 3 DESCRIPTION OF BENEFICIARIES BUSINESS ....................................................................................... 4 DESCRIPTIVE FINDINGS ..................................................................................................................... 5-9 72 Background & Methodology List based sampling, using the take all approach was used given the relatively small population. A total of 35 beneficiaries were expected to participate, in the final analysis 24 were successfully reached. Reasons for non-participation included refusal to participate (n=2), unable to contact (n=9). A structured questionnaire was used to gather and record the data. All interviews were achieved via Computer Aided Telephone Interviewing (CAPI). The list of beneficiaries was supplied by M&E in collaboration with SEBI. Quality Assurance Measures During The Data Collection MRSL employed a number of different measures to ensure data quality during the collection phase including: • Interviewer Training – each and every single person assigned to this project participated in a training seminar to ensure there was full understanding of the questionnaire. This involved a thorough review of the questionnaire and explanation of all instructions given. The training also featured role plays by each interviewer in order to ensure that they fully understood the tool. • Audio recording of the interviews allowed MRSL to listen to approximately 30% of all completed interviews in order to validate its accuracy and ultimate quality. Description of Beneficiaries Businesses Before SEBI Since SEBI % Change Since SEBI Average number of persons employed: • Full time • Part time 7 3 4 9 5 4 +29% +67% 0% Total number of persons employed: • Full time • Part time 168 69 99 214 108 106 +27% +57% +7% Average number of volunteers used* 23 27 +17% *Most rely on volunteers (n=17) 73 Descriptive Findings Q. To the best of your recollection how did you find out about the JN SEBI Program? Half of the beneficiaries gained awareness of the program through either word of mouth (n=7) or the JN Foundation direction (n=5). Traditional media, it is recalled, also had a major role to play (n=6). Other means by which awareness was obtained included Expos/trade shows (n=3), social media (n=2) and RADA (n=1). Q. During which year did you first become involved in the JN SEBI program? Most of the beneficiaries reported being a part of SEBI between the years 2016 to current (n=18). Only one of the beneficiaries surveyed reported being involved since inception in 2012, while the remaining five (5) beneficiaries became involved between 2014 and 2015. Q. Approximately how many different workshops/seminars held by SEBI did you or a representative from your business participate in? All participants attended at least six (6) different SEBI workshops/seminars. Beneficiaries reported participating in an average of 14 different such workshops/seminars. The maximum number participated in by a single beneficiary was thirty (30). Q. What would you say were the top three things learnt from SEBI Workshops/Seminars? Marketing/visibility and brand development was by far the single most frequently mentioned learning among beneficiaries (n=17). Completing the top three learnings were the importance of accounting (n=9) and effective management (n=7). Other information gleaned by participation in SEBI programs included: • Social Impact • Packaging • Sales 74 • Fundraising • Customer Service • Cost Sharing • Pricing • Networking • The importance of good governance • The definition of Social Enterprise Q. To what extent have you been able to apply what you learnt? The majority of beneficiaries reported being able to apply most to all things they learnt fromtheir participation in the SEBI workshop. Nine (9) out of the twenty four (24) beneficiaries interviewed, reported that they have been able to apply everything they learnt while eight (8) reported that they have been able to apply most things. While none of the beneficiaries reported an inability to apply anything there were seven (7) who agreed that they have been able to apply only some things. Lack of financing (n=2) and irrelevance of the information shared (n=2) are the two main barriers to application of the information learned through the SEBI program. Other barriers cited include time, lack of human resources and in the process of doing so. Q. How much do you agree or disagree with each of the following statements about SEBI? The SEBI program is highly regarded by its beneficiaries. This is noted in their attitude towards the perceived benefits of the program to them. The program is reported to be least beneficial in terms of cross promotion and referencing among social enterprises. Beneficiaries also do not credit their sustenance in business to SEBI. Of significance however is the benefit of the workshops to their business as well as its contribution to the development of Jamaica. Attitude Statement Agreement Scale N My involvement in the SEBI programme has helped me to ensure the financial strength of my business Strongly Agree 15 Moderately Agree 5 Neither Agree or Disagree 2 Moderately Disagree 2 Total 24 75 Attitude Statement Agreement Scale N The training I have received in the SEBI workshops have been very helpful to my business Strongly Agree 21 Moderately Agree 2 Neither Agree or Disagree 1 Total 24 The training I have received through SEBI was understandable, of enough depth, and of high quality. Strongly Agree 17 Moderately Agree 6 Moderately Disagree 1 Total 24 The mentorship and 1-on-1 assistance I have received through SEBI has been very useful to my business. Strongly Agree 13 Moderately Agree 8 Moderately Disagree 2 Strongly Disagree 1 Total 24 My involvement in SEBI has provided me with valuable networking opportunities (opportunities to meet other persons) that have been helpful to raise awareness about my business. Strongly Agree 19 Moderately Agree 2 Neither Agree or Disagree 2 Moderately Disagree 1 Total 24 My involvement in SEBI has provided me with valuable networking opportunities (opportunities to meet other persons) that helped my business find new clients. Strongly Agree 13 Moderately Agree 5 Neither Agree or Disagree 2 Moderately Disagree 4 Total 24 My involvement in SEBI has provided me with valuable opportunities to meet other businesses that face challenges like those my business faces. Strongly Agree 19 Moderately Agree 4 Strongly Disagree 1 Total 24 Social enterprises that I met through SEBI have been a good source of sales, referrals, and cross-promotion. Strongly Agree 7 Moderately Agree 12 Neither Agree or Disagree 3 Moderately Disagree 1 Strongly Disagree 1 Total 24 The time commitment required to be meaningfully involved in SEBI activities was reasonable and possible for me. Strongly Agree 17 Moderately Agree 4 Neither Agree or Disagree 2 Moderately Disagree 1 Total 24 76 Attitude Statement Agreement Scale N The tools and materials shared by SEBI are relevant to my business needs Strongly Agree 16 Moderately Agree 6 Moderately Disagree 2 Total 24 The tools and materials shared by SEBI are easy to understand and use in my business Strongly Agree 16 Moderately Agree 6 Neither Agree or Disagree 2 Total 24 The tools and materials shared in the SEBI workshops will be useful to my business over the long term. Strongly Agree 18 Moderately Agree 4 Neither Agree or Disagree 2 Total 24 If it were not for the SEBI program I probably would not be in business for this long. Strongly Agree 3 Moderately Agree 6 Neither Agree or Disagree 7 Moderately Disagree 6 Strongly Disagree 2 Total 24 The SEBI program is important for the development of Jamaica Strongly Agree 22 Moderately Agree 2 Total 24 Q. And since being in the SEBI program, have you seen any increase in sales or number of customers that you have? All respondents with the exception on one, reported that they have seen an increase in their sales since participating in the SEBI Program. Q. Have you heard about the incubator program offered under the SEBI program? Similarly, all with the exception of two (2) respondents claimed awareness of the SEBI incubator program. One was definitely not aware and the other was unsure. The JN Foundation was almost the main source of information about the incubator program (n=12). The traditional media (n=4) 77 and business colleagues (n=3) were other common sources among those aware. Most of those who were aware of the incubator program also applied to be a part of it (n=15). But seven (7) did not seek to apply. All applications with the exception of one were accepted. Where there was rejection this seem to have been a valid case as the respondent admitted to uncertainty about the social mission of the business. The large majority of those whose application were accepted (n=12) reported that participation in the incubator was effective. There was no one single reason opinion presented as to the drivers of the efficacy of the program. However reasons for its efficacy were: • Relevance/applicability of training received • Technical support supplied • Networking opportunities • The branding/marketing techniques learnt • The experience gained • The program put us under the microscope to analyse the company handling cash flow, the company theme, rebranding. • The exposure via social media and heightened public awareness • We learned from what other businesses were doing Among the two respondents who considered the program ineffective, the explanation given for their views was that they were not paired with a mentor or company that understood their business as well as lack of funding. Q. Have you heard about the accelerator program offered under the SEBI program? Beneficiaries were not as aware of the accelerator program as they were of the incubator program. Seventeen (17) respondents claimed awareness while six (6) were not aware. The other respondent was not sure. The JN Foundation was again the most likely source by which awareness of the Accelerator program was gained (n=11). Business colleagues (n=4) was another common source. Other sources of information about the program were traditional media (n=1) and internet (n=1). But (7) did not seek to apply. All applications with the exception of one were accepted. 78 Limited space was the reason given for rejection. All beneficiaries who participated in the program also reported that their involvement in the program was effective for their business. Q. To what extent would you recommend or not the continuation of the SEBI program? All beneficiaries reported that they would recommend the continuation of the SEBI program. In fact 21 of the 24 interviewed said they would strongly recommend while the remaining three said they would moderately recommend its continuation. Q. What would be your top three recommendations to the management of the SEBI program for improving its effectiveness and impact for businesses? Ten of the twenty four persons interviewed had no recommendations. Among those who did however, funding assistance was the most common recommendation. Improvements with regard to the mentorship program was also frequently cited. Other recommendations presented included: • Access to workshop notes where workshops are missed • Be more deliberate in the selection of participants as some participants social mission was not clearly defined • Continue focus on promotions • Continue to employ persons who are easy to work with • Create an online learning opportunity for accounting • Shorter duration for workshops • Earlier assignment of a mentor • Better selection of mentors ensuring they understand the mentee’s business • Practical information for financial management • More workshops outside of Kingston • Workshop Presentations should be more relevant and practical to participants’ businesses • Systems to evaluate how businesses are progressing Page 1 of 8 JN Foundation Individual Statement of Difference to ME&A’s Final Evaluation Report on the Social Enterprise Boost Initiative Project Reference: Cooperative Agreement AID-532-A-12-00004 Summary findings on the Evaluation:  The evaluation is heavily opinion based. The opinions of the evaluator are not validated by data.  There are a preponderance of throwaway statements by the writer, with no data or validated information to back up claims. A typical example is on page 8 which states that there is evidence of something, but does not offer any. Interventions improved the quality of the operating environment of the SEs as they were able to incorporate what they had learned from specific workshops into their everyday activities. However, evidence further suggests that this component was most impactful because of the stage of development of the incubator and accelerator beneficiaries when they were selected.  Key persons from the project design and implementation phase of SEBI were not interviewed.  Evaluators cited a lack of time to reaching and interview key persons.  Underlying tone of negativity throughout the document.  Much of the evaluation focuses on rural impact, which is not a key focus area of the project. However, having focused on rural impact, the evaluation still lacked findings which are presented within project documentation.  One glaring omission from the entire evaluation is the impact that SEBI has had on people living with disabilities. 20% of the SE’s under SEBI are operated by people living with a disability. The accomplishments to this group are immense, so much so, that one of the stories carried by USAID worldwide was about a social enterprise disability group from SEBI (Deaf Can).  This also shows a lack of knowledge with regards to the reach and impact of SEBI. Throughout this document, there are other examples of this knowledge gap by the evaluators.  At the review meeting on December 10, the evaluator presenting to the group listed different strategies for promoting social entrepreneurship in Jamaica. He listed things such as participation in fairs and expos, promoting social entrepreneurship in schools and at church, and through plays and theatre. Apart from plays and theatres, all other suggestions have been taking place for considerable amounts of time. This again pointed to a lack of knowledge about the ecosystem.  Market Research Services Limited Survey conducted a part of the evaluation research, with the results presented as an appendix. There is little mention or analysis of the findings from this survey within the report.  At the review presentation held on December 10th, one of the evaluators stated that it took two years to commence project activities. Project agreement was signed in 2012. Training commenced 2013. Page 2 of 8  The evaluation approaches certain items from a one dimensional position. When speaking about impact on rural Jamaica, they fail to acknowledge the national agencies that now use social enterprise as a methodology for community economic development, and focus only on whether someone in a rural community understands the definition of a social enterprise. The enabling environment has been created, the awareness amongst national agencies was raised, local and global markets were exposed to rural social enterprises, and a large % of the projects supported through the project are rural. The research must be fulsome to be representative of the impact. Access to Finance. Pg. 20 Final Report The final report indicated that the SEBI beneficiaries will not be viable if financing is not injected into these enterprises. Prior to SEBI, financial institutions did not know about social enterprise. Through the intense advocacy work done by SEBI, financial institutions are not only aware of social enterprises, they are opening up access to financing to social enterprises. SEBI has engaged the Development Bank of Jamaica (DBJ) to create a special financing product to support social enterprises. The persons engaged at the DBJ include Milverton Reynolds, Managing Director, Christopher Brown, General Manager and Dianna DaCosta, Manager of Business Analytics and Research. SEBI and the DBJ collaborated on research into the financing needs of social enterprise, the result of which was used to develop a financing product for social organizations and social enterprises. This research was conducted by Market Research Services Limited, which is a member of the evaluation team. The Board of the DBJ approved the inclusion of social enterprises in its development strategy and as a result, some call for financing from the MSME sector now include social enterprises. It was important for SEBI to lobby the DBJ because development financing to the commercial banks originate from the DBJ. The JN Small Business Loans are open to financing social enterprises and was a major sponsor of the SEBI Summit in January 2017. The JN Group has deliberately sought out and offered bank accounts to the SEBI social enterprises as a first step to building banking relationships so that they can be put on a path to accessing financing. The DBJ was also a sponsor of the SEBI Summit as was the Inter-American Development Bank. Other financial institutions that have entered the social enterprise sector are Victoria Mutual Building Society which has partnered with British Council to do a Social Enterprise in Schools project. The National Commercial Bank has been engaging social enterprises and has selected SEBI social enterprise Deaf Can! Coffee as one of its brand ambassadors. The Jamaica Stock Exchange (JSE) has launched a social stock exchange which will target social projects from not for profit organizations and social enterprises. The JN Foundation, because of the work of SEBI was invited to be member of the board of the JSE Social Stock Exchange and SEBI was a key influencer for the development of this initiative. The JN Foundation conducted its own social exchange at the 2017 SEBI Summit, by hosting the Jamaica Social Exchange, a marketplace for trading in impact and business, which was attended by over 700 people. SEBI took the social enterprises in its incubator and accelerator programmes through several capacity development training including accessing financing. The enterprises went through book keeping and accounting training and each member of the second incubator was given the Quick Books software with training so that they can begin the process of organization their business finances. One of the SEBI Page 3 of 8 Accelerator members, Seretse Small, owner of Avant Academy of Music stated that they were able to raise J$23 million in angel financing because of the training and mentorship of SEBI which allowed them to reorganize their business and improve their accounting. Before SEBI, no financial institution knew of social enterprise. Today, the JN Bank, JN Small Business Loans, Development Bank of Jamaica, Victoria Mutual Building Society, The Jamaica Stock Exchange and National Commercial Bank are all engaging social enterprises in various ways. SEBI has introduced the financial institutions to social enterprise which are opening up access to financing. The social enterprises have been empowered, through the various capacity development activities, to seek out financing at appropriate stages of their development. Operational Efficiency – Pg 20 The report states that Only 7 of the 23 direct beneficiaries interviewed indicated an increase in operational efficiency of some shape or form due to the interventions. However, the Market Research Services Limited survey results indicate that the response to the question “since being in the SEBI program, have you seen any increase in sales or number of customers that you have?” . The response - All respondents with the exception on one, reported that they have seen an increase in their sales since participating in the SEBI Program. This is an example of the lack of validation with data available. SEBI reports and documentation also indicate the growth in revenue, employment numbers and market size. Needs Assessment of Social Enterprises. Page ii Final Report SEBI team members had meetings with each of the enterprises as part of the selection process for the incubator and accelerator programmes. During these initial meetings, the team development an understanding of each enterprise and identified specific gaps which would be addressed in a more targeted way. For example, when SEBI met with Bunkers Hill Cultural Xperience which was operating an attraction targeting locals and tourist, we discovered that they did not have the appropriate license to operate. SEBI assisted Bunkers Hill cultural Xperience to obtain the appropriate licenses to operate. In addition to licensing, significant product development was required at Bunkers Hill to address safety, service and management. Industry expert mentors were assigned to Bunkers Hill in both the incubator and accelerator programmes. The attraction was unknown prior to SEBI, so the project invested in brand development of the enterprise and targeted visibility campaigns in traditional and social media. An Open Day was held at Bunkers Hill where over six major tour operators participated. Bunkers Hill Cultural Xperience confirmed that they have signed a contract with Caribic Vacations. The project used this approach to developing the enterprises in its incubator and accelerator programme. While, there were general capacity development activities, enterprises that required specialized assistance were assigned mentors who coached them through targeted business improvements. Among the enterprises that were assigned mentors were: PSDO Mobility Center, Bunkers Hill Cultural Xperience, Nexx Stepps, Avant Academy of Music, Inzzpire 365, Bresheh, UCA Marketing, JAD Binders, Image Training Centre, Deaf Can! Coffee, Music Inc, Petals N Roots and Superior Crafts & More. Page 4 of 8 “The evaluation found that, at macro level, after six years of SEBI, there is still no legislative framework in place in Jamaica to support SEs” pg. iii of Final evaluation Report SEBI’s objective was to create an enabling environment for social enterprises. One of the activities in the 2016-2017 approved work plan towards achieving this objective was to influence government policy to include social enterprise. Research commissioned by SEBI in 2015 and conducted by Dr. Kadamawe K’nife of UWI Office of Social Entrepreneurship found that social enterprise was not included in any government policy which was a barrier to the development of a social enterprise sector in Jamaica. SEBI sought to address the recommendation of this research and joined forces with the Planning Institute of Jamaica and the Ministry of Commerce, Agriculture and Fisheries to create policy framework for social enterprise. After careful assessment, all stakeholders agreed to lobby government to include social enterprise in Jamaica’s revised MSME & Entrepreneurship Policy. Jamaica’s Cabinet has confirmed this policy inclusion. It is also important to note that included in the revised MSME & Entrepreneurship Policy (pg 21) is a Jamaica definition of social enterprise as well as criteria for identifying social enterprises. The policy strategy of social enterprise was put through a wide stakeholder consultation by the responsible government ministry, Ministry of Industry, Commerce, Agriculture and Fisheries in Kingston and selected rural parishes. Mr. Clifford Spencer of the Ministry of Commerce, Agriculture and Fisheries noted in the review meeting held with the evaluators on December 10, 2018 that policy is a component of Jamaica’s legislative framework. SEBI specifically targeted policy inclusion in its work plan and has achieved this. Creating an enabling environment for social enterprises or any business should not be limited to legislative framework because an enabling environment refers to the ease of doing business and entails building out an eco-system. In building out a social enterprise eco-system, SEBI has engaged the academia: UWI’s Office of Social Entrepreneurship has conducted sector research for SEBI and UTECH’s Technology Innovation Center invited SEBI’s Project Manager to sit on its evaluation committee as it seeks to model social enterprise with the addition of social value creation to its target group. SEBI has brought the Government of Jamaica into the social enterprise sector which includes the Planning Institute of Jamaica; Ministry of Commerce, Agriculture and Fisheries; Development Bank of Jamaica, Jamaica Social Investment Fund and Ministry of Culture, Gender and Sports. Important to the Eco-system are private sector organizations. SEBI social enterprises have partnered with the private sector to grow their business. Among these are Deaf Can! Coffee, Nexx Stepps, UCA Marketing, JAD Binders, 360 Recycle, EducaTours, Bartley’s All in Wood, EcoFarms, etc. SEBI created a level of awareness about social enterprise in Jamaica which is making private sector organizations more open to doing business with social enterprises. Before SEBI, private sector organizations did not know about social enterprise and would not consider doing business with them. Consumers are pivotal to the eco-system and SEBI strategically identified social consumers who had the ability and willingness to buy from social enterprises. To state that after six years of SEBI there is still no legislative framework is a misrepresentation of the project objective and also of the work done in the sector towards policy inclusion and building an eco￾system. Government representatives and stakeholders noted their objection to this statement in the December 10, 2018 meeting with the evaluators. Page 5 of 8 The Buy Social Campaign was incorrectly described as a social media campaign (pg. iii). The Buy Social Campaign was not a social media campaign. Social media was one channel used in the campaign, in addition to print media, digital boards, newspapers, radio and television interviews, bus wrap and integration into all SEBI events. The buy social campaign was explained in the quarterly report for October to December 2017 and updates on the campaign providing in all subsequent reports. Multiple communication channels were used to ensure that the message reached a wide target audience. The project had an initial budget of J$6.2 million to design and implement a consumer awareness campaign. SEBI assessed the products of the social enterprises and determine the consumer group that would be most likely to buy from social enterprises. SEBI also contracted a professional marketing company, The LAB, to support the building out of the campaign. Based on all the expert assessment the target audience was selected and the communication channels that would appeal to the target audience used. SEBI has the laid the groundwork for the emergence of social consumers in Jamaica which is pioneering work for the emerging social enterprise sector. Exclusion of Rural Parishes Pg. 21 Final Report Throughout the report, it was suggested that SEBI excluded rural parishes. Six of the ten social enterprises in the first incubator which ran from 2013 to 2015 were from rural parishes. These include: Daughters of Indigo (St. Thomas), Network of Women (Manchester), Grotto Juice Company (Savanna-la-Mar), Mustard Seed Communities/ MSC Farms (St. Catherine), The Ulster Springs Womens’ Group (Trelawny) and The Source Savanna-la-Mar (Westmoreland), Eight of the 21 social enterprises in the 2nd incubator programme (2016-2017) were from rural parishes. These included: Montego Bay Marine Park (St. James), MSC Farms (St. Catherine), The Source Savanna￾la-Mar (Westmoreland), Bunkers Hill Cultural Xperiece (Trelawny), PSDO Mobility Center (St. Catherine), Link Your Purpose (Manchester), Napier Patterns and Moulds (St. Catherine) and Bartley’s All in Wood (Manchester). The fact is that 7 out of the 14 social enterprises in the accelerator programme (March 2018-November 2018) are located in rural parishes. These include Nexx Stepps (St, Thomas), Bunkers Hill Cultural Experience (Trelawny) EcoFarms (Manchester), Link Your Purpose (Manchester), Inzzpire 365 (St. Thomas), Bartley’s All in Wood (Manchester) and Live Wire Act (St. James). SEBI held 2 networking events to build awareness in St. James; presented on social enterprise at diaspora conferences in St. James followed by study tours to the rural SE’s such at The Source, Bethel Town Sorrel Factor, Hanover Bee Farmers and Beeston Springs Community Tours; and supported rural social enterprises, Nexx Stepps and Bunkers Hill Cultural Experience (Trelawny) in hosting Open Days as part of building awareness about social enterprises and stimulating demand for the products from social enterprises. In partnership with the World Bank, SEBI provided capacity building training about social enterprise to business support organizations in June 2017, such as the Social Development Commission (SDC) and Jamaica Social Investment Funds. The SDC and JSIF are the two government agencies that provide extensive support to rural community based organizations. This training would ensure that the SDC Page 6 of 8 officers understood social enterprise and could take the model to the 14 parishes it serve. This collaboration with the World Bank to train business enabling organizations was highlighted in the July 2017 to September 2017 quarterly report. As a result of partnerships with agencies with national reach, JSIF, PIOJ et al all factor in social enterprise to their community development strategies and approaches. The call for applications to participate in the SEBI incubator and accelerator programmes was published in the major newspaper and shared with business support organizations and social media. Social enterprises from all across Jamaica were given the same opportunity to participate in SEBI. Once the applications were received and enterprises short listed, SEBI team members travelled across Jamaica to assess the enterprises. During the selection process, SEBI conducted verification visits to applicants in St. Elizabeth, Manchester, St. James, St. Thomas, Westmoreland, St. Catherine, Trelawny, Clarendon, Kingston and St. Andrew. Overall survival is perceived as paramount as opposed to advice on what to do rather than how to do it—one of SEBI’s fault lines (pg. iii) This statement appears subjective and based on the opinion of a limited number of social enterprises in the SEBI incubator and accelerator programme. SEBI delivered an Entrepreneur empowerment programme where social enterprises were exposed to methodologies and approaches to make better decisions towards their own development. The learning styles of each of the enterprises differ so SEBI used a number of media and approaches to reach this diverse group of social enterprises. One social enterprise in particular insisted that SEBI customized the training around her specific manufacturing business. The majority of the enterprise in the SEBI incubator and accelerator programme gave positive feedback on SEBI’s capacity development training and other activities. This was confirmed in the evaluation report under consideration and on the feedback forms completed at the end of the capacity development trainings. Using one social enterprise’s feedback to bias SEBI’s capacity development activities is not an objective approach to an evaluation. Data Availability and Data Quality. The evaluators indicated that SEBI could not produce some quarterly reports and annual reports. SEBI was required to submit quarterly reports to USAID Jamaica. SEBI has submitted all the required quarterly reports and annual reports to the USAID Jamaica Agreement Officer Representative. The evaluators did not indicate that any reports were not received to the implementer throughout this evaluation process. The implementer had one introductory meeting with the lead evaluator, Colin MacClean and did not see or obtain request for missing data from Colin. One additional brief meeting with a second member of the evaluation team was held with the implementer. SEBI was also required to develop a monitoring, evaluation and learning plan for submission to USAID Jamaica. In addition to submitting a monitoring, evaluation and learning plan, the project submitted a monitoring, evaluation and learning report to USAID Jamaica which was approved. SEBI has done evaluation reports at the end of the first phase, as well as after the two year extension to 2017. These reports could have been submitted to the evaluators if gaps in report or further enquiries were made of the implementer. Page 7 of 8 “The ET found that the selection and identification of beneficiaries was not highly effective or appropriate. Those SEs selected to participate in the incubator program were not selected in a structured manner”. Extracted from pg. 9 of the final report. This statement is completely incorrect. SEBI advertised calls for application in the major print media, as well as through social media. The calls were also shared by organizations in the space. Applications were received and a committee evaluated the applications received. A short list was generated and site visits were conducted to the short listed applicants. Based on the findings at the site visit a final selection was made. This assumption could have been clarified if the evaluators had asked the SEBI team about the selection process. This feedback was provided by the implementer after receiving the draft report, however, the clarification was not considered in the final report presented. “Consequently, only 7 of the 21 incubators were able to graduate to the accelerator component of the program” pg. 9. This statement is a misrepresentation of the decisions made in the project. SEBI was scheduled to end December 2017. USAID Jamaica indicated a willingness to extend the project to 2018 and as a result SEBI submitted activities that would be implemented in 2018. SEBI specifically stated that it would implement an accelerator with 12-15 social enterprises. SEBI wanted to reach more social enterprises and therefore could not offer all the opportunity to participate to its incubator members. As a result, seven incubator members were transferred to the accelerator and seven participation spots were made available to new social enterprises. This is a biased statement and is not based on any decision made by the implementer or USAID Jamaica as the evaluators did not test this assumption by asking for clarification on this. “Although a working group was established by MICAF to develop a definition for SEs in collaboration with key stakeholders, the MSME policy framework references two different definitions, neither of which indicates that one or the other is the ‘formal’ definition” –pg. 15 Final Report. The Government of Jamaica representatives at the evaluation presentation meeting held on December 10, 2018 confirmed that the MSME & Entrepreneurship Policy contains one definition for social enterprise Jamaica. The PIOJ and MICAF representatives further advised that the Government of Jamaica held wide scale stakeholder consultation in several parishes and the definition presented in the MSME & entrepreneurship Policy has been accepted as the local definition for social enterprise. At the evaluation review meeting, the evaluator challenged the definition, and seemed to use this to attempt to discredit the work done by MICAF and local stakeholders. “One of the more revealing findings from this evaluation concerns the lack of knowledge by any of the parties interviewed, including the MICAF who at the time of the interview were awaiting feedback from SEBI on the status of SEJA, of what was being proposed for SEJA or what it would mean for potential beneficiaries and for the SE sector in Jamaica in general. While MICAF and PIOJ have led discussions via a Policy Working Group on MSMEs in Jamaica, including an Action List with options for SEJA, the ET found no evidence that GOJ knows what a proposed institutional framework for SEJA would encompass” pg. 19 Final Report The representatives who were interviewed from the Planning Institute of Jamaica (PIOJ) and the Ministry of Commerce, Agriculture and Fisheries (MICAF) strongly denounced this statement. In fact, the Page 8 of 8 representative from MICAF declared at the presentation meeting held by the evaluators on December 10, 2018, that he was misquoted and misrepresented by the evaluators in this report. He further shared that this misrepresentation of his statement added to the bias in this report. The MICAF representatives denouncement of this statement and his misrepresentation was witnessed by at least six representatives from USAID Jamaica.