TECHNICAL REPORT Submitted to: Higher Education for Development (HED) Monday, August 18, 2014 Report prepared by: Theodore J. Sofianos, PhD Edu-Sol International, LLC Palm Coast, Florida 32164 Phone: (386) 503-0362 Email: t.sofianos@gmail.com Therese Fellner, PhD Evaluation Research Services, LLC Racine, WI 53402 Phone: (262) 930-8695 Email: tfellner@wi.rr.com Barbados: Supporting Entrepreneurs through the JOBS Initiative (Job Opportunity for Business Scale Up) SUMMATIVE IMPLEMENTATION AND OUTCOME EVALUATION June 2014 Table of Contents Glossary of terms and acronyms Executive Summary…………………………………………………...................................... 1 Introduction ……..………………………………………….…………………………………. 5 Desk Review ……..……………………………………….…………………………………... 7 Project Implementation: Pre Re-Scope………………………………………………... 11 Project Re-Scope………………………………………………………………………. 12 Project Monitoring…………………………………………………………………….. 14 Project Implementation: Post Re-Scope………………………………………………. 15 Project Evaluation……………………………………………………………………... 16 Literature Review……………………………………………………………………………... 17 Historical Perspective ………………………………………………………………… 17 The National Context………………………………………………………………….. 18 The Labor Market……………………………………………………………………... 20 Higher Education and Government’s Role in Entrepreneurship……………………… 22 Curriculum Usage……………………………………………………………………... 24 RFA and Curriculum Design………………………………………………………….. 28 DACUM process………………………………………………………………………. 29 Pedagogical Practices…………………………………………………………………. 31 Evaluation Methodology……………………………………………………………………… 34 Study Design and Management……………………………………………………….. 34 Questionnaire Design………………………………………………………………….. 35 Sampling Design………………………………………………………………………. 36 Data Collection Procedures…………………………………………………………… 37 Management of Data………………………………………………………………….. 38 Limitations of the Study………………………………………………………………. 39 Findings……………………………………………………………………………………….. 40 Human Capacity Building for Faculty………………………………………………… 42 Institutional Capacity Building………………………………………………………... 44 Research Initiatives……………………………………………………………………. 57 Project Administration and Award Management……………………………………... 59 Discussion of Findings………………………………………………………………………... 60 Efficiency of external stakeholder engagement …………………………………......... 62 Relevance of external stakeholder engagement and the intervention content………… 67 Effectiveness of the intervention……………………………………………………… 74 Potential for lasting effects……………………………………………………………. 79 Recommendation for Sustainability…………………………………………………………... 82 Original RFA Expectations……………………………………………………………. 82 Stakeholder Interviews………………………………………………………………... 83 Summative Evaluation………………………………………………………………… 85 Conclusions……………………………………………………………………………………. 85 Primary conclusions from the evaluation consultants………………………………… 86 References……………………………………………………………………………………... 88 JOBS Project Management Personnel List……………………………………………………. 90 Stakeholder Interview List…………………………………………………………………….. 91 Stakeholder Interview and Focus Group List…………………………………………………. 92 Participant Observation List ………………………………………………………………….. 93 Tables…..……………………………………………………………………………………… 94 Figures…..…………………………………………………………………………………….. 95 Appendix 1 - Qualitative Research Materials: Stakeholder Interview Protocol; Focus Group Discussion Guide; Statement of Informed Consent; Participant Information Sheet……………………………………………….. 96 Appendix 2 - Survey Research Materials: Invitations for Faculty Symposium Survey Tool; Invitation for Mentoring Symposium Survey Tool; Invitation for the Train the Trainer Survey Tool 2012; Introductory 2012 Summary; Mentoring Symposium Survey Tool Introductory 2013 Summary; Train the Trainer Survey Tool Introductory 2013 Summary.......................................... 102 Glossary of Terms and Acronyms BCC Barbados Community College BEF Barbados Entrepreneur’s Foundation BMA Barbados Manufacturing Association BIDC Barbados Investment and Development Corporation BYBT Barbados Youth and Business Trust CANTA Caribbean Association of National Training Agencies CARICOM Caribbean Community CASH Cultural Entrepreneurship, Alternative Energy Initiatives, Social Entrepreneurship, and High-Tech Ventures CHSB Cave Hill School of Business COHSOD CARICOM Council for Human and Social Development CQI Continuous Quality Improvement CSME Caribbean Single Market Economy DACUM Designing a Curriculum GA Graduate Assistantships GDP Gross Domestic Product GEM Global Entrepreneurship Monitor GNP Gross National Product GSU Governor’s State University HED Higher Education for Development HOTS Higher Order Thinking Skills IU Indiana University JOBS Job Opportunity for Business Start Up KSB Kelley School of Business LLP Lean Launch Pad M & E Monitoring and Evaluation (M & E) plan MBA Master’s Degree in Business Administration MSc Master of Science Degree NTA National Training Agency OECS Organization for Eastern Caribbean States PAC Program Advisory Committee PMP Partnership Management Plan PI Principal Investigator PIP Project Implementation Plan RF Results Framework RFA Request for Applications SBA Small Business Association SVG St. Vincent & The Grenadines US United States USAID United States Agency for International Development UWI University of the West Indies SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 1 | P a g e Executive Summary The USAID’s Job Opportunity for Business Scale Up (JOBS) is an initiative to promote post-secondary skills training in entrepreneurship through partnerships among U.S. and developing country institutions of higher education. The Barbados JOBS partnership between the Indiana University Kelley School of Business (IU-KSB) and the Cave Hill School of Business at the University of the West Indies (CHSB-UWI) is focused on the growth of an entrepreneurial culture and diversifying the service oriented economies in Barbados and neighboring countries. The partnership worked to establish the Cave Hill School of Business Centre for Enterprise and Entrepreneurship at the University of the West Indies as the hub for innovative business training and growth in the Eastern Caribbean by concentrating on two objectives: (1) To enhance CHSB's institutional capacity to serve as an entrepreneurship center for excellence in Barbados and the Organization of Eastern Caribbean States (OECS), and (2) To strengthen entrepreneurship knowledge and skills among diverse stakeholder groups in Barbados and the OECS through outreach and engagement. The American Council on Education office of Higher Education for Development (ACE/HED) contracted with two independent consultants to conduct a summative implementation and outcome evaluation of the JOBS partnership project in Barbados. The objectives of the evaluation were to assess the: (1) efficiency of external stakeholder engagement, (2) relevance of external stakeholder engagement and the intervention content, (3) effectiveness of the intervention, (4) potential for lasting effects, and (5) recommendations for partnership sustainability. Stakeholder interviews were conducted onsite at the Cave Hill School of Business from June 19, 2014 to June 24, 2014. A total of 25 stakeholders representing a multidisciplinary group of college, business, student, and governmental personnel affiliated with the project were identified through a desk review. Focus groups were also conducted for SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 2 | P a g e the beneficiaries of the JOBS Project, specifically for: (1) students involved with the Student Consultancy Engagement activity, (2) small businesses involved with the Student Consultancy Engagement activity, (3) students involved with the graduate assistantships, (4) Lean Launch Pad Exploration Workshop participants, and (5) Lean Launch Pad 10-week Workshop participants. Following the onsite fieldwork, an additional stakeholder interview was conducted with the HED Program Officer, as well as the distribution of electronic surveys to the participants of the JOBS Faculty Symposium, JOBS Mentoring Symposium, and the JOBS Train the Trainer program. Evaluation findings demonstrate that projected targets (audiences and beneficiaries served) in Human Capacity Building were achieved through Faculty Symposiums and Faculty Development activities. The projected targets (audiences and beneficiaries served) in Institutional Capacity Development ranged from no achievement (development of curriculum and upgrading of existing curricula, business incubator), to minimal achievement (student internships), to moderate achievement (mentoring), and lastly complete achievement (short￾term development programs, workshops and certificate programs, and Train-the Trainer). Projected targets in in Research Initiatives (case studies) were considered to be partially achieved through the delivery of multiple case studies and submission of one article for publication review to a publishing house, however projected targets (audiences and beneficiaries served) were not achieved in terms of faculty fellowships. Findings also demonstrated that the JOBS Project experienced numerous challenges throughout the project lifecycle that impacted the delivery of activities and the overall effectiveness, impact, and sustainability of the JOBS Project. From an administrative and management standpoint, there was also noted variance in project scope and award duration from the initial RFA released on May 7, 2010 through the project re-scope in February 2012. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 3 | P a g e JOBS Project partners worked collaboratively to engage external stakeholders and ensure that their concerns and priorities were incorporated into the JOBS Project implementation planning and delivery. CHSB played the lead role in transferring the input from meetings and advisory committees into program customization to meet the needs of Barbados and the OECS. The intervention content appeared to be responsive to external stakeholder needs, directly aligned with national and local priorities, and rooted in industry best practices. However, there is uncertainty as to the true level of impact and value of the content delivered as ongoing formative evaluation mechanisms were not in place to appropriately and accurately measure these constructs. If the project was implemented properly, the model could reproduce itself propagating an entirely new and broad business culture, thus lending itself to economic independence for those who desired a self-employment pathway; this model could also become self￾perpetuating as it would collaterally employ other Barbadians and OECS business start-ups. Overall, evaluation findings support JOBS project accomplishments in measuring contributions towards strategic objectives by using USAID standard indicators (curriculum new or revised, research/joint projects, short-term training) as well as custom indicators (training materials developed/adapted, short-course non-student/faculty participants, faculty coached in case-study development and use), in addition to the enhancement of CHSB's institutional capacity and strengthening of entrepreneurship knowledge and skills to better serve Barbados and the OECS. Recommendations for sustainability include exploring opportunities for collaboration with Barbados Community College; continuing to track the progress of the JOBS project participants; work with the BIDC and further develop the portal and to perhaps interrelate both portals (CHSB and BIDC); routinely recalibrate the entrepreneurship curriculum; hold various entrepreneurship expos that disseminate program awareness and acumen; continue SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 4 | P a g e pedagogical exchanges between US and Barbadian professors and administrators; consistently liaise with government officials to better express success which will inevitably translate to program needs (i.e. funding); and lastly, to create Program Advisory Committees that consist of local, regional (and if possible) international entrepreneurs, academicians, administrators and government officials to further develop entrepreneurship objectives and to highlight and promote that CHSB-UWI is the regional education destination for burgeoning entrepreneurs. Conclusions in part from the JOBS project were: that the Train the Trainer program and materials used were a value-added outcome for Barbados and the OECS and should be prioritized for continuation and funding; the JOBS Project communication issues was identified and rooted in cultural considerations, or lack thereof, among project partners; mores systematic and comprehensive evaluation mechanisms and tools need to be developed and in place prior to project implementation to ensure appropriate and effective monitoring and evaluation; to include as the geographic scope of work, Barbados and all of the 7 countries of the OECS, was ambitious and unrealistic; and engagement of local higher education institutions (BCC, UWI) has the opportunity to advance the foundation laid by the JOBS Project. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 5 | P a g e Introduction In 1991, The Centre for Management Development (CMD), now the Cave Hill School of Business (CHSB), was started with a grant from USAID Barbados and considerable support from the University of the West Indies and the private sectors in Barbados and the Eastern Caribbean. It began as a pilot program and in 2003, it received another grant from USAID Barbados to administer an Internship and Scholarship program along with the academic programs. Given a 20 year foundation on which to build, USAID Barbados, in collaboration with Higher Education for Development (HED), launched a competition in August 2010 to select a United States Higher Education Partner to support a partnership focused on strengthening institutional capacity of CHSB-UWI for supporting entrepreneurs and entrepreneurship in Barbados and the OECS. IU-KSB, with programs consistently ranked among the best in the nation (U.S. News and World Report Best Colleges, 2013), was selected as the U.S. partner to administer and manage the award. HED receives funding from USAID’s Bureau for Economic Growth, Education and Environment, Office of Education (E3/ED), USAID’s functional and Regional Bureaus and worldwide Missions, and the U.S. Department of State to support higher education partnerships to advance global development, economic growth, good governance, and healthy societies. These partnerships provide training, applied research, academic program development, program evaluation, policy analysis, and program implementation, which are critical to furthering the U.S. government’s (USG) foreign assistance goals. The scope of this report evaluates and assesses the effectiveness of the entrepreneurship project with regard to the CHSB and OECS; concurrently, the report will measure the longitudinal intervention effects that were cited for the partnership whose mission SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 6 | P a g e was to develop, cultivate and improve the entrepreneurship education model and framework at CHSB. The partnership was selected and thus arranged between IU-KSB and CHSB-UWI through a collaborative effort between the Government of Barbados (GOB), and USAID/HED. The undertaking was to develop, implement and facilitate an entrepreneurship model that was feasible, curriculum-driven and sustainable via internal and external relationships with education and business stakeholders. On a broader scale, the partnership’s charge was to build capacity for entrepreneurship methods and practice via the CHSB’s Centre for Enterprise and Entrepreneurship. The objectives were to achieve the following entrepreneurship-building curriculum and best￾practice methods by incorporating treatments, and upon completion, to expand the program’s successful methodologies and practices to the OECS: Human Capacity Building of Faculty Institutional Capacity Building  Academic Programs‐development of curriculum in the area of entrepreneurship  Short‐Term Programs‐development of short term programs in entrepreneurship  Internships  Mentoring  Virtual Incubator and Portal  Entrepreneurial Training Series  Entrepreneurial Competitions Research Initiatives (Case Studies) It is also important to understand that the students who were to be solicited were non￾traditional students (i.e. students who were not seeking a degree-to-employment via customary educative pathways). SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 7 | P a g e Desk Review Prior to departing for fieldwork, the evaluation consultants conducted a comprehensive desk review to inform their understanding of the overall framework and objectives of the JOBS partnership. HED provided the consultants with relevant project documents and progress reports, dated from May 7, 2010 through March 31, 2014, to guide the partnership review. The report content as follows presents a chronological sequence of the partnership development, project implementation and subsequent re-scope, project monitoring, and project evaluation to orient readers to the JOBS project cycle, from inception through delivery to beneficiaries. Partnership Development USAID. USAID has worked in the Eastern Caribbean for more than three decades implementing programs to promote economic growth, improve the education system, and meet other development needs in the region. (Source: http://www.usaid.gov/barbados/history). USAID supports the growth of the economies of the countries of the Eastern Caribbean and seeks the private sector's active involvement in curriculum-building relative to the demands of the labor market. By doing so, USAID hopes to improve the opportunities for youth to successfully enter the job market or create their own sustainable businesses, and ultimately contribute to economic growth and regional security. (Source: http://www.usaid.gov/economic-growth-1). USAID Barbados has a history of collaboration spanning more than twenty years, with focus on building institutional capacity of the University of West Indies, Cave Hill School of Business. Higher Education for Development. Higher Education for Development (HED) works in close partnership with the United States Agency for International Development (USAID) and operates with the advice and counsel of the six major U.S. higher education associations to support the engagement of higher education in development issues worldwide. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 8 | P a g e HED manages innovative partnerships that join U.S. colleges and universities with institutions of higher learning in developing nations. In doing so, HED supports and facilitates the engagement of U.S. higher education in global development. (Source: http://www.acenet.edu/higher-education/topics/Pages/higher-education-development.aspx). On May 7, 2010, Higher Education for Development (HED) issued an announcement for one award of up to $1,350,000 for a three-year period for a higher education partnership with CHSB-UWI in Barbados, contingent on the approval of United States Agency for International Development funding. Based on the results of a mid-course assessment, (“review of progress”) in year three, funding for a fourth and fifth year of partnership activities may be awarded. (Source: Request for Applications (RFA): Barbados: Supporting Entrepreneurs through the JOBS Initiative). The competition focused on the selection of an US Higher Education Partner for the Jobs Opportunity for Business Scale Up (JOBS) Partnership Award and Management, designed to strengthen institutional capacity of CHSB-UWI in support of entrepreneurs and entrepreneurship. According to the Request for Applications (RFA), the JOBS Partnership Award stemmed from a USAID worldwide initiative to promote post-secondary skills training. Post￾secondary education institutions were encouraged to include core courses and competencies in business such as planning and management, financial planning and market analysis for their graduates. The intention was to prepare graduates who have an entrepreneurial spirit, whatever their area of study, to consider starting their own small business, to create jobs or to fill positions in growth sectors. (Source: Request for Applications (RFA): Barbados: Supporting Entrepreneurs through the JOBS Initiative, Background). The RFA anticipated that the partnership would expand after the first year to include the UWI Department of Management Studies, the Barbados Community College, and other SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 9 | P a g e regional stakeholders as appropriate. As reported in the RFA, CHSB-UWI conducted a survey of enterprise and entrepreneurship programs in the region, resulting in the identification of gaps in the development and enhancement of enterprise and entrepreneurship. Such gaps included: 1. The creation of an entrepreneurial mindset/attitude 2. Capacity building for institutions and individuals 3. Filling inadequacies in education, training and development - with a special focus on alternative energy initiatives, high-tech ventures, social and cultural entrepreneurship 4. Expansion of research 5. Internships and other experiential learning/activities (Source: Request for Applications (RFA): Barbados: Supporting Entrepreneurs through the JOBS Initiative, Partnership Description). As a result, USAID, HED, and CHSB-UWI developed performance objectives to specifically address the five gaps. Performance objectives were identified as follows: 1. Provide CHSB faculty and facilitators with short and long-term knowledge, skills and training resulting in their improved ability to teach students and advise local citizens critical components of entrepreneurial and enterprise development. 2. Introduce updated degree programs and course curriculum at CHSB that will strengthen the Entrepreneurial streams in the Executive MBA and workshop and certificate programs, based on international promising practices to train future entrepreneurs. 3. Work with the CHSB to adopt elements of the Vincent HoSang Entrepreneurship Incubation Programme at the Mona School of Business at the University of West Indies in Jamaica, as a way to create the CHSB Incubation Programme. 4. Provide undergraduate students experiential learning opportunities through the UWI Student Entrepreneurial Empowerment Development (SEED) Project resulting in these students eventually launching and operating their own businesses or measurably enhancing existing small and medium sized enterprises (SMEs). 5. Provide faculty and students with knowledge and skills related to applied research techniques that focus on needs assessments resulting in gathering information for case studies and promising practices that highlight innovation, creativity, and wealth creation. 6. Enhance the capability of the Centre for Enterprise and Entrepreneurship at CHSB to: SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 10 | P a g e  Offer short-term programs resulting in enhanced skills and knowledge of practicing entrepreneurs;  Organize and design intensive incubator experiences that share learning opportunities among U.S. and Caribbean residents; and,  Provide access to an Information Portal resulting in increased opportunities for support of business start-ups. (Source: Request for Applications (RFA): Barbados: Supporting Entrepreneurs through the JOBS Initiative, Performance Objectives). On August 16, 2010, The American Council on Education, Office of Higher Education for Development (ACE/HED) issued a Program Description Including: Competition to Select U.S. Higher Education Partner Partnership Award and Management in which ACE/HED requested a period of performance for the Associate Award of 48 months. The performance period included three months for start up to cover the review of applications and negotiation of a sub cooperative agreement so that partnership activities could begin, the partnership award period, and a three-month period for close-out. The partnership award was identified as a 42-month term (January 1, 2011 through June 30, 2014) with the last year of planned funding for the Associate Award as Fiscal Year 2014, with all activities ending by September 30, 2014. This timeframe identified in the program description was inconsistent with the timeline presented in the RFA released in May 2010. Indiana University. The Indiana University (IU) Kelley School of Business (KSB) competed and was selected for the JOBS Partnership Award and Management. By December 2010, a draft sub-award agreement with IU was in place for a 36-month, January 1, 2011 - December 31, 2013 project. IU-KSB proposed to “provide faculty capacity building, curriculum development, short term development programs, development of research techniques for faculty, and project monitoring and evaluation to the CHSB-UWI to better enable the development of a comprehensive entrepreneurship program for MBA and undergraduate students that would help strengthen the entrepreneurial environment of SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 11 | P a g e Barbados.” IU-KSB proposed activities to address each performance objective as articulated in the RFA. Project Implementation: Pre Re-Scope HED conducted a monitoring visit February 7th - 9 th, 2011 at CHSB to convene project partners on project implementation. This event also marked a perceived misalignment on budget line items, specifically the request for more travel money for CHSB staff development, more travel money for OECS outreach, and more money for internships in the OECS. Additionally, there was a sense on behalf of CHSB that their partnership concerns as related to distribution of award funds were not being taken seriously. Strategies for moving forward such as milestones written into the RFA and IU work plan, including the scheduling of all activities for the faculty development, online training forum, information portal, Train the Trainers, curriculum development, and a feeder system for the existing incubator in the community were agreed upon and the parties prepared for project launch. A JOBS Project Coordinator was hired as the sole JOBS staff member at the CHSB in March 2011 and was responsible for organizing the project, writing and submitting reports, administering project activities, and designing and conducting trainings and workshops. CHSB provided support services as needed (e.g. room set-ups, print outs, etc.). The first quarter of project activities, April 1 - June 30, 2011, produced a faculty inventory and needs assessment, CHSB-UWI faculty visit and orientation at Indiana University, presentation and discussion of JOBS initiative with key entrepreneurship stakeholders in Barbados, incubator needs assessment, and completion of the new Diploma and MBA programs in Entrepreneurship for CHSB. However, the project experienced significant delays over the next six months, July - December 2011, due to administrative processes and subsequent budgetary obstacles between CHSB and the IU Grants Office, particularly for the payment of vendors residing in Barbados and the OECS. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 12 | P a g e In January 2012, the CHSB implemented the JOBS Entrepreneur’s Survival Series consisting of a set of four workshops addressing knowledge gaps identified by entrepreneurship agencies in Barbados. Participants consisted of entrepreneurs, nascent entrepreneurs, employees of small businesses, and individuals working with agencies charged with assisting entrepreneurs and promoting entrepreneurship. However, partnership communication challenges and cultural norm barriers persisted and led to a formalized need to revisit the project work plan and budget. Project Re-Scope February 2012 marked what became known as the “project re-scope” in which IU￾KSB and CHSB-UWI partners met to revise the JOBS Project to more effectively meet the current economic and entrepreneurial development needs of Barbados and the OECS. The main findings highlighted the need to further emphasize three areas of the partnership program in order to attain its proposed objectives. (Source: Request for Applications (RFA): Barbados: Supporting Entrepreneurs through the JOBS Initiative). Faculty Development. After an initial training visit by CHSB faculty to IU-KSB in Year 1 and a subsequent faculty inventory and assessment, it was determined that it would be beneficial for the faculty to experience and attend a wider range of entrepreneurial training workshops and conferences from other US institutions. Faculty would be selected each year to attend professional development training events and then return to CHSB-UWI to disseminate that information to other faculty, and add value to their knowledge, skills, and current teaching and research methods. Entrepreneurship Training and Services. The JOBS team determined that instead of creating another traditional (brick and mortar) incubator, there would be a renewed focus on training potential clientele, entrepreneurs and managers. By designing a “Train the Trainer” workshop program to be delivered across the OECS region, JOBS would equip SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 13 | P a g e trainers from the OECS countries to deliver entrepreneurial skills. CHSB and OECS training centers would serve as key resources for entrepreneurs and business development. Youth Program Initiatives. Entrepreneurship is still seen as a career path for individuals that are not prepared for advanced education and are most likely traditionally unemployable. Therefore, the JOBS team would place a greater emphasis on developing the entrepreneurial spirit and the reputation of entrepreneurship as a viable career option among the youth in Barbados and the OECS. The re-scope placed an emphasis on youth initiatives that included student internships, mentorship, youth business competitions and the use of both traditional education and grassroots efforts to further foster a culture of entrepreneurship. Other supporting components of the JOBS partnership. Components included virtual incubation; online portal, curriculum development; case writing/publication for the Global Entrepreneurship Monitor (GEM); marketing and communications; and monitoring and evaluation. It was also this point in time when HED extended project activities with a 6-month no￾cost extension from the original sub-award end date of December 31, 2013 to June 30, 2014.Additionally, the JOBS project re-scope required adjustments in monitoring and evaluation protocol prompted by the shift in USAID Evaluation Policy released in January 2011 followed by a new online reporting system (Partnership Results & Information Management Engine, PRIME). Consequently, JOBS Project partners began converting the original results framework into a three-part monitoring and evaluation (M & E) plan. The updated M & E plan included a results framework (RF) reflective of both USAID standard and JOBS custom indicators; a partnership management plan (PMP) detailing data sources and methods for data collection and analysis; and a partnership implementation plan (PIP) identifying project objectives, outputs, activities, and related timeline. (Source: Evaluation– Learning from Experience, USAID Evaluation Policy, January 2011). SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 14 | P a g e As a result of the project re-scope and the necessary programmatic and budget revisions among project partners, the implementation of JOBS project activities for the CHSB halted again for a second time from March 2012 - August 2012. Project partners expressed uncertainty regarding which activities would be approved as allowable costs. Project Monitoring HED conducted a monitoring visit in May 2012 to assess partnership progress to date, inclusive of project re-scope provisions, PRIME, and the updated Results Framework and performance management tools. Reflection upon the articulated partner challenges rooted in cultural differences, quality of communication between the partners, and its impact on project administration and implementation was facilitated by HED personnel. Differences in work styles and roles and responsibilities within each institution were also discussed. Agreement was reached to add personnel to the JOBS team at CHSB to facilitate project success and respond to the accelerated revised schedule of partnership activities to make up for lost project time (12 months in total), while still maximizing impact by affecting multiple academic years (2012-2013 and early 2013-2014). Project partners also acknowledged the importance of building relationships—assimilating two institutions and people with fairly different cultures, work and communication styles—prior to project implementation. In late January 2013, the CHSB-UWI and IU-KSB teams participated in a coordination meeting held in Washington, D.C. at HED. The meeting provided critical opportunities for joint project monitoring, reporting and teambuilding. The HED/IU/CHSB teams also discussed and worked on customization of the partnership Monitoring & Evaluation tools developed by HED, including the Results Framework, Performance Management Plan and Partnership Implementation Plan. A separate session focused on strategy for JOBS partnership sustainability, impact and contributions beyond 2014. This joint SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 15 | P a g e management meeting was regarded by all participants as an important milestone in the effective administration of the JOBS partnership. The revision of the Results Framework was finalized in April 2013 and provided project partners with a clearer direction for the remaining 14 months of the JOBS project. The original Results Framework and subsequent revision is presented in Table 1 below. TABLE 1 Results Framework Revision Original Results Framework December 2010 Updated Results Framework April 2013 Outcome 1: Faculty have improved ability to teach students and advise local citizens on enterprise development. Outcome 1.1: Improved faculty and facilitators' capacity to design entrepreneurship development programs, conduct training, and/or advise entrepreneurs. Outcome 1.2: Enhanced CHSB's ability to offer updated degree programs and course curriculum. Outcome 2: CHSB has increased the number and quality of academic and non-academic program offerings in entrepreneurship. Outcome 2.1: Increased access to customized entrepreneurship training, tools and techniques for participants Outcome 2.2: New and/or enhanced experiential opportunities for students in entrepreneurship practice Outcome 3: CHSB has enhanced outreach capacity to offer entrepreneurial training and advisory services to the region. NOTE: Outcome 3 in the Original Results Framework became embedded into Outcome 2.2. in the updated Results Framework. Project Implementation: Post Re-Scope Once programmatic and budgetary revisions of the JOBS project re-scope were finalized in August 2012, CHSB-UWI resumed project implementation by delivering Train the Trainer workshops to Barbados and OECS stakeholders in St. Vincent, Grenada, St. Lucia, St. Kitts and Nevis, Antigua, and Dominica. These workshops were delivered between September- December 2012. CHSB-UWI also offered the first JOBS Mentorship Symposium during this time period (October 2012) designed for business mentors residing in Barbados, in addition to a pilot internship program during the January 2013 semester break, pairing UWI SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 16 | P a g e students with businesses that were entrepreneurial in nature, or with agencies that worked with entrepreneurs. Following the January 2013 coordination meeting in Washington, D.C., project activities resumed with and IU/UWI Student Consultancy Engagement Project and Case Study Workshop in March 2013; delivery of the Train the Trainer workshops from May – September, 2013 to Grenada, Barbados, St. Vincent, St. Lucia, Antigua, and Dominica stakeholders; a second JOBS Mentorship Symposium, Student Internship program, Graduate Assistantships, and a Faculty Symposium. In November 2013, the Lean Launch Pad: Opportunities for Exploration Workshop was initiated, followed by the Lean Launch Pad Entry Competition in March 2014 and the IU-KSB, Governor’s State University (GSU), and another UWI Student Consultancy Engagement. Overall, the desk review indicated that 16 months out of the 42-month partnership award were relatively inactive (inclusive of the 6-month no cost extension), essentially leaving 26 months of time to accomplish project objectives. Aforementioned partnership challenges and the necessary timing for JOBS project activities to work around the various holidays and school semesters in Barbados and all the islands in the OECS were primary factors for the periods of inactivity. Project Evaluation Although identified as an annual function in the IU-KSB work plan, budget narrative (“provide oversight to the evaluation part of the project, ensuring that goals and benchmarks are being met”), and in 3 separate budget line items (Personnel - Evaluation Consultant at 5% effort; Other Direct Costs - External Evaluator; Other Direct Costs - Project Management and Evaluation), there is no evidence that any summative or formative evaluation was conducted until the end of the project during an onsite visit from May 21 - May 27, 2014. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 17 | P a g e The evaluation cited two omissions in the project, 1) a lack of direct measures by which to assess the attainment of the project’s goal(s); and 2) a lack of plans for maintenance and continuity of the advances made by the JOBS Project. It was recommended that metrics related to project objectives be established in the planning phase, before any of the intervention’s activities or experiences were implemented to ensure clear direction for formative and summative project evaluation. Short-term gains, particularly the enhancement of institutional and human capacity of the CHSB-UWI with respect to entrepreneurship were noted, however the IU-KSB evaluation relayed reservations about the long-term effectiveness of the project. Literature Review A literature review was conducted in order to identify relevant, best practices for entrepreneurship education while also accounting for the contextualization of the literature review for Barbados and the OECS. The review presents a sequence of literature beginning with a historical perspective followed by the national context, Barbadian labor force, and the role of higher education and governmental in entrepreneurship. Curriculum development and pedagogical practices are featured aspects. Historical Perspective Entrepreneurship is evident in our history for thousands of years. Before any good and/or service could be monetized, trade and barter were the basic tenets for a burgeoning entrepreneur (Bygrave & Zacharackis, 2008). As this term has evolved, there should be a distinct reference made between salesmanship and entrepreneurship: in some cases the salesman is the entrepreneur, but not in all. In many cases, the salesman works for the entrepreneur. Entrepreneurs are leaders willing to take risk and exercise initiative, taking advantage of opportunities in the market by planning, organizing and making use of resources, SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 18 | P a g e often by innovating new or improving existing products (Johnson, 2005). This is not often accomplished without the aid of education, educative resources and design (Kuratko, 2009). The National Context As is cited in the National Content Standards for Entrepreneurship Education: wealth and a high majority of jobs are created by small businesses started by entrepreneurially￾minded individuals, many of whom go on to create large businesses. People exposed to entrepreneurship frequently express that they have more opportunity to exercise creative freedoms, higher self-esteem, and an overall greater sense of control over their own lives (Drucker, 1985). As a result, many experienced business persons, political leaders, economists, and educators believe that fostering a robust entrepreneurial culture will maximize individual and collective economic and social success on a local, national, and global scale (Crane & Meyer, 2011). Entrepreneurship curriculum integration has become a national priority for a number of nations. Cited in the National Content Standards for Entrepreneurship Education (2014), the following standards and indicators are essential to understand the formative evaluation process, and subsequently, to gauge pre- and post-results: Integration of the Standards into the curriculum, whether as subject-specific content or as context for a broader range of educational learning objectives, can help address many of the issues facing… education programs. Educators who can demonstrate the link between incorporating standards and achieving identified goals by many of these issues will be more successful in implementing the standards into their curriculum. Standards-based education requires that instruction be based upon a clear and complete listing of the skills and knowledge the student should be able to demonstrate at the completion of the instructional program. This context is in the form of standards and performance indicators that should be used to develop curriculum appropriate for the learner and program. Standards set out a commonality of content and performance for entrepreneurship education programs. Mastery of academics is the basis for life-long learning. Standards (-based education) can contribute to this mastery by providing the context for learning and applying academic skills and knowledge. Contextual learning theory states that learning takes place when students can process new information or knowledge in such a way that it makes sense to them in their own frame of reference. Students learn best when they SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 19 | P a g e can make the connection between what they are learning and how that knowledge will be used. Educators who use a standards-based approach to define context for the teaching and learning of academic skills are using applications that are familiar to students of any age. Performance-based assessment requires that students apply the skills and knowledge they have gained in such a way as to demonstrate mastery. Standards provide guidance for the design of many performance-based assessment activities, both as context for the application of academic skills and to demonstrate mastery of specific entrepreneurship standards and performance indicators. Whether it's communication arts in the writing and presenting of the business plan, mathematics in the analysis of the financial statement and projections, science in the development of prototype products, economics in the understanding of the effect of supply and demand on potential sales, or marketing and business skills required for success as an entrepreneur, these standards can be used to design performance-based assessments for a wide range of subjects and students at all levels. 21st Century Learning Skills emphasize academic skills and learning skills, as well as curriculum content that has been largely overlooked in the past but is essential to the 21st century. The Standards can assist educators in developing a curriculum which addresses these and the other essential elements of learning for the 21st century. In addition to gaining academic skills as described above, entrepreneurship education develops the skills to learn. Information and communications technology skills are acquired as students conduct research, communicate with others, and prepare written and visual presentations of their work. Thinking, problem-solving, interpersonal, and self-directional skills are inherent in a curriculum that includes the standards and performance indicators. By using the real world of entrepreneurship to apply 21st Century skills, students are learning the content and context necessary to adapt to our changing world. Workforce Development programs can benefit from the implementation of a standards-based approach. Standards can serve as a basis to prepare people for creating their own businesses by establishing the skills and knowledge that are required for success. Programs can focus their resources on meeting the needs of the clients identified by review of standards and performance indicators, thereby best utilizing the limited resources to meet specific client needs. In addition to preparing individuals for their own futures as entrepreneurs, standards can serve as a basis for programs which develop employee skills and knowledge to make them more effective and efficient. Understanding the components of business ownership and acquiring the entrepreneurial spirit increases employee productivity, improves customer satisfaction, and moves business forward. Using a set of given standards as a framework for program curriculum will assist local communities, states and nations to better compete in the world marketplace through building a skilled workforce with life-long learning skills and work training. Higher Education programs in colleges and universities will find the standards this organization supports, uses and proliferate helpful for establishing a commonality of program content across departments and institutions. Standards and performance indicators also provide a basis for accountability of learning goals required for effective program management and review. Standards will allow various programs and SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 20 | P a g e departments to integrate the appropriate components to support their curricular offerings and degree programs. Institutions can achieve leadership in economic development and support one-stop career training offerings by using the Standards and related programs as a vehicle for community outreach. Barbadian and OECS stakeholders must have access and input to curriculum design and implementation as is cited in the Standards. In order to continue high caliber content delivery and refresh it routinely (as is depicted in the Curriculum Process Oval on p. 27), standards and performance indicators must be a staple ingredient. There will be no CQI if the stakeholders are not periodically informed and educated to the curriculum design (and/or redesign). Responsiveness to the lack of entrepreneurial impetus should be considered critical to the success of any depleted numbers regarding upstarts and entrepreneurship. Employers, business owners, and industry leaders must be engaged throughout any advisement strategies to ensure that curriculum alignment is in keeping with economic growth opportunities. The Labor Market The Barbados Statistical Service (as referenced in the next table) does contain a very good account of indicator-data such as Gross Domestic Product (GDP), Gross National Product (GNP), unemployment rates, etc., but the analyses this consultancy has undertaken will be problematic since it is evident that there was minimal (if any) baseline data for the JOBS project to carry out formative to summative evaluation applications; thus, there is no longitudinal tracer studies and, as yet, there is no instrument for reverse tracer studies. On all accounts, this consultancy has learned that Barbadians on the whole do not envision themselves as entrepreneurs; and subsequently, the Government has taken steps toward the development of demand-driven educational services including improved educator preparation programs, career planning, quality assurance, and improved provision of second￾chance education and training. The latter is certainly evident at CHSB-UWI given the new pedagogical strategies, implements and programs associated with entrepreneurship. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 21 | P a g e The following table describes a startling dynamic and trend as to why the Government of Barbados had to act and enlist the knowledge, expertise, curricula and best practices from its leading academic institutions as a way to bolster enterprise and economic growth. Table 2. Labour Force Participation Rate (%) 2003-2013 As per the Barbados Statistical Service: The labor force participation rate is a vital tool used to assess the “active” portion (those persons of working age who are either (i) employed or (ii) unemployed but actively seeking employment) of a country’s adult population. This rate decreased by some 2.6 percentage points over a 10 year period, moving from 69.6% in 2003, to 66.7% in 2013. This decrease was also reflected by a general downward trend being observed, particularly between the years 2005 and 2010. Over the period referenced, the rate reached a maximum of 69.6% in 2005, whilst the minimum of 66.2% was attained in 2012. The biggest decrease in this rate was observed between the years 2005 and 2006, when a decline of some 1.8 percentage points was recorded. It is noted that working for oneself is a valued and worthy endeavor throughout any western business community; with it comes job satisfaction. Primarily because of SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 22 | P a g e industrialization, we have taken antiquated job-performance tasks and extrapolated them from what once was a customary, European-Mercantile, cottage-industry setting; thus, its metamorphosis and evolution to a more productive underlying principle: a mechanized world bearing a surfeit of urban metropolises swarming with a cornucopia of business enterprises and organizations. Entrepreneurship, in essence, is how people develop innovative ideas and take them to market. It obviously takes a certain mentality, but it also is dependent upon a number of factors, especially a country’s educational infrastructure and environment. Barbados has a unique opportunity because of its diaspora and geographical location; thus, importation and exportation—that have direct relationships to entrepreneurship—are ways of life. International best practice continually demonstrates work-placed learning – that it is essential in gaining work experience in today’s job market. The Youth Build Trust (YBT) and the Barbados Investment and Development Corporation (BIDC), with the support of government, have essential work ahead of them to enable, support, engage, entice and further advance entrepreneurial tendencies, and those tendencies to expand, propagate and manifest themselves into a robust, entrepreneurial culture for all business and industry sectors. Higher Education and Government’s Role in Entrepreneurship In 1989, members of the Caribbean Community (CARICOM) decided on an integrated development strategy to chart their way into the 21st century. One of its key features was to deepen economic integration by advancing beyond a common market towards a Caribbean Single Market and Economy (CSME). The free movement of labor will be achieved through measures such as removing obstacles to the business development process (Miller, 1988). Key elements of this include the free movement of goods, services, capital and labor; the latter cannot be achieved without the cultivation of entrepreneurship (via educational pathways) and the endorsement of government. In 1990, CARICOM articulated a position on the SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 23 | P a g e development of Technical and Vocational Education and Training (TVET) in the region. This prompted the national training agencies of Jamaica, Barbados and Trinidad and Tobago to establish the Caribbean Association of National Training Agencies (CANTA). By agreement between these founding members, the National Training Agency (NTA) of any CARICOM member state may become a member of CANTA. CANTA was given its mandated authority through the CARICOM Council for Human and Social Development (COHSOD) as the lead body for TVET in October 2003 with three main goals:  To support the establishment and functioning of National Training Agencies;  To promote the development of a competitive regional workforce; and  To facilitate the free movement of certified skilled workers within the CARICOM Single Market and Economy. There is insight and acumen when considering these three main goals: training that was to occur at secondary and postsecondary levels (e.g. Samuel Jackman Prescod Polytechnic Institute) was not only for skills training. For instance, SJPP has developed PACs (program advisory committees) who are business owners and managers in various business and industry sectors. The supposition was for those who received training and developed their skills in various trades and technical areas, also realized the advantageous position to being a business owner – an entrepreneur. This supposition was researched and vetted in 2010 via an education sectors study that strived to understand the relationship between secondary and postsecondary education techniques, curricula, pedagogies, implements, and their effect on job creation and enterprise. A medium term development strategy was developed for 2010 - 2014 which set out to outline a broad framework of policies and programs for increasing skills, jobs, and to overcome the Great Recession. The plan was used to guide the country as it emerged from what turned out to be a global recession. An important feature of the plan was its focus on maintaining macroeconomic stability through sustainable fiscal and debt management. The SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 24 | P a g e plan promoted the need for new areas of growth in education to assist in broadening the following strategies:  An increased effort was created to further develop the small business sector through technical support and ensuring that the necessary policies were put in place to assist small businesses and create employment.  The plan sought to create new enterprises that could earn foreign exchange through higher exports.  In the international business sector, government was seeking to expand and double taxation treaty arrangements with countries within the various continents while also engaging in other initiatives to expand the sector.  In the area of tourism, the plan outlined measures to further develop and expand the tourism product.  In agriculture, emphasis was to be placed on a greater use of technology and ensuring food security.  In the energy sector, advancement of alternative forms of energy such as wind, waste to energy and solar was urged and encouraged. The latter is consistent (specifically the first three bullets) with the direction government has taken, and it directly connects the expertise of CHSB to business development and growth via educative pathways to promote economic impendence for the entire region. Curriculum Usage Most entrepreneurship coursework is designed to introduce students to the many aspects of self-employment (Drucker, 1985). Curriculum is tailored to encourage the student to identify their own personal assets and define a plan of action for their future studies or career path. Aspects of business that are normally covered include: a) What is an entrepreneur and what is the pathway to become one; b) What experiences (educative and experiential) may help one become an entrepreneur; c) What types of businesses would fit the model of economic supply and demand for a chosen field; d) How does one prepare to be their own supervisor; SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 25 | P a g e e) How do small businesses have a global impact? General learning outcomes include, but are not totally respective of:  Define entrepreneurship and distinguish between entrepreneurship and small business management  Identify types and characteristics of entrepreneurs  Understand the advantages of working for oneself compared to working for others  Sources of new enterprise ideas and methods of generating new enterprise ideas  Market and competition analysis  Legal aspects of ownership  Employer/ employee legalities  Sources of capital/ finance  Identifying and utilizing best practice models and support systems for entrepreneurs  Product comparison and analysis  Import and export principles  Methods of pricing products and services, selling, advertising, and sales promotion  Appropriate business ethics  Development of a business plan The list cited is only part and parcel of the pedagogical implements for learning entrepreneurship principles and practices. Continuous Quality Improvement (CQI) must be the goal for any curricula to evolve. As businesses and business models have ebbs and flows, so does curriculum development and refinement. Thus, the curriculum design process relies on profile information as a starting point. It is the organized preparation of whatever is going to be taught at a given time in a given year. In some cases, similarities exist overseas with other academic and technical programmatic plans that may be present in principle, but they are not either being employed or the infrastructure may not be conducive to programs slated SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 26 | P a g e for a particular program’s delivery; it is important to recognize the aforementioned before designing coursework that may not be associated with a goodness of fit model. Curriculum specialists through various educative seminars/conferences have created the structural themes necessary to assist in the development of the focal points of curricula. The basic tenets are comprised of essential ideas, like supply and demand, as well as the relationships between them. Such relationships make sure that students are able to be aware of these associative relationships and linkages throughout the curricula spectrum (e.g. cross-curricula planning). Curriculum development is basically founded on the following four ideas: 1. To determine what educational principles must be reached via educative means. 2. The best way of deciding what kinds of learning experiences are apt to be valuable in reaching these objectives. 3. The best way of putting in order familiarity of learning for effective instruction. 4. The best way of evaluating aforementioned familiarity of learning (learning experiences). The following curricula process oval model depicts a continuous quality enhancement loop for the benefit of curriculum delivery: SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 27 | P a g e FIGURE 1 Key Elements Curriculum Process Oval – Entrepreneurship Concentration and Sustainability Therefore, the goal of entrepreneurship curriculum should be to strengthen the pedagogical framework and identify strategic learning and reinforcement measures to enhance the impact of entrepreneurship programs in ensuring a successful education-to-work transition. Part of the on-going practices, but not readily seen in the process oval are to:  Provide an overall description and analysis of regional businesses and their connection locally, then globally;  Develop recommendations for strengthening the policy and institutional framework for the articulation of secondary to post-secondary curricula and programs;  Identify the most cost-effective technical education modalities at the secondary and postsecondary levels, make recommendations for updating their programs and improve their articulation with labor market needs;  Make recommendations for strengthening monitoring and analytical capacities to gather and use labor market data to inform educators, business and government officials regarding policies and priorities. 1. Identify 5000-6000 ENT curriculum 2. Stake￾holders are part of process vetting curricula 3. Brainstorm possible experiential applications 4. Solicit business participation (e.g. practicums; motivational talk series) 5. Bridge theory with practicum via stake-holder meetings 6. Assess/ evaluate the process onsite 7. Make recommen￾dations. Adapt. Adjust. 8. Refine the Design (On-going evaluation) CQI as the basis for pivoting SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 28 | P a g e RFA and Curriculum Design As per the Request for Applications, the notion to broaden the awareness/ cognizance, engagement and positioning of entrepreneurship in Barbados and the OECS should start via an educative platform. As the Country of Barbados has an extremely high literacy rate, it is a logical step to develop program choices and upgrade teaching methodologies and principles for various technical, vocational and professional job sectors. There are several institutions that focus on entrepreneurship in the United States. Their programs are designed to incorporate a number of business components such as: management, entrepreneurship, corporate innovation, human resource management, organizational behavior, organizational theory, strategic management, consulting, leadership, and international business. Literature suggests that the accomplishment of the goals of the JOBS partnership were dependent on alignment of curriculum design; for example: (1) To provide faculty capacity building; (2) To develop an entrepreneurship concentration for graduate students, executive education certificates, undergraduate degrees, and an entrepreneurship minor for undergraduates; (3) To establish a business incubator program that will consist of incubator training, a business incubator, a venture forum, undergraduate internships, and a mentoring program; (4) To develop research techniques for faculty so that case studies can be disseminated through an online portal; and (5) To develop a project monitoring, evaluation, and sustainability plan. All five goals are unachievable if the curriculum design is not aligned with the social, environmental and economic factors of the country. Entrepreneurial ventures (and venture capital companies) are not a part of the national fabric for Barbados or the OECS. Currently the unemployment rate is 13.2%. Barbados has approximately 250,000 people, with the majority being females (10,098 more than males). The GDP is $7.2B, but the inflation rate is relatively low (Barbados Statistical Service, 2014). SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 29 | P a g e The government worked with various international agencies and functionaries to allow for a more open and friendly business-opportunity environment; in essence, to stimulate and entice business growth and entrepreneurship. Based on the Terms of Reference and the RFA, it was overwhelmingly obvious that the government has looked to the educational sectors for the spread of entrepreneurship, and thus, (given time, effort and dedication), the reduction of unemployment and the increase in GDP and exportation. DACUM Process A typical process at key junctures while engaging business and industry stakeholders is known as the DACUM process (Developing a Curriculum); a tool that can be used to establish skills standards, competencies, program requirements, and course sequences for an institution’s program(s). Using the DACUM process increases the relevance of academic programs by aligning skill standards to required job competencies and to better prepare students for employment. First developed by Ohio State University in the 1970s, DACUMs draw information from industry to determine the skills and knowledge employees in various positions will need to carry out their duties. Highly skilled, current employees are involved to determine the nature of their daily work and the type of experience and knowledge they utilize. Educators then use the information obtained to create the courses necessary to provide the skills and training needed. The DACUM process can successfully assist in the design of certification, licensure, and degree programs at technical schools and universities throughout the OECS. The following schematic offers entry and exit strategies for curriculum delivery post-DACUM process: SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 30 | P a g e FIGURE 2 Transition/Exit Model from Secondary Education to Post-Secondary Education MA/ ENG/ Bus. Mgt. Components Transition Model (e.g. MBTI test) Transition/ Exit Model from Secondary Education  Post-Secondary Education (e.g. career education, certification, undergraduate, graduate) Career Education Program Level 1 Business Management Component Certification/ Certificate Program Level 2 MA/ ENG/ Bus. Mgt. Components Undergraduate Program Level-n UWI/ CHSB *This model consists of multiple exit points. The function of the model may be modified based on a given career path. (interest survey) Business Track Platform (repeat interest survey) reinforced *discovery UWI/ CHSB Business Training Programs (e.g. career ed; certification; undergraduate, graduate ) (with MA/ ENG Bus. Mgt. components relatable to program selected) Barbados Secondary Ed.: Gen. Ed. MA/ENG SCI/Hist LOTSHOTS (With emphasis and offerings for business coursework) knowledge knowledge discovery reinforcement Gauge community interest; bus. refresher; CHSB marketing platform = = Non-degree seeker; proficiency in entrepreneurship; business outreach component; additional CHSB marketing platform = Academic degree seeker; concentration in entrepreneurship; business outreach component; additional CHSB marketing platform Graduate Program Level-n UWI/ CHSB = Graduate degree seeker; expertise in entrepreneurship, management and finance; business outreach component; additional CHSB marketing platform = Continuous Quality Improvement (CQI) Taxonomical Progression  LOTS: Lower Order Thinking Skills HOTS: Higher Order Thinking Skills SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 31 | P a g e A compilation of literature reporting on essential components for systems of institutional effectiveness (Baer, L., et al, 2008; Bergquist, W.A., 1992; Chance, S., 2010; Norris, D. M. et al 1991) identify systems for curriculum design and redesign as sharing five components that are systematically connected as a comprehensive process for continuous quality improvement (CQI). The five components are (1) Strategic Planning, (2) Annual Planning, (3) Department Planning, (4) Budget Planning, and (5) Evaluation/Assessment. The system is designed to be participatory and cyclical. For each component of the planning system, a participatory structure should be designed to ensure broad involvement of the stakeholders. All areas of the institute, including all academic programs and support areas, should be included; this will translate to a much more unified approach for curriculum delivery, and all participants will share a common goal and vision once the process is completed. Actually, the process repeats itself cyclically, but the initial process is all￾inclusive. Participants self-identify if they choose to stay on the curriculum committee or others associated with curriculum design and deployment. Pedagogical Practices Certain key questions are constantly looming when an institution or educational system starts to inwardly look at how students are engaged in learning; how tutelage and reinforcement is delivered; how a particular content needs to be shaped relative to a society or region? As stated when using the DACUM process as a tool, there are certain questions to be answered for the curriculum to be shaped, and simultaneously, have stakeholder invested whilst the process is taking place. The following questions are typical in the process of forming pedagogical practices for a given discipline, and many other questions would be borne out of collegial exchanges and open dialogue:  What are the range of qualifications and programs being offered in each institution to promote access and progression? SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 32 | P a g e  How well prepared are students for entering into tertiary education in terms of basic technical entry skills, core skills and literacy/numeracy?  What teacher competencies and capabilities in terms of entrepreneurial content and business management do professors and instructors hold in order to effectively teach the subject?  To what extent does the OECS system allow for effective management of institutions in terms of responsibility for internal quality assurance, curriculum delivery, finance and staff recruitment and development? (How well is senior staff capabilities aligned in terms of knowledge, understanding, skills and attributes to provide professional leadership?)  Have the main challenges been identified and analyzed for universities to understand their capacity in order to meet demand? (What strategies and investment are needed to deliver the proper training in the field?)  Has the existing teacher training practices and methodologies been analyzed to suggest changes in line with the adopted entrepreneurship curriculum? Pedagogical strategies may vary from institution to institution, from country to country, region to region, but how we learn will always be via V.A.R.K: visual, aural, reading/writing, kinesthetic (Fleming and Mills, 1992). Pedagogical methods that involve differentiated instruction have been found to assist with retention and reinforcement. Assignments should move students from the lower levels of Bloom's Taxonomy of Learning Domains (which is to remember and understand) to higher levels (to apply and analyze), and finally, at its zenith (to evaluate and create). The instructional designer and instructor need to be deliberate about helping students achieve higher levels of learning, perhaps more so than in the face-to-face environment where opportunities for discussion help students achieve higher levels of thinking more spontaneously (Barbe, Milone & Swassing, 1979). Learner goals might be better served when students contribute to a common project, such as a group blog, where students can post work items from their communities that are relevant to a topic, concept, or theory being discussed in the course. Because entrepreneurship does not occur in a vacuum, neither should the teaching and learning dynamic; for example: those who already are trying to move an entrepreneurial idea SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 33 | P a g e forward do not usually do so without the input of others (whether subliminally or intentionally). The idea takes place due to a litany of psychological, social, cultural and environmental factors; this is a universally accepted construct that lends itself to pedagogical strategies and implements, and also flows through the learning curve as an idea develops. Therefore, in the world of work we learn through best practices and information we internalize from many aspects; and in many cases, we learn from those around us; we do this routinely in the business world and it’s known as collaboration. Hence, there are eight sound pedagogical practices when teaching curriculum in general, and undoubtedly entrepreneurship curriculum: 1. Instructor collaboration (peer-to-peer review). 2. Coordination and cooperation with the instructional designer/curriculum expert. 3. Ensure the learning objectives are proved out via the anticipatory – that they are answered at the lesson’s conclusion (Madeline Hunter, 1967). 4. Be cognizant of how to best engage students with the material (motivational sets) 5. Use differentiated communication techniques. 6. Consistent messages should be evident (e.g. outcomes should be predetermined). 7. Reduction in the amount of online lectures. 8. The curriculum should evolve and therefore be altered and revised routinely. Pedagogy, learning methods, and their implements are an evolving and ongoing choreography. But when teaching entrepreneurship, the teaching methods will rapidly correlate to the Higher Order Thinking Skills (HOTS) given the content and critical thinking strategies one must engage in order to develop sound business practices that guard against investment failure and high risk. Thus, entrepreneurship pedagogical practices are aligned with key critical thinking strategies. Critical thinking is metacognitive in that it involves thinking about one’s thinking. It begins when one can reflect on his/her cognitive strategies, and ask key questions that are the basis of critical thinking strategies: understanding that SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 34 | P a g e certain questions need to be addressed; that the practice of self-rationalization occurs (and being as objective as possible); and to authentically believe in the results of our rationalization (Nosich, 2012). It can be argued that those who take their idea to market are already critical thinkers. But the foresight of recognizing that an academic component is necessary (in most cases) to hone and build upon that idea is paramount as it will equate to a better economic model that transcends entrepreneurial tendencies and translates to an entrepreneurial culture. Evaluation Methodology Study Design and Management The project was designed to gather data from JOBS project stakeholders and to assess the: (1) efficiency of external stakeholder engagement, (2) relevance of external stakeholder engagement and the intervention content, (3) effectiveness of the intervention, (4) potential for lasting effects, and (5) recommendations for partnership sustainability. Based on the desk and literature review by Dr. Sofianos and Dr. Fellner, an evaluation methodology and data collection plan that aligned with HED’s evaluation needs was proposed and presented to HED for review prior to conducting activities in the field. The evaluation methodology was a mixed methods approach, described by Johnson et al (2007) as the combination of elements of qualitative and quantitative methods (i.e. such as qualitative and quantitative data collection, analysis, inference techniques) for the purpose of breadth and depth of project understanding and impact. The data collection plan (pre-field work) consisted of stakeholder interviews and focus group sessions. Once onsite, additional methods including participant observation and survey research were added to address perceived gaps in assessing project impact. Standards of qualitative research were employed in conducting this project. The interview protocol design, focus group questionnaire design, data collection, data analysis, and writing the report were provided by Dr. Theodore Sofianos, Edu-Sol International, LLC, SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 35 | P a g e Daytona Beach, Florida and Dr. Therese Fellner, Evaluation Research Services, Racine, Wisconsin. Staff from the Cave Hill School of Business assisted with scheduling interviews, focus groups, and arranging for facility space. Following onsite fieldwork, survey questionnaire design was provided by Dr. Fellner and administered by staff from the Cave Hill School of Business. Questionnaire Design Stakeholder Interviews. The interview protocol for the JOBS Project stakeholders was developed by Dr. Fellner and adapted accordingly to meet the needs of Higher Education for Development. Stakeholders answered questions regarding the efficiency of external stakeholder engagement, the relevance of external stakeholder engagement and the development intervention content, the effectiveness of the intervention, the likelihood of lasting effects, and recommendations for partnership sustainability. See Appendix 1. Project Beneficiaries Focus Groups. Focus group protocol for the JOBS Project stakeholders was developed by Dr. Fellner and adapted accordingly to meet the needs of Higher Education for Development. Stakeholders answered questions about JOBS Project awareness, motivation to participate in the intervention, the efficiency of external stakeholder engagement, relevance of external stakeholder engagement and the development intervention content, the effectiveness of the intervention, likelihood of lasting effects, and recommendations for partnership sustainability. See Appendix 1. Project Beneficiaries Survey Research. The interview questions for the JOBS Project stakeholders (JOBS Faculty Symposium, JOBS Mentoring Symposium, and the JOBS Train the Trainer program) was developed by Dr. Fellner and adapted accordingly to meet the needs of Higher Education for Development. Stakeholders answered questions regarding the efficiency of external stakeholder engagement, the relevance of external stakeholder engagement and the development intervention content, the effectiveness of the intervention, SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 36 | P a g e the likelihood of lasting effects, and recommendations for partnership sustainability. See Appendix 2. Sampling Design Stakeholder Interviews. JOBS stakeholders were identified through the desk review process and collaboratively with HED and Cave Hill School of Business (CHSB) representatives. Stakeholders were categorized as college stakeholders, specifically CHSB, University of West Indies (UWI), Indiana University (IU); government stakeholders, Barbados Investment and Development Corporation (BIDC), USAID Barbados, Higher Education for Development (HED); and student stakeholders. Stakeholders were contacted by CHSB and appointments were scheduled during the onsite visit. Drs. Therese Fellner and Theodore Sofianos conducted all the interview appointments with each stakeholder. Stakeholder interviews were scheduled for 60-90 minute blocks of time and conducted onsite at CHSB. Two of the interviews were conducted via telephone. Out of the 20 identified individual stakeholders, 11 (55%) participated in the onsite interview process. All data was digitally recorded at the time of the interview, and interviewer notations were made as appropriate. Project Beneficiaries Focus Groups. JOBS project beneficiaries were also identified through the desk review process and collaboratively with HED. Project beneficiaries were categorized as student, faculty, and small business beneficiaries. Project beneficiaries were also contacted by CHSB and appointments were scheduled during the onsite visit. Drs. Therese Fellner and Theodore Sofianos conducted all the focus group appointments. Focus group sessions were scheduled for 60 minute blocks of time and conducted at the CHSB-UWI. All data was digitally recorded at the time of the focus group, and interviewer notations were made as appropriate. Complimentary food and non-alcoholic beverages were offered during the 60 minute session. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 37 | P a g e Upon arrival, project beneficiaries were asked to read and sign a Participant Information Sheet prior to completing a Statement of Informed Consent. See Appendix 1. These statements informed participants that their participation was voluntary, they could choose to leave at any time, and that they agreed to have the conversation digitally recorded. Participants were assured that their conversations and identities would be kept confidential, and informed that the focus group sessions would center on their opinions and experiences of participating in a JOBS training event. Project Beneficiaries Survey Research. JOBS project beneficiaries were identified by CHSB staff through sign-in sheets and were categorized by their country where they participated in the training interventions. JOBS Faculty Symposium and Mentoring Symposium project beneficiaries were contacted by CHSB via email informing them of the project end and inviting their participation in a survey regarding their experience with the JOBS project. Using email addresses supplied by CHSB, Dr. Therese Fellner contacted 2012 and 2013 JOBS Train the Trainer project beneficiaries via email, also informing them of the project end and inviting their participation in a survey regarding their experience with the JOBS project. See Appendix 2. Data Collection Procedures Stakeholder Interviews. A total of 18 interviews were conducted with individuals representing JOBS Partnership Administration and Management, Students: Project Beneficiaries, Faculty/Instructors: Project Beneficiaries, and USAID Barbados and the Eastern Caribbean. Project Beneficiaries Focus Groups. A total of 7 individuals participated in 3 different focus groups representing Student and Small Business Beneficiaries through the JOBS Lean Launch Pad 10-week program, Lean Launch Pad – Opportunities for Exploration, and the MBA Student Consultancy Engagement project. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 38 | P a g e Project Beneficiaries Participant Observation. A total of 6 teams of Small Business beneficiaries representing 16 individuals were observed during the Lean Launch Pad 10-week program celebration. The event allowed for a closer familiarity with JOBS beneficiaries and the direct impact of the intervention. Project Beneficiaries Survey Research. A total of 12 emails with an unique survey link were sent to individuals who participated in and benefitted from the JOBS Faculty Symposium which garnered only 1 response; 40 emails with an unique survey link were sent to individuals who participated in and benefitted from the JOBS Mentoring Symposium which garnered only 3 responses; and 117 emails with unique survey links were sent to individuals who participated in and benefitted from the JOBS Train the Trainer program. Nineteen percent (19%) of the Train the Trainer participant emails were returned as undeliverable; Fifteen percent (15%) of the remaining beneficiaries responded to the survey. Management of the Data Stakeholder Interviews and Project Beneficiaries Focus Groups. Stakeholder interviews and focus groups discussions were managed by Dr. Theodore Sofianos and Dr. Therese Fellner. Dr. Theodore Sofianos transcribed all interviews and focus group recorded notes. Transcriptions were coded using an open-ended coding scheme allowing for themes and trends in the data to surface, followed by the search for patterns in the coded data to categorize them by similarity (things happen the same way); difference (they happen in predictably different ways); frequency (they happen often or seldom); sequence (they happen in a certain order); correspondence (they happen in relation to other activities or events); and causation (one appears to cause another) (Saldana, J. 2009). Project Beneficiaries Participant Observation. Participant Observation data was managed using the same coding and categorization schema as described for the Stakeholder Interviews and Project Beneficiaries Focus Groups. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 39 | P a g e Project Beneficiaries Survey Research. The survey research data collected was quantitative in nature and managed using designated URLs which included an evaluation link from the participant email solicitation. Data was transmitted electronically, kept private and confidential, and stored on servers located in the United States. Limitations of the Study The study focused on the perceptions of and experience with the Barbados JOBS partnership between IU-KSB and CHSB-UWI. 1. The study was restricted to the JOBS stakeholders as identified by Higher Education for Development (HED), Cave Hill School of Business, and the evaluation consultants. 2. The duration of the study was limited to the time period of June 1 - July 14, 2014. 3. Desk review materials were provided by HED although evaluation consultants also utilized open source and public domain content for data analysis. 4. Project progress reports were only available through March 31, 2014 as the JOBS partnership was still active when the evaluation study was conducted. 5. Onsite data collection was limited to 4 business days, June 19-20 and 23-24, 2014. 6. Offsite data collection was limited to 10 business days, June 30 - July 14, 2014. 7. Offsite data collection required project beneficiaries to respond to events and activities that occurred as long as 22 months in the past. 8. Evaluation of project activities is restricted to a maximum of 26 months out of the 42 month award period due to numerous project delays. 9. Stakeholder interview and focus group data only reflects those stakeholders available and willing to participate during the onsite data collection. 10. Evaluation consultants for the Summative Implementation and Outcome Evaluation of the JOBS Project were given a 27 day threshold to complete all work. 11. The findings, conclusions, and implications of the study were limited to, and based upon, the analysis of the data obtained from the desk review, stakeholder interviews, beneficiaries’ focus groups, participant observation, and survey research results. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 40 | P a g e Findings In order to be comprehensive in presenting study findings, the evaluation team linked the initial framework of human capacity building for faculty, institutional capacity building, and research initiatives to the performance objectives, project approaches, and project targets to fully assess JOBS Project fidelity, effectiveness, and impact. FIGURE 3 Linkage of framework to JOBS Project Evaluation Consequently, this Findings Section focuses on the alignment of identified gaps in the development of enterprise and entrepreneurship in Barbados and the OECS with original project concept, project targets and milestones for specified audiences and beneficiaries served (as detailed in the RFA) in comparison to the actual implementation and outcomes of project activities. Additionally, qualitative data gathered during the onsite fieldwork through stakeholder interviews and beneficiary focus group sessions is integrated with the aforementioned quantitative data evaluation to further contextualize the JOBS Project findings. Human Capacity Building Performance Objectives (e.g.project targets) Intstitutional Capacity Project Fidelity, Effecvtiveness & Impact Development of Enterprise and Entrepreneurship alignment of Identified gaps SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 41 | P a g e FIGURE 4 Framework for analyzing core JOBS project areas FIGURE 5 Relationship between JOBS objectives and project areas Human Capacity Building for Faculty •Faculty Development •Professional Development Institutional Capacity Building •Academic Programs •Short‐Term Programs •Internships •Mentoring •Business Incubator/Virutal Incubator and Portal •Entrepreneurial Training Series •Entrepreneurial Competions Research Initiatives •Case Studies •Publications •Fellowships Objective 1: Enhance CHSB's institutional capacity to serve as an entrepreneurship center for excellence in Barbados and the OECS. Human Capacity Building for Faculty Faculty Development Professional Development Institutional Capacity Building Academic Programs Research Initiatives Case Studies Publications Fellowships SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 42 | P a g e FIGURE 6 Relationship between JOBS objectives and project areas Human Capacity Building of Faculty JOBS Faculty Symposium. JOBS Partnership faculty and instructors from IU-KSB, CHSB, and UWI were considered key to the project, with the overall projected targets for Faculty Development seeking to impact the work of 10 faculty per year over the 42-month project, for a total 30 over the life of the project per the RFA. Summative evaluation demonstrates that the JOBS project accomplished this goal through the JOBS Faculty Symposium I (August 2013) which reached 43 participants and introduced faculty to concepts of the Entrepreneurial Mindset and Innovative Thinking (GAP #1), built awareness and capacity to incorporate entrepreneurial principles into their current methodology, and encouraged innovative thinking to nurture the entrepreneurial drive of the younger generation. However, no data was collected from any of the participants on intervention effectiveness and/or was impact data collected post-training to understand the application or use of knowledge gained from the JOBS Faculty Symposium. Although an attempt was made Objective 2: Strengthen entrepreneurship knowledge and skills among diverse stakeholder groups in Barbados and the OECS through outreach and engagement. Institutional Capacity Building Academic Programs Short‐Term Programs Internships Mentoring Business Incubator/Virutal Incubator and Portal Entrepreneurial Training Series Entrepreneurial Competions SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 43 | P a g e post-fieldwork to collect effectiveness and impact data from individuals who participated in the JOBS Faculty Symposium, there was such a low return rate which rendered data collected as unusable. Stakeholder interviews provided limited assessment as to the effectiveness of the Faculty Symposium as an intervention strategy to enhance CHSB's institutional capacity to serve as an entrepreneurship center for excellence in Barbados and the OECS. “The symposium was very successful with IU since the speakers were sourced to increase the entrepreneurship curriculum knowledge – how to-s and what works.” Faculty Professional Development. As a result of the project re-scope in February 2012, additional human capacity building activities providing professional development for CHSB and UWI faculty and administrators were incorporated into the project. The expectation was for faculty to attend professional development training, return to CHSB to disseminate the new information to other faculty, and add value to their knowledge, skills, and current teaching and research methods (GAPS #2, #4). The summative implementation and outcome evaluation demonstrates 12 CHSB and UWI faculty/administrators attended 9 professional development events in the US from February 2012 – June 2014. No data was collected from any of the participants to understand how faculty disseminated information to other faculty, and added value to their knowledge, skills, and current teaching and research methods. Stakeholder interviews, however, did indicate examples of knowledge transfer and value to the beneficiaries. “The faculty who went through the process benefited greatly. It gave a body of knowledge as to how and incorporate entrepreneurship in their courses.” “The partnership with CHSB had significant value to me as I was able to conduct several workshops on entrepreneurship to students in Secondary Schools, The National Organization for Women and the Ministry of Education (USAID Project) for youths. The project results can be described as follows: 1. Improved knowledge of trainers on business practices and management; 2. Increased in the number of persons of the benefits of entrepreneurship; 3. The development of business plans by students for review in a national competition; and 4. The capacity of business support SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 44 | P a g e organizations in St. Vincent and the Grenadines has increased by the training of key persons in entrepreneurship and business planning.” Table 3 Human Capacity Building of Faculty Focus Area Projected Target Activity Actual Target Gap(s) addressed Objective(s) addressed Faculty Development 30 Faculty Symposium 43 #1 #1 Professional Development US Events/Trainings 12 #2, #4 #1, #2 Institutional Capacity Building Project activities to build Institutional Capacity Development consisted of 6 distinct areas, namely the development of curriculum and upgrading of existing curricula, short-term development programs, workshops and certificate programs, a business incubator, student internships, and mentoring (GAPS #2, #3, #4). Academic Programs ‐ Development of curriculum in the area of entrepreneurship. There were five proposed academic program outcomes as a result of the partnership. The accomplishment of each outcome was evaluated independently, and later aggregated to assess the area in its entirety. A total of 250 students served by these new and upgraded academic programs was projected by Year Three of the JOBS project. The summative implementation and outcome evaluation demonstrates that the proposed curriculum development was not in place by the end of the JOBS project (June 30, 2014) and since the outcome of curriculum development was attached to projected beneficiaries to be served - the 250 students to be served by the new, upgraded academic program - minimal achievement was noted. Furthermore, the remaining four academic program outcomes were not achieved during the course of the project:  Establish an entrepreneurship module in its first-year orientation program.  Create an entrepreneurial minor for undergraduates. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 45 | P a g e  Provide creative learning experiences for faculty (as it relates to the development of curriculum in the area of entrepreneurship).  Development of an online certificate program. In 2011, the CHSB launched two new academic programs in business administration with a concentration on entrepreneurship, the Executive Diploma in Entrepreneurship and a Masters in Business Administration (MBA) in Entrepreneurship. Table 4 Institutional Capacity Building: Development of curriculum in the area of entrepreneurship Focus Area Projected Target Activity Actual Target Gap(s) addressed Objective(s) Addressed Design of a full concentration for graduate students. 250 students reached Masters in Entrepreneurship and Innovation 0 #2, #3, #4 #1 Development of an online certificate program. Executive Diploma in Entrepreneurship 0 #2, #3, #4 Establish an entrepreneurship module in its first-year orientation program. N/A 0 N/A Create an entrepreneurial minor for undergraduates. N/A 0 N/A Provide creative learning experiences for faculty. N/A 0 N/A However, due to lack of enrollment, the graduate program was redesigned as a MBA in Entrepreneurship and Innovation and submitted to the UWI Academic Board for approval in 2014. Stakeholder interviews offered some insight into the delays of developing curriculum and/or upgrading of existing curricula: “A diploma did not go forward because they took it to possible participants and it was not a sell. Market didn’t bear the fruit at the time or weren’t interested in the subject.” “Non-achievement in some areas was because they were beyond CHSB control for example, approvals for various programs…” SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 46 | P a g e “More marketing and thoughtful strategic positioning needed to happen. For instance, only one person signed up for the first MBA. Don’t just throw something out there, but do the research before doing the introduction of a program.” “Academic program was initially a debacle. Too fast of a delivery. Too soon. Little communication. IU was top down.” Short‐Term Programs ‐ development of short term programs in entrepreneurship. The short-term development programs included 4 distinct project activities, each with a separate scope of work and targeted number of participants (GAP #3). The overall projected targets for the workshops and/or certificate programs proposed to serve 200 participants a year by year three. Like the academic programs, the accomplishment of each outcome was evaluated independently, and later aggregated to evaluate short-term development programs in its entirety. The summative implementation and outcome evaluation demonstrates a total of 405 participants were served from January 2012 – June 2014 and considered this Institutional Capacity Development area to be accomplished. Student Consultancy Engagements. Student Consultancy Engagements provided another short-term development opportunity for both UWI and IU students. In 2013, a total of 10 MBA students from IU and Master of Science (MSc) students from UWI worked collaboratively on projects to assist start-up companies part of the UWI technology transfer program. Students engaged in experiential learning opportunities and gained exposure to operating in the international and Caribbean environment. The next year in March 2014, the second Student Consultancy Engagement took place and involved 17 student-consultant participants from UWI, IU, and Governor’s State University (GSU). Student teams were given the opportunity to apply their theoretical knowledge and assist five Barbadian businesses by giving sound business advice and guiding them onto successful paths. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 47 | P a g e Stakeholder interviews regarded the Student Consultancy Engagements as positive experiences for both student groups and small business owners and cited the short-term training as a valuable component of the JOBS Project that should be sustained. “In terms of how the students interacted, very good outcomes. As far as the businesses listening to the students’ recommendations, a dose of reality may have rubbed them the wrong way.” “Try to maintain a link with IU. Somehow maybe UWI could continue the exchange. Enormous value on both ends. International opportunities on both ends. Great to understand the economies and opportunities on both ends. Continue the facilitation and teams for both.” “Student engagements went above expectations. They were a huge value add for businesses who received consulting, and the teamwork that came out of the student repartee and how well they worked together was wonderful. It was a huge amount of responsibility and pressure. Hope that it will continue in some way.” “The student engagement piece was most important and needs to continue. Most businesses do not have the funds to hire consultants, and the students fulfilled that niche; they came in and made cognizant, successful recommendations. This process inevitably becomes an asset to steer a business in a different direction and will eventually help the company grow. It shows the value of the program by seeing a successful pivot by the business that benefited. It could also help the programs at CHSB grow.” “The opportunities the IU students had were so many more than UWI. The IU students were better prepared for entry. The peer-to-peer gaps did exist, but one of the biggest impacts was the ambition quotient: The IU students were good examples for the UWI students.” Lean Launch Pad. As the result of a human capacity building professional development activity in 2012 and 2013 attended by CHSB and UWI faculty, in November 2013 the JOBS Project offered The Lean Launch Pad: Opportunities for Exploration Workshop, designed to gauge interest in the LLP model. The workshop was attended by 29 persons and served as the culmination of Trepfest, Barbados’ first Entrepreneurship festival held during Global Entrepreneurship Week 2013. In March 2014, the JOBS Project offered the first Lean Launch Pad Information Session and Mixer held at the Cave Hill School of Business. The promotional campaign attracted seventeen (17) teams, fourteen (14) of which were selected to pitch to the panel of SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 48 | P a g e judges and observers, for a total of 47 individuals. A total of 6 teams of 4 persons each went on to participate in the 10-week program. Stakeholders believed the Lean Launch Pad program to be impactful and driven by CHSB’s identification of best practices and interest in replicating this model through the JOBS Project. “LLP is definitely going to leave a legacy…teaching (the) LLP method will be a lasting vestige.” “LLP was a good example. IU did not have any weigh in. It was all CHSB for that one. Most others, the content and direction came from IU. But both faculties were able to add and revise the curriculum.” “The Lean Launch Pad was added since it was useful in taking those who have ideas and taking it to the next level. It got people through this structured methodology so that they may begin a business.” “LLP was making informed and educated pivots. The process helped the team realize their downfalls as to review, revise, and come at it again.” “Through LLP, examples were set as to what needed to be done for product alignment – environmentally, socially, culturally; what permeates all cultures is when you bring a cultural piece of food – such as Indian, Mexican, etc.; therefore, need to source that need.” “The LLP was the one large item that has come out as a sustainable and laudable effort.” CASH Workshops. Although not provided as newly developed curriculum as suggested in the original RFA, the short-term training workshops did focus on topics of Cultural Entrepreneurship, Alternative Energy Initiatives, Social Entrepreneurship, and High-Tech Ventures (CASH). CASH 3-hour workshops were offered once in each focus area during May and June 2014, the final months of the JOBS Project. All workshops were held in Barbados and served a total of 35 individuals. Graduate Assistantships (GA). Another short-term development opportunity for UWI students launched from April-September 2013 were Graduate Assistantships (GA). These opportunities were offered to two post-graduate students currently enrolled at the University SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 49 | P a g e of the West Indies specifically to support academic and administrative functions of the JOBS project at the CHSB. Focus group and individual interviews of current and former GAs provided insight into their experience, particularly since the scope of work was predominantly administrative. For many students, being a JOBS Graduate Assistant (Custom Indicator #7) did provide some with “hands-on learning with entrepreneurship and educational development programs, including research, business management, events, and training opportunities.” The role was also designed to facilitate time management and organizational skills, both considered to be critical project management techniques. Students shared what they found valuable in GA program content. “Extremely valuable because I never worked in the business environment.” “Planning and strategy, project management came into play to a large degree.” “The program provided a great networking cadre of likeminded persons to connect and grow together. As well as grow the idea.” Other students had hoped that the Graduate Assistantship would provide more direct training in entrepreneurship and understanding of developing educational programs, as well as noting the dynamics of introducing new programs at the university level. “Learned about planning and organizing. Not the most entertaining job. Knew it was administrative job, but thought it was going to be more. In my view, it wasn’t content￾relevant.” “Explains why it’s so hard to garner innovation and support. Laborious process. Requires a lot of patience.” “Train the Trainer” workshop program. Over the course of the project, 150 individuals were trained in Barbados, Antigua, Dominica, Grenada, St. Kitts and Nevis, St. Lucia, and St. Vincent using the “Introduction to Entrepreneurship” toolkit created by IU￾KSB in 2011 for the JOBS Project and customized for use within the region by Paul Pounder, lecturer at UWI, and Brian Toppin, Program Manager at the CHSB. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 50 | P a g e Table 5 Institutional Capacity Building: Short‐Term Programs ‐ development of short term programs in entrepreneurship Focus Area Projected Target Activity Actual Target Gap(s) addressed Objective(s) addressed Short-term development program 200 #3 #1, #2 2013 Student Consultancy Engagement 10 2014 Student Consultancy Engagement 17 Lean Launch Pad: Opportunities for Exploration Workshop 29 Lean Launch Pad Information Session 47 Lean Launch Pad 10-week Program 20 CASH Workshops 35 Graduate Assistantship 11 Train the Trainer 150 TOTAL 320 Many stakeholders believed the Introduction to Entrepreneurship toolkit, that provided the content and curriculum for the Train the Trainer program, was a significant program accomplishment. “Ready-to-go, stand-alone training program adapted for Barbados and the OECS for cultural relevance. CHSB trained over 100 people with that toolkit. They also trained local colleges with that toolkit who had the capacity to put on those programs. They worked with the Chambers and other pro-business entities. The training was broken up into logical sections in order to educate the burgeoning entrepreneurs.” “Those who participated in the Train the Trainer workshop were also a success story and the fact that they learned how to use the toolkit was of their own impetus.” “CHSB and Indiana University provided a very good overview of the project within the islands and within St. Vincent and the Grenadines. In St. Vincent and the Grenadines, they collaborated with Invest SVG to create an awareness of entrepreneurship. The content of the training was very good and adequately represented the best practices in facilitation. The instruction of the training was of the highest quality and provided trainers within practical application and lessons learned.” Following the completion of fieldwork, survey research data designed to collect effectiveness and impact was sent to individuals who participated in and benefitted from the SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 51 | P a g e JOBS 2012 and 2013 Train the Trainer programs. Survey respondents attending the 2012 Train the Trainer program represented beneficiaries in Antigua, Dominica, St. Kitts and Nevis, and St. Lucia involved in a variety of job functions including business management, consulting, business development, and accounting. Participants rated the training as excellent (67%) and very good (33%); 50% rated the information as extremely useful and 50% as very useful; 67% believed their skills improved a great deal because of the training with 33% responding with a lot of skill improvement; 67% felt that at least one-half or more of the training content was new to them; and a minimum of 38 small businesses have been trained as a result of the intervention. The 2012 Train the Trainer program received a Net Promoter Score1 of 83%, which is consistent with “world class” products and services (Reichheld, 2011). Survey respondents attending the 2013 Train the Trainer program represented beneficiaries in Antigua, Barbados, Dominica, Grenada, and St. Lucia involved in a variety of job functions including advertising and marketing, business, consulting, education, and training. Participants rated the training as excellent (14%) and very good (86%); 33% rated the information as extremely useful and 50% as very useful; 43% believed their skills improved a great deal because of the training with 29% responding with a lot of skill improvement; 57% felt that at least one-half or more of the training content was new to them; a minimum of 35 small businesses have been trained as a result of the intervention. The 2013 Train the Trainer program received a Net Promoter Score of 29%, which is consistent with average products and services (Reichheld, 2011). 1 Net Promoter, developed by Satmetrix, Bain & Company and Fred Reichheld, is now the worldwide standard for organizations to measure, understand, and improve their customer experience. The Net Promoter Score, or NPS®, is a straightforward metric that helps companies and every employee understand and be accountable for how they engage with customers. Employees at all levels of the organization understand it, opening the door to customer-centric change and improved business performance. (www.netpromoter.com) SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 52 | P a g e Furthermore, the survey research also provided data demonstrating application and dissemination of knowledge, increased awareness, and intervention effectiveness and impact beyond Barbados to the OECS. This is the only JOBS Project intervention treatment evaluated inclusive of direct measures by which to assess the attainment of the project’s goal. Table 6 Institutional Capacity Building: Train the Trainer Focus Area Projected Target Activity Actual Target Gap(s) addressed Objective(s) addressed Introduction to Entrepreneurship N/A Barbados training 5 #5 #1, #2 St. Vincent training 5 Grenada training 5 St. Lucia training 9 St. Kitts and Nevis training 19 Antigua training 12 Dominica training 11 Grenada training 14 Barbados training 17 St. Vincent training 15 St. Lucia training 10 St. Kitts training 4 Antigua training 13 Dominica training 11 Internships. The overall projected targets for the student internships sought to serve 175 persons per year by Year Four of the JOBS Project. The summative implementation and outcome evaluation demonstrates a total of 12 student participants were served from January 2013 – August 2013 and considered this JOBS project activity to be minimally accomplished (GAP #5). Although originally envisioned to “develop a set of field-specific proposals—from institutes to venture capital funds—that would allow entrepreneurship to direct change and growth in culture and the arts, health care, education, and business,” the internships were more general in nature. In January 2013, a total of four UWI students completed the JOBS internship program having been paired with businesses that were entrepreneurial in nature, or SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 53 | P a g e with agencies that worked with entrepreneurs. Internships lasted four weeks, during the UWI semester break. A second internship program was offered in July – August 2013 to support increased knowledge in entrepreneurial skills affecting the overall business climate in the region (Barbados and the OECS). Although a total of eight (8) students were successfully placed at various small and medium- sized businesses in Barbados, only six (6) internships were completed. No data was collected from any of the participants to understand the impact of the internships on individual knowledge gain in entrepreneurial skills. Stakeholder interviews referenced the internship component of the original RFA as being too ambitious and not necessarily rooted in a solid understanding, yet still providing value to beneficiaries. “The original request stated that the project was to support 220 internships. But that became much less (the original RFA was too ambitious). There wasn’t enough interest or understanding to support that figure. But those who went through it were very satisfied.” Table 7 Institutional Capacity Building: Internships Focus Area Projected Target Activity Actual Target Gap(s) addressed Objective(s) addressed Develop a set of field￾specific proposals that would allow entrepreneurship to direct change and growth. 175 Internship I 4 #5 Internship II 6 #1, #2 Total 10 Mentoring. The overall projected targets for the mentoring program was to serve 150 individuals persons per year by Year Four of the JOBS Project. The summative implementation and outcome evaluation demonstrates a total of 71 participants were served from October 2012-June 2013 and considered this JOBS project activity to be partially accomplished (GAP #5). SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 54 | P a g e In alignment with the original concept, CHSB worked with the Barbados Youth Business Trust to develop and provide mentoring opportunities for students ready to launch their business ideas. Two JOBS Mentorship Symposiums were offered to practicing business mentors with varying levels of mentorship experience. The objectives of the symposium were to provide concepts, processes and tools to enable qualified mentors be more effective in mentoring emerging entrepreneurs; enhance the ability of mentors to advise and support entrepreneurs and entrepreneurial teams; and create opportunities for mentors to network and establish linkages. Stakeholder interviews described the JOBS Mentorship Symposiums as issue-driven and developed in partnership with the BYBT. “Mentorship symposium to develop (as per the BYBT) wanted to develop capacity based on mentorship (well attended). People got the opportunity for mentoring as a more formal way to learn from the program.” Table 8 Institutional Capacity Building: Mentoring Focus Area Projected Target Activity Actual Target Gap(s) addressed Objective(s) addressed Work with the Barbados Youth Business Trust to develop and provide mentoring opportunities for students ready to launch their business ideas. 150 JOBS Mentorship Symposium I 35 JOBS #5 #1, #2 Mentorship Symposium II 36 Total 150 71 However, no data was collected from any of the participants in order to assess the accomplishment of symposium objectives. Instead, participants responded to the value of program content, knowledge gain and/or benefit, usefulness of materials, quality of facilitator, newness of the information provided, and overall program effectiveness. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 55 | P a g e Table 9 Institutional Capacity Building: Mentoring Symposiums WORKSHOP Program Content Knowledge Gain/Benefit Useful Material Quality of Facilitator New information Program Effectiveness JOBS Mentorship Symposium II 70% 90% 84% 83% 58% 75% PEERS Mentorship Symposium 87% 87% 87% 100% 88% 100% Like the JOBS Faculty Symposium, an attempt was made post-fieldwork to collect effectiveness and impact data from individuals who participated in the JOBS Mentoring Symposium. A total of 3 beneficiaries responded (less than 1% return rate), producing a range of intervention impact and effectiveness from low to high. It is worthy to note, however, that participants shared symposium content with as many as 9 additional small businesses beyond the short-term training – demonstrating some application of knowledge and knowledge dissemination. Business Incubator/ Virtual Incubator and Portal. As a result of the February 2012 project re-scope, the business incubator initiative originally conceived to establish and/or refine existing business incubator programs in Barbados and in the OECS intending to serve 20 companies per year by Year Two of the JOBS Project, the business incubator initiative was shifted to become a virtual incubator portal based on findings of the baseline assessment. The virtual incubator portal was to provide access to tools and foster entrepreneurship networks in Barbados and the OECS. In January 2014, the BIDC was selected to participate in the JOBS Virtual Incubator Program. A Memorandum of Understanding was created and sent to the BIDC for review and signing. IU-KSB completed the portal for virtual incubator support and forwarded to CHSB in February 2014. Transition to a new host site was essential as the portal was being supported by a US service funded by the JOBS Project. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 56 | P a g e In March 2014, the CHSB JOBS team met with the BIDC to discuss the status of the MOU and facilitate the execution of the agreement. As a result, the MOU was expanded to include CHSB support to the BIDC in order to foster client companies’ participation in other business development programs. Following MOU signing, it was anticipated that the support portal would expand and incorporate additional partners in the OECS. As of June 30, 2014 the MOU was not in place. Table 10 Institutional Capacity Building: Business Incubator Focus Area Projected Target Activity Actual Target Gap(s) addressed Objective(s) addressed Establish and/or refine existing business incubator programs 20 companies N/A 0 Provide access to an #5 #1, #2 Information Portal resulting in increased opportunities for support of business start-ups. N/A Virtual incubator 0 Total 0 Entrepreneurial Training Series. The Entrepreneur’s Survival Series was offered in January – February 2012 and drew 85 participants (mix of entrepreneurs, budding entrepreneurs, current employees and persons working with agencies designed to assist entrepreneurs and promote entrepreneurship) and provided materials in practical approaches for establishing the real cost of a product or service and the determination of pricing strategies; cash flow management; global opportunities; and developing new marketing approaches to promote their businesses. Stakeholders cited the Entrepreneur’s Survival Series as an initial step in engaging entrepreneurs and small business owners in Barbados. “The Survival Series for entrepreneurship was designed to help people work the marketing end of the small business development; it also was to go over small business budgeting, pricing products, trade, and cash flow. CHSB charged a nominal amount for this series so that way it put a value on the program. A certification of completion at the end was given. This helped to start a connection with entrepreneurs and business organizations.” SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 57 | P a g e Table 11 Institutional Capacity Building: Entrepreneurial Training Series Focus Area Projected Target Activity Actual Target Gap(s) addressed Objective(s) addressed Entrepreneurial Training Series Entrepreneur’s Survival Series 85 #3 #1, #2 Entrepreneurial Competitions. Entrepreneurship competitions were another activity (and JOBS Project Custom Indicator) that experienced numerous postponements over the course of 18 months. Administrative and procedural issues were cited in progress reports as preventing the Business Plan Competitions from running as scheduled. The Young Entrepreneurs Competition (open to students at tertiary institutions in Barbados and the OECS) and the IDEA Competition (Business Plan/Feasibility Study, open to anyone) were scheduled to be held in coordination with the Barbados Entrepreneurship Summit in fall 2013. Although the Idea Competition did finally launch in Barbados and throughout the OECS (November 2013) and ran for a period of six weeks, no entries were received. Stakeholder interviews provided limited insight into the ongoing delays for the Entrepreneurship Competitions and reasons for failing to achieve projected target outcomes for the activity. “They (CHSB) had the entrepreneur competition and there were no applicants. Perhaps not a targeted population.” Research Initiatives (Case Studies) Per the original RFA, CHSB worked to expand its range of enterprise and entrepreneurship research initiatives with guidance from IU-KSB. Research was to reflect promising practices and documentation of successful models and approaches. Project targets focused on the delivery of three case studies, one semi-annual publication to include best practices, and 10 fellowships per year beginning in Year 3. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 58 | P a g e In March 2013, Dr. Patricia McDougall from IU led a case-writing workshop for selected persons from Barbados and the OECS. Participants were assigned to prepare and submit a case-study on entrepreneurship that could be used to support teaching. As a follow-up in August 2013, Dr. McDougall presented a Case Study Workshop that shared information on the Entrepreneurial Mindset and Teaching with Cases as part of Global Entrepreneurship Week. Members of the Barbados Youth Business Trust benefitted from the Teaching with Cases lecture, while trainers, UWI and CHSB facilitators and lecturers also benefitted from the Teaching with Cases lecture. A total of 40 individuals participated in this Institutional Capacity Development event. The summative implementation and outcome evaluation demonstrates the completion of six case studies and one case study inclusive of best practices submitted to the IVEY Publishing House for publication review. No fellowships in alignment with research initiatives as originally envisioned were reported as transpiring. Research initiatives for the JOBS Project were considered to be partially accomplished (GAP #5). From July – September 2011, baseline research on entrepreneurial activities in Barbados and the OECS was conducted by Systematic Marketing and Research Services, Inc. with report release in October 2011. No data was collected from any of the individuals writing and/or submitting case studies, or from any of the participants in the Teaching with Cases Lecture to understand the impact of the content delivery and the extent to which it influenced or guided future action and practice. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 59 | P a g e Table 12 Research Initiatives Focus Area Projected Target Activity Actual Target Gap(s) addressed Objective(s) addressed Entrepreneurial activities in Barbados and OECS Baseline research #4 #1 Case studies 0 Case writing workshop 7 Case studies 3 Delivery of case studies 6 Case studies 0 Teaching with cases lecture 40 Semi-annual publication 1 Submission for publication 1 Fellowships 10 N/A 0 Total 14 54 Project Administration and Award Management Given CHSB’s tenured relationship with USAID Barbados and previous awards directly routed to CHSB, the JOBS Project Administration and Award Management process and protocol was a new system and, according to stakeholder interviews, contributed to communication challenges. “Previous projects, the award went to Cave Hill, but in this case, the process was different. Funds were awarded to an US university, then came to CHSB. Anticipation was that there would be a collaborative process, and emphasis was on the milestones, exchanges, and building capacity. IU managing the project from the US and “overseeing” what CHSB was doing didn’t seem like the norm.” “Initially it (partnership) was cooperative…tension occurred due to financial issues with USAID (used to be that CHSB got the money outright). New model was that funds came through IU and therefore this was viewed as CHSB being subservient to IU due to this financial model.” “IU was contracted by HED and was structured for things to flow up from CHSB to IU to HED. Not the usual way. CHSB was on the ground implementer; IU provided content expertise.” “CHSB has had many other donor-funded projects (with USAID seed funding); this project was different in terms of working with USAID through IU. Prior, the relationship’s expectations and communication strategies were not a challenge.” SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 60 | P a g e “It was not a good workflow. Went from CHSB to UWI to HED. It was edited by IU, but the invoices flowed directly upward.” “Nuances were the primary difficulty. Certain expectations were evident on both accounts from both parties. They had a work plan, but CHSB was used to having a lot of flexibility (prior to new HED model) and some activities were very prescribed.” Upon review of the series of announcements regarding the JOBS project, it was also noted that there was variance from the initial RFA released on May 7, 2010 through the project re￾scope in February 2012. Table 13 demonstrates the shifts in award scope during the life cycle of the project. Table 13 Project Administration and Award Management Source Date Amount Duration Award Timeline Project activities end RFA May 7, 2010 $1.35 million 42-month January 1, 2011 – June 30, 2014* *Based on the results of a mid￾course assessment, in year three, funding for a fourth and fifth year of partnership activities may be awarded. September 30, 2014 Program Description August 16, 2010 $1.35 million 42-month January 1, 2011 – June 30, 2014** **48 month performance period to include 3 months for start-up (application review and sub cooperative agreement negotiation), and a 3-month period for close-out. June 30, 2014 HED Sub￾award with IU February20 11 $1.35 million 36-month Feb. 28, 2011 – December 31, 2013 December 31, 2013 HED Sub￾award Modification with IU July 2012 $1.35 million 42-month Feb. 28, 2011 – June 30, 2014 June 30, 2014 Discussion of Findings The framework of the Discussion of Findings is categorized into 4 distinct areas of evaluation: (1) Efficiency of external stakeholder engagement. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 61 | P a g e  Nature and extent of external stakeholder engagement in providing input on partnership implementation.  Nature and extent of collaboration and coordination with other collaborators.  Levels of satisfaction with services and products by all stakeholders groups. (2) Relevance of external stakeholder engagement and the intervention content.  Extent to which the intervention content was aligned with national and local priorities, funder strategies, and recipients’ needs.  Extent to which the program was conducive to local ownership.  Extent to which the intervention was contextualized culturally, socially and environmentally.  Extent to which the program reflected commonly accepted best practices in entrepreneurship.  Extent to which activities and outputs were linked logically to program objectives. (3) Effectiveness of the intervention  Achievement of outputs and outcomes.  Positive or negative and intended or unintended effects of the intervention.  Extent to which the results attributable to the intervention.  Major factors influencing the achievement or non-achievement of results. (4) Potential for lasting effects.  Likelihood of the impact as a result of the intervention.  Likelihood that the intervention effect will continue over time after funding has ceased.  Major factors that are likely to influence lasting effects of the intervention. Stakeholders interviewed represented a multidisciplinary group of college and university administrators, faculty, and students, as well as representatives from USAID Barbados and HED. Stakeholders answered questions regarding their role in the JOBS SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 62 | P a g e project, how the project worked, the engagement of stakeholders, project content and effectiveness, project impact, and partnership sustainability. Stakeholder roles and responsibilities included:  JOBS Project Management – CHSB  JOBS Project Management – IU  JOBS Project Management – HED  JOBS Faculty Trainer  USAID Authorized Organizational Representatives  JOBS Principal Investigators (PI)  UWI Faculty Members  CHSB Faculty Members  CHSB Administrators  CHSB JOBS Training Facilitator  Lean Launch Pad Facilitators  Student Engagement Consulting Faculty Leads Efficiency of external stakeholder engagement In order to evaluate the nature and extent of the project collaboration and coordination, data was collected from stakeholders on management practices and partnership implementation. First, stakeholders shared their perspectives on how the project worked, the roles and responsibilities of partners, and the coordination of project activities. Project logistics, roles and responsibilities, and coordination of project activities. “CHSB and IU developed an initial work plan. Once the project commenced, the work plan was re-drafted. Why re-drafted? Because the first one was not realistic. Needs were not congruent. Monitoring and evaluations were confusing.” “Record keeping was also an issue. Process was being designed while project was beginning. It isn’t possible to measure all outcomes from baseline since there was none.” SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 63 | P a g e “CHSB didn’t think IU would have a role in the management of the project. University of West Indies was supposed to be the beneficiary of the project.” “Based on the roles, CHSB was a very strong partner. Although they are very independent and could do a considerable amount on their own, they still benefited from IU’s guidance. But in other instances, not very much guidance was accepted. In the end, CHSB was able to customize and adapt the program accordingly. They assumed they were in the driver’s seat from beginning and knew what they were doing.” Partnership communication. As noted earlier in the evaluation, communication among project partners posed challenges to project implementation and ongoing delivery of intervention activities. Stakeholders discussed communication among project partners and the challenges they faced. “CHSB felt they were being treated as a pupil as opposed to a colleague. They did need the training, although not to the point where CHSB didn’t know the basics. “ “The project went through a phase of adjustments to understand each other. First year was a rocky road…The role was not as smooth as wanted…USAID Barbados was a bit of a referee to ensure that all parties would work together to understand each other’s ways of doing things…Both sides needed to understand cultural issues. All through HED filter. Ongoing dialogue occurred behind the scenes to ensure that all partners were working together.” “The cooperation at the outset could be typified as rocky. But regarding the change management side of things that was necessary to go through in order to have a proper orientation. In the beginning, it was like being force-fed by IU (very prescriptive). Horizontal communication was there, but not to the type and quality of what needed to be in place. The cultural norm for IU was to be more facilitative. It became apparent that CHSB was going to have to make the IU personnel understand that a top down model was not going to work at CHSB. It was also imperative that the investment of money be secured for change management on the front end. At first it had an element of artificiality.” “Majority on of the problems were at the beginning…in general, communication was good once it got going. They didn’t realize the cultural communication differences between each other.” “Cultural differences on both sides. Tenets, cultural mores, innuendos, and also the rollout was done kind of as you go (not well structured and/or communicated). Not as well planned out for CHSB as it needed to be. Needed more of a direction. Maybe it couldn’t be that way, but that could just be happenstance based on the different business cultures not understanding one another…also there was not enough flexibility.” SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 64 | P a g e “CHSB has had many other donor-funded projects (with USAID seed funding); this project was different in terms of working with USAID through IU. Prior, the relationship’s expectations and communication strategies were not a challenge.” “Year 3 was a more of a smooth year, but processes and systems were not in place up front. USAID system of doing things was very different than IU’s. The issues were embedded in the administrative qualities of other IU Project Managers.” “Turnover at HED didn’t help…” “Communication was definitely an area of weakness at the beginning. Cultural interpretations perhaps in error; they may be underestimated this dynamic’s importance.” Communication strategies that were used among the team that facilitated collaboration included various modes such as email, Skype, and teleconference calls. Several stakeholders underscored the importance of meeting face-to-face with all partners as pivotal points in the project. Staffing size, management meetings, and reporting processes were identified as factors influencing type and quality of partner communication. “IU, CH, and HED met in Miami in January 2012… The meeting exposed that there were times when intended communication patterns were not perceived as such; this was just a disconnect based on business practices and the intersection of communication practices…Midway point in Miami also served as a way to figure out planning whereby everyone could get together and discuss the issue; this was very productive, to move the time line around as to what had to be reframed. Talking it through face to face was very helpful.” “There were more face-to-face meetings in DC in 2013 between IU, HED and CHSB so they could sit and work out a foundational work plan.” “In January 2013, they (CHSB, IU and UWI) met in DC and (before that) in Miami – then things really started to gel and become connected. A term was devised: “AI” - Appreciate inquiry; this helped to understand how the confluence of communication needed to come together and built the understanding with CHSB, IU and UWI.” “They (core team of 3-4 persons) had monthly and specific management calls. They did it purposely to be collaborative. One team would set the agenda, and the other team would set up the minutes. And then flip it. They did this relatively early in the process to have a more collaborative relationship.” “Quarterly reports. Institutionalized the reporting strategies. Telephone and other communication devices for various discussions on deliverables.” SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 65 | P a g e “CHSB received additional personnel – 3.5 persons attributed to grant funds. This certainly helped with the load…Both parties needed to agree on a reporting strategy. Report submissions were not outlined to CHSB. Billable hours were an issue. This did not translate well since time sheets were not tallied, etc. This was a cultural barrier.” “10 personnel of CHSB in 2011 went to IU to learn about how to deliver the program. It was very beneficial. There needed to be more mutual academic and programmatic exchanges from the initial portion of grant implementation.” “USAID helped get IU to work with more cultural awareness.” Stakeholder identification. Depending on the type of intervention being delivered, stakeholders were identified accordingly and by the appropriate JOBS project partner. CHSB and IU worked together to strategically engage UWI faculty. Press releases were designed to garner support, and organizations and agencies who would be in the position to support capacity building for the JOBS program were targeted for outreach. “Together they (CHSB and IU) sought out faculty in UWI who were entrepreneur￾focused, who were economics-focused and trained and/or specialized in entrepreneurship courses. They moved forward with engagement of business stakeholders.” “There were some key organizations (Barbados Entrepreneur’s Foundation, Barbados Youth and Business Trust) that CHSB had good contacts…and the faculty that were interested are the ones who were encouraged.” “He (UWI Faculty Member) had those contacts because he used to be an entrepreneur himself.” “CHSB helped to contact faculty members from UWI who were teaching entrepreneurship…helped to select business partners…liaised with the BYBT, the manufacturers association, and met with them to see how CHSB could serve the budding entrepreneurs and entrepreneur wannabees.” “Stakeholder identification was divided according to the program. For the Train the Trainer, stakeholders were Barbados and OECS oriented toward entrepreneurship and was self-identification for the most part; for Faculty Development activities, individuals were made aware of the event via marketing; for Case study work, faculty who were interested in entrepreneurship were approached by CHSB; for GAs, internships, mentorships, students were identified by CHSB and by individual faculty members.” Stakeholder engagement. JOBS Project stakeholders cited meetings and advisory committees as mechanisms the partnership had in place for external stakeholders to provide SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 66 | P a g e input into implementation planning and delivery of JOBS training, activities, and events. Additionally, external stakeholders were able to recognize shared missions of regional organization and opportunities for collaboration with the JOBS Project. “Input from stakeholders occurred through various meetings. Many stakeholders were not involved intimately before. Then they saw who was in the room and began the discussion. The local SBA recognized this was helping. IDB was also running alongside this project, FINPYME – was a series of training programs that was going on at the same time. They offered diagnostic tools, technical assistance, and the end result was that you could apply for a loan. Now there was a shared mission and vision by all stakeholders. Stakeholders saw a benefit. They in turn liked the Lean Launch program. And naturally they wanted to get involved with the FINPYME for loans.” “A stakeholders committee to (e.g. Program Advisory Committee) was incorporated and recommended for part of the program delivery.” In order to ensure that stakeholder concerns and priorities were incorporated into the JOBS Project implementation planning and delivery, CHSB played the lead role in transferring the input from meetings and advisory committees into program customization to meet the needs of Barbados and the OECS. There was a sense that consultations occurred for the short-term development programs and that information gathered was being used to tailor and customize content. “That was the role of CHSB. BYBT had a relationship with IU. Small enterprise with youth seemed to have a lot of street credit with the community. But how to use them in the model was with CHSB.” “IU took the lead in materials, but they (CHSB and UWI faculty) took the lead as far as the customization to each of the OECS countries. They adjusted the content to jive with the country… This took place locally as it should.” “A lot of consultation. A lot events needed to be coordinated. How the program should be tailored, fashioned was left to the academic experts.” When it came to identifying and selecting participants for the JOBS training events, workshops, and other program activities, the scope of the project to serve Barbados and each of the countries in the OECS provided additional challenges for the JOBS Project. Stakeholders interviewed noted the differences among the countries in the OECS and the need SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 67 | P a g e for each country to be served as a separate entity as compared to “the OECS.” Consequently, the belief surfaced that the most significant project impact would be in Barbados. The Lean Launch Pad was the only project activity where participation was contingent upon a competition-based structure. “Next steps were to identify participants for activities…with Train the Trainer, the businesses could nominate the individuals. But with CASH, anyone could come. Faculty was targeted. Lean Launch Pad used a “Shark Tank” model. All driven from CHSB.” “OECS…St Vincent’s is similar (to Barbados). Dominica…you need to look more toward entrepreneurship since they are so rural; there is no choice for them. All the islands want to have this kind of grant opportunity to find out what they should be centering on. Grant money is not new; but with JOBS it goes into real activities. But most impact will be here in Barbados.” “They (CHSB) drew people in by working with stakeholders through the BYBT. They tried to garner a list of mentors via a symposium, the CASH series, and the Survival Series. Interested business people throughout Barbados got involved.” “Used the team dynamic. It was like a pre-competition to see how the teams collaborated.” “Tried to target a group. Advertisements went out, registrations came in. LLP was really the only one where choosing was done.” “Very responsive. CHSB tried to find interest among the various communities and populations. They actively sought out the participants for the activities – but the problem was that target marketing may have been absent.” Relevance of external stakeholder engagement and the intervention content In order to evaluate the relevance of external stakeholder engagement and the intervention content, additional data was collected from stakeholders on the design and contextualization of program content. Although stakeholders believed the intervention content to be responsive to external stakeholder needs, there was uncertainty as to the true level of impact and value of the content delivered. “In terms of how the students interacted, very good outcomes. As far as the businesses listening to the students’ recommendations, a dose of reality may have rubbed them the wrong way.” SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 68 | P a g e “The value of what they were tasked to do was essential… this kind of model responds to the needs of business. Academia meets practice through this model. Great blend. And this was also a connection to UWI rather than just have it at CHSB. It opens up opportunity. It served student population, awareness, effectiveness of student consulting, but there is a need to make sure the businesses are getting the value and that the reports from business owners are mostly positive… Once the 4 facilitators came together and spearheaded the program in tandem with continuous quality improvement (CQI), it showed they were on the right path.” “(The responsive to stakeholder interests and needs was) pretty high. The key was the existing mindset of CHSB.” “Very responsive. There could have been additional surveys, but more effort was needed during the course of the program as to how the course was changing. If not compensated, faculty will be not dedicated. To do more activities, there was not sufficient speed for what needed to be done and evaluative measures were left behind. Project lost almost a year’s time and yet HED wanted the money to be spent.” “Very effective. Dissemination of program’s uniqueness and importance to the government’s direction was a huge plus. IU and CHSB knew that the content needed to be highly customized for the island population.” In terms of enabling local ownership, the perception of stakeholders varied depending upon which intervention was being delivered to stakeholders. Another variable cited was the stage of the business owner, specifically nascent entrepreneurs, solo-entrepreneurs or “cottage industry” entrepreneurs, micro-business owners (1-10 employees), and existing small business owners. The greatest impact on enabling local ownership, according to stakeholders interviewed, appeared to be with developing new and fresh business start-ups. “Most focus was on those who needed improvement as entrepreneurs. But it also focused on people who had an idea. Virtual incubator was seen as assisting with the latter. An Idea Competition was not very well attended and did not work out well. The Lean Launch Pad was added since it was useful in taking those who have ideas and taking it to the next level. It got people through this structured methodology so that they may begin a business. There was a distinction between quasi-entrepreneur and a new wannabe entrepreneur.” “LLP was making informed and educated pivots. The process helped the team realize their downfalls as to review, revise, and come at it again. What a particular product matters to the diaspora… students needed to take the steps necessary to understand those differences.” “Ideas took shape and form. Business ideas seemed to promote an excitement to build capacity and to create awareness about the program.” SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 69 | P a g e “It was certainly evident with the LLP. Great to have media involved. Using empirical values to show valuation and generated a lot of interest. It also generated some strong testimonials.” “It got people to engage in a new product or service offering. Support mechanisms were helped by the project and elevated CHSB because of the training modules. Via the project, there are more resources for area businesses already, but the initial start￾ups need the real mentoring and financial resources. The small start-ups and youth were the ones who benefited from better training and now have the info on the island.” “A lot of the activities of the grant were oriented to those who didn’t have yet a business, but were considering, but designed to bring them in… whether students are in nascent stages. Needed to understand how an entrepreneurship works and decide if that was the career path you wanted.” Alignment with national and local priorities. Overwhelmingly, all stakeholders believed the intervention content was directly aligned with national and local priorities. Many cited the impetus of the JOBS initiative as a direct response to the Barbados and OECS governments calling for investments in supporting and developing entrepreneurship. Furthermore, stakeholders regarded the JOBS Project as a successful strategy in building capacity by engaging educational institutions in a lead role. “(JOBS) was borne out of an escalating unemployment rate. The government recognized the need to quell those figures. BEF was trying to be a catalyst to build more, or attracting operations of external businesses to Barbados. The government put that into a national document. JOBS idea was to strengthen the educational outlets roles for entrepreneurship. The question was how to increase this capacity, and that goes right into the theme of JOBS. The project did align with national priorities. Overall, we achieved what was needed.” “It (JOBS) was in sync with national priorities. Government has weighed in and been supportive. The governments of Grenada, St. Lucia, and Barbados were supportive of the entrepreneurship project. Barbados was to be the hub, and this initiative (JOBS) was seen as the key and way forward.” “Yes, without a doubt. The project did align with national priorities. Achieved overall what was needed.” “Definite alignment….It will aid in reducing the failure rate via education. Anything on a government level should be to facilitate a more educative, proactive, and less fearful pro-business environment.” SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 70 | P a g e “It (JOBS) is very much so (in alignment). Government thrust is for entrepreneurship, especially with the recession. Supporting more entrepreneurship and innovation. We must be focused on educating our country to become employers and not employees.” “The project did align with national priorities via entrepreneurship. Perhaps the government doesn’t get what entrepreneurship actually is. It’s not just one person selling nuts on the side of the road, but that it’s growing scaling a business upward to employ and be a small to large scale business on the island. How to really support entre in a meaningful way is the goal. It’s not just an upstart, but how to sustain and maintain and support the business owner.” “The project did align with national priorities very well. Achieved overall what was needed. This was noticed from the BIDC (as filtered through the partners here) and BYBT, for they have given kudos.” “To a great extent. The RFA was set up so it would be 100% responsive to the partnership’s effectiveness exploring job entrepreneur opportunities. They had to mesh with local entrepreneurs with academic partnerships and then to guide them through a process for the objectives to be realized.” Project fidelity. When considering whether program activities were implemented according to the original design and intention of the RFA, stakeholders identified the necessity of being able to “pivot” as a result of the realization along the way that the existing environment was not conducive to (small) businesses in general, and that project partners were either not prepared to implement project activities or did not have the infrastructure (i.e. adequate personnel) in place to do so. Furthermore, the communication challenges and subsequent delays in activities referenced earlier in this evaluation were also considered to effect overall program implementation. “The biggest flaw was the business plan competition. They ran newspaper articles. But didn’t have an attractive prize. In retrospect, no business plan competition came to fruition. Timing was also an issue. No compensation was a detractor.” “Time was a problem in order to get certain things done. And in some cases there were added benefits…some things were discovered that were not part of the original plan (i.e. CASH series) that the participants benefitted from.” “Would have been more likely to succeed (in implementing project activities) if had hired additional personnel.” “USAID selected CHSB, but were they ready? The re-scope acknowledged the fact that there were not students on campus, even though faculty was abundant and ready. And this was because of the structure and that CHSB was not an equal to KSB, and it SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 71 | P a g e created tension due to this. It reverberated forward into the re-scope... Perhaps if they had more confidence working together, they could have done things so the 6-12 months wouldn’t have been lost. But that wasn’t the case.” “In terms of things that work, methods and strategies of things that have worked have been implemented: Ideas were reworked and then established. As they reached the end of the financial report, accruals and projections come into HED with significant spending shortfalls. Needed to spend. Now IU and CH had to merge that into the following year. Re-budgeting exercises were of the norm, but we didn’t have to redo the work plan, just move the timeline.” Industry Best Practices. As presented in the Literature Review, entrepreneurship and entrepreneurial curriculum is not as prevalent in mainstream academia, and the selection of IU-KSB was purposeful in order to provide for the integration of commonly accepted best practices in entrepreneurship and entrepreneurial curriculum. Also referenced in the Literature Review is the international best practice in entrepreneurship and entrepreneurial curriculum that demonstrates work-placed or experiential learning considered to be essential in gaining work experience in today’s job market. Stakeholders interviewed universally agreed that the JOBS Project was indeed rooted in best practices and cited IU-KSB as the project partner and the JOBS faculty development project activities as the primary resources. Although framed around best practices in the discipline, there was still a need for the intervention to be contextualized culturally, socially and environmentally for project beneficiaries. All customization took place at the local level with CHSB. “The project gets an A+…best practices were used to support the training of staff.” “(JOBS) initiatives showcased best practices. What was needed was to increase faculty acumen in their discipline; this allowed CHSB and stakeholders to build skills and collaborate and not be in a vacuum. Shared visions and shared goals was the key to success. CHSB went to Darden and saw the model at the school; this helped (faculty) teach the content.” “IU was used to push through for the whole process (of best practices). Darden School’s learning sessions were also a testament to best practices for cultivating an entrepreneurial culture. Significant transfer of good sound approaches and CHSB benefited from these interactions. Darden was the school in NY that helped the team benefit.” SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 72 | P a g e “In order to design them (project activities), those who did design them certainly used best practices. In terms of things that work, methods and strategies of things that have worked were implemented. Ideas were reworked and then implemented.” “(Project activities) aligned with the core values of entrepreneurship.” “High value. Using cutting edge content and training methods to address original objectives has been done.” “Reflected it (best practices) rather well. The idea of taking coaching development as an entrepreneur and bring it through the business plan fits best practices in this area.” “Through LLP, examples were set as to what needed to be done for product alignment – environmentally, socially, culturally; what permeates all cultures is when you bring a cultural piece of food – such as Indian, Mexican, etc.; therefore, need to source that need.” “The project did do a good job of putting best practices in place at CHSB.” JOBS Project activities and outputs were regarded by stakeholders to be logically linked to program objectives, namely to enhance CHSB's institutional capacity to serve as an entrepreneurship center for excellence in Barbados and the OECS, and to strengthen entrepreneurship knowledge and skills among diverse stakeholder groups in Barbados and the OECS through outreach and engagement. However, it was also noted that although activities and outputs were linked to program objectives that the necessary metrics to validate the linkage and subsequent effectiveness of project activities was a consistent gap throughout the project. Once again, stakeholders cited the project’s administrative challenges and restructuring as impacting project activities and outputs. “At no any point were there real metrics. Mostly anecdotal. But immediate impact was noted as a treatment via tangible application.” “Yes – certainly (activities and outputs linked to program objectives).” “Yes. Because a lot of time was spent through planning that that would occur. Barriers were removed in the Miami re-scope.” “Fairly well logically linked. When meeting with DC, CH and IU, we were to deal with new system (results framework) for a better roll-out for the project. IU did develop the draft for this. It never got fully completed. Focus was on activities.” SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 73 | P a g e “Yes and no. After the 1st year, they went thru a big restructuring for monitoring and evaluation purposes. This was something they had to communicate to overcome. The old school partnership started with a previous system. But the new program plan had to deviate from that old school of thought. And so they tried to use performance monitoring-type of measures to ensure program succession, sustainability, progression, etc.” Stakeholder recommendations. Recommendations by stakeholders as it related to intervention content focused predominantly on the short-term development trainings, workshops, and seminars and underscored previous findings regarding the range of needs for the target beneficiaries (nascent entrepreneurs, solo-entrepreneurs, micro-business owners, and existing small business owners). Additionally, stakeholders reiterated the need for content to be contextualized culturally, socially and environmentally for project beneficiaries. “Program content with regard to cultural, environmental, and social dynamics - adaptations (to content were necessary) with tax systems – Value Added Tax (VAT). No LLCs here; different business laws. CHSB provided supplemental indigenous information.” “The symposium was very successful with IU since the speakers were sourced to increase the entrepreneurship curriculum knowledge and how to implement a program for entrepreneurship.” “(The) university system is different in Barbados. Traditions are different. Socially different. Differences should have been met sooner in the process of negotiation. But it was not.” “No recommended changes for the Student Consultancy Engagement except UWI students needed to be better prepared.” “Could have perhaps given some different versions of the training material (i.e. Cash Management, sticking it into a cigar box, a bank, and finance opportunities). Perhaps levels of curriculum. Could be too basic or too advanced. Not just one prescription.” “Even though the programs were developed from IU, they were given to CH to review and edit the “template.” CHSB reworked the curricula and had to make adaptations to things to make them more relevant to the island nation’s capacity, mentality, etc.” “Nuances and differences around the island nations were profound, and that needed to be discovered by IU. IU thought initially they would give the curricula to CHSB, and the other satellite islands. They (OECS) are not part of Barbados and IU needed to understand that. And each template was to be re-worked according to how other OECSs operated. IU was framing it differently and didn’t see that there was NOT a SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 74 | P a g e seamless transition between curriculum and other cultural, social islands issues would factor into the equation.” “For LLP, CHSB followed the script from the education handbook but had to adapt the script to deal with the passive aggressive persona of Caribbean students. They are more inhibited. Presentation skills should be moved up in the rotation of curricula.” Effectiveness of the intervention Due to the lack of evaluation measures assessing project activity impact and effectiveness throughout the course of the JOBS Project, it was imperative to collect qualitative data that specifically addressed intervention effectiveness and project results from stakeholders directly involved in the project. Through the lens of value, achievement of results, project effects, and impact, stakeholders provided their perception of the effectiveness of the intervention. Project accomplishments. Stakeholders described numerous value-added attributes of the JOBS Project that were directly related to USAID standard indicators (curriculum new or revised, research/joint projects, short-term training) as well as custom indicators (training materials developed/adapted, short-course non-student/faculty participants, faculty coached in case-study development and use). Identified project value also supported the JOBS Project objectives of enhancing CHSB's institutional capacity and strengthening entrepreneurship knowledge and skills. “Jointly created curricula and a way forward. The partnership jointly delivered reports and solutions as well in some cases. CHSB did fine on their own for delivery of content. And the case writing workshop worked.” “There was immense value to the institution and to who we serve. A center of excellence for entrepreneurship. The other value is internal resources. The relationships that were created helped for notoriety (of CHSB’s courses offered and the element of quality) based on what has transpired. The project has brought about a measure of respect and thinking as to how people see this as a cutting edge school and destination for business courses.” “For students, this project opened up some opportunities. For current entrepreneurs, they now see CHSB as a resource and networking. For (external) stakeholders, through the project there was a deepening of the relationships. Networking was a key offshoot of the project.” SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 75 | P a g e “CHSB is now set apart due to connectivity with the private sector and overall community. Perceptibility and positioning has helped immensely.” “Knowledge (was a big value). One (case) study addressed issues of how to expand entrepreneurship in Caribbean and the fear of business ownership (in the Caribbean). The country’s psychology is a big part of how businesses are aligned. Opportunity is when skill meets luck.” “Faculty development (was of significant value). Attendance at NCIIA and LLP…Team of 4 went and saw how the process (LLP) could be transferred. All (CHSB and UWI faculty) helped to get the LLP off the ground.” “It took some time, but students (Student Consultancy Engagements) got the message. They also needed to look at licensing, getting product into different markets, and beginning to understand the message that export is a part of the process. Part of the need is to tie in those diaspora markets… in trying to get students to see opportunities, and trying to expose them to the business model that is obvious to exportation.” “Try to maintain a link with IU. Somehow maybe UWI could continue the exchange (Student Consultancy Engagements). Enormous value on both ends. International opportunities on both ends. Great to understand the economies and opportunities on both ends. Continue the facilitation and teams for both.” “IU is a new international connection (for CHSB) and a potential academic partner. (Faculty Development provided) international experiences to faculty members and (Student Consultancy Engagements) provided superb experiential practices to students.” “The faculty who went through the process benefited greatly. It gave a body of knowledge as to how and incorporate entrepreneurship in their courses.” As indicated by stakeholders when reflecting upon project value, the outputs and outcomes considered to be achieved through the JOBS Project closely parallel the cited value￾added attributes of the JOBS Project. Specifically, the USAID standard indicators of new, research projects, and short-term training. “Curriculum new and new programs! MBA and entrepreneurship programs. Questionnaires online.” “Now a Masters in Innovation has been approved. CHSB is waiting to see how subscription level is.” “Finally – now have an MBA entrepreneurship program. Men may have some great ideas, but it’s been the high energy 18 year olds that don’t want to be in the classroom. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 76 | P a g e So the ideas was to offer 10 week non-credit courses to affect this population and to go to the vocational HSs and recruit, or from BCC.” “Outreach/extension = Got out of it what was needed. Training/short-term = Big plus. Got that done. New programs = The MBA program.” “Modest gains in research/joint projects. 3 cases so far. More to come. Now there’s an incentive to write a case since UWI will compensate.” “The outreach to garner more students to be interested for the GAs was not adequate. Small number as to how much they should have had within the JOBS program.” “In general, did everything, but to a lesser extent. Couldn’t meet target goals as far as enough of those interested in the Barbadian audience. Couldn’t meet up to what the demand that was proposed.” Stakeholders described numerous positive effects of the intervention, both in terms of programmatic outcomes for beneficiaries and relationally among JOBS Project colleagues. Unintended effects of the intervention also included positive results evidenced through the Student Consultancy Engagements. “Helped to build credibility of CHSB. CHSB already knows about business and entrepreneurship and has the tools and resources to achieve this. For those people who came to learn, they heard it from word of mouth. JOBS is also associated with IDB, and IDB has a good track record in building capacity and assisting with educational endeavors overseas.” “Collecting data for GEM has helped the public viewing CHSB in a different light given the CHSB stakeholders and the fostering of the CHSB current supporters.” “Knowledge to spread out throughout the island.” “JOBS has been talked about a lot. The business community knows about the JOBS initiative. CHSB and JOBS are almost synonymous. It was promoted and marketed along with the partnership with IU and it has elevated the discussion throughout many sectors.” “Relationship value for future friendships, business relationships, etc.” “CHSB Senior staff is really good. First rate. Very organized and connected (to local and regional communities/networks). Passion among the Bajsans to be creative but needs the support to follow through.” “Not expected to have the partners work so well in the end after all that had originally happened. In general, there are some definite benefits… but the negatives were all about the first year.” SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 77 | P a g e “Student engagements went above IU’s expectations. Huge value added for businesses who received consulting, and the teamwork that came out of the student repartee and how well they worked together was wonderful. It was a huge amount of responsibility and pressure. IU hopes that it will continue in some way.” When asked to what had proven to be the most impactful component of the JOBS Project, stakeholders agreed that capacity building, greater awareness, and partnership with UWI faculty were the most promising and rewarding results. “Country that is sensitized to entrepreneurship. Taken the cynicism and Skeptical portion out. Make it business friendly. Change policies are important.” “By building capacity, people will be affected by the contagion. Long term effects will stay with those who have been through the process.” “The great awareness. Importance of entre and innovation was on the periphery. This relationship has brought about a method and given a systematic approach to academia, business science, and experiential/ practical approach. A lot of retrenchments and shrinking of economy and this programs help with that.” “Global Entrepreneurship Monitor (GEM) experience will continue.” “GEM report impact needs to be highlighted. Considered this an area to be pre￾eminent for policy makers and other governments to realize what has transpired at CHSB-UWI with the collaboration of KSB.” “Teaching learning material that was left.” “Ability to take IU’s content back and use it was probably the biggest contribution.” “Short term training (Train the trainer, short courses). Exposure occurred regarding the program throughout the business community. This advanced the profile of CHSB. It increased awareness and gave the opportunity for engagement of BIDC, and other stakeholders. Created positioning of CHSB and substantial capacity building.” “The student engagement piece was most important and needs to continue. Most businesses do not have the funds to hire consultants, and the students fulfilled that niche; they came in and made cognizant, successful recommendations. This process inevitably becomes an asset to steer a business in a different direction and will eventually help the company grow. It shows the value of the program by seeing a successful pivot by the business that benefited. It could also help the programs at CHSB grow.” Academia engagement. As found in the Literature Review, the engagement of academia in strengthening entrepreneurship knowledge and skills for diverse stakeholder groups was expressed as one of the major factors influencing the achievement of results (and SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 78 | P a g e continued impact) of the JOBS Project. Commitment and persistence of JOBS Project Administrators and staff was also cited as a major factor in achieving results. “Academicians were engaged due to the contagious nature of passion for the program.” “As far as whole UWI campus, only 3-4 faculty. If government wants to drive it (entrepreneurship), they’ll need more personnel. And a fundamental need to have an entrepreneurship faculty at the CHSB.” “Persistence (all partners). CHSB had the option of opting out, but this did not happen. UWI faculty were intelligent and committed, but the personnel capacity was stretched.” “Building capacity has been impactful. Having an academic institution such as CHSB that is behind business communication and business growth goes a long way. Engaging the students authentically through this program and offer courses will benefit the island community.” Non-achievement of results. Factors considered to influence the non-achievement of results were regarded as “beyond CHSB control” and related to the project’s administrative challenges (delays in project implementation, staffing needs) as notated earlier in this evaluation. In particular, IU-KSB needed to understand the structure of CHSB (Executive Education, not traditional students, etc.) and the related implications for access to faculty. “Need dedicated faculty to the project – control of resources (enforcement of roles, deliverables).” “IU and HED didn’t realize that the nature of CHSB was Executive Education. There weren’t dedicated faculty selected from UWI or from CHSB to have that base, cadre, from the beginning.” “In general, the project was a bit too ambitious and perhaps too many ingredients in the mixing bowl. The measure of the interventions started too late as well.” “Not enough staff, and not enough resources pushed things back. Do they (CHSB) need more staff? Why not hire more? IU was holding purse strings though, and the lack of communication for hiring key positions didn’t help to bridge that gap.” Although stakeholders contributed to what they believed to be results and positive effects of the JOBS Project intervention activities, many also noted that there was a lack of SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 79 | P a g e outcome assessment to inform the project and follow-up with participants to understand use of knowledge. “At no any point were there real metrics. Mostly anecdotal. But immediate impact was noted as a treatment via tangible application.” “Application of knowledge not able to be assessed. There was a more robust assessment of intervention would have been appropriate. Distractions contributed to this gap.” “If job metrics were to be measured, it should have been done prior. Entrepreneurship is weak, lacking and therefore this activity engenders new skills.” “As far as a follow-up to individuals to talk to those in the Survivor Series, none done. As far any respondents for the Train the Trainer series, nothing formative or usable.” Potential for lasting effects Based upon implementation progress to date, stakeholders were asked what they believed was likely to be the long term impact as a result of the JOBS program. Data provided reinforced the accomplishment of one of the two primary objectives, namely to enhance CHSB's institutional capacity to serve as an entrepreneurship center for excellence in Barbados and the OECS, in addition to the pivotal role played by academicians – specifically CHSB and UWI. “CHSB rebranded in this area (entrepreneurship). More partners coming to CHSB. More opportunities for interested parties. GEM report excites business, government and education personnel.” “CHSB might be the umbrella for that unification of all pro-entrepreneurship relationships.” “Implication is that so many will have benefitted from CHSB programs, new resources, and entrepreneurial orientation. More business will have been created by those who came from CHSB.” “CHSB now has higher profile and more prominent status for entrepreneurship delivery and programs. It moved them up a few rungs.” “Students are taught to obtain public sector jobs, so this project was an attitudinal adjustment – counter cultural norms. Once this spreads, OECS will really own these business concepts and practices.” “The LLP was the one large item that has come out as a sustainable and laudable effort…Lifetime Management Technologies has also heard of the LLP series. These SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 80 | P a g e are entrepreneurs hearing about other start-ups that have heard the good and the bad of the program and the business landscape.” “Increased appetite for entrepreneurship. Coincided with the government’s transition to entrepreneurship. Academics get interested and get involved, it gets infectious. The academicians are the positive contagion.” “Momentum of academics are and should be involved. Once the buy-in took place, the role of the management was a major factor. Having that in the (CHSB) leadership (entrepreneurship mind-set and experience) was a bonus and factor.” “Once you get the academics excited and on track, they start to work on joint projects together and the method starts to expand. CHSB owns this now. The excitement is evident.” “Academics are not as open to change as people in business are. Not always the case, but with entrepreneurship – the drive and momentum for making something, building something, and the profits are there, people get excited.” “Once it was advanced via the academics, it went well. Relationships were cultivated throughout the project.” “Research methodology to show how academicians can work with practitioners, and almost become unified in the mission, to bridge the partnership between education, business, government, their funding arms, as pioneers for action in the entrepreneurship arena.” “Private Sector Association (PSA) is tied to the Chamber of Commerce. PSA is about improving private sector for economic growth. As part as the government mandate, since they academics were very focused on publishing, the strategic plan to have better outreach to the private sector and the community.” “There is an academic practitioner divide. Need to close the gap. The private sector will help to grow this rally for entrepreneurs.” “Could have a huge impact specifically on the economy…Having success in various businesses will give credence to education in entre.” “Improvements in intellectual capacity, curriculum. Students will have great relationships out of this. Personal and collegial.” “Solid lasting legacy for CSHB within its current space in the marketplace. Would need to go through a realignment with UWI to evolve to a research entity.” “Faculty’s awareness has been elevated.” Stakeholder perception of the likelihood of program effects continuing over time after JOBS funding has ceased produced varied responses. The Introduction to Entrepreneurship toolkit, used as the basis of the Train the Trainer short term development activity, was SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 81 | P a g e described as a tangible program effect as was the new curriculum (MBA in Entrepreneurship and Innovation, potential to integrate LLP). There was also a sense that the CHSB’s role in the GEM research and report would continue and was a direct result of the JOBS project. “Toolkits will continue to be used. Research directions will continue. GEM research. JOBS provided capacity building – CHSB is continuing the work. GEM will continue to receive articles from CHSB faculty.” “LLP will become (in some variation) a part of the curriculum.” “Curricula will live on. Knowledge will live on. Train the trainer model will be available. Some faculty will continue supporting the body of knowledge and curriculum development.” “Some aspects but not all. Both institutions (IU and CHSB) have a willingness to collaborate in various ways into the future. All the resources developed will continue to be delivered and offered. The raw materials are there. Certainly many aspects will be continued.” The continuation of JOBS Project activities–and their continued effects–proved to be more questionable for stakeholders. “Depends on funding. Incubator will require funding. LLP will require funding. Not going to be taught without funding.” “The outreach to garner more students for the GAs was not adequate. Small number as to how much they should have had within the JOBS program. A good outcome for CHSB - They do believe they need to use their own money to bring this program up to speed.” “They (CHSB) wanted to have a sampling of companies at different levels of implementation – and some would be able to pay.” “CHSB will have to charge. Money will come from tuition. Nominal amounts. Seek out foundations.” “Initially, faculty was to share learning, and then to build the case studies into their own coursework. It was a great experiment, but it looks like that the program won’t last without the proper funding.” “It (LLP) has to be paid for by participants, but that shouldn’t be a key issue since it’s of great value. But the faculty has to be compensated in order to be sustained for this program to continue. Once the train the trainer part has been completed, there has to have a level of compensation for the instructor to continue.” “The model will not be sustainable due to the funding drying up. There will be some real challenges. One thing realized: a real divide between an MBA program here (CHSB) and the US.” SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 82 | P a g e “Challenge for sustainability (will be) once the money dries up. If a business owner offers certain workshop/session, the question now is do we do it for free or is it a nominal charge?” “Rule of thumb, you can send a number of personnel down here (CHSB), but we have to have the specialists here to run the program. The model is very good that we’ve produced, but we need the opportunity to demonstrate Recommendations for Sustainability Recommendations for sustainability of the Barbados JOBS Initiative are presented using three reference frameworks, each with its own relevant value system and rationale for consideration supported by qualitative data collection. The first series of recommendations are consistent with original expectations found in the RFA issued by Higher Education for Development in 2010. Original RFA Expectations Relevance of Program Design Explore opportunities for collaboration with Barbados Community College. USAID and CHSB anticipated that the partnership would expand after the first year to include the UWI Department of Management Studies, the Barbados Community College, and others as appropriate to the partnership design and activities. While small gains were made with UWI, neither the desk review nor onsite stakeholder interviews indicated any outreach or collaborative partnerships with BCC related to the JOBS Project. BCC’s Industry Service Unit offers several certificate courses designed to assist business and industry, inclusive of application to entrepreneurship and small-business start-ups. Relevant courses include a Cottage Industries Program, Quick Books Pro, Taxation and Tax Management, and Web Applications Programming & Design. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 83 | P a g e Results-Based Management Continue to track the progress of the JOBS project (key demographic and psychographic measurements about participants. CHSB has the ability to continue engagement and track progress of the JOBS project through interventions such as the Train the Trainer and Faculty Research Initiatives (specifically the GEM Research) that extended beyond Barbados into the OECS. Financial Resource Planning and Management 4 th and 5th year of funding. In the original RFA, the foresight was present to provide funding for a fourth and fifth year of partnership activities based on the results of a mid￾course assessment, (“review of progress”) in year three. Given the many starts and stops of the JOBS Project which essentially abbreviated the partnership award by 18-months, recommendations for sustainability would have been to revisit the opportunity for a fourth and fifth year of partnership activities to ensure project impact and planning for continuation of interventions. Stakeholder interviews reinforced this recommendation. “Negative would be to end now. The project was to further the capacity of CHSB to make it the premier school of entrepreneurship in the eastern Caribbean. A shame that we’ll have no real hard data to see this through. More time was needed.” Stakeholder Interviews The second series of recommendations come from JOBS project partners themselves. Stakeholders were asked to provide recommendations they believed would improve the likelihood of JOBS program sustainability. Upon reflection, some stakeholders offered more conceptual recommendations while others were more specific and focused on the JOBS Project. Results-Based Management “Pursue first on a small scale, and if positive, work it forward and grow the program or coursework.” SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 84 | P a g e “To have originally and to have set up a sustainability plan – to think through that issue of evaluation and assessment from the beginning.” Stakeholder Engagement “Spend the time in the beginning to make sure that all parties are on the same page.” “Cultural comparisons are made and understood – especially on the front end. And the vision is shared.” “Need to look at internship opportunities that are seeded and continued. And to work on student engagement. The Pathways for the entrepreneurship model should be conveyed across the campus so that experiential learning is also taking place.” “Continue relationships with IU and other countries.” “There is a need also to partner with the Government of St. Vincent and the Grenadines for creating higher levels of awareness of entrepreneurship especially within the secondary school system.” “There is need for the offering of a certificate in entrepreneurship by partnering with training institutions and community groups in St. Vincent and the Grenadines.” Non-Financial Resource Planning and Management “Put more effort into marketing (thoughtful, strategic positioning) – (i.e., Master’s in Entrepreneurship and Innovation).” “Quickly repeat the programs for brand awareness. Do more consumer research for needs and market. Need to research what level and programs are best for delivery – with regard to the overall population.” “Significant issues that were academically challenging as well. What needs to happen is once building capacity with new ideas, cases, curriculum, your creating sustainability right there.” “Administrative and programmatic congruency and expectations are in alignment.” “Continued intervention by training and developing a cadre of professionals in the field of entrepreneurship given the limited available skillsets and training in this area.” Financial Resource Planning and Management “Development and IDB funds should continue, but be targeted.” “CHSB needs to have their OWN personnel such as faculty that can be utilized as (needed). CHSB dedicated resources (faculty – trainers, program delivered).” “Continue the Survivor Series and LLP. Those can continue through the proper sponsorship and through the private sector (and government sponsorship as well).” SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 85 | P a g e Summative Evaluation The final series of recommendations are comprehensive and a direct result of the summative implementation and outcome evaluation of the JOBS Project. In addition to the aforementioned recommendations, Drs. Sofianos and Fellner propose additional recommendations for consideration based on the findings of the summative implementation and outcome evaluation of the JOBS Project. Relevance of Program Design  Apply key elements of the curricula oval mode to the CHSB-UWI Masters of Entrepreneurship and Innovation degree.  Utilize the DACUM process in customizing entrepreneurship and business development training for Barbados and the OECS.  Design Student Consultancy Engagements as semester long, credit based experiential learning experiences for UWI students. Results-Based Management  Integrate standard-based education and performance based assessment to provide the basis for accountability in providing entrepreneurs and small business owners with training and support. Institutionalization  Explore opportunities to continue delivery of the Lean Launch Pad program for entrepreneurs throughout the business life cycle (nascent entrepreneurs, solo-entrepreneurs or “cottage industry” entrepreneurs, micro-business owners (1-10 employees), and existing small business owners). Conclusions A key approach to the collaborative agreement for the project would have been to promote institutional effectiveness by establishing internal processes for ongoing evaluation and assessment. An initiation of an annual planning cycle that had each administrative unit and academic department establishing annual goals and monitoring progress could have positioned the project better for successful monitoring and evaluation. Curriculum design was via a top-down model. But to CHSB’s credit, they valued the expertise of IU-KSB and began SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 86 | P a g e their own curriculum redesign for their entrepreneurship coursework; in reality, this is a process coupled to the institutional effectiveness process to focus resources on identifying areas where improvements could be made, setting clear objectives or targets for improvement, monitoring achievements, and providing feedback that identified best practices. Primary conclusions from the evaluation consultants Train the Trainer Program and Materials (Toolkits) were a value-added outcome for professionals in Barbados and the OECS and should be prioritized for continuation and funding, as necessary. The culturally customized model, rooted in best practices and a direct result of the partnership with Indiana University Kelley School of Business, exists to sustain the program through the University of West Indies Cave Hill School of Business network. Lean Launch Pad Program and Materials were a value-added outcome for entrepreneurs and small business owners in Barbados and the OECS and should also be prioritized for continuation and funding, as necessary. Geographic scope of work to include Barbados and all of the 6 countries of the OECS was ambitious and unrealistic when considering that each of the 7 countries have their own respective, independent governments and cultural nuances. “Culture will eat strategy for breakfast.” (Peter Drucker, n.d.). The greatest challenges of the JOBS Project were identified as rooted in cultural considerations, or lack thereof, among project partners. The quote by Peter Drucker underscores the importance of relationship building to thoughtfully account for beliefs, values, behaviors, and norms of internal and external partners prior to project implementation. Funding sources, as well as project grantees, need to play a formal role in facilitating the integration of cultural, social, and environmentally considerations. Evaluation mechanisms and tools need to be developed and in place prior to project implementation to ensure the ongoing assessment (formative evaluation) of objectives, SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 87 | P a g e outputs, and outcomes with the alignment of project goal(s) and scope of work. In addition, formal summative evaluations on a regular cycle should be employed to respond to intermediate measures of project impact and effectiveness. Engagement of local higher education institutions and faculty members (BCC, UWI) has the opportunity to advance the foundation laid by the JOBS Project, particularly the Lean Launch Pad, and link academics to the business community as pivotal to the entrepreneurial ecosystem of Barbados and the OECS. Several of the faculty members who received training through the JOBS Project and/or served in a leadership role have departed from UWI and CHSB, leaving knowledge vacancies for sustainability opportunities. Creating awareness of CHSB resources for entrepreneurs and small business owners is essential in demonstrating value and content customization for Barbados and the OECS to build ongoing interest and participation. Registration and recruitment initiatives must be evident. Strategic linkage to governmental priorities and funding support to facilitate the aforementioned conclusions would not only leverage the groundwork of the JOBS Project but also serve as the tangible foundation of the economic development vision for the region. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 88 | P a g e References Baer, L., Duin, A., and Ramaley, J. (2008). Smart Change. Planning for Higher Education, 36(2),5-16. Barbados Statistical Service (BSS). (2014). Retrieved July 3, 2014 from http://www.barstats.gov.bb/glossary/ Barbe, WB., R.H., Swassing & Milone, M.N. (1979). Teaching Through Modality Strengths: Concepts and Practices. Zaner-Bloser, Columbus, OH. Bergquist, W.A. (1992). The Four Cultures of the Academy. San Francisco: Jossey-Bass. Bygrave, W. D. & Zacharakis, A. (2008). Entrepreneurship. John Wiley & Sons, Inc. Chance, S. (2010). Strategic by Design: Iterative Approaches to Educational Planning. Planning for Higher Education, 38(2), 40–54. Crane, F.G. & Meyer, M. H. (2011). Entrepreneurship: An Innovator’s Guide to Startups and Corporate Ventures. Sage Publications, Inc. Drucker, P. F. (1985). Innovation and Entrepreneurship. New York: Harper and Rowe Publishers, Inc. Drucker, P. F. (n.d.). The Drucker Institute. Fleming, N. & Mills, C, (1992). Not Another Inventory, Rather a Catalyst for Reflection, To Improve the Academy, Vol. 11, Page 137. Hunter, M. (1967). Retention Theory for Teachers: A Programmed Book. Tip Publications Johnson, D. P. M. (2005). A Glossary of Political Economy Terms. Auburn University. Johnson et al (2007). Toward a Definition of Mixed Methods Research. Journal of Mixed Methods Research 2007; 1; 112. Kuratko, D. F. (2009). Entrepreneurship: Theory, Process, and Practice. Mason, Ohio: South-Western Cengage Learning. Miller, R. L. (1988). Economics. New York: Scribner Laidlaw. National Content Standards for Entrepreneurship Education. Retrieved June 30th, 2014 from http://www.entre-ed.org/Standards_Toolkit/curriculum_edu_issues.htm Norris, D. M. and Poulton, N.L. (1991). A Guide for New Planners. Ann Arbor, Michigan: The Society for College and University Planning. Nosich, G. (2012). Learning to Think Things Through: A Guide to Critical Thinking Across the Curriculum, 4 th ed. Pearson Education, Inc., Boston, MA. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 89 | P a g e Reichheld, F. (2011). The Ultimate Question 2.0: How Net Promoter Companies Thrive in a Customer-Driven World. Harvard Business Review Press; Rev Exp edition. Saldana, J. (2009). The coding manual for qualitative researchers. Los Angeles, CA: SAGE. U.S. News and World Report Best Colleges 2014 (September 10, 2013). Retrieved from http://colleges.usnews.rankingsandreviews.com/best-colleges SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 90 | P a g e JOBS Project Management Personnel List Cave Hill School of Business Dr. Jeannine Comma JOBS Partnership Director Cave Hill School of Business University of the West Indies Marjorie Wharton Director for Strategic Development Cave Hill School of Business University of the West Indies Audra Stevenson Drakes JOBS Partnership Coordinator Cave Hill School of Business University of the West Indies Indiana University Dr. Bruce Jaffe JOBS Partnership Director Kelley School of Business Indiana University Jonathan Crum JOBS Partnership Coordinator Kelley School of Business Indiana University Higher Education for Development Diana Páez-Cook Program Officer, Latin America/Caribbean Higher Education for Development (HED) SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 91 | P a g e Stakeholder Interview List Partnership Administration/Management Dr. Jeannine Comma JOBS Partnership Director Cave Hill School of Business University of the West Indies Dr. Bruce Jaffe JOBS Partnership Director Kelley School of Business Indiana University Audra Stevenson Drakes JOBS Partnership Coordinator Cave Hill School of Business University of the West Indies Jonathan Crum JOBS Partnership Coordinator Kelley School of Business Indiana University Marjorie Wharton Director for Strategic Development Cave Hill School of Business University of the West Indies Faculty/Instructors/Trainers: Project Beneficiaries Dwayne Devonish Lecturer in Management Science University of the West Indies Dr. Chris Hiller Technology Transfer/Innovation Coordinator University of the West Indies Chris Harper Cave Hill School of Business University of the West Indies Susan Branker Greene Cave Hill School of Business University of the West Indies Curtis B. Dennie Management Consultant Antigua & Barbuda Students: Project Beneficiaries Jenelle King Graduate Assistant Cave Hill School of Business University of West Indies Egbert Irving Graduate Assistant Cave Hill School of Business University of West Indies Dr. Jason Marshall Graduate Assistant Cave Hill School of Business University of West Indies Onya West Graduate Assistant Cave Hill School of Business University of West Indies USAID Mansfield Blackwood Development Specialist USAID Barbados and the Eastern Caribbean Walter Doetsch Office Director USAID Barbados and the Eastern Caribbean HED Diana Páez-Cook Program Officer, Latin America/Caribbean Higher Education for Development (HED) SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 92 | P a g e Stakeholder Interview and Focus Group List Student Consulting/Engagement Small Business: Project Beneficiary #1 Students: Project Beneficiary #1 Students: Project Beneficiary #2 Lean Launch Pad Exploration Workshop Small Business: Project Beneficiary #1 Small Business: Project Beneficiary #2 Small Business: Project Beneficiary #3 Lean Launch Pad 10 Week Small Business: Project Beneficiary #1 Small Business: Project Beneficiary #2 Graduate Assistantships Students: Project Beneficiary #1 Students: Project Beneficiary #2 Students: Project Beneficiary #3 Students: Project Beneficiary #4 SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 93 | P a g e Participant Observation List Lean Launch Pad 10 Week Small Business: Project Beneficiary #1 Small Business: Project Beneficiary #2 Small Business: Project Beneficiary #3 Small Business: Project Beneficiary #4 Small Business: Project Beneficiary #5 Small Business: Project Beneficiary #6 Small Business: Project Beneficiary #7 Small Business: Project Beneficiary #8 SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 94 | P a g e Tables Page 15, Table 1: Results Framework Revision Page 21, Table 2. Labour Force Participation Rate (%) 2003-2013 Page 44, Table 3. Human Capacity Building of Faculty Page 45, Table 4. Institutional Capacity Building: Development of curriculum in the area of entrepreneurship Page 50, Table 5. Institutional Capacity Building: Short‐Term Programs ‐ development of short term programs in entrepreneurship Page 52, Table 6. Institutional Capacity Building: Train the Trainer Page 53, Table 7. Institutional Capacity Building: Internships Page 54, Table 8. Institutional Capacity Building: Mentoring Page 55, Table 9. Institutional Capacity Building: Mentoring Symposiums Page 56, Table 10. Institutional Capacity Building: Business Incubator Page 57, Table 11. Institutional Capacity Building: Entrepreneurial Training Series Page 59, Table 12. Research Initiatives Page 60, Table 13. Project Administration and Award Management SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 95 | P a g e Figures Page 27, FIGURE 1: Key Elements Curriculum Process Oval – Entrepreneurship Concentration and Sustainability Page 30, FIGURE 2: Transition/Exit Model from Secondary Education to Post-Secondary Education Page 40, FIGURE 3: Linkage of framework to JOBS Project Evaluation Page 41, FIGURE 4: Framework for analyzing core JOBS project areas Page 41, FIGURE 5: Relationship between JOBS objectives and project areas Page 42, FIGURE 6: Relationship between JOBS objectives and project areas SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 96 | P a g e Appendix 1 – Qualitative Research Materials Barbados: Supporting Entrepreneurs through the JOBS Initiative (Job Opportunity for Business Scale Up) SUMMATIVE IMPLEMENTATION AND OUTCOME EVALUATION: Key Stakeholder Interview Protocol Background 1. What was your role in the JOBS project? 2. What are the major services you provided/oversaw? 3. At what point in time did you join the project? 4. Talk about how the project worked. a. What was the role of Cave Hill School of Business? University of West Indies? Indiana University Kelley School of Business? b. Discuss communication among project partners. c. Describe the strategies used among the team that facilitated collaboration. What worked? What didn’t work? d. How role did each partner play in the coordination of project activities (trainings, workshops, faculty development, etc..)? What worked? What didn’t work? 5. Describe who the stakeholders were and how they were identified? 6. What mechanisms did the partnership have in place for stakeholders to provide input into implementation planning and delivery? 7. How did the partnership ensure that stakeholder concerns and/or priorities were being incorporated into implementation planning and delivery? 8. How did the partnership identify and select participants for activities? 9. In your opinion, how responsive was the partnership to stakeholder interests and needs? 10. In your opinion, how conducive was the partnership to enabling local ownership? 11. To what extent was the intervention content aligned with national and local priorities, funder strategies, and recipients’ needs? SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 97 | P a g e 13. In your opinion, were program activities implemented according to the original design? 14. To what extent did the program reflect commonly accepted best practices in entrepreneurship? 15. To what extent were activities and outputs linked logically to program objectives? 16. What recommendations would you suggest for program content? (i.e., topic of trainings, resource materials development, etc.) 17. What was the value of the partnership to you? 18. In your opinion, have outputs and outcomes been achieved? 19. What were the major factors influencing the achievement/non-achievement of results? 20. What are the positive/negative and intended/unintended effects of the intervention? 21. To what extent, do you perceive these results as being attributable to the program? 22. What has proven to be the most impactful component of the JOBS Project? 23. Given implementation progress to date, what do you think is likely to be the long term impact as a result of the program? 24. Please share how likely do you think the program effects will continue over time after funding has ceased? 25. What recommendations would you make to improve the likelihood of program sustainability? SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 98 | P a g e Barbados: Supporting Entrepreneurs through the JOBS Initiative (Job Opportunity for Business Scale Up) SUMMATIVE IMPLEMENTATION AND OUTCOME EVALUATION: Focus Group Facilitator Guide – Graduate Assistantship (Students) Distribute the Participant Information form and the Statement of Informed Consent for completion before the session starts. Collect and verify that both forms are completed. INTRODUCTION  Welcome participants and introduce yourself.  “The purpose of this project is to understand your experience with the JOBS Initiative project activities.”  “We value the information you will be sharing with us today and want to make sure that we capture everything, so we will be digitally recording during the session. Is everyone OK with that?”  “Here are the ground rules: o Only one person talks at a time, o No taping on the table, o Speak from your own experiences, and o Respect others’ comments as being from their own experiences.”  “In the interest of time and giving all the opportunity to share, I may have to interrupt at times to keep the discussion moving along.”  ANY QUESTIONS? 1. Tell us your name, what your role was with the graduate assistantship/student consulting engagement (student or business owner)/Lean Launch Pad activities, and what made you decide to participate in the focus group?? (OPENING) 2. Think back to the graduate assistantship/student consulting engagement (student or business owner)/Lean Launch Pad activities. How did you first hear about the project? (INTRODUCTORY) 3. What motivated you to get involved in the project? (TRANSITION) 4. For this next series of questions, think back to your experience with the graduate assistantship/student consulting engagement (student or business owner)/Lean Launch Pad activities: (KEY QUESTIONS) SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 99 | P a g e Relevance of external stakeholder engagement and the intervention content a. Were you satisfied with the program content? (i.e., topic of trainings, resource materials development, etc.) PROBES: Cultural acceptability. PROBES: Social acceptability. PROBES: Environmental acceptability. Effectiveness of the intervention b. How valuable did you find the program content? PROBES: PROBES: PROBES: c. What are you types of things are you doing differently as a result your participation? Potential lasting effect of the intervention and provide recommendations for partnership sustainability d. To what extent will your experience influence future decision making? PROBES: PROBES: e. Of all the things we discussed, what would your recommendations be for continuing the graduate assistantship/student consulting engagement (student or business owner)/Lean Launch Pad opportunities? (ENDING QUESTION) THANK THEM FOR THEIR TIME. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 100 | P a g e Barbados: Supporting Entrepreneurs through the JOBS Initiative (Job Opportunity for Business Scale Up) SUMMATIVE IMPLEMENTATION AND OUTCOME EVALUATION CONSENT FORM Focus Group: Graduate assistantship/student consulting engagement (student or business owner)/Lean Launch Pad beneficiaries. Please initial box 1. I confirm that I have read and understand the information sheet for the above study and have had the opportunity to ask questions. 2. I understand that my participation is voluntary and that I am free to withdraw at any time, without giving reason. 3. I agree to take part in the above study. Please check box Yes No 4. I agree to respect the anonymity of other focus group participants. 5. I agree to the focus group being manually recorded. 6. I agree to the use of anonymized quotes in publications. 7. I agree that my data gathered in this study may be stored (after it has been anonymized) and used for future research. Name of Participant Date Signature Name of Focus Group Facilitator Date Signature SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 101 | P a g e Barbados: Supporting Entrepreneurs through the JOBS Initiative (Job Opportunity for Business Scale Up) SUMMATIVE IMPLEMENTATION AND OUTCOME EVALUATION PARTICIPANT INFORMATION SHEET Focus Group: Graduate assistantship/student consulting engagement (student or business owner)/Lean Launch Pad beneficiaries You have been invited to take part in a focus group study. Before you decide whether or not to take part, it is important for you to understand why the study is being done and what it will involve. Please take time to read the following information carefully. USAID’s Job Opportunity for Business Scale Up (JOBS) is an initiative to promote post￾secondary skills training in entrepreneurship through partnerships among U.S. and developing country institutions of higher education. The Barbados JOBS partnership between the Kelley School of Business at Indiana University and the Cave Hill School of Business (CHSB) at the University of the West Indies focuses on the growth of an entrepreneurial culture and diversifying the service oriented economies in Barbados and neighboring countries. The partnership worked to establish the Cave Hill School of Business Centre for Enterprise and Entrepreneurship at the University of the West Indies as the hub for innovative business training and growth in the Eastern Caribbean. The purpose of this study is to understand how effective the partnership was in enhancing CHSB's institutional capacity to serve as an entrepreneurship center for excellence in Barbados and the Organization of Eastern Caribbean States (OECS) and in strengthening entrepreneurship knowledge and skills. It is up to you to decide whether or not to take part. If you do decide to take part you will be given this information sheet to keep and be asked to sign a consent form. If you decide to take part you are still free to withdraw at any time and without giving a reason. Regardless of whether you choose to take part or not in the study, there will be no impact on your relationship with Cave Hill School of Business or the University of West Indies. As part of the focus group session, the facilitator will ask a series of questions to the group and ask for your response. All discussions will be digitally recorded to ensure that participant responses and conversations are documented accurately and completely. The focus group session will run approximately 60 minutes. There are no physical or mental risks to you as a participant in this study. While confidentiality cannot be guaranteed due to the focus group setting, all participants are asked to respect the privacy and anonymity of other participants. Higher Education for Development (HED) will be responsible for maintaining the confidentiality, privacy and anonymity for the collection, storage and publication of any materials associated with the focus group sessions. You can contact Audra Stevenson-Drakes, JOBS Project Manager, at audra.stevenson@cavehill.uwi.edu or via phone at 246-417-0244 for further information on this study or if you have any concerns about the way in which the study has been conducted. Thank you for taking time to read the information sheet. Name of Participant Date Signature SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 102 | P a g e Appendix 2 – Survey Research Materials June 30, 2014 Dear Participant, The Cave Hill School of Business, University of the West Indies (CHSB) would like to thank you for your participation in the USAID Job Opportunities for Business Scale Up (JOBS) Initiative over the past three years. The project will end on 30th June 2014, and our donor agency USAID seeks to gather direct feedback about the impact and sustainability of the overall project. Our records show that you were a participant in the Faculty Symposium which took place at the Cave Hill School of Business. We have been requested to forward the following survey to you which can be accessed online through the url: https://www.surveymonkey.com/s/7CKKZCY. Please submit your feedback to this survey regarding the Faculty Symposium held for the JOBS Project. Should you encounter any difficulty accessing the document, feel free to contact Therese Fellner at tfellner@wi.rr.com. Thank you, in advance, for your support. Your assistance will help to inform funding for similar projects in the future and will be accessed by representatives from the donor agencies. Kind Regards, Cave Hill School of Business The University of the West Indies (Formerly Centre for Management Development) Cave Hill Campus St. Michael BARBADOS Email: chsb.jobs6@gmail.com Phone: 246.238.0518 Website: www.uwichsb.org SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 103 | P a g e June 30, 2014 Dear Participant, The Cave Hill School of Business, University of the West Indies (CHSB) would like to thank you for your participation in the USAID Job Opportunities for Business Scale-up (JOBS) Initiative over the past three years. The project will end on 30th June 2014, and our donor agency USAID seeks to gather direct feedback about the impact and sustainability of the overall project. Our records show that you were a participant in the Mentorship Symposium which took place at the Cave Hill School of Business. We have been requested to forward the following survey to you which can be accessed online through the url: https://www.surveymonkey.com/s/YFGSVVQ. Please submit your feedback to this survey regarding the Mentorship Symposiums held for the JOBS Project. Should you encounter any difficulty accessing the document, feel free to contact Therese Fellner at tfellner@wi.rr.com. Thank you, in advance, for your support. Your assistance will help to inform funding for similar projects in the future and will be accessed by representatives from the donor agencies. Kind Regards, Cave Hill School of Business The University of the West Indies (Formerly Centre for Management Development) Cave Hill Campus St. Michael BARBADOS Email: chsb.jobs6@gmail.com Phone: 246.238.0518 Website: www.uwichsb.org SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 104 | P a g e July 14, 2014 Dear Participant, Thank you for your participation in the USAID Job Opportunities for Business Scale-up (JOBS) Initiative over the past three years with the Cave Hill School of Business. The project ended on 30th June 2014, and the donor agency USAID seeks to gather direct feedback about the impact and sustainability of the overall project. Records show that you were a participant in the 2012 Train the Trainer event which took place in Barbados and throughout the OECS. If you would, please take 5 minutes and provide your feedback on the training at https://www.surveymonkey.com/s/BKZSWS9 Should you encounter any difficulty accessing the document, feel free to contact me, Therese Fellner, as the External Evaluation Consultant for HED at tfellner@wi.rr.com. Thank you, in advance, for your support. Your assistance will help to inform funding for similar projects in the future and will be accessed by representatives from the donor agencies. Kind Regards, Therese Therese Fellner, PhD Evaluation Research Services tfellner@wi.rr.com 262.930.8695 SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 105 | P a g e JOBS 2012 Train the Trainer Survey Tool Introductory Summary USAID’s Job Opportunity for Business Start Up (JOBS) was an initiative to promote post-secondary skills training in entrepreneurship through partnerships among U.S. and developing country institutions of higher education. The Barbados JOBS partnership between the Kelley School of Business at Indiana University and the Cave Hill School of Business at the University of the West Indies focused on the growth of an entrepreneurial culture and diversifying the service oriented economies in Barbados and neighboring countries. The partnership worked to establish the Cave Hill School of Business Centre for Enterprise and Entrepreneurship at the University of the West Indies as the hub for innovative business training and growth in the Eastern Caribbean by enhancing CHSB's institutional capacity to serve as an entrepreneurship center for excellence in Barbados and the Organization of Eastern Caribbean States (OECS), and to strengthen entrepreneurship knowledge and skills among diverse stakeholder groups in Barbados and the OECS through outreach and engagement. As a direct beneficiary of the Barbados JOBS Project, you are invited to share your experience with the Train the Trainer event you attended in 2012. Completion of the survey should take no longer than 5 minutes. Your participation is appreciated. The purpose of this survey is to understand your satisfaction with training, whether others have been trained as a result, and how likely you will continue to apply the knowledge gained from the training and in what ways. There are no potential risks or benefits to participating and you can stop participating in the study at any point. All respondents' identities will be kept confidential and will not link to any survey responses when using data. Please contact project evaluators, Dr. Theo Sofianos, t.sofianos@gmail.com or Dr. Therese Fellner, tfellner@wi.rr.com with any questions. Thank you for your time. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 106 | P a g e JOBS 2012 Train the Trainer Survey Tool 1. In what country do you reside? Antigua Barbados Dominica Grenada St. Kitts and Nevis St. Lucia St. Vincent Other (please specify) 2. Which of the following best describe your job function? Accounting Advertising / Marketing Business Development Consulting Education Finance General Business Human Resources Information Technology Legal Management Public Relations Training Other (please specify) 3. Overall, how would you rate the Train-the-Trainer event? Excellent Very good Fairly good Mildly good Not good at all 4. How useful was the information presented at the training? Extremely useful Very useful Somewhat useful Slightly useful Not useful at all 5. How much have your skills improved because of the training? A great deal A lot A moderate amount A little None at all 6. How much of the information presented at the training was new to you? All of it Most of it About half of it Some of it None of it 7. How many other individuals have you trained as a result? 0 1-3 4-6 7-10 More than 10 Other (please specify) 8. How likely is it that you would recommend this training to a friend or colleague? Not at all likely - 0 1 2 3 4 5 6 7 8 9 10 Extremely likely SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 107 | P a g e JOBS 2013 Mentoring Symposium Survey Tool Introductory Summary USAID’s Job Opportunity for Business Scale Up (JOBS) was an initiative to promote post￾secondary skills training in entrepreneurship through partnerships among U.S. and developing country institutions of higher education. The Barbados JOBS partnership between the Kelley School of Business at Indiana University and the Cave Hill School of Business at the University of the West Indies focused on the growth of an entrepreneurial culture and diversifying the service oriented economies in Barbados and neighboring countries. The partnership worked to establish the Cave Hill School of Business Centre for Enterprise and Entrepreneurship at the University of the West Indies as the hub for innovative business training and growth in the Eastern Caribbean by enhancing CHSB's institutional capacity to serve as an entrepreneurship center for excellence in Barbados and the Organization of Eastern Caribbean States (OECS), and to strengthen entrepreneurship knowledge and skills among diverse stakeholder groups in Barbados and the OECS through outreach and engagement. As a direct beneficiary of the Barbados JOBS Project, you are invited to share your experience with the June 2013 Mentoring Symposium you attended. Completion of the survey should take no longer than 5 minutes. Your participation is appreciated. The purpose of this survey is to understand your satisfaction with the mentoring symposium and the impact the training had on your ability to advise and support entrepreneurs and entrepreneurial teams. There are no potential risks or benefits to participating and you can stop participating in the study at any point. All respondents' identities will be kept confidential and will not link to any survey responses when using data. Please contact project evaluators, Dr. Theo Sofianos, t.sofianos@gmail.com or Dr. Therese Fellner, tfellner@wi.rr.com with any questions. Thank you for your time. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 108 | P a g e JOBS 2013 Mentoring Symposium Survey Tool 1. In what country do you reside? Antigua Barbados Dominica Grenada St. Kitts and Nevis St. Lucia St. Vincent Other (please specify) 2. Overall, how would you rate the Mentoring Symposium? Excellent Very good Fairly good Mildly good Not good at all 3. How useful was the information presented at the Mentoring Symposium? Extremely useful Very useful Somewhat useful Slightly useful Not useful at all 4. How much have your skills improved because of the Mentoring Symposium? A great deal A lot A moderate amount A little None at all 5. How much of the information presented at the Mentoring Symposium was new to you? All of it Most of it About half of it Some of it None of it 6. With how many businesses have you shared symposium information? 0 1-3 4-6 7-10 More than 10 Other (please specify) 7. The symposium... Provided concepts, processes, and tools to enable qualified mentors to be more effective in mentoring emerging entrepreneurs. Strongly disagree Disagree Neither Disagree nor Agree Agree Strongly Agree Enhanced the ability of mentors to advise and support entrepreneurs and entrepreneurial teams. Strongly disagree Disagree Neither Disagree nor Agree Agree Strongly Agree Created opportunities for mentors to network and establish linkages. Strongly disagree Disagree Neither Disagree nor Agree Agree Strongly Agree 8. How likely is it that you would recommend this training to a friend or colleague? Not at all likely - 0 1 2 3 4 5 6 7 8 9 10 Extremely likely SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 109 | P a g e JOBS 2013 Train the Trainer Survey Tool Introductory Summary USAID’s Job Opportunity for Business Start Up (JOBS) was an initiative to promote post-secondary skills training in entrepreneurship through partnerships among U.S. and developing country institutions of higher education. The Barbados JOBS partnership between the Kelley School of Business at Indiana University and the Cave Hill School of Business at the University of the West Indies focused on the growth of an entrepreneurial culture and diversifying the service oriented economies in Barbados and neighboring countries. The partnership worked to establish the Cave Hill School of Business Centre for Enterprise and Entrepreneurship at the University of the West Indies as the hub for innovative business training and growth in the Eastern Caribbean by enhancing CHSB's institutional capacity to serve as an entrepreneurship center for excellence in Barbados and the Organization of Eastern Caribbean States (OECS), and to strengthen entrepreneurship knowledge and skills among diverse stakeholder groups in Barbados and the OECS through outreach and engagement. As a direct beneficiary of the Barbados JOBS Project, you are invited to share your experience with the Train the Trainer event you attended in 2013. Completion of the survey should take no longer than 5 minutes. Your participation is appreciated. The purpose of this survey is to understand your satisfaction with training, whether others have been trained as a result, and how likely you will continue to apply the knowledge gained from the training and in what ways. There are no potential risks or benefits to participating and you can stop participating in the study at any point. All respondents' identities will be kept confidential and will not link to any survey responses when using data. Please contact project evaluators, Dr. Theo Sofianos, t.sofianos@gmail.com or Dr. Therese Fellner, tfellner@wi.rr.com with any questions. Thank you for your time. SUMMATIVE IMPLEMENTATION/OUTCOME EVALUATION: BARBADOS JOBS 110 | P a g e JOBS 2013 Train the Trainer Survey Tool 1. In what country do you reside? Antigua Barbados Dominica Grenada St. Kitts and Nevis St. Lucia St. Vincent Other (please specify) 2. Which of the following best describe your job function? Accounting Advertising / Marketing Business Development Consulting Education Finance General Business Human Resources Information Technology Legal Management Public Relations Training Other (please specify) 3. Overall, how would you rate the Train-the-Trainer event? Excellent Very good Fairly good Mildly good Not good at all 4. How useful was the information presented at the training? Extremely useful Very useful Somewhat useful Slightly useful Not useful at all 5. How much have your skills improved because of the training? A great deal A lot A moderate amount A little None at all 6. How much of the information presented at the training was new to you? All of it Most of it About half of it Some of it None of it 7. How many other individuals have you trained as a result? 0 1-3 4-6 7-10 More than 10 Other (please specify) 8. How likely is it that you would recommend this training to a friend or colleague? Not at all likely - 0 1 2 3 4 5 6 7 8 9 10 Extremely likely