February 2019 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by the Ethiopia Performance Monitoring and Evaluation Service (EPMES) of Social Impact, Inc. EVALUATION Youth Potential Activity Youth Cohort Study February 2019 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by the Ethiopia Performance Monitoring and Evaluation Service (EPMES) of Social Impact, Inc. DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. EVALUATION Youth Potential Activity Youth Cohort Study A cohort study to measure the results of an Activity that empowered youth through workforce development and livelihood support February 2019 Evaluation Mechanism Number: AID-663-C-16-00010-EPMES Ethiopia Performance Monitoring and Evaluation Service (EPMES) Activity for USAID/Ethiopia Cover photo by Social Impact, Inc. Authors Deepika Chawla, Team Lead Jeffrey Tines (endline) Basab Dasgupta Worku Ambelu Hika Alemu Ermias Assefa Eshetu Estera Barbarasa (midline) i Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report ABSTRACT This multi-round Youth Cohort Study was conducted as part of the evaluation of the Building the Potential of Youth (Youth Potential) Activity under the broader Ethiopia Performance Monitoring and Evaluation Service (EPMES) Activity. The study measures Activity outcomes and changes over 12 months for a subset of Youth Potential beneficiaries from four Ethiopian regions. The study uses a panel design and qualitative and quantitative approaches to measure changes in income, employment status, work readiness skills, knowledge of and access to employment/entrepreneurial resources, and functional literacy and numeracy skills. Data were collected at three points of the study—baseline, midline and endline—through surveys, focus group discussions, key informant interviews, and informational meetings. The study findings show that 12 months after participation in the Activity, a significantly larger number of participants were employed (69 percent) and had a higher income (71 percent) than at baseline. Of those employed, 88 percent were self-employed, with agriculture and trading as the primary occupations. Participants reported that they had greater self-confidence and a better work ethic and felt more empowered in searching for employment, creating self-employment, and obtaining complete information. ii Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report ACKNOWLEDGEMENTS The Youth Cohort Study is the result of the cooperation of several organizations and individuals, and the team would like to take this opportunity to thank them all. We would like to start by expressing our appreciation of the U.S. Agency for International Development (USAID)/Ethiopia and the implementing team for the Building the Potential of Youth (Youth Potential) Activity for helping facilitate the data collection. For each phase – baseline, midline and endline – of this study, Save the Children (SC) and its partners implementing Youth Potential enabled the study team to access the participants as well as the implementing team on the ground and conduct meetings with government officials at the regional, woreda, and kebele levels. Our thanks to the focus group participants, the beneficiaries of the Activity and their guardians, the technical, vocational, and educational training (TVET) personnel and the local entrepreneurs that are providing training to the Activity participants, and all those who took the time out of their busy schedules to sit down with us for interviews and informational meetings and shared their thoughts and opinions about the Activity. The Social Impact, Inc. (SI) team in Ethiopia and at the headquarters office has been a tremendous support, stepping in as needed with solutions to any and every problem. The team of surveyors from the Sub-Saharan Africa Research Center (SART) has proved to be invaluable in collecting quality survey data. The members of the study team worked together as a whole and their knowledge of the Ethiopian context and the various technical insights they provided have made the data collected that much better and more robust. Finally, we would like to acknowledge the drivers of the team vehicles who, despite the very rough terrain, drove us patiently to various corners of Amhara, Afar, Oromia, and Tigray regions. Without the help and valuable contribution of each group of people mentioned here, and several whom we may have missed, it would not have been possible to complete this exercise and the writing of this report. On behalf of the EPMES YCS team Deepika Chawla (Team Lead) EPMES YCS Team SI staff Consultants Basab Dasgupta Deepika Chawla (Team Lead) Worku Ambelu Ermias Assefa Eshetu Hika Alemu Estera Barbarasa (midline only) Francis Okello (baseline) Jeffrey Tines (endline only) iii Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report CONTENTS Abstract ........................................................................................................................................... i Acknowledgements ....................................................................................................................... ii Contents ........................................................................................................................................ iii Lists of Tables and Figures .......................................................................................................... iv Acronyms ....................................................................................................................................... v Geographic Coverage .................................................................................................................. vi Executive Summary .....................................................................................................................vii Introduction ................................................................................................................................... 1 Youth Cohort Study ..................................................................................................................................................... 2 Study Objectives ............................................................................................................................................................ 2 Methodology................................................................................................................................................................... 3 Study Limitations ........................................................................................................................................................... 5 Main Findings ................................................................................................................................. 7 Training, Coaching, and Mentoring ........................................................................................................................... 7 Employment Status of Study Participants .............................................................................................................. 10 Income, Expenditures, and Assets ........................................................................................................................... 20 Utilization of Health Services ................................................................................................................................... 27 Self-Perception of Work Readiness ........................................................................................................................ 29 Knowledge of and Access to Employment and Entrepreneurial Resources ................................................. 31 Functional Literacy and Numeracy Skills ............................................................................................................... 32 Conclusions .................................................................................................................................. 34 Recommendations ...................................................................................................................... 36 Annexes ........................................................................................................................................ 40 Annex I: YCS Statement of Work ............................................................................................. 41 Annex II: Youth Potential Activity Results Framework .......................................................... 62 Annex III: YCS Baseline Study – Summary .............................................................................. 63 Annex IV: YCS Midline Study – Summary ................................................................................ 72 Annex V: List of Research Questions and Indicators ............................................................... 83 Annex VI: Sources of Data ......................................................................................................... 87 Annex VII: Data Collection Details ........................................................................................... 89 Annex VIII: Data Collection Instruments ................................................................................. 95 Annex IX: Data Tables and Graphs ......................................................................................... 126 Study Team Disclosures of Conflict of Interest ..................................................................... 137 Study Team Profiles .................................................................................................................. 144 Annex XI: Post-Evaluation Action Plan .................................................................................. 151 iv Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report LISTS OF TABLES AND FIGURES Table 1: Employment status at baseline and endline (proportion responding) .....................................................................10 Table 2: Employment status at baseline, midline, and endline (proportion responding, by sex)......................................11 Table 3: Employment status at baseline, midline, and endline (proportion responding, by region) ................................12 Table 4: Employment status at baseline, midline, and endline (proportion responding, by education level) ................12 Table 5: Type of self-employment at baseline and endline (proportion of self-employed responding) .........................15 Table 6: Change in employment status and sector from baseline to midline to endline ...................................................16 Table 7: Views on whether agriculture provided good employment opportunities ...........................................................17 Table 8: Change in field of work of the employed between baseline, midline, and endline ..............................................18 Table 9: Change in income between baseline, midline, and endline ........................................................................................20 Table 10: Change in income between baseline, midline, and endline, by sex ........................................................................21 Table 11: Change in income between baseline, midline, and endline, by region ..................................................................21 Table 12: Change in income between baseline, midline, and endline, by education ............................................................22 Table 13: Percentage reporting expenditure on various items, baseline, midline, and endline ........................................23 Table 14: Percentage reporting expenditure on various items, baseline, midline and endline, by region .....................24 Table 15: Spending on assets at baseline, midline, and endline ................................................................................................. 25 Table 16: Health and wellness indicators at baseline, midline, and endline (proportion reporting), by change in income .....................................................................................................................................................................................................28 Table 17: Participants’ self-perception of work readiness ..........................................................................................................30 Table 18: Avenues of search for employment – baseline to endline .......................................................................................31 Table 19: Entrepreneurial resources required, by current employment status ...................................................................32 Table 20: Change in level of functional literacy and numeracy, baseline to midline to endline ........................................33 Table 21: Post-evaluation action plan ........................................................................................................................................... 157 Figure 1: Respondents looking for work, by employment status (%) ......................................................................................11 Figure 2: Distribution of employment status over eight months for a subgroup of survey participants (n=81) .........14 Figure 3: Distribution of monthly mean and median income for a subgroup of survey participants (n=75) ................22 Figure 4: Change in respondents incurring expenditure, proportion baseline to midline and midline to endline, by item (%) ...................................................................................................................................................................................................23 Figure 5: Distribution of monthly mean and median income and expenditure for a subgroup of survey participants (n=83).......................................................................................................................................................................................................25 Figure 6: Reasons for obtaining loans for a subgroup of survey participants, by region, age group, and sex (n=81) . 26 v Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report ACRONYMS BL Baseline BYOB Be Your Own Boss BOLSA Bureau of Labor and Social Affairs BoWA Bureau of Women’s Affairs COR Contracting Officer’s Representative EDC Education Development Center EL Endline EPMES Ethiopia Performance Monitoring and Evaluation Service FCE Facilitator for Change Ethiopia FGD Focus Group Discussion GDP Gross domestic product GoE Government of Ethiopia GTP Growth and Transformation Plan HUNDEE HUNDEE-Oromo Grassroots Development Initiative ICT Information and Communications Technology KII Key Informant Interview ML Midline MOU Memorandum of Understanding NGO Non-governmental organization PADet Professional Alliance for Development in Ethiopia PDP Personal Development Plan PYD Positive Youth Development REST Relief Society of Tigray SART Sub-Saharan Africa Research Center SC Save the Children, Inc. SES Socio-Economic Status SI Social Impact, Inc. SNNPR Southern Nations, Nationalities, and Peoples' Region TTI Technical Training Institutions TVET Technical and Vocational Education and Training USAID United States Agency for International Development USD United States Dollar WBL Work-based learning WRN! Work Ready Now! WR Plus Work Ready Plus YCS Youth Cohort Study YES Youth Economic Strengthening YIA Youth in Action YP Youth Potential vi Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report GEOGRAPHIC COVERAGE *Youth Potential Activity has been implemented in select woredas in six regions – Afar, Amhara, Oromia, SNNPR, Somali, and Tigray. Please note that over the period of data collection (April 2017 to May 2018), in the SNNPR region, the Youth Potential Activity was initially active in Hammer and Esirano areas but then shifted focus to offering services in Hawassa, Zuria, and Dawro areas instead. The map above shows the 30 woredas where the activity was implemented across all six regions, as well as the woredas selected for this evaluation. vii Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report EXECUTIVE SUMMARY Building the Potential of Youth (Youth Potential [YP]), a five-year Activity initiated in 2015, provides rural Ethiopian youth (ages 15–29) with access to workforce development and livelihood support to strengthen their skills and increase their earning potential. Designed to reach 34,537 “unemployed or underemployed rural youth,” the Activity enrolls a group of male and female youth every three months. YP is implemented under a cooperative agreement between the United States Agency for International Development (USAID) and a group headed by Save the Children (SC).1 In its fourth and current year of implementation, it is providing services in 30 woredas in Afar; Amhara; Oromia; Somali; Southern Nations, Nationalities, and Peoples’ Region (SNNPR); and Tigray regions. In 2017, USAID/Ethiopia contracted with Social Impact, Inc. (SI) to implement the Ethiopia Performance Monitoring and Evaluation Service (EPMES) Activity. Under EPMES, USAID requested an independent performance evaluation of YP. The evaluation included three activities: (i) a baseline study, (ii) performance monitoring and a data system review, and (iii) a multi-round cohort study of beneficiaries henceforth referred to as the Youth Cohort Study (YCS). This report is the final report of the multi￾round YCS and incorporates findings from all three rounds. YCS aims to examine whether youth who receive workforce development and livelihood support through YP earn a higher income and acquire the necessary skills, knowledge, and social capital to achieve economic self-sufficiency over the longer term. YCS addresses the following broad evaluation questions:2  To what extent has the income and employment status of study participants changed between enrollment in the Youth Potential Activity and the twelve months following enrollment? Does increased income lead to improved health outcomes among study participants?  To what extent has participation in the Youth Potential Activity improved the work-readiness skills of study participants?  To what extent has participation in the Youth Potential Activity improved knowledge of and access to employment/entrepreneurial resources for study participants?  To what extent has participation in the Youth Potential Activity improved functional literacy and numeracy skills of study participants?  How do the outcomes addressed in the questions above vary by participant characteristics (e.g., sex, age, ethnic/linguistic group,3 socioeconomic status [SES], marital status, parenting status) and by service delivery partner?4 The study will inform stakeholders, USAID, the Government of Ethiopia (GoE), and YP implementers about YP and its progress toward near-term outputs and outcomes. In addition, it will demonstrate long-term contributions of YP initiatives on a youth cohort of youth and highlight major strengths and weaknesses of YP initiatives to enable any necessary adjustments and inform the design of similar activities. 1 Cooperative Agreement # AID-663-A-15-00006; the implementation team includes Save the Children (SC) and its partners Education Development Center, Inc. (EDC), HUNDEE-Oromo Grassroots Development Initiative (HUNDEE), Professional Alliance for Development in Ethiopia (PADet), Relief Society of Tigray (REST), and Facilitator for Change Ethiopia (FCE). 2 From the EPMES YCS statement of work. 3 In discussion with USAID, it was agreed to drop this variable due to political sensitivities. 4 Given that the YP activities in each of the regions are implemented by a different service delivery partner, region was used as a proxy for service delivery partner. viii Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Study Approach To address the evaluation questions, YCS used a panel design to follow a subset of one cohort (enrolled in March/April 2017) of YP participants from select woredas and kebeles in Afar, Amhara, Oromia, and Tigray over 12 months to gain insights into changes in their employment and income status. The regions, and the woredas and kebeles within the regions, were purposefully selected based on four criteria: (i) each region was covered, (ii) each service delivery partner on the SC team was included, (iii) the pastoral-productive mix was representative of all woredas in which YP was being implemented, (iv) woredas represented all the years for which YP was being implemented. Because YP covers eligible youth in specific locations/kebeles within the woredas and is implemented over an extended period, a panel design was used to assess whether YP is associated with expected changes in outcomes. Quantitative and qualitative data were collected through surveys and focus group discussions (FGDs) with a sample of participants, which were supplemented by FGDs with parents and guardians and key informant interviews (KIIs) with implementation staff and other stakeholders on various indicators to document changes over the year following YP enrollment. Data were collected and analyzed at three times: (i) baseline, in the month of YP enrollment (collected in April/May 2017); (ii) midline, at six months after enrollment (collected in September/October 2017); and (iii) endline, at 12 months after enrollment (collected in March/April 2018). Income and expenditure data were also collected periodically (using monthly financial diary interviews) for a purposive subset of the cohort. To ensure comparability, data collection instruments used at baseline were also used at midline and endline, but questions were added at midline and endline as needed to assess YP impacts. Data collected at endline were compared with data from baseline (and midline, as necessary) to measure changes in participant outcomes and YP’s impacts on them. Data were disaggregated by sex, age, education level, and region. The variable “region” was used as a proxy both for the different service delivery partners and for the pastoral/productive nature of the target populations. Starting with a cohort of 670 youth, the endline survey yielded data on 458 participants, for a net response rate of 68 percent.5 During the year-long study, qualitative data were collected through 100 FGDs conducted with youth cohort participants (77) and their parents/guardians (23) from seven woredas in the four regions. KIIs were conducted with a select group of government officials; microfinance officials; TVET and TTI administrators; employers; and work-based learning (WBL) providers and YP staff at the regional, woreda, and kebele levels. Interviews were also held with the SC staff responsible for oversight and implementation. Limitations This study has several limitations. First, the data used in assessing YP contribution are based on self￾reporting. Although using a combination of methods and triangulating data from other sources such as key stakeholder interviews could have mitigated the resultant bias, the likelihood remains of a response￾shift bias caused by a change in the participants’ frame of reference across measurement points. This change introduces the possibility of different responses to the same set of questions administered at different times, thus compromising the quality and robustness of inferences drawn from intertemporal comparisons. Second, the inability, despite multiple requests, to access data on underemployed youth—a category created by YP based on certain criteria—at the time of sample selection for the YCS made it difficult to address the findings for the youth under this category separately from those employed. Third, given that there are no standard definitions of the terms “underemployment” or “better employment” the evaluation tracked the self-reported employment status of all study participants and used proxy measures, such as satisfaction with employment, search for new employment, and change in income, as 5 This was anticipated, and a cushion was built into the sample to allow for 40 percent attrition over the year. A sample size of 360–400 was deemed to be adequate (and was also required by the contract), which was achieved by selecting an initial sample size of 670 beneficiaries that resulted in data on 458 respondents. ix Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report indicators of improvement in employment status. Fourth, data were disaggregated by region as a proxy for service delivery partner and pastoral/productive area. Even though further disaggregation within region by rural and semi-urban areas or by smaller administrative units could have provided greater insights, data were not disaggregated to this level because introducing too much granularity into the analysis could lead to erroneous conclusions. Fifth, sustaining participation in regular monthly financial diary interviews for collection of income and income-use data was challenging because of participants’ desire to be compensated for their time. As a result, financial data were collected for only eight months instead of the planned 12 months, which could have introduced inaccuracies in estimates of income and its use. And finally, because one year is too short a time to expect changes in health care–seeking behavior or health outcomes, it is difficult to relate any change in health outcomes to YP interventions. Main Findings Employment increased by more than 40 percent between YP enrollment and 12 months following enrollment. Self-employment saw the most growth—from 41 percent of study respondents at baseline to 60.5 percent at endline—with wage employment rising from 6.9 percent to 8.5 percent, a statistically non-significant. Employment grew among both male and female study respondents and in all four regions between baseline and endline, with the greatest increase in Tigray and Afar, both of which started from a low base. Study respondents at all education levels reported an increase in employment between baseline and endline, with the greatest increase among those with some college/university education (from 29 percent to 73 percent), followed by those with no schooling (from 34 percent to 75 percent) and those that had completed technical and vocational education and training (TVET) (from 45 percent to 84 percent). Among the self-employed, there was a marked shift between baseline and endline away from working in family business toward working in self-owned enterprises. About two-thirds of the self-employed were engaged in agriculture because it was the most widely available income-generating activity (business), although many believed that it did not offer the best earning potential or return for the effort required. The vast majority of study respondents (71 percent) reported an increase in income between YP enrollment and 12 months following enrollment. Although incomes of both male and female study respondents rose, more males (76 percent) than females (66 percent) benefited from the increase. Among the regions, Tigray had the largest proportion of respondents whose incomes rose, followed by Oromia, Amhara, and Afar. Across education levels, two-thirds of the study respondents who had completed TVET reported an increase in income, followed by 63 percent of those with no schooling and under 60 percent in all other education categories. In line with increasing income, there was a marked increase in the number of study respondents spending their incomes on housing, food, utilities, education, health care, investments, and savings. The average amount saved by study respondents increased by more than 80 percent, from US$137 (3,135 Birr) to US$251 (5,757 Birr). The largest increases were in Tigray, where the proportion of respondents spending on consumption and investment increased twenty-fold for spending on utilities, six-fold on rent and lease, threefold on food and education, and twice or more on most other items. Increases in other regions were relatively modest. A significantly larger proportion of study respondents reported incurring health care expenditures for themselves or their families at 12 months following YP enrollment, but the change in average monthly spending was not significant, as reflected by the monthly financial diary interviews. There is also some evidence of improvement in the health of study respondents whose incomes rose: 90 percent reported that their health had improved during the year, a significantly higher proportion than those whose income fell during this period (78 percent). Likewise, the incidence of illness was lower among those whose income rose between baseline and endline, with 28 percent of these respondents reporting having an illness in the recall period of six months, compared with 43 percent of respondents whose x Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report incomes fell. Among the regions, study respondents in Afar reported the largest improvements in health outcomes between baseline and endline, and Tigray the least. More respondents at lower education levels than those with higher (TVET/college) education levels reported improved health outcomes over six months. Both male and female respondents reported taking care of their health, and whereas men reported feeling generally less stressed, less anxious, and less angry, women reported feeling generally happier and healthier. Although not statistically significant, these differences are the result of a culmination of the effects of YP participation, such as having a more positive outlook, being engaged in work that resulted in a higher income and having a support network of peers. As reported above, these effects were more pronounced among participants with lower education levels. Study respondents’ work readiness skills improved significantly between YP enrollment and 12 months following enrollment, especially in expertise in managing a business, proficiency in writing job applications, and knowledge of where and how to search for a job and obtain financing (even if they have not succeeded in obtaining financing). Study participants reported being able to get better and more complete information and feeling more empowered in searching for employment and creating self￾employment because of their training. Those looking for wage employment reported being more aware of the various places where they could apply for jobs. Likewise, those looking to be engaged in self￾employment reported being more aware of how to obtain the resources needed to start their own enterprise. Conclusions YP has been implemented well and has succeeded in helping youth increase their incomes and assets through employment or better employment. Work Ready Now! (WRN!) and Work Ready Plus (WR Plus) training was sufficiently provided for YP beneficiaries followed in this study. However, coaching and mentorship was not provided to the level that addressed all beneficiaries, mainly because of the late start of the mentorship program but also because of the low number of facilitators relative to the number of beneficiaries, high level of staff turnover, logistical challenges of reaching out to all beneficiaries spread out over long distances, and cultural challenges of benefiting from local mentors. A significant increase in self-employment was recorded among study respondents between YP enrollment and 12 months following enrollment. Sectoral diversification, however, was limited, and the majority of beneficiaries remained engaged in agriculture even though many believed it was not the best income-generating activity. Constraints that prevented youth from entering other sectors of self￾employment were access to start-up and working capital, land, and working space. A related challenge was the ability of the participants to organize themselves into groups, for which loans were relatively easier to obtain. Female participants faced specific challenges in branching out into new areas, especially those considered male domains. Increase in wage employment during this period was very limited, which—though not out of the expectation—is striking given that several opportunities remained untapped in many sectors and regions even as several beneficiaries remained unemployed after the training. The YCS findings suggest that YP provided the study beneficiaries with life skills and helped them develop a more positive attitude toward work, especially self-employment, and encouraged the participants to remain in their communities instead of migrating to the cities or even to other countries. The average income of the study cohort increased in the 12 months following YP enrollment, but increases were not large enough to result in major changes in the beneficiaries’ lifestyle. Keeping pace with rising incomes was the huge increase in savings, which doubled within the year following YP enrollment. Expenditures on necessities such as housing, food, utilities, health care, and education also rose significantly, leaving enough money for only modest increases in lifestyle consumption items such as clothing, mobile phones, radios, and so forth. Further, although the income of female beneficiaries has also increased, the change has been much less than that for male beneficiaries. Interestingly, those with xi Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report no education gained as much as those with TVET education, a testimony to the opportunities that self￾employment provided. One year is too short a period for big changes in health care–seeking behavior and in health outcomes—and none were seen in the 12 months following YP enrollment. However, the increase in income was associated with greater attention of beneficiaries to their health and that of their families, and with better health and lower levels of illness. As a result of peer group support and availability of someone to discuss their anxieties and problems with, beneficiaries—especially those with lower education levels—reported feeling lower stress levels and greater satisfaction with their employment status, leading to a general feeling of well-being and overall better health outcomes. YP strengthened participants’ confidence and work ethic and improved their self-perception of work readiness skills. Participants developed skills related to communication, work habits and conduct, teamwork, goal setting, job searching, and so forth and used them to expand the scope of their ongoing activities. However, with the passage of time and the hardships caused by the ongoing drought, the initial euphoria of having received training and being part of a coaching group was beginning to wear off, leaving some participants feeling less confident and more uncertain of their abilities. Recommendations With one year to go, several urgently important actions must be taken to maximize YP contribution toward increasing employment and income among youth in Ethiopia. In order to have an increased contribution toward that, there is a need to strengthen coaching and mentoring of youth, involvement of parents and guardians improve access to financial and marketing opportunities, and place more emphasis on the building of skills and access to employment opportunities for female youth. 1 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report INTRODUCTION Ethiopia, the second most populous nation in Africa with a population of about 102 million (2016), is the fastest growing economy in the region. Buoyed by the huge expansion in agriculture, construction, and services, and supported by increases in public investments, Ethiopia’s gross domestic product (GDP) has risen by more than 10 percent year-on-year since 2005–06, and poverty rates have fallen from 55.3 percent at the turn of the century to about 30 percent in 2017. At this rate, Ethiopia is on track to become a lower-middle-income country by 2025.6 Recent trends in economic growth notwithstanding, the average income in Ethiopia is very low (US$ 783 in 2017) and roughly 30 percent of Ethiopians live in poverty and are vulnerable to food insecurity.7 Literacy rates among the country’s youth between 15 and 29 years of age, which account for 29 percent of the country’s population, are low (47 percent for women and 63 percent for men) and are even lower if the entire population over 15 years of age is included (28 percent for women, 49 percent for men).8 Although access to education has increased in recent years and primary school enrollment has quadrupled, improvements in learning outcomes and quality of education remain elusive. In addition, Ethiopia faces huge challenges in maternal mortality, nutrition, and gender equality. Sustaining the double-digit economic growth and accelerating poverty reduction and food security require progress in job creation and improved governance, which are the foundational pillars of the five￾year Growth and Transformation Plan (GTP) of the Government of Ethiopia (GoE). GTP I was implemented between 2010–11 and 2014–15, a period that saw a steady increase in GDP, falling rates of unemployment, and the transition of the labor force from agriculture to construction and services.9 GoE is currently implementing GTP 2, covering the period 2015–16 to 2019–20, and is mobilizing internal and external resources through bilateral and multilateral cooperation. In supporting Ethiopia’s ambitious socioeconomic agenda, the United States Agency for International Development (USAID) launched the Building the Potential of Youth (YP) Activity in 2015 with a focus on providing unemployed and underemployed youth ages 15 to 29 with tools needed to succeed, while building the ability of national and local institutions to deliver demand-driven soft and hard skills. Designed to help youth in rural areas and towns obtain the abilities, knowledge, and social capital to increase their income and long-term economic self-sufficiency, this support aims at providing rural youth with literacy, numeracy, and life skills training alongside vocational and entrepreneurship development activities through partnerships with training, small business, and microfinance institutions. The five-year (January 2015 – December 2019), US$17.3 million, YP Activity is being implemented under a Cooperative Agreement (# AID-663-A-15-00006) between USAID and a group headed by Save the Children, Inc. (SC), including Education Development Center, Inc. (EDC), HUNDEE-Oromo Grassroots Development Initiative (HUNDEE), Professional Alliance for Development in Ethiopia (PADet), Relief Society of Tigray (REST), and Facilitator for Change Ethiopia (FCE). Awarded on January 1, 2015, and running until December 2019, YP is currently in its fourth year of implementation. YP has three objectives (see Annex 2 for the results framework): 6 The World Bank. 2018. Ethiopia – Country Overview. Accessed on June 1, 2018 at http://www.worldbank.org/en/country/ethiopia/overview 7 Ibid. 8 UNESCO. 2018. UIS Data for the Sustainable Development Goals. Accessed on June 4, 2018 at http://www.uis.unesco.org/Education/Documents/literacy-statistics-trends-1985-2015.pdf 9 The World Bank. 2017. Ethiopia – Country Partnership Framework for the Period FY18-FY22 (English). Accessed on June 1, 2018 at http://documents.worldbank.org/curated/en/202771504883944180/Ethiopia-Country-partnership-framework-for-the￾period-FY18-FY22 2 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report  Develop a rigorous evidence base to inform USAID and its development partners, including youth, GoE, and nongovernmental organizations, on appropriate, effective, and sustainable market-driven youth workforce development interventions.  Improve youth access to market-driven skills and experiences to support the transition to safe and viable employment.  Coordinate delivery among local institutions of quality market-driven services connecting underserved youth, especially women, to employment and income opportunities. Targeted to reach 34,537 unemployed or underemployed rural youth ages 15–29, YP focuses on the most vulnerable sections of society, particularly women, and those transitioning out of pastoralism. YP is achieving its objectives by:  Ensuring that training and service providers consider the market relevance and target curricula and programs for meeting the skills needs of youth participants.10  Offering tailored technical and life skills training to youth to create more viable livelihood prospects.  Expanding approaches and offerings in work-based learning (WBL), including employer visits, organized job shadowing, short-term employment, and internships/WBL opportunities, that will lead to workforce-ready and employable youth with practical experience.  Building Youth Service Provider Networks and providing a cost-effective approach to enhance support service access for remote communities and disadvantaged groups. YP is being implemented in six regions of Ethiopia (Afar, Amhara, Oromia, SNNPR, Somali, and Tigray). While the regions (and woredas within the regions) are selected in consultation with USAID, GoE, and other stakeholders, the kebeles from where the participants are drawn are nominated by the woreda administration. SC and its partners have split among themselves YP implementation across the six regions, with SC, in addition to its role as a leading partner, assuming responsibility for Afar, SNNPR, and Somali; PADet for Amhara; HUNDEE for Oromia; and REST for the Tigray region. A map showing the location of current activities is placed in the section titled Geographic Coverage. As of the end of December 2017, YP had provided life skills training, coaching, and mentoring to more than 18,312 youth (47 percent female), of which 8,183 youth are engaged in new or improved employment (43 percent female; 66 percent engaged in self-employment).11 YOUTH COHORT STUDY USAID/Ethiopia has contracted with Social Impact, Inc. (SI) to conduct an independent performance evaluation of YP. Carried out under the umbrella of USAID/Ethiopia Performance Monitoring and Evaluation Service (EPMES), which provides USAID/Ethiopia with project/activity external evaluation, performance monitoring, and performance management support services, the YP performance evaluation includes (i) baseline, midline, and endline data collection, analysis, and reporting; (ii) analysis of SC and its partners’ data collection, storage, analysis, and reporting systems; and (iii) a multi-round Youth Cohort Study (YCS) focused on a subset of youth participants and geographic implementation areas. This report presents the findings of the YCS. STUDY OBJECTIVES YCS aims to examine whether youth who received workforce development/livelihood support through YP attained increased income and a higher level of the necessary skills, knowledge, and social capital to achieve economic self-sufficiency over the longer term. The premise underlying YP is that youth 10 The terms “participants” and “beneficiaries” are used interchangeably in this report. 11 Youth Potential Activity – Annual Report PYII1; Youth Potential Activity – Quarterly Report – PY4 Quarter 1. 3 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report participation in training programs that build capacity and work readiness skills will result in better skills, including in functional literacy and numeracy, which in turn will better prepare them to find employment or engage in self-employment, leading to higher income and better health outcomes (see Activity Results Framework in Annex 2). Specifically, YCS addresses the following broad questions identified by USAID in the statement of work (See Annex 1):  To what extent has the income and employment status of study participants changed between enrollment in the YP Activity and the twelve months following enrollment? Does increased income lead to improved health outcomes among study participants?  To what extent has participation in the YP Activity improved the work-readiness skills of study participants?  To what extent has participation in the YP Activity improved knowledge of and access to employment/entrepreneurial resources for study participants?  To what extent has participation in the YP Activity improved functional literacy and numeracy skills of study participants?  How do the outcomes addressed in the questions above vary by participant characteristics (e.g., sex, age, ethnic/linguistic group, SES, marital status, parenting status) and by service delivery partner? The study will inform stakeholders, USAID, the Government of Ethiopia (GoE), and YP implementers about YP and its progress toward near-term outputs and outcomes. In addition, it will demonstrate long-term contributions of YP initiatives on a youth cohort and highlight major strengths and weaknesses of YP initiatives to enable any necessary adjustments and inform the design of similar activities. Summaries of the baseline and midline studies completed as part of the YCS are included in Annexes III and IV, respectively. This final YCS report combines the findings of the endline study with the previous two studies to provide a comprehensive view of YP contributions. . METHODOLOGY Study Design To monitor the dynamics of poverty and movements in and out of the labor market, the YCS adopted a panel design in which repeated measures were collected from the same sample at three different times 6 months apart over 12 months. A mixture of qualitative and quantitative methods was employed for the collection of primary and secondary data (see Annex VI for sources of data). Primary data were collected through (i) a survey of a cohort of YP participants; (ii) focus group discussions (FGDs) with a subgroup of these participants; (iii) FGDs with the parents/guardians of the participants; (iv) key informant interviews (KIIs) with stakeholders, including government officials, employers, training providers, and YP implementers; and (v) monthly interviews of a subgroup of participants from the March/April 2017 cohort to monitor employment status and use of income (financial diary). Secondary data were collected from a review of documents, including programmatic reports and databases already in place for YP, and were triangulated against the primary data to obtain a more complete picture. Sampling Of the six regions in which YP is being implemented, data were collected from a sample of participants (cohort) drawn from Afar, Amhara, Oromia, and Tigray regions. The statement of work for YCS suggested that data be collected from Amhara, Oromia, Somali, and Tigray regions. However, due to instability in the Somali region, it was, in consultation with USAID and SC, substituted with the Afar region, which, like Somali, is a pastoral region and where, as in Somali, YP is being implemented by SC. The sample size for the quantitative survey of YP participants was set by the statement of work of YCS at 360-400 youth at endline. Working backward and setting the attrition rate at 20 percent between 4 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report baseline and midline and then again between midline and endline suggested a baseline sample size of 560–600 youth trainees. The 560–600 youth trainees were selected from woredas and kebeles from within the four regions through a multistep stratification process to ensure that the selected woredas and kebeles were representative of all woredas and kebeles in all six YP regions. Across the six regions, YP is being implemented in 30 woredas, of which 37 percent are classified as pastoral and 63 percent as productive. Within the four regions identified for data collection, YP is being implemented in 21 woredas, 24 percent of which are pastoral. Since each woreda enrolled 40–50 participants each month, 13 woredas were selected on the criteria that (i) each of the four regions selected was covered, (ii) each service delivery partner was included, (iii) the pastoral-productive mix was representative of all woredas in which YP was being implemented, and (iv) the woredas represented all the years for which YP was being implemented. Since there were only six pastoral woredas in the four regions, the study oversampled them and included five pastoral woredas. Overall, five pastoral (38 percent) and eight productive (62 percent) woredas were selected from the four regions as follows: Afar (three pastoral), Amhara (four productive), Oromia (two productive and two pastoral), and Tigray (two productive). Data were collected from all participants who enrolled in YP in March–April 2017 in the selected woredas, yielding a cohort of 670 YP participants at baseline. Financial monthly diary data, which were illustrative of changes in the spending habits of YP beneficiaries, were collected through monthly interviews conducted over eight months from a purposive sample of YP participants from one training group (20–25 participants) from each region, pastoral and productive, and from four woredas (one per region) that were easy to access from the perspective of data collectors. Qualitative data were collected from a purposive sample of half of the woredas from each region, for a total of seven woredas (two each in Afar, Amhara, and Oromia and one in Tigray). Over the year-long study, data were collected through 100 FGDs conducted with participants and their parents and guardians and 172 KIIs (see Annex VII for details). Many of the KIIs were conducted with the same individuals at baseline, midline, and endline. Similarly, many of the FGD participants were common between baseline, midline, and endline. Data Collection Qualitative and quantitative data were collected at three points: (i) baseline, in the month of YP enrollment (collected in April/May 2017); (ii) midline, at six months after enrollment (collected in September/October 2017); and (iii) endline, at 12 months after enrollment (collected in March/April 2018) (see Annex X for composition of the team). Qualitative data were collected by the study team over four weeks each at baseline, midline, and endline. FGDs with participants and their parents and guardians were conducted in the local language, with the assistance of translators fluent in the regional language as well as English. KIIs were conducted in the regional language or in English, depending on the preference of the respondent. Quantitative data were collected through surveys administered by trained data collectors who used handheld electronic devices to ensure rapid and accurate data collection. The survey was translated into the regional languages prior to administration. Income and expenditure data were also collected periodically (using monthly interviews) from a purposive subset of the cohort for eight months. At each round, data were collected using the same methods and tools, with only minimal revisions to tools to improve on the clarity of questions and to fill evaluative information gaps identified during the baseline or to collect data that were likely to be available only at 6 months or 12 months after enrollment. Starting with a cohort of 670 youth, the endline survey yielded data on 458 participants, for a net response rate of 68 percent. Of the 670 participants surveyed at baseline, 582 responded (response rate 87 percent). These 582 respondents from the baseline were surveyed again at midline, and the second 5 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report round of primary data were obtained on 533 participants (response rate 92 percent). And finally, these 533 participants were surveyed at endline, and the third and final round of primary data were obtained on 458 participants (response rate 86 percent) (see Annex VII for sample distribution). The overall attrition rate was lower than anticipated and well within the buffer provided for in the sampling. Attrition over the year was not limited to any one subgroup. In addition, the demographic, education, and employment profile of the 458 participants that stayed until the endline survey was very similar to the profile of the 582 participants that responded at the time of baseline,12 suggesting that the sample was not compromised in any way due to the loss of 124 participants between baseline and endline. Fifty-seven percent of study participants were male, and two-thirds were in the 19–24 age group at both baseline and endline. Half of the participants had completed at least Grade 9, three-quarters could read and write fluently, and a little over nine-tenths could add simple numbers—proportions that stayed consistent over the year. More than 95 percent of the study participants were native to the region where they were living at the time of the study, and almost all spoke the language of the region (see Annex IX, Table XI for sample profiles). Data Analysis A combination of quantitative and qualitative methods was used to analyze the data at every stage of collection. Quantitative data were analyzed using descriptive univariate analysis, which involved computation of means, frequencies, and cross-tabulations to provide a description of the sample as well as outputs and outcomes following the intervention. Data were prepared to allow for comparisons between baseline and endline as well as for univariate and multivariate analysis and regressions as necessary. Qualitative data from a review of documents, interviews, and FGDs were coded into categories and classifications, and the emerging patterns and themes were analyzed in the context of the evaluation questions to document change over time. Where possible, data were disaggregated by participant characteristics, including sex, age, and region (which also served as a proxy for(i) the service delivery/implementing partners and (ii) pastoral/productive regions). Tests of significance were carried out as needed to determine whether the data underlying the change from baseline to endline were likely to be associated with YP and not likely to occur randomly or by chance. STUDY LIMITATIONS  The assessment of YP contribution on YCS participants is based on self-reporting, both in quantitative surveys as well as in FGDs. Although using a combination of methods and triangulating with data from other sources such as key stakeholder interviews could have mitigated the resultant bias, the likelihood remains of a response-shift bias caused by a change in the participants’ frame of reference across measurement points. This change would result in different responses to the same set of questions administered at different times, thus changing the bias at each measurement point.13  YP targets unemployed or underemployed youth, a category created by YP based on certain criteria, and participants were assigned to one of three categories—employed, unemployed, or underemployed—at the time of enrollment. Although it is assumed that all participants who reported their status as employed at baseline were actually underemployed, lack of access to these data, despite multiple requests at the time of sample selection, has made it difficult to address the findings for the underemployed youth separately from those categorized as employed. 12 Table with descriptive statistics of the baseline, midline, and endline samples placed in Annex IX. 13 See, for instance, R. Rosenman, V. Tennekoon, and L. Hill. 2011. “Measuring Bias in Self-Reported Data.” International Journal of Behavior Healthcare Research Oct; 2(4): 320–332. 6 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report  Given that there are no standard definitions of the terms “underemployment” or “better employment,” this evaluation tracks the self-reported employment status for all study participants and does not disaggregate outcomes for the underemployed or better employed. Indicators such as satisfaction with employment, ongoing search for employment, and change in income are used as measures of improvement in employment status.  Data have been disaggregated by region as a proxy for service delivery partner and pastoral/ productive area. Even though further disaggregation within region by rural and semi-urban areas or by smaller administrative units could provide greater insights, data have not been disaggregated to this level because introducing too much granularity into the analysis could lead to erroneous conclusions.  Sustaining participation in regular monthly interviews for the collection of income and income use data (financial diary) was challenging because of the desire by participants to be compensated. As a result, financial data were collected for only 8 months instead of the planned 12 months, which could have resulted in inaccuracies in arriving at estimates of income and its use.  Although some conclusions have been posited, one year is too short a time to expect changes in health care–seeking behavior or health outcomes. It is difficult, therefore, to relate any change in health outcomes to YP interventions. 7 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report MAIN FINDINGS The study examined participation in training, coaching, and mentoring and changes between baseline, midline, and endline in participants’ employment and income status, utilization of health care, self￾perception of work readiness skills, knowledge of and access to employment/entrepreneurial resources, and literacy and numeracy skills.14 TRAINING, COACHING, AND MENTORING Participation in training programs By endline, all study respondents from the March/April 2017 cohort had completed WRN! or WR Plus, the basic components of the YP training program.15 All participants in Amhara, Oromia, and Tigray had attended the WRN! training, and in Afar, 59 percent had attended WRN! and the rest had attended the WR Plus program. Although developing a Personal Development Plan (PDP) was an integral part of the WRN! training and was designed to guide the activities offered to the participants, only about half of respondents overall (ranging from 24 percent in Afar to 65 percent in Tigray, 55 percent male and 41 percent female) reported having developed one. Most respondents who had developed a PDP reported following the plan (84 percent male, 77 percent female), even though there was some confusion on the period covered by the PDP (three months to five years). Most respondents (79 percent male, 72 percent female) reported that they reviewed the PDP periodically on their own, whereas others reviewed it with their parents/guardians, spouses, partners, and friends. FGD participants presented a different picture, suggesting that almost all had developed a PDP during the training. Interviews with facilitators confirmed this reporting. However, most participants did not use the PDPs as a guide to implementing what was planned. Participants, especially those still unemployed, reported that “they were beginning to revisit their PDPs and to follow them and update them to their current situation” (male participant, Tigray). Many also said that knowing what they know now, “they would have been more careful about what they wrote in their PDP” (female participant, Amhara), a sentiment also expressed by a few of the facilitators who said that if they could go back in time, they would spend more time reviewing the PDPs to ensure they were viable. Regions varied widely in the proportion of study participants that had taken the Youth in Action (YIA) and Be Your Own Boss (BYOB) training.16 Sixty-eight percent of respondents in Oromia had attended 14 Throughout this analysis, differences in outcomes are assessed in terms of “statistical significance”—that is, differences that are not attributed to chance. Technically, this means that if there really is no difference between two outcomes or two statistics, there is a low probability of getting a result that large or larger. To determine whether the observed difference is statistically significant, we compute the p-value, or the probability of observing the difference if no difference exists. If the p￾value is less than a predetermined level, called the significance level, the difference is assessed as being statistically significant. Throughout this study, the significance level of 1 percent is set as the cut-off for strong significance and 5 percent as the cut-off for weak significance. However, we note that statistical significance does not mean necessarily mean practical significance, and a low p-value indicating a difference is statistically significant does not necessarily mean that the difference is important. For this reason, where the outcome numbers are close, we also calculate the “effect size,” which is simply the magnitude of the difference. The magnitude can be expressed as the “absolute effect size,” or the difference between the mean outcomes in two different intervention groups. Absolute effect size does not consider the variability in scores, in that not every subject achieved the average outcome. Variability can be accounted for by taking the difference between the two groups and dividing it by the standard deviation of one of the groups, also known as “Cohen’s d.” Where the absolute effect size is small (Cohen’s d<.2), both substantive significance (Cohen’s d) and statistical significance (p-value) are calculated and reported. 15 WRN! and WR Plus training (specifically for participants with no/low levels of literacy) provided participants with soft skills in interpersonal communication, work habits, and conduct and developed their leadership skills. Participants were expected to develop a personal development plan at the end of the training. 16 PYD, also called the Youth in Action (YIA) training, prepared participants for the next phase by encouraging self-reflection as well as focus on life objectives and goals. BYOB (Be Your Own Boss) training provided trainees who wished to start their own businesses with skills in developing business and work plans, basic bookkeeping, assistance with loan applications, and so forth. 8 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report the YIA training, compared with none in Afar. Similarly, the proportion of respondents that had attended BYOB varied from 24 percent in Afar to 89 percent in Tigray. This disparity was corroborated by interviews with YP staff, who indicated that BYOB training was offered only to participants who had decided to set up their own business as indicated in their PDPs. In addition, the point at which the training was offered varied across regions and by woreda within regions, with some offering BYOB immediately after the WRN! training and others offering it a few months down the road depending on available resources and the need to meet outreach targets.17 Approximately one-tenth of the survey respondents reported having benefited from YP scholarships to attend hard-skills technical training provided by external providers (e.g., technical and vocational education and training [TVET] and technical training institutes [TTIs] or private providers) and WBL opportunities. Training programs in agriculture, animal fattening, and poultry farming were selected equally by all participants regardless of region and sex, but more males opted for tailoring, driving, bakery, and construction work and more females for hairdressing, hotel and tourism industry information and communications technology (ICT), and computer skills. Training in agriculture and animal husbandry, as well as provision of coffee and tea services, were popular with participants across all education levels. Tailoring was popular among those with lower education levels, whereas the more educated preferred bakery, marketing, tourism, hairdressing, and ICT. FGD data and KIIs conducted in the regions supported these findings. Trainings in hard and soft skills through TVETs, TTIs, or private providers (tailoring, hairdressing, etc.) were fee-based (where tuition fee was paid to the provider by YP on behalf of the participants) and offered in accordance with the YP plan to provide training to about 7 percent of beneficiaries from each region. Beneficiaries for this initiative were selected based on their PDPs, evidence of their continuing interest and capability, and budget availability. In Amhara and Afar, for instance, training for many beneficiaries was delayed by several months because of unavailability of funds and competing priorities (according to KIIs at midline and endline as well as a review of YP records). Training program implementation varied across regions but was within YP guidelines. For instance, in Amhara, YP offered grants to private providers to train the participants with the understanding that the provider would in turn offer start-up help/capital to the trainees. In other regions, like Tigray, recognizing that not all training needed to be two to three months in duration, YP staff arranged for shorter training with lower tuition, enabling it to cover a larger number of participants with the same budget. Participation in internships/work-based training YP offered participants exposure to future work through job shadowing and WBL. Job shadowing was provided as a half- or full-day activity as part of the WRN! training to expose participants to self￾employment or other activities they may be considering. WBL activities, for the most part, ranged from three days to two weeks in duration. Only 8 percent of the survey respondents overall (n=39), mostly from Tigray and Amhara, reported participating in job shadowing and WBL. Most of these activities were less than two weeks long and the decision on where the training would take place was taken by the facilitator in consultation with the participants and their parents/guardians. Interviews with YP staff and FGDs with participants indicated that the four regions had operationalized WBL in different ways. In Amhara, job shadowing was offered as a part of the WRN! program, whereas in Oromia, it was offered only to those planning to go for wage employment. Interviews with the YP regional manager in Oromia indicated that although job shadowing was planned to be offered to 30 percent of participants, only 17 percent had been placed to date (overall for the program). In Amhara, YP staff reported that they had taken participants to see best practices in agriculture as part of job 17 To meet outreach targets, resources were diverted to train new participants, and BYOB training for existing participants was postponed to the next quarter/financial year. 9 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report shadowing; however, the cost of transportation for all participants has made them reconsider the approach. In addition, they were helping provide apprenticeships for the participants through arrangements with parents and family members. In Afar, the terms job shadowing and WBL were used interchangeably. Even though none of the survey participants from Afar reported participating in any kind of WBL activity, during FGDs, participants in Chifra (Afar) reported that they had shadowed a farmer to learn best practices for growing tomatoes and onions. A female FGD participant in Amhara reported that following a job shadowing visit with a trader in a city, she was able to expand her inventory and increase her income. FGD participants indicated that these programs had been very beneficial and that they had learned from observing “a business similar to [their] own.” Interviews with YP staff revealed that because job shadowing and WBL opportunities were few, YP had enlisted the support of various stakeholders—including government agencies (such as Youth and Sports, Children’s and Women’s Affairs, Labor, Micro and Small-Scale Enterprise, municipalities, TVETs, and others) and nongovernmental organizations (such as microfinance and private organizations)—to identify and create internship/WBL opportunities for the beneficiaries. Over the past year, YP had collaborated with several ongoing activities and programs in the region, such as the sugar factory in Amhara and laundry center in Afar, to leverage resources to support the participants. YP had established agreements and memoranda of understanding with several of these organizations. The agreement with the sugar factory, for instance, was that YP would provide capacity building for the management staff in exchange for internships/WBL opportunities for its beneficiaries. With TVET, the agreement was that YP would provide the machines for the center and TVET would provide hands-on experience to the participants.18 Each of these initiatives has met with varying levels of success. Coaching and mentoring The facilitators provided coaching in one-on-one and group coaching sessions and through information exchange with other programs, such as the SC Youth Economic Strengthening (YES) program. Only 16 percent of survey respondents reported receiving coaching and 7 percent reported receiving mentoring, with youth from Afar and Oromia reporting limited or no access to coaching and mentoring at endline. The frequency of coaching varied across regions from at least once a month (44 percent) to once every six months (32 percent). A majority (78 percent) of the participants that had received coaching reported that it helped them problem-solve and learn from others and that it served as a venue for socialization. Participants engaged in wage employment reported receiving guidance on finding employment opportunities, completing a job application, and writing a resume, and the self-employed reported that the coach/facilitator had helped them identify self-employment opportunities, develop business plans, connect with a loan institution, and even form a group. Similarly, of those that reported receiving mentoring, the majority (75 percent) reported learning new skills and doing better at work because of the mentoring. A little over half (56 percent) also reported having learned to manage their accounts better, and a similar number indicated having found a new direction and area of focus in their work. FGDs and interviews with facilitators reaffirmed the benefits of coaching and noted that group coaching also provided the participants with opportunities to learn from their peers and held them accountable for continued savings. As one participant noted, “the facilitators come, encourage us, check on monthly savings, and talk about the challenges faced.” But not everyone was satisfied with the level of engagement, as one young person from Yabelo Woreda in Oromia commented, “the training was good, but we are left with only the training.” Interviews with YP staff revealed several challenges related to the coaching program, such as the heavy workload of the facilitators and woreda officers, distance, rough terrain, and the growing number of 18 However, these arrangements had limited success, and the YP staff are exploring alternatives with private training providers whereby block grants to the providers could cover the cost of training and include WBL opportunities for participants to gain experience. 10 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report participants. In some cases, facilitators/coaches took up to four hours to get to hard-to-reach kebeles by walking or using public transport. YP had purchased motorbikes for staff use, but most did not have licenses to drive them. Because YP kept coaching open to all beneficiaries, the number of participants per facilitator also grew, which made it difficult for the facilitators to do full justice to all their trainees. The facilitators did not always have enough time for following up with individual coaching. In Tigray, for example, the staff prioritized training and pretraining selection activities over the provision of post￾training follow-up and coaching and offered training in the morning and coaching in the afternoon to as many participants as could be reached. Further, because the coaches met with the participants for group coaching fairly regularly, new materials were needed to keep the attention and interest of the participants. The situation was further exacerbated by staff turnover. In Yabelo (Oromia), for instance, at endline the implementing team had a difficult time finding replacements for two facilitators who had been covering 11 kebeles but had resigned a couple of months ago. A similar situation was noted in Afar (baseline) and Amhara (midline). FGDs revealed that participants found the YP mentoring program useful. The mentoring program, which started in October 2017, was very new and had been utilized at endline by only 7 percent of the participants, partly because of the program’s novelty and partly because, as interviews with YP staff revealed, participants did not think highly of the expertise of the mentors from their own communities and were thus hesitant to go to them. Distance and logistics posed additional challenges, as beneficiaries found it difficult to get to the mentors in kebeles (and residences within kebeles) that were spread out within woredas. Although the users of mentoring services reported benefiting from the experience, there was no evidence to suggest that the lack of mentoring had adversely affected the achievement of expected results. EMPLOYMENT STATUS OF STUDY PARTICIPANTS19 Employment status Employment rose by more than 40 percent between YP enrollment and 12 months following enrollment. As Table 1 shows, 69 percent of study respondents at endline were employed, (statistically) significantly higher than 48 percent at baseline (p<.01). Self-employment saw the most growth—from 41 percent at baseline to 60.5 percent at endline. The difference is both statistically significant (p<.01) and nontrivial in effect size (Cohen’s d=.264). Wage employment among study respondents rose from 6.9 percent at baseline to 8.5 percent at endline, but the change was small (Cohen’s d=.044) and not statistically significant (p=.16). Correspondingly, only 27.5 percent of the study respondents were unemployed (seeking work) at endline, a statistically significant decrease from 45.4 percent at baseline (p<.01) and modest in effect size (Cohen’s d=-.251). Survey data indicate an initial spurt in employment at the midline point, which was tempered, however, by the time of endline data collection. Table 1: Employment status at baseline and endline (proportion responding) Employment Status Baseline (n=582) Midline (n=533) Endline (n=458) p-value* Effect Size In school or training 4.6% 2.1% 2.8% .17 .079 Paid/wage/paid-in kind employed 6.9% 8.8% 8.5% .16 .044 Self-employed 41.0% 64.9% 60.5% <.01 .264 Unemployed (seeking work) 45.4% 22.5% 27.5% <.01 -.251 Unemployed (not seeking work) 2.0% 1.7% 0.7% .08 -.035 *Paired t-test and effect size are computed on the common sample of 458 study respondents at endline and baseline. 19 This section addresses the research question: To what extent has study participants’ employment status changed between YP enrollment and 12 months following enrollment? 11 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Source: Authors’ calculations, based on YCS baseline, midline, and endline survey data. A sizeable proportion of the respondents in each period was looking for new employment or for a change in employment. As Figure 1 shows, two-thirds of the wage-employed at baseline were looking to change their employment. This proportion fell to 55 percent at midline but rose again to 62 percent at endline. There was much less desire for movement among the self-employed, with only about one-third of the self-employed at baseline looking for other forms of employment, rising to 41 percent at midline and declining to 36 percent at endline. The proportion of unemployed looking for employment increased over the year, from 32 percent at baseline to 46 percent at midline and 52 percent at endline. Figure 1: Respondents looking for work, by employment status (%) Source: Authors’ calculations, based on YCS baseline, midline, and endline survey data. Both male and female study respondents reported an increase in employment at endline compared with baseline. As Table 2 shows, 79 percent of male respondents were employed at endline, significantly more than 54 percent at baseline (p<.01). Likewise, 57 percent of female study respondents at endline were employed, significantly more than 40 percent at baseline (p<.01). Self-employment saw the most growth—from 45 percent of study respondents at baseline to 68 percent at endline for males (p<.01, Cohen’s d=.212) and from 36 percent at baseline to 51 percent at endline for females (p<.01, Cohen’s d=.304). Wage employment among male and female study respondents rose from 9 percent and 4 percent, respectively, at baseline to 11 percent and 6 percent, respectively, at endline, but neither of the changes was substantively significant (Cogen’s d=.022 for males and .059 for females) or statistically significant (p=.18 for males and .59 for females). Correspondingly, 18 percent of male and 40 percent of female study respondents were unemployed (seeking work) at endline, a significant decline from the baseline values of 39 percent for male and 54 percent for female respondents (p<.01 and effect size nontrivial in both cases). Table 2: Employment status at baseline, midline, and endline (proportion responding, by sex) Male Female Baseline (n=332) Midline (n=304) Endline (n=261) p￾value* Effect Size Baseline (n=250) Midline (n=229) Endline (n=197) p- value* Effect Size In school or training 5% 2% 3% .20 -.024 4% 3% 3% .57 -.048 Paid wage/in kind employed 9% 10% 11% .18 .022 4% 7% 6% .59 .059 Self-employed 45% 69% 68% <.001 .212 36% 59% 51% <.001 .304 Unemployed (seeking work) 39% 18% 18% <.001 -.194 54% 29% 40% <.001 -.296 12 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Male Female Baseline (n=332) Midline (n=304) Endline (n=261) p￾value* Effect Size Baseline (n=250) Midline (n=229) Endline (n=197) p- value* Effect Size Unemployed (not seeking work) 3% 1.0% 0% <.05 -.026 3% 2% 2% .48 -.044 *Paired t-test and effect size are computed on the common sample of 261 male and 197 female respondents at endline and baseline. Source: Authors’ calculations, based on YCS baseline, midline, and endline survey data. All four regions reported increases in the proportion of study respondents that were employed at endline compared with baseline. As shown in Table 3, the largest increase was reported in Tigray, where the proportion of employed study respondents increased threefold, from 24 percent at baseline to 70 percent at endline (change large and statistically significant, p<.01). This was followed by Afar (17 percent at baseline to 47 percent at endline, change large and statistically significant, p<.01), Amhara (65 percent at baseline to 83 percent at endline; change modest but statistically significant p<.01), and Oromia (60 percent at baseline to 68 percent at endline; change small and statistically not significant). Increases in the proportion of self-employed explained most of the increases in employment in all regions. Table 3: Employment status at baseline, midline, and endline (proportion responding, by region) Employment Status Afar Amhara Oromia Tigray BL (n=108) ML (n=95) EL (n=87) BL (n=200) ML (n=172) EL (n=133) BL (n=178) ML (n=171) EL (n=155) BL (n=96) ML (n=95) EL (n=83) In school or training 8% 1% 1% 5% 5% 7% 8% 1% 2% Paid/wage/paid-in kind employed 2% 5% 5% 13% 11% 12% 5% 8% 6% 3% 10% 12% Self-employed 15% 50% 43% 52% 70% 71% 55% 60% 63% 21% 80% 58% Unemployed (seeking work) 69% 45% 53% 32% 18% 15% 34% 24% 24% 68% 10% 28% Unemployed (not seeking work) 7% 2% 1% 2% 1% 3% 1% 0% 2% Source: Authors’ calculations, based on YCS baseline, midline, and endline survey data. Study respondents at all education levels reported an increase in employment between baseline and endline. As Table 4 shows, study respondents who went to college experienced the largest gains in employment (from 29 percent to 73 percent), followed by those who had no schooling or attended traditional schools (from 34 percent at baseline to 75 percent at endline) and those who had completed TVET (from 45 percent at baseline to 84 percent at endline). Self-employment was the driver of this increase in employment, rising significantly across all education levels. In comparison, wage employment went up significantly at endline compared with baseline only for those with college or university degrees (27 percent at endline compared with 12 percent at baseline); the increase was neither large nor statistically significant at other education levels. Table 4: Employment status at baseline, midline, and endline (proportion responding, by education level) No School/ Traditional Grade 1 to 5 Grade 6 to 8 Grade 9 to 12 Completed TVET College/ University BL ML EL BL ML EL BL ML EL BL ML EL BL ML EL BL ML EL n= 32 n= 32 n= 24 n= 85 n= 62 n= 50 n= 140 n=132 n= 108 n= 279 n= 257 n= 227 n= 29 n= 39 n= 38 n= 17 n= 11 n= 11 In Percentages In school or training 0 0 2 2 5 0 0 0 2 2 5 0 0 0 2 2 5 0 Paid/wage/ paid-in kind employed 0 10 5 10 18 18 0 10 5 10 18 18 0 10 5 10 18 18 13 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Self-employed 7 5 74 72 61 51 36 75 74 72 61 51 36 75 74 72 61 51 36 Unemployed (seeking work) 2 5 16 20 25 23 45 25 16 20 25 23 45 25 16 20 25 23 45 Unemployed (not seeking work) 0 0 1 2 3 0 0 0 1 2 3 0 0 0 1 2 3 0 Source: Authors’ calculations, based on YCS baseline, midline, and endline survey data. The analysis suggests that YP contributed to a significant proportion of the increase in the likelihood of employment in the 12-month period between baseline and endline. Logit regression analysis finds that the likelihood of the study respondents being employed rose from 49 percent at baseline to 69 percent at endline. Using the Fairlie20 technique to decompose this difference of 20 percent suggests that 45 percent of this difference (i.e., 9 percentage points) can be explained by changes in demographic characteristics, education level, and spatial variations due to unobserved factors at regional level,21 and the remaining 55 percent of this difference (i.e., 11 percentage points) is contributed by YP as well as by changes due to unobserved factors.22 No difference is premised on the intertemporal influence of unobserved factors since the same respondents are surveyed in baseline and endline and are expected to be exposed to shocks, if any, the same way (see Tables X6 and X7 in Annex IX for details). Similar trends were noted among the subsample of respondents tracked through monthly financial diary interviews. A majority of the respondents, both male and female, indicated that they were self-employed (Figure 2), while a small proportion reported being engaged in wage-based employment or working for their families as unpaid employees. A small number also reported holding secondary jobs; these respondents reported working an average of 22–25 days per month in their primary jobs and 4–5 days on their secondary jobs. The average number of days worked varied by region (ranging from a high of 26 days per month in Afar to a low of 18 days per month in Oromia). 20 Fairlie, R. W. (Nov. 2003) An Extension of the Blinder-Oaxaca Decomposition Technique to Logit and Probit Models http://www.econ.yale.edu/growth_pdf/cdp873.pdf 21 Regional dummies are used to control for spatial variation of the unobserved factors, such as influence of unobserved socioeconomic differences or heterogeneity across regions. 22 The Fairlie decomposition method is a widely used technique for identifying and quantifying the separate contributions of group differences in measurable characteristics on binary outcomes. The decomposition technique involves comparing the employment status of the same respondents across two time periods, baseline and endline, and identifying changes that occurred in employment between the two periods of time, baseline and endline. 14 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Figure 2: Distribution of employment status over eight months for a subgroup of survey participants (n=81) Source: Authors’ calculations, based on data from monthly (financial diary) interviews with a select group of participants. One significant change between YP enrollment and 12 months following enrollment was the big increase in self-employment in income-generating activities or business (the terms “income-generating activity” and “business” are used interchangeably in this document). Individual self-employment has emerged as the dominant employment modality for study respondents across gender and geographical regions, and both survey results and FGDs confirmed this preference. Principal reasons given by study respondents for preferring self-employment over wage employment were (i) greater income-generating/earning potential, (ii) better and continuous source of income, and (iii) satisfaction of being one’s own “boss” and not having to report to somebody else. In fact, there were cases of YP participants leaving their jobs and seeking business opportunities. One such example is that of two participants from Amhara, who had worked as Bajaj drivers until recently but had left their jobs and were unemployed and awaiting the approval of a loan to purchase a Bajaj (at the time of endline data collection).23 FGD participants also noted that the number of available wage employment opportunities were limited, despite the success stories of some participants who had benefited from the YP training and had found jobs. One such example is from Afar, where 75 participants who were trained in the early days of YP to become wage￾employed Afar language teachers successfully transitioned upon completion of the training program to become classroom teachers with the Bureau of Education. However, this intervention was implemented only one time, despite the relatively large number of youths who were helped by it to find wage employment. KIIs indicated that several wage employment opportunities in the regions had not been filled up. Interviews with representatives from stakeholder groups such as the Bureau of Labor and Social Affairs (BOLSA), TVET service providers, and other stakeholder groups across the four regions indicated that numerous untapped wage employment opportunities were available in areas such as business, accounting, ICT and secretarial, management, and mobile telephone repair (Afar); light manufacturing, construction, food processing, water systems, and hospitality (Amhara); and automotive, accounting, and network installation (Tigray). Similarly, there were untapped wage employment opportunities in the 23 A Bajaj is a popular brand of motorcycle rickshaw (manufactured by the Bajaj Auto company in India), and is the predominant form of transportation/taxi service in many cities of Ethiopia. 15 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report agriculture value chain across all the regions. In fact, several TVET service providers reported being approached by employers for help with recruitment of future employees (Tigray, Amhara), prompting the TVET providers to target their training programs toward the existing job opportunities. However, the exceptional level of collaboration in Tigray between BOLSA and YP did not exist elsewhere, and many participants who were potentially interested in wage employment opportunities did not approach BOLSA to seek support in finding wage employment. Table 5: Type of self-employment at baseline and endline (proportion of self-employed responding) Type of self-employment Self-employed Baseline (n=238) Self-employed Midline (n=348) Self-employed Endline (n=277) p-value* BL-EL Effect size Self-owned 66% 74% 79% <0.01 0.223 Family owned 31% 15% 16% <0.01 -0.241 Group owned 4% 11% 5% 0.57 0.022 *paired t-test and effect size are computed on the common sample of 197 study respondents at endline and baseline Source: Authors’ calculations, based on YCS baseline, midline, and endline survey data. Among the self-employed, there was a marked shift between baseline and endline away from working in family business toward working in self-owned enterprises. As Table 5 shows, among the self-employed, the proportion of study respondents working in self-owned businesses rose from 66 percent at baseline to 79 percent at endline, a modest (Cohen’s d=.223) but statistically significant change (p<.01). This was compensated by a decline in the proportion of study respondents engaged in family-owned businesses from 31 percent at baseline to 16 percent at endline, a modest (Cohen’s d=-.241) but statistically significant change (p<.01). The trend first noted at midline, six months after YP training, continued with some ups and downs at endline. This trend is also supported by data from the monthly financial diaries (see Figure 2). The trend was common across male and female respondents, and both moved away from working in family-owned to self-owned businesses. Among the regions, the most marked shift away from family-owned to self-owned businesses was in Tigray, where 90 percent of study respondents worked in self-owned businesses at endline compared with 60 percent at baseline, and 2 percent worked in family￾owned businesses at endline compared with 40 percent at baseline. Marked changes were also reported in Oromia, where 78 percent and 20 percent of study respondents worked in self-owned and family￾owned business, respectively, at endline compared with 57 percent and 41 percent, respectively, at baseline. Similar shifts in Amhara and Afar were relatively more modest. Also noted was a move toward group-based activities at midline, which did not continue at endline, a finding supported by data from FGDs where the participants reported a lack of trust and a common goal as reasons for not engaging in group-owned employment. In addition, as shown in Table 6, movement was also noted from unemployment to employment and from one type of employment to another. 16 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Table 6: Change in employment status and sector from baseline to midline to endline Baseline Midline Baseline Total Midline Total Difference Unemployed (including in school) Agriculture Trading Manufacturing Daily Labor Services Professional Unemployed (including in school) 94 70 25 5 11 21 6 232 117 -115 Agriculture 12 75 10 0 0 6 1 104 160 56 Trading 4 3 32 0 0 6 1 46 75 29 Manufacturing 0 2 0 3 1 2 0 8 12 4 Daily labor 2 6 1 2 8 3 0 22 20 -2 Services 4 4 7 2 0 24 2 43 62 19 Professional 1 0 0 0 0 0 2 3 12 9 Total 117 160 75 12 20 62 12 458 458 Midline Endline Midline Total Endline Total Difference Unemployed (including in school) 72 29 4 0 3 6 3 117 142 25 Agriculture 33 100 11 2 7 4 3 160 159 -1 Trading 21 13 32 2 0 7 0 75 56 -19 Manufacturing 1 1 0 5 2 2 1 12 15 3 Daily labor 4 2 1 2 9 0 2 20 21 1 Services 9 12 7 4 0 29 1 62 50 -12 Professional 2 2 1 0 0 2 5 12 15 3 Total 142 159 56 15 21 50 15 458 458 Note: This table presents the dynamic process of employment status and sheds light on change in employment status and intersectoral mobility from baseline to midline and from midline to endline. The rows represent the participants’ employment status at baseline and midline, respectively, while the columns show their changed status at midline and endline. For example, the table suggests that of the 458 respondents that completed the survey from baseline to endline, 94 respondents were unemployed at baseline and remained so at midline. Among the remaining 138 unemployed in baseline, 70 found jobs in agriculture, 21 in trading, 5 in manufacturing, 11 in daily labor jobs, 21 in service sectors, and 6 in professional jobs. Contrary to this trend, 12 respondents who were employed in agriculture during baseline became unemployed during midline; 75 of the remaining 92 are still in agriculture, while 10 moved to trading and 6 to service sector and one to professional sector. Similarly, the rows for midline to endline reflect movement in the past six months. Source: YCS baseline, midline, and endline survey data. Most study respondents preferred self-employment to wage employment, and most self-employed were engaged in agriculture because it was the most widely available income-generating activity, even if many believed that it did not offer the best earning potential. Sixty-three percent of study respondents who were self-employed at endline were engaged in agriculture and livestock production and agriculture￾related trading activities, very similar to 62 percent at baseline. Trading in consumer products, the next largest category, employed 13 percent of study respondents at endline and 11 percent at baseline (Table X4 in Annex IX). Twenty-three percent of those employed at endline had a secondary job, compared with 30 percent at midline and 35 percent at baseline. Data from monthly financial diary interviews for a select subgroup of participants also supported the above findings and indicated that participants were self-employed in areas such as agriculture, petty trading, and provision of services (such as tea/coffee). FGDs reinforced the preference for agriculture-related activities among the self-employed. Participants noted that “agriculture is a good opportunity because there are no other viable opportunities,” that “agriculture is a good opportunity because it helps to feed the country,” and “most of us have a strong background in the agricultural sector.” Those who did not agree that agriculture provided the best 17 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report opportunities noted that it did not generate a high income and was very labor intensive. Participant quotes in Table 7 summarize the FGDs. Table 7: Views on whether agriculture provided good employment opportunities Arguments in support Arguments against ● “Agriculture is a good option because it is productive; it is possible to work in this area with low inputs and in the absence of loans” (Tigray). ● ¨If fertilizer and land is available, then agriculture is a good opportunity¨(Oromia). ● “All believe that agriculture, particularly poultry farming, is a good opportunity” (Tigray). ● “All believe that agriculture is a good opportunity because it is part of their background as animal breeders; also, maize production feeds both animals as well as families” (Afar). ● “Lack of resources prevent good production; it is tiresome and requires investments/costs, and youth will not change in agriculture” (Oromia). ● “Even though we participate in potato production, the largest share of the profit margin goes to the brokers. As a result, we do not benefit from agricultural production as much as we should” (Oromia). ● “Lack of land and access to financial resources are key reasons why many of the youth do not believe that agriculture is a good opportunity” (Oromia). Source: YCS endline FGDs. Several FGD participants noted that they were involved in agricultural activities because of the training and support they received from YP, though not all agreed. Several FGD participants reported an increase in their income levels since becoming engaged in YP, either because their productive capacity had been enhanced due to technical and hard-skills training or because they had expanded their portfolios of agricultural activities in which they were engaged because of the support and guidance provided by YP. As one FGD participant in Afar put it, “[they] used to grow maize, now diversified activities within agriculture, including animal fattening. They sell coffee and sugar, and the profits cover their expenses - income has improved.” In Oromia (Yabelo), however, many FGD participants stated that they had not experienced any improvement in their agricultural productivity that could be attributed to YP. They stated that “although many remain engaged in self-employment agricultural activities, they have yet to experience the type of positive developments that others have experienced in other regions of the country because of the lack of support.” Although unrest in the target regions was not a major factor, drought was cited as a reason for unemployment by several FGD participants engaged in agricultural activities. Profile of those employed or unemployed An in-depth examination of the data to determine the profile of those employed and unemployed over the 12 months demonstrated some interesting similarities and differences among the groups. Most participants that reported being employed at baseline, midline, and endline (constant employment; 38 percent of the respondents at endline) were male, lived in Amhara and Oromia regions, were more likely to be older (in the 25+ age group), and living independently. They were also more likely to be engaged in agriculture and agriculture-related activities. Similarly, those who were unemployed at baseline but found employment after training through YP (employed post-training; 21 percent) were more likely to be male, in the 19–24 age range, and living with their families in Tigray. Those with insecure employment (unemployed at baseline and employed either at midline or endline not both; 27 percent) were equally likely to be male or female, in the 19–24 age range, living with their families in Oromia and Afar. The chronic unemployed (those who were unemployed at baseline and remained unemployed after training; 14 percent) were most likely to be female, younger, in the 15–18 and 19–24 age group and living with parents in Afar and Oromia. Education level did not appear to make a difference to the employment status of the participants (see Annex IX, Table X5 for details). Most participants with insecure employment were engaged in seasonal activities, including agriculture and daily labor. 18 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Gender differences in employment There were huge differences between employed male and female study respondents in their fields of work at both baseline and endline. As Table 8 shows, 65 percent and 62 percent of employed male respondents worked in agriculture-related activities at endline and baseline, respectively, compared with 42 percent and 39 percent employed female respondents at endline and baseline, respectively. Similarly, more than one-fifth of all employed female respondents were engaged in trading at endline and baseline, compared with 5 percent or less of employed males. There was a spike in the proportion of participants reporting they were engaged in trading at midline; however, at endline this number had again fallen to a proportion similar to baseline. Likewise, a much larger proportion of employed female respondents worked in the services sector than employed males. However, the gap in the proportions of males and females employed in the services sector and as daily labor has narrowed over the 12 months following YP enrollment, and fewer males are employed as daily labor (narrowing the gap from 9 percentage points at baseline to 2 at endline) and more employed females have left the services sector to take up professional jobs (narrowing the gap with employed males in the services sector from 17 percentage points at baseline to 5 at endline). Table 8: Change in field of work of the employed between baseline, midline, and endline Field of Work Male Female Baseline (n=182) Midline (n=242) Endline (n=205) Baseline (n=103) Midline (n=153) Endline (n=111) Agriculture 62% 61% 65% 39% 26% 42% Trading 3% 10% 5% 22% 40% 23% Manufacturing & Construction 5% 5% 6% 2% 1% 3% Daily Labor 14% 6% 7% 5% 6% 5% Services 9% 11% 11% 26% 20% 16% Professional 3% 2% 4% 0 5% 6% Other 4% 4% 2% 5% 3% 5% Source: Authors’ calculations, based on YCS baseline, midline, and endline survey data. Female FGD participants across the four regions stated that YP had provided them with critical soft skills, such as managing clients, planning income-generating activities/businesses, and financially managing such activities, that allowed them to expand the number of income-generating activities in which they were engaged. As a result, they had a more positive attitude toward work and were reaping the benefits of hard work. In addition, although they were still making decisions together with their families, female participants noted that they had a voice in the discussions about their future. Numerous female participants reported expanding what they were doing before (particularly for those engaged in animal farming and coffee/tea shops) or branching out into new activities. Both FGD participants and key informants indicated that the prevalent biases and stereotypes continued to have an impact on women’s ability to obtain loans and employment in areas not considered to be typical female domains. The greatest challenge that they were confronted with was gaining access to start-up capital. In Oromia (Wolmera), for example, the evaluation team found that only 2 of the 12 in the cohort group could access financial support to expand or start a service-oriented activity, such as a hair-dressing salon. Two other members of the same focus group were unemployed because they could not raise start-up capital, and the remaining eight had migrated to Arab countries to work as domestic workers. Satisfaction with employment Study respondents reported a marked increase in satisfaction with their chosen employment between YP enrollment and 12 months following enrollment. Compared with 44 percent of those employed (wage or self-employed) who were satisfied with their employment at baseline, 61 percent of employed 19 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report study respondents reported being satisfied with their jobs at endline.24 More female (65 percent) than male (60 percent) respondents reported being satisfied with their employment at endline, compared with 39 percent of males and 53 percent of females at baseline. Among age groups, the 25–29 age group was the most satisfied (67 percent) at endline; at baseline, the 15–18 age group reported being most satisfied (56 percent). In terms of education level, the largest increase in satisfaction with employment situation between baseline (40 percent) and endline (88 percent) was reported by those who had attended some college or university, followed by those with TVET education, who reported an increase in satisfaction from 46 percent at baseline to 72 percent at endline. The largest percentage of study respondents who were mostly unsatisfied or unsatisfied with their current employment situation at endline were in Amhara (28 percent); across the other three regions, less than 20 percent of the participants expressed being unsatisfied with their current jobs. FGDs yielded mixed results. On the one hand, participants noted that they were not satisfied with their current jobs (at endline) and maintained that YP had had limited impact on their employment status, since training provided through WRN, PYD, and BYOB did not help them improve their incomes and employment status (FGD participants in Yabelo, Oromia). They suggested that YP training should be complemented by post-training support, such as hard-skills training for more participants (beyond what is currently offered and more evenly distributed across cohorts), start-up capital, and/or access to financial support, in addition to linkages to financial institutions as currently offered. Some FGD participants in Afar, Oromia (Wolmera), and Amhara (South Achefer) also stated that there had been almost no change in their employment status, and as a result, they were not satisfied with their current employment situation. They also expressed their concern about the lack of post-training support, especially start-up capital, provided by YP. On the other hand, many study respondents in Afar, Amhara, and Tigray expressed satisfaction with their employment and noted that YP training had helped them develop relevant work-related skills in planning, accessing financial support, management, and customer service skills and that their incomes had increased. Reasons for being without work Lack of access to finance was cited by most study respondents as the main reason for being without work 12 months after enrolling in YP. Compared with only 3 percent of study respondents at baseline, 51 percent listed lack of finance as the main reason for being out of work at endline. At baseline, most respondents (62 percent) cited lack of employment opportunities as the reason for unemployment, but at endline only 32 percent identified it as a reason for no employment. The proportion of respondents citing mismatch between skills and job opportunities as the reason for unemployment also fell, from 30 percent at baseline to 19 percent at endline. At the same time, the proportion of respondents citing lack of education as a reason for unemployment went up from 21 percent at baseline to 31 percent at endline, suggesting a greater awareness of the importance of high levels of literacy and numeracy. These results were supported by FGDs, in which study respondents across the four regions identified four main reasons for lack of employment: (i) lack of access to finance/working capital, (ii) lack of land and/or working space, (iii) lack of start-up materials (equipment, tools, etc.), and (iv) lack of hard-skills training. FGD participants in Amhara and Oromia stated that lack of access to hard-skills training was one of the important reasons for high unemployment among youth. The reasons identified by the FGD participants in Amhara and Oromia were consistent with the statement from BOLSA in Afar region that youth in YP were not being provided with marketable skills such as construction, metal work, woodwork, computer, and driving license as determined by the local labor market needs. Parents and guardians of participants in Oromia identified many of the same reasons as above for high unemployment. The current drought situation also had an impact on participants’ employment status. As one parent in the Afar region reported, “my daughter used to work with her brother on chat farming, but this has stopped due to drought,” leaving her unemployed. Several participants also made similar statements. KIIs (with BOLSA 24 Satisfaction as noted here includes all those that reported being either “satisfied” or being “mostly satisfied.” 20 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report in Oromia, Bureau of Agriculture in Tigray, Bureau of Youth and Sport in Tigray, and Women’s Affairs in Oromia) reinforced these conclusions. KIIs with government officials25 suggested skills mismatch as the main reason for unemployment. As stated above, BOLSA officials in Afar stated that YP was not equipping youth with skill sets for occupations (e.g., woodworking, mild industry) within economic sectors that had the greatest demand for skilled workers. Other officials in Afar and Amhara stated that YP provides soft-skills training but does not conduct examinations and provide certificates at the end of the training programs. As a result, the skills are not marketable outside the region. This same argument was repeated by BOLSA officials in Tigray, who also noted that YP did not provide sufficient hard-skills training for participants. Officials of the Bureau of Women’s Affairs (BoWA) in Yabelo had a similar position, stating that YP placed insufficient emphasis on creating job opportunities for women. BoWA officials maintained that YP focused more on male participants than on female participants as the target group of its activities. INCOME, EXPENDITURES, AND ASSETS26 Income Level of Study Participants The vast majority of study respondents reported an increase in income between YP enrollment and 12 months following enrollment. As Table 9 shows, 71 percent of respondents reported an increase in income at endline compared with baseline. Incomes of 61 percent of respondents grew between baseline and midline—i.e., in the first six months following enrollment—and incomes of 55 percent grew between midline and endline. Overall, incomes of 13 % of respondents decreased between baseline and endline, while 12 percent reported no change. Incomes of 4 percent of respondents rose between baseline and midline but fell between midline and endline. Table 9: Change in income between baseline, midline, and endline Between Baseline and Midline (n=533) Between Midline and Endline (n=458) Between Baseline and Endline (n=458) Increased 61% 55% 71% Decreased 13% 12% 13% Stayed the same 26% 33% 12% Increased at midline, fell at endline 4% Source: Authors’ calculations, based on YCS baseline, midline, and endline survey data. Incomes of both male and female study respondents rose in the 12 months following YP enrollment, but more males than females benefited from an income increase. As Table 10 shows, incomes of three￾quarters of male respondents and two-thirds of female respondents rose between baseline and endline. Incomes of more female respondents (15 percent) than male respondents (11 percent) fell over the year, and incomes of more female respondents (15 percent) than male respondents (10 percent) remained unchanged. This finding is reinforced by regression analysis, which suggests that regardless of the type and sector of employment, male participants were more likely than female participants to report increase in income. As Table X6 (Annex IX) shows, male respondents were 42 percent and 23 percent more likely than their female counterparts to report an increase in income at midline and endline, respectively. 25 Government officials from the Bureau of Sports, Bureau of Women’s and Children’s Affairs, and Bureau of Labor sit on the committee for selection of YP participants. However, they are not a part of the team that designs the YP curriculum, and in some cases, as reflected in KIIs, do not entirely agree with the YP training. 26 This section addresses the research question: To what extent has study participants’ income changed between YP enrollment and the 12 months following enrollment? 21 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Table 10: Change in income between baseline, midline, and endline, by sex Male Female Change Between Baseline and Midline (n=304) Between Midline and Endline (n=261) Between Baseline and Endline (n=261) Between Baseline and Midline (n=229) Between Midline and Endline (n=197) Between Baseline and Endline (n=197) Increased 65% 60% 76% 56% 49% 66% Decreased 13% 10% 11% 14% 15% 15% Stayed the same 22% 30% 10% 30% 36% 15% Increased at midline, fell at endline 0% 0% 4% 0% 0% 4% Source: Authors’ calculations, based on YCS baseline, midline, and endline survey data. Among the four study regions, Tigray had the largest proportion of respondents whose incomes rose in the 12 months following YP enrollment. As Table 11 shows, incomes of 81 percent of respondents from Tigray rose at endline compared with baseline. Among other regions, incomes rose for 72 percent of respondents in Oromia, 68 percent in Amhara, and 66 percent in Afar during these 12 months. Incomes of 18 percent of respondents from Afar fell between baseline and endline, followed by Amhara (15 percent), Oromia (12 percent), and Tigray (4 percent). This finding is reinforced by regression analysis, which shows that compared with respondents from Tigray, the likelihood of reporting an increase in income is significantly lower in Amhara, Oromia, and Afar (see Annex IX, Table X6 and Figure X1 for details). Table 11: Change in income between baseline, midline, and endline, by region Afar Amhara Oromia Tigray Change BL-ML n=95 ML-EL n=87 BL-EL n=87 BL-ML n=172 ML-EL n=133 BL-EL n=133 BL-ML n=171 ML-EL n=155 BL-EL n=155 BL-ML n=95 ML-EL n=83 BL-EL n=83 Increased 63% 41% 66% 54% 55% 68% 56% 55% 72% 81% 71% 81% Decreased 14% 17% 18% 16% 13% 15% 18% 12% 12% 1% 7% 4% Stayed the same 23% 41% 10% 30% 32% 13% 27% 34% 12% 18% 22% 12% Increased at midline, fell at endline 0% 0% 6% 0% 0% 4% 0% 0% 4% 0% 0% 4% Source: Authors’ calculations, based on YCS baseline, midline, and endline survey data. FGDs with study respondents confirm the spatial heterogeneity in incomes reported above. In Tigray and Amhara, most FGD participants stated that they had experienced an increase in their income over the past six months. In Amhara, participants noted that their incomes had risen by 25 percent to 75 percent, but those in petty trading and with coffee shops said that “the inflation was having a negative effect on their income as the customers did not wish to pay more for the goods.” In Oromia (Wolmera and Yabelo), however, some FGD participants stated that they had not experienced an increase in income in the past six months. Afar also had a mixed response, in that some experienced an increase in income whereas others did not. Many FGD participants in Afar, especially the women, reported that the ongoing drought had had a negative effect on their income, but some were more positive and noted that “if things continue like this we will be okay,” a finding also corroborated by their parents and guardians. Some parents and guardians in FGDs in Amhara reported that their wards’ income had risen and that they were saving more, while others said that although their wards had businesses, they did not yet have an income or that it was too soon to tell if income had increased. KIIs with facilitators in several woredas indicated that the beneficiaries’ income had increased “as reflected by their clothes, larger savings, better living and generally improved appearance.” Last but not least, given that most of those employed are working in agriculture and rearing livestock, the seasonality of the activity and its dependence on weather conditions may also affect the outcomes. Analysis of monthly income data 22 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report collected from monthly (financial diary) interviews with a select subsample of survey respondents support this premise of seasonal ups and downs in income even as the overall trend is for an increase in income (Figure 3). Figure 3: Distribution of monthly mean and median income for a subgroup of survey participants (n=75) Source: Authors’ calculations, based on data from monthly (financial diary) interviews with a select group of participants. Two-thirds of the respondents who had completed TVET reported an increase in income, followed by 63 percent of those with no school, and less than 60 percent in all other education categories. As Table 12 shows, half of the respondents with a college education or with Grades 9 to 12 level of schooling reported either no change or a decline in income. Table 12: Change in income between baseline, midline, and endline, by education No School/Trad. Ed. Grade 1 to 5 Grade 6 to 8 Grade 9 to 12 Completed TVET College/Univ. Change BL-ML n=32 ML-EL n=24 BL-ML n=62 ML-EL n=50 BL-ML n=132 ML-EL n=108 BL-ML n=257 ML-EL n=227 BL-ML n=39 ML-EL n=38 BL-ML n=11 ML-EL n=11 Increased 63% 63% 65% 54% 59% 59% 61% 51% 64% 68% 45% 45% Decreased 6% 13% 13% 10% 20% 11% 13% 14% 3% 11% 0% 9% Stayed the same 31% 25% 23% 36% 20% 30% 26% 35% 33% 21% 55% 45% Source: Authors’ calculations, based on YCS baseline, midline, and endline survey data. Expenditure There was a marked increase in the 12 months following YP enrollment in the number of respondents spending on consumption items aimed at increasing their quality of living as well as investment items aimed at increasing future income. As Table 13 shows, significantly more respondents at endline were spending on rent/lease, food, electricity and water, education, health care, investments, and savings than at baseline (p<.01 in all cases), though the effect size was small in the case of expenditures on electricity and water, education, health care, and savings. After an initial decline between baseline and midline, the proportion of respondents spending on electricity, household goods, clothing, family planning, and transportation rose significantly between midline and endline. The proportion of respondents spending their incomes on savings and investments in business rose sharply at midline compared with baseline and kept rising to endline but at a slower rate. On the other hand, the growth in the proportion of respondents spending on education and support to their family was very modest at midline compared with baseline but picked up at endline compared with midline. Fewer respondents were spending on clothing and family planning at endline than at baseline, but the difference was neither large nor significant. 23 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Table 13: Percentage reporting expenditure on various items, baseline, midline, and endline Baseline (n=582) Midline (n=533) Endline (n=458) p-value* Effect Size Rent/Lease 11.9% 19.5% 26.9% <.01 0.241 Food and related goods 47.9% 57.8% 68.6% <.01 0.30 Electricity and water 17.2% 15.2% 24.7% <.01 0.114 Education/school fees 4.5% 4.9% 11.1% <.01 0.153 Healthcare 22.0% 29.1% 32.5% <.01 0.154 Investment in business 8.8% 18.4% 19.7% <.01 0.231 Savings 43.1% 55.9% 57.0% <.01 0.184 Household goods 33.3% 19.5% 39.1% .02 0.102 Clothing 87.6% 82.0% 84.9% .06 -0.072 Support to other family members 12.7% 13.5% 17.7% .10 0.068 Taxation 8.8% 9.2% 10.9% .32 0.037 Contraceptive and family planning 3.4% 2.1% 2.8% .39 -0.023 Transportation 82.6% 80.7% 85.2% .53 0.016 *Paired t-test and effect size computed on the common sample of 458 study respondents at endline and baseline. Source: Authors’ calculations, based on YCS baseline, midline, and endline survey data. Between midline and endline, the proportion of study participants increased in spending across all items of consumption and investment for which data were collected. Figure 4 plots the percentage point difference between midline and baseline and between endline and midline, with a positive (negative) value indicating an increase (decrease) in the proportion of spenders for that item. As the figure shows, the proportion of respondents spending on transportation, family planning, clothing, household goods, and utilities fell at midline compared with baseline. The increases between midline and endline compensated for this decline in all categories except family planning and clothing, which were the only two categories that saw a fall in the proportion of spenders.27 Figure 4: Change in respondents incurring expenditure, proportion baseline to midline and midline to endline, by item (%) Source: Authors’ calculations, based on YCS baseline, midline, endline survey data. 27 The decrease in expenditures on family planning could perhaps be attributed to the availability of family planning products free of charge through health clinics (reports by FGD participants). -15% -10% -5% 0% 5% 10% 15% 20% Respondents incurring expenditure proportion, by item Midline to Endline Baseline to Midline Transportation Support to family members Clothing Taxation Household goods Savings Healthcare Education/school fees Food Rent/lease Family planning Investment Electricity & water 24 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Changes in the 12 months following YP enrollment in the proportion of study respondents spending on consumption and investment items were much more modest in Afar than in other regions. As Table 14 shows, the proportion of respondents in Afar spending on rent/lease, electricity and water, education, and investments went up at midline but fell at endline compared with baseline, whereas all other regions recorded marked increases. The largest increases were in Tigray, where the proportion of respondents spending on consumption and investment increased twenty-fold for spending on utilities, six-fold on rent and lease, threefold on food and education, and twice or more on most other items of spending. Increases in Amhara and Oromia were relatively modest. Table 14: Percentage reporting expenditure on various items, baseline, midline and endline, by region Afar Amhara Oromia Tigray BL n=108 ML n=95 EL n=87 BL n=200 ML n=172 EL n=133 BL n=178 ML n=171 EL n=155 BL n=96 ML n=95 EL n=83 Rent/Lease 8% 16% 6% 20% 25% 48% 8% 10% 16% 6% 31% 35% Food & related goods 48% 60% 64% 56% 59% 83% 58% 64% 76% 13% 41% 36% Electricity and water 33% 24% 22% 21% 18% 38% 12% 9% 17% 1% 13% 20% Education/school fees 9% 6% 7% - 3% 8% 6% 5% 15% 5% 6% 14% Healthcare 33% 24% 33% 28% 42% 38% 17% 17% 29% 6% 33% 29% Investment in business 7% 12% 6% 14% 14% 35% 5% 4% 6% 7% 59% 35% Savings 8% 8% 8% 52% 71% 76% 61% 49% 55% 31% 89% 82% Household goods 33% - 21% 36% - 51% 38% - 33% 20% - 51% Clothing 85% 68% 83% 86% 84% 83% 92% 87% 92% 86% 83% 77% Support to other family members 5% 6% 3% 13% 15% 23% 25% 13% 16% - 19% 27% Taxation 4% 9% 2% 4% 8% 20% 20% 11% 10% 3% 8% 7% Contraceptive and family planning 4% 1% - 6% 4% 8% 2% 2% 1% - 0% 1% Transportation 72% 71% 77% 84% 88% 89% 89% 77% 87% 79% 83% 83% Source: Authors’ calculations, based on YCS baseline and endline survey data. Monthly expenditure data (financial diary) collected over eight months from a small subsample of YP participants suggest that expenditures incurred varied from month to month and did not necessarily reflect the respondents’ income and ability to spend (Figure 5). Expenditures remained fairly constant with small changes while the income may have varied substantially. Most respondents incurred expenditures on purchase of livestock, goats, and sheep; investment in business; and savings, followed by purchase of clothing and food and household items. 25 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Figure 5: Distribution of monthly mean and median income and expenditure for a subgroup of survey participants (n=83) Source: Authors’ calculations, based on data from monthly interviews (financial diary) with a select group of participants. FGDs with respondents confirmed the spending patterns reported above. In Tigray, participants reported changes in the variety and content of food consumed and some reported an increase in milk intake, while in Amhara (South Achefer) some reported higher intake of vegetables. FGD participants in Amhara stated that they were spending between 20 percent and 40 percent of their income on their families, while others reported using the increase in their incomes to pay back loans, save more money, and reinvest in their own businesses. Some female FGD participants in Afar who experienced an increase in their income stated that they were spending more on their families and husbands; however, they also stated that drought was affecting their ability to spend. Participants also reported expenditures on building assets and using the savings as start-up capital. For instance, one female participant in Oromia (Wolmera) who participated in a hard-skills training program offered by a TVET center had invested a portion of the increase in her income in the establishment of a beauty salon. Parents in FGDs also reported that the youths were spending more to reinvest in business and pay back loans as well as in increased savings. Assets There was a modest change in spending on assets in the 12 months following YP enrollment. As Table 15 shows, there was a slight increase in the proportion of respondents spending on houses, animals, mobile phones, land, television, and cars/motorcycles/bicycles, but the change was neither large nor statistically significant. Likewise, there was a modest decline in the proportion spending on radio/tape recorders and animal-drawn carts, but the change was small and not statistically significant. Spending on assets owned at the family level was very similar and follows the same pattern. These findings were corroborated in FGDs, in which participants and their parents and guardians reported investing money from their income into building assets and expanding their businesses. Table 15: Spending on assets at baseline, midline, and endline Owned by Individual Owned by Family BL n=582 ML n=533 EL n=458 BL n=582 ML n=533 EL n=458 House 20% 18% 22% 86% 73% 76% Animals 48% 57% 51% 83% 73% 75% Mobile Telephones 82% 87% 85% 66% 60% 69% Land for farming, grazing, etc. 19% 24% 21% 70% 66% 62% 26 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Owned by Individual Owned by Family BL n=582 ML n=533 EL n=458 BL n=582 ML n=533 EL n=458 Radio/Tape recorder/MP 3 player 20% 18% 17% 41% 32% 33% Television 4% 5% 7% 18% 13% 15% Animal-drawn cart 2% 1% 1% 4% 4% 3% Car/truck/motor cycle/bicycle* 3% 4% 5% 3% 5% 4% Other 1.5% 2% 1.5% 2% 2% 4% *Note: This does not include the purchase of Bajaj scooters mentioned earlier in this report, since at the time of data collection, participants were still waiting for the loans to be approved and for the vehicles to be delivered to them. Source: Authors’ calculations, based on YCS baseline, midline, and endline survey data. These findings were also supported by data from the monthly financial diary interviews with a small subsample of participants. In addition, the monthly data showed that immediately after participation in the WRN! training, the participants had obtained loans from microfinance institutions and family and friends as capital for starting a new business or supporting existing businesses. Participants also obtained loans, to a lesser extent, to support purchase of assets and pay for health care and for household expenditures (Figure 6). Figure 6: Reasons for obtaining loans for a subgroup of survey participants, by region, age group, and sex (n=81) Source: Authors’ calculations, based on data from monthly interviews (financial diary) with a select group of participants. Savings The average amount saved by respondents rose by more than 80 percent in the 12 months following YP enrollment. At baseline, 73 percent of employed and 42 percent of unemployed respondents were putting money away in savings.28 At endline, the proportion of the employed that was saving went up modestly to 77 percent while that of unemployed remained unchanged. During this one-year period, however, the average saving amount went up significantly (p<.01), from 3,135 Birr at baseline (US$137, median 1200 Birr/US$52) to 5,757 Birr at endline (US$251; median of 3,000 Birr/US$131). The largest number of those reporting saving money were from Tigray (88 percent), followed by Amhara (78 percent), Oromia (66 percent) and Afar (25 percent). Of those that reported saving, 46 percent reported that they put away money a few times a year on a seasonal basis, another 39 percent did so monthly, and the rest as money became available, on a weekly or daily basis. Although there were no 28 The source of funds among the unemployed included pocket money from their families, money earned from odd-jobs, etc. 27 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report differences in saving patterns between young men and women, those with no/low levels of education were more likely to save on a seasonal basis (62 percent) whereas those with a college education were more likely to save monthly. Most participants (80 percent) reported keeping their savings in a bank or microfinance institution, and 16 percent overall (highest in Afar at 36 percent) reported keeping their savings at home. A few also reported keeping their money in saving groups, traditionally known as “EQUIB” or YES Groups. FGDs with participants supported these findings, with most respondents indicating that they were saving regularly. The participants reported a tacit encouragement to save from their coaches, who were always likely to ask how much they had saved since the last time they met. As one FGD participant in Amhara put it, “the facilitators come, encourage us, check on monthly savings and talk about the challenges faced.” The participants also reported that they put away money in savings at harvest time or when they sold the animals they were fattening for this purpose. A few participants reported that they were not saving as they were spending the money on buying clothes (41 percent), returning loans, and investing in a business (9.9 percent), or had no money to save. These findings were also corroborated by their parents and guardians. UTILIZATION OF HEALTH SERVICES29 A significantly larger proportion of study respondents reported incurring health care expenditures for themselves or their families at 12 months following YP enrollment on treatment of simple seasonal illnesses such as common cold and diarrhea to more serious ones like heart problems, dialysis, and surgeries, but the change in average monthly spending was not significant (data from monthly financial diary interviews). Survey data show that one-third (33 percent) of study respondents incurred spending on health care compared with one-fifth (22 percent) at baseline, which is statistically significant (p<.01) although the effect size is small (Cohen’s d=.154). The average spending at endline on health was US$27 (617 Birr) per month (median expenditure US$17.5, 400 Birr),30 higher than health expenditures incurred at midline (US$17 average and US$11 median), but lower than the US$30 (685 Birr) average expenditure reported at baseline. The difference is modest (effect size -2.16) but is not statistically significant. At endline, health-related expenditures ranged from 40 Birr to 8,000 Birr (US$1.8 to US$352), with almost three-quarters of the respondents spending less than 500 Birr (US$22) per month. Less than one-tenth of all respondents reported spending more than 1,000 Birr (US$45) per month on health care. Although one year is a short timeframe in which to expect big changes in health care–seeking patterns, there is some evidence of greater utilization of health services and larger expenditures in the one year following YP enrollment. Three-quarters of respondents (76 percent) reported visiting a health care provider (health care worker, nurse, or doctor) one to two times in the recall period of six months (no change from baseline). Thirty-one percent of the participants reported taking a family member to a provider. Those who sought treatment reported going to a government hospital, health center, or government health post. A very small proportion (1 percent) of respondents reported going to a traditional healer. Almost half of the respondents reported receiving medicines (49 percent), female contraceptives (14 percent), male contraceptives (5 percent), and advice (34 percent) from the provider (all, except obtaining female contraceptives, significantly more than the baseline, p<.01). Of those that did not need health care, over half indicated that if necessary, they would go to a government facility (67 percent hospital; 26 percent facility). Fifty-three percent of those reporting an illness took one to seven days off from work to recuperate. These findings are also supported by data from the monthly financial 29 Responds to the research question: Does higher income lead to better health outcomes among study participants? 30 Exchange rate 22.9 Birr = 1 USD (using rate from April 2017 to allow for comparability with baseline). 28 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report diary interviews, where the female respondents indicated they were likely to take between one to five days off from work and the male respondents reported taking two to seven days. There is also some evidence of improvement in the health of study respondents whose incomes increased in the 12 months following YP enrollment. As shown in Table 16, 90 percent of respondents whose incomes rose between baseline and endline stated that they were taking better care of their health, insignificantly different from 86 percent of respondents whose income fell between baseline and endline. However, 90 percent of respondents whose incomes rose between baseline and endline reported that their health had improved, significantly better than 78 percent of respondents whose income fell during this period (p<.01, effect size modest). A significantly smaller proportion of respondents whose incomes rose between baseline and endline (28 percent) reported having an illness in the recall period of six months, compared with 43 percent of respondents whose incomes fell (p<.01, effect size modest). There were no statistically significant differences or differences in effect size between study respondents whose incomes had increased since baseline and those whose incomes had decreased in feeling less stressful (p=.29), less anxious (p=.09), less angry (p=.02), and getting fewer headaches (p=.42), though the former were significantly more likely to feel happy than the respondents whose incomes had decreased (p<.01). Table 16: Health and wellness indicators at baseline, midline, and endline (proportion reporting), by change in income Income Change Baseline to Midline n=533 Income Change Midline to Endline n=458 Income Change Baseline to Endline n=458 Change BL-EL p￾values* Increase n=325 Decrease n=71 No change n=137 Increase n=253 Decrease n=56 No change n=149 Increase n=327 Decrease n=58 No change n=54 Increased BL to ML; decreased ML to EL n=19 Take care of health 84% 62% 73% 92% 84% 80% 90% 86% 74% 79% p=.20 Health improved 88% 85% 75% 89% 75% 86% 90% 78% 78% 74% p<.01 Illness (6 months) 33% 39% 30% 25% 46% 38% 28% 43% 41% 37% p=.01 Less stressed 91% 75% 73% 88% 71% 82% 87% 85% 74% 58% p=.29 Less anxious 90% 76% 71% 90% 70% 81% 89% 83% 72% 53% p=.09 Get fewer headaches 78% 63% 59% 76% 66% 76% 77% 76% 65% 53% p=.42 Less angry 89% 72% 75% 84% 75% 72% 83% 71% 67% 79% p=.02 Happier 93% 83% 85% 94% 73% 83% 93% 79% 78% 63% P<.01 Source: Authors’ calculations, based on YCS baseline, midline, and endline survey data. Among the regions, study respondents in Afar reported the largest improvements in health outcomes between baseline and endline, and Tigray the least. More respondents at lower education levels reported feeling better than those with higher (TVET/college) education levels. Both male and female respondents reported taking care of their health, and whereas men reported feeling generally less stressed, less anxious, and less angry, women reported feeling generally happier and healthier. The results from the survey analysis were reinforced in FGDs with study participants. Many FGD participants noted that following participation in the WRN! training and because of the ongoing group and individual follow-on coaching, “they were taking better care of their health,” had “learned to be more patient,” were “less anxious about the future,” and “less quick to anger.” They were also working harder, which together with being better organized and having more money, was enabling them to take 29 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report better care of themselves and their personal hygiene. FGD participants also reported that their food intake had improved in variety and content, and some reported drinking milk regularly. In the words of one participant from the Tigray region, “we are working, engaged, eating better, so not getting sick.” Several participants also reported that they had decreased their consumption of alcohol, tobacco, and chat and were spending more time with their families. This change was corroborated by parents and guardians, who observed that because YP participants had a social network and a facilitator who was regularly checking on them, they had better mental health and had developed a positive attitude toward life. They gave examples of their wards now “talking with family members” and “socializing more with their friends.” The fact that the beneficiaries were busy and engaged in productive work also helped enhance the general feeling of well-being. SELF-PERCEPTION OF WORK READINESS31 Study respondents’ work readiness improved significantly between YP enrollment during 6 and 12 months following enrollment. As Table 17 shows, respondents perceived improvements in several areas of work readiness at endline compared with baseline, especially in knowledge of where and how to search for a job, expertise in managing a business, knowledge of how and where to obtain loans, knowledge of how and where to obtain financing, and proficiency in writing job applications, all of which were statistically significant (p<.01 or better). Changes reported in other areas were not statistically significant. Improvements in work readiness among male respondents mirrored the overall patterns; among female respondents, however, improvements between baseline and endline were statistically significant only for knowledge of where and how to search for a job, knowledge of how and where to obtain loans, and knowledge of how and where to obtain financing. Male respondents assessed their work readiness at higher levels than did female respondents at baseline, midline, and endline. 31 Responds to the research question: To what extent has participation in the YP Activity improved the work-readiness skills of study participants? 30 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Table 17: Participants’ self-perception of work readiness (mean scores computed by assigning the values of 5 to “very good” and 1 to “very poor” self-rating) Overall Male Female (n=458) (n=261) (n=197) Baseline Midline Endline Baseline Midline Endline Baseline Midline Endline Knowledge of how and where to look for a job 3.2 3.3 3.6 2.7 2.9 3.1 2.5 2.5 2.5 Expertise in managing a business 3.4 3.4 3.6 2.7 2.9 3.1 2.5 2.5 2.5 Knowledge of resources for obtaining Loans 3.5 3.5 3.8 2.7 2.9 3.1 2.5 2.5 2.5 Knowledge of how and where to access finance, land, etc. 3.0 3.2 3.4 2.7 2.9 3.1 2.5 2.5 2.5 Proficiency in writing job applications 2.6 2.7 2.8 2.7 2.9 3.1 2.5 2.5 2.5 Knowledge of how to access markets 3.6 3.5 3.7 2.7 2.9 3.1 2.5 2.5 2.5 Willing and able to work hard 4.1 4.2 4.2 4.2 4.3 4.3 4.0 4.1 4.1 Expertise in communicating with suppliers, financiers, customers, etc. 4.4 4.5 4.3 4.4 4.5 4.4 4.2 4.5 4.2 Skill in developing business and person development plans 3.9 3.5 3.8 3.9 4.0 4.0 3.7 3.8 3.5 Knowledge of starting a business 3.6 3.5 3.6 2.7 2.9 3.1 2.5 2.5 2.5 Knowledge of how and where to obtain licenses and permits 3.3 3.9 3.4 3.5 3.6 3.2 3.3 3.1 3.4 Ability to work in teams 3.9 4.0 4.0 4.1 4.1 4.1 3.7 3.9 3.8 Confidence and belief in own ability to succeed in the workplace 4.4 4.4 4.5 4.5 4.4 4.3 4.3 4.3 4.5 *Paired t-test, carried out on the common sample of 458 study respondents at baseline, midline, and endline. Source: Authors’ calculations, based on YCS baseline, midline, and endline surveys. Survey respondents at midline and endline reported feeling more confident of their abilities, especially in managing customers, spending money according to a plan, and managing time better. They felt they had better marketing skills and were able to solve their problems. They also reported that they were more persistent, worked for more days of the week, were respectful of other jobs and customers, and had better marketing skills (Annex IX, Table X8). FGDs with study participants from all four regions supported these findings. During FGDs, participants reported that the training “helped [me] to prepare for work” by giving them confidence in their own abilities and providing them with the skills, knowledge, and motivation to start their own businesses or improve their existing businesses; skills to find a job; and the skills to better manage their finances and the encouragement to save on a regular basis. FGD participants noted that there was a change in their attitude since baseline and that at endline they were “more respectful towards all work,” had learned “not to differentiate between big and small jobs, all jobs [could] be managed,” and that they had “hope, once you start with something you can continue with it.” They reported that the training had helped them be more motivated for work, work for longer hours, and be more productive with higher income. The participants noted that following their YP participation, they were working and living according to a plan and had a better idea of how to run a business and be independent. They also reported being more respectful and better at handling their customers. FGD participants that were unemployed reported revisiting their PDPs to see how they could change their 31 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report situation. Parents and guardians participating in FGDs supported the self-perceptions of the study participants. They reported that the youth showed positive changes after WRN! training, which included being better organized, improved time management, not staying out late at night, not being idle, and so forth. Parents from a focus group in Tigray commented that the youth now “have skills in running a business,” “have better communication skills,” and were now “generating additional income.” They further noted that the youths were “busy with work, no more drinking,” and that they were “changed person[s], paying back loans.” Parents noted that in addition to their own activities, the youth were helping the parents/guardians with their work. KNOWLEDGE OF AND ACCESS TO EMPLOYMENT AND ENTREPRENEURIAL RESOURCES32 There was a modest change in the different ways participants used to search for wage employment between YP enrollment and 12 months following enrollment. As Table 18 shows, seeking assistance of friends and family, attending job fairs, registering with public employment offices and private employment agencies, and direct applications to employers were the job search avenues most commonly used by respondents looking for employment (or new employment) at baseline, in that order, and the same pattern persisted at endline. Although the same avenues were preferred by both male and female respondents looking for employment (or new employment) at both baseline and endline, significantly more male participants registered with public employment offices and private employment agencies at baseline and endline than did female participants (p<.01, modest effect size). Likewise, significantly more male participants than female participants looked for jobs at work sites and firms and applied directly to jobs (p<.01, modest effect size), perhaps a function of their greater mobility. Seeking assistance from friends and relatives was the option used by most respondents looking for employment (or new employment) in Tigray and Amhara, attending job fairs the preferred option in Afar, and registration with the public employment office the preferred option in Oromia. Table 18: Avenues of search for employment – baseline to endline All Male Female BL n=205 ML n=232 EL n=195 BL n=142 ML n=149 EL n=117 BL n=63 ML n=83 EL n=78 Seeking assistance of friends and relatives 46% 65% 59% 54% 66% 61% 29% 61% 56% Attending job fairs 48% 47% 38% 47% 46% 41% 51% 51% 33% Registering with public employment office 20% 20% 25% 18% 24% 30% 24% 13% 18% Checking at work sites/firms etc. 16% 14% 20% 20% 15% 24% 8% 12% 14% Direct application to employers 23% 13% 16% 25% 15% 22% 19% 7% 6% Registering with private employment agencies 12% 9% 9% 14% 10% 12% 8% 7% 5% Getting more education/training 3% 5% 3% 2% 5% 2% 5% 4% 6% Placing/answering newspaper ads. 5% 6% 2% 8% 6% 2% 0% 5% 1% Answering internet advertisements 2% 2% 2% 3% 2% 3% 0% 1% 0% Source: Authors’ calculations, based on YCS baseline, midline, and endline surveys. The use of entrepreneurial resources varied according to whether the respondent was wage employed, self-employed, or unemployed, but the pattern by and large remained unchanged between baseline and endline. As Table 19 shows, the felt need for information about where and how to raise financing was expressed by most respondents at endline, midline, and baseline, especially by the wage employed (presumably seeking to move to self-employment). Assistance from friends and family was used most by 32 Addresses the research question: To what extent has participation in the YP Activity improved knowledge of and access to employment/entrepreneurial resources for study participants? 32 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report those in wage employment at midline and by the unemployed at endline and baseline. The desire to learn new skills was expressed most by the wage employed at baseline and endline. Similar patterns were observed among male and female respondents. Across regions, financing for start-up and implementation of the planned income-generating activity or business was listed as the most important resource required in Amhara and Oromia at both endline and baseline. Learning new skills was listed by most respondents in Amhara and Tigray, though it fell in Amhara between baseline and endline. Table 19: Entrepreneurial resources required, by current employment status Baseline n=582 Midline n=533 Endline n=458 Wage Emp n=40 Self Emp n=238 Unemp n=277 Wage Emp n=47 Self Emp n=346 Unemp n=129 Wage Emp n=39 Self Emp n=277 Unemp n=129 Assistance from friends & family 20% 32% 38% 57% 43% 40% 28% 32% 42% Land, equipment, etc. 53% 34% 26% 49% 49% 30% 59% 47% 31% Financing 80% 85% 64% 81% 72% 64% 90% 84% 63% Permits & licenses 25% 29% 18% 9% 9% 22% 15% 19% 9% New skills 73% 54% 51% 47% 44% 50% 72% 54% 36% Source: Authors’ calculations, based on YCS baseline, midline, and endline surveys. FGDs with study respondents confirm these findings. Almost all respondents reported that they were looking to create self-employment, and respondents across all regions, especially in Amhara and Oromia, were aware of the need for arranging finance for creating self-employment (reported by 79 percent). The need to learn new skills ranked somewhat higher than the need to locate land, space, and equipment (43 percent), as well as seeking the assistance from family, friends, and colleagues. The need to obtain permits and licenses was also reported by a small proportion. As indicated elsewhere in this report (Figure 6), data from the monthly financial diary interviews showed that following the WRN! training, participants obtained loans through microfinance institutions (especially in Tigray) and tapped family and friends to raise capital for start-ups and for expansion of existing ventures. Most of these loans were obtained in the months immediately following the training, indicating use of training. FUNCTIONAL LITERACY AND NUMERACY SKILLS33 Functional literacy and numeracy did not change much between baseline and endline, largely because the youth cohort selected for training at baseline already had high levels of literacy and numeracy, but also because YP did not provide any training directed at improving literacy and numeracy.34 As Table 20 shows, 86 percent of YP participants at endline could read and write with fluency, compared with 85 percent at baseline. Likewise, at endline, 96 percent of YP participants could recognize numbers and 91 percent could perform simple addition, compared with 97 percent and 93 percent, respectively, at baseline. None of these differences is statistically significant. These findings are supported by FGDs, in which most participants noted that there was no change in their literacy and numeracy skills. Multiple young men and women across all four regions stated, “I knew how to read and write before joining the 33 Addresses the research question: To what extent has participation in the YP Activity improved functional literacy and numeracy skills of study participants? 34 The YP description on the USAID website reads: “Rural youth will receive literacy, numeracy, and life skills training alongside vocational and entrepreneurship development activities through partnerships with training, small-business, and micro-finance institutions” (ref: https://www.usaid.gov/ethiopa/education). However, review of YP documents as well as interviews with YP management showed that YP was focused only on life skills training alongside vocational and entrepreneurship activities and was not providing literacy or numeracy training. 33 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Potential training, and there has been no change in my ability.” At the same time, they noted that the training and preparation of PDPs and business plans reinforced their reading and writing skills. Participants in Amhara and Tigray noted that they “keep records and do some form of book-keeping” and use their reading, writing, and math skills regularly in managing accounts, preparing budgets, and tracking inputs and outputs, and over time had become more comfortable with the use of these skills. Table 20: Change in level of functional literacy and numeracy, baseline to midline to endline Literacy and Numeracy Baseline (n=458) Midline (n=458) Endline (n=458) Difference BL-EL p-value Effect size Functional Literacy Can’t read & write 8.3% 10.5% 7.2% -1.1% .32 -0.021 Can read & write with difficulty 7.2% 7.9% 6.8% -0.4% .72 -0.009 Can read & write with some fluency 13.1% 7.6% 7.6% -5.5% <.01 -0.194 Can read & write very easily 71.4% 74.0% 78.4% 7.0% <.01 0.104 Functional Numeracy Recognize numbers 96.7% 93.7% 96.3% -0.4% .67 -0.010 Add simple numbers 93.0% 91.0% 91.0% -2.0% .11 -0.039 Subtract simple numbers 86.5% 88.6% 89.1% 2.6% <.05 0.045 Multiply simple numbers 84.9% 83.8% 88.2% 3.3% <.02 0.055 Divide simple numbers 83.8% 83.6% 88.0% 4.1% <.01 0.068 Source: Authors’ calculations, based on YCS baseline, midline, and endline surveys. Significant differences were observed between baseline and endline in the ability of YP participants to perform multiplication and division of numbers. Eighty-eight percent of YP participants at endline could multiply simple numbers, higher than 85 percent at baseline, a difference that is small (Cohen’s d=.055) but statistically significant (p<.02). Similarly, 88 percent of YP participants at endline could divide simple numbers, a difference that is small (Cohen’s d=.068) but statistically significant (p<.01). 34 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report CONCLUSIONS Youth Potential Activity has been implemented well and has generally succeeded in helping youth increase their assets and incomes through employment or better employment to achieve economic self￾sufficiency. The YCS findings suggest that YP provided the study beneficiaries—unemployed or underemployed youth—with life skills and helped them develop a more positive attitude toward work, especially self-employment, and encouraged the participants to remain in their communities instead of migrating to the cities or even to other countries. WRN! and WR Plus training were sufficiently provided for the YP beneficiaries that were followed in this study. However, coaching and mentorship was not provided to the level that addressed all beneficiaries, mainly because of the late start of the mentorship program but also because of the low number of facilitators relative to the number of beneficiaries, high level of staff turnover, logistical challenges of reaching out to all beneficiaries spread out over long distances, and cultural challenges of benefiting from local mentors. Although all beneficiaries were required to develop PDPs and those who have used them attest to their usefulness, the broader application of the plan was minimal, with neither the facilitators nor the beneficiaries using them optimally to follow up on coaching and further enhancing personal development. A significant increase in self-employment was recorded among study respondents between the time of YP enrollment and 12 months following enrollment. Increase in wage employment during this period was very limited, which, though not out of the expectation, is striking given that several opportunities remained untapped in many sectors and regions even as several beneficiaries remained unemployed after the training. Further, though the increase in self-employment is encouraging, sectoral diversification was very limited, and the majority of beneficiaries remained engaged in agriculture even though many believed that it was not the best income-generating activity or business. Constraints that prevented youth from entering other sectors of self-employment were access to start-up and working capital, land, and working space. A related challenge was the ability of the participants to organize themselves into groups. Loans were relatively easier to obtain for group-based self-employment, but participants reported problems in organizing themselves in groups. Female participants faced specific challenges in branching out into new areas, especially those that were considered to be the domains of men. The average income of the study cohort rose in the 12 months following YP enrollment, but increases were not large enough to result in major changes in the beneficiaries’ lifestyle. Keeping pace with rising incomes was the huge increase in savings, which doubled within the year following YP enrollment. Expenditures on necessities such as housing, food, utilities, health care, and education also rose significantly, leaving enough money for only modest increases on lifestyle consumption items, such as clothing, mobile phones, radios, and so forth. Further, although the income of female beneficiaries has also increased, the change has been much less than that for male beneficiaries. Interestingly, those with no education gained as much as those with technical and vocational education, a testimony to the opportunities that self-employment provided. One year is too short a period for big changes in health care–seeking behavior or in health outcomes, and none were seen in the 12 months following YP enrollment. However, the increase in income was associated with greater attention of beneficiaries to their own health as well as that of their families, and with lower levels of stress and illness. YP strengthened participants’ confidence and work ethic and improved their self-perception of work readiness skills. Participants developed skills related to communication, work habits and conduct, teamwork, goal setting, job searching, and so forth and used them to expand the scope of their ongoing activities. However, with time and the hardships caused by the ongoing drought, the initial euphoria of 35 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report having received training and being part of a coaching group was beginning to wear off, leaving some participants feeling less confident and more uncertain of their abilities. Study participants reported both feeling more empowered in searching for employment and creating self-employment and being able to get better and more complete information because of their training. Those looking for wage employment reported being more aware of the various places where they could apply for jobs. Likewise, those looking to be engaged in self-employment reported being more aware of the process for obtaining the resources needed to start and/or expand their own enterprises. YP participation did not affect functional literacy and numeracy because the cohort already had fairly high levels of literacy and numeracy at the time of enrollment. However, the use of reading and writing skills during training and in the preparation of PDPs and business plans reinforced these abilities. Participants also reported that managing their finances, developing budgets, and monitoring their inputs and profits had improved their ability to do simple addition and subtraction. 36 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report RECOMMENDATIONS The YCS study suggests several important actions that must be taken to maximize YP’s remaining contribution to increasing employment and income among youth in Ethiopia. The study also provides some recommendations for the design of future activities based on the YP experience. General Recommendations for YP Strengthen coaching and mentoring: Coaching and mentoring programs are key to YP’s success; all YP beneficiaries must have access to coaches (facilitators) and mentors. Specific action steps include the following:  Hire more facilitators and train them in the art of coaching.  Establish the maximum number of beneficiaries that can be attached to a facilitator at any time and rebalance facilitators’ workload and responsibilities accordingly.  Identify and train more mentors, especially in far-flung areas, so that distance does not become a barrier to providing mentorship services to all beneficiaries.  Educate participants and prospective mentors alike in ways that the mentor-mentee relationship should work to maximize the benefits to the participant.  Institute the practice of publicly recognizing and honoring facilitators and mentors so that they are seen as critically important in guiding the beneficiaries in achieving their employment and income targets. Increase the involvement of parents and guardians: Parents and guardians of YP participants play a key role in helping their wards achieve their self-employment goals, especially in arranging for the needed initial financing. However, this involvement is tentative and unplanned, and parents are not fully aware of market requirements and opportunities to help their wards set up income-generating activities and businesses. Greater involvement of parents and guardians in selected YP elements together with better information sharing about the training programs and their intended benefits would help reinforce and further strengthen the role parents and guardians can play to achieve the necessary outcomes. Specific action steps include the following:  Educate parents about the essential things to know to effectively support their wards.  Periodically share information about YP and the progress of their wards.  Involve parents in designing the essential elements of training so that there is ownership of training, leading to parental support for youth after training. Diversify access to new market opportunities: Most YP participants who are self-employed are engaged in traditional fields of agriculture and trading, and very few are engaged in other activities where there may be good potential opportunities. Diversification of market opportunities would help beneficiaries expand their earning potential. Specific action steps include the following:  Maintain and regularly update an inventory of market opportunities in both traditional and new areas, such as in the agriculture supply chain space, and share with beneficiaries as they become available.  Establish linkages with ongoing government and donor-funded programs and activities and help beneficiaries get greater access to nontraditional opportunities.  Establish linkages with the private sector to help beneficiaries obtain access to training and internship opportunities. 37 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Ensure that Personal Development Plans developed by the beneficiaries are viable: Personal Development Plans form the backbone of training and are developed as a roadmap for the beneficiaries to follow as they start with activities after training. Plans are good only to the extent they are viable. Specific action steps include the following:  Review the personal development plans with an eye to implementation.  To the extent possible, include a cost-benefit analysis in the proposed plan. Provide greater support and training on the usefulness of mentoring by community members: YP beneficiaries have not utilized the mentoring offered due to preconceived biases against mentors from within the community. Some specific action steps to encourage use of mentoring include the following:  Educate parents and communities to support mentoring by community members.  Educate youth on the advantages of mentoring by those from within the community.  Support mentors with training on how to provide mentoring. Help beneficiaries obtain financing: Availability of funding is a basic requirement for self-employment, both for start-up and for later expansion. Over the past year, there has been a general relaxation of policies related to loan requirements and the introduction of new government loan programs specifically targeting youth in all regions, but more needs to be done to ease the capital markets and improve access to funding opportunities. Specific action steps include the following:  Educate beneficiaries about the various public and private sources and schemes of financing, eligibility requirements, and how and where to apply.  Continue advocacy for public and private financing programs and an easing of the collateral requirements for YP participants.  Provide specialized training and counseling to beneficiaries to avail group loan opportunities. Strengthen job placement services: Strengthening job placement services could help YP participants find more and better wage employment opportunities at the end of their training. YP could assume the responsibility to better help its participants find wage employment upon completion of training and to reduce some pressure from the limited self-employment options available. Specific action steps include the following:  Collect and share information about job opportunities at the woreda and regional levels, map participants’ skills with jobs, and aggressively advertise the availability of skills to potential employers.  Strengthen relationships with private and government employers about current and future job opportunities and align the YP training. Strengthen linkages with existing government programs and donor-funded initiatives designed to promote employment: Establishing and strengthening linkages with government programs and donor￾funded initiatives could help participants access these programs and initiatives and optimally use available resources. Specific action steps include the following:  Identify government programs designed to promote wage and self-employment and strengthen linkages with relevant public officials and make connections for beneficiaries to access these initiatives.  Identify donor-funded initiatives, such as USAID’s Feed the Future Initiative, and strengthen linkages with relevant staff running these initiatives to help YP participants find better wage or self-employment opportunities at the end of their training. 38 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report  Identify private companies and strengthen linkages with these companies and their appointed staff to assist YP participants find better wage or self-employment opportunities at the end of their training at these companies. Continue evaluation of YP: One year is a very short time to study YP’s long-term effects. The evaluation team recommends the following:  Conduct a follow-up study of the YCS cohort to understand long-term outcomes.  Include a greater focus on the mentorship component in the final YP evaluation.  Include a comparative study of participants in the final evaluation as YP changes its approach to coaching from offering life-of-project coaching to coaching limited to six months to understand its effect.  Evaluate YP components such as YES groups that are still in a nascent stage. Specific Recommendations for Female Participants While all the above recommendations apply for both male and female participants, some specific recommendations for female participants are listed below. Improve access for females to new market opportunities.  Establish linkages with ongoing government and donor-funded programs and activities and help beneficiaries get greater access to nontraditional opportunities, especially for females.  Establish linkages with the private sector to help female beneficiaries obtain access to training and internship opportunities. Foster a culture supportive of female employment in general, and in nontraditional roles in particular.  Educate parents on gender equality and the importance of encouraging girls to explore nontraditional avenues of work.  Work with the Bureau of Women’s and Children’s Affairs to ensure training programs are supportive of female youth. Foster community support for female youth in nontraditional employment/roles.  Educate parents on gender equality and the importance of encouraging girls to explore nontraditional avenues of work.  Work with the Bureau of Women’s and Children’s Affairs to ensure training programs are supportive of female youth.  Foster community support for female youth in nontraditional employment/roles. Train on group work as a means of improving access to loans/financing.  Train all participants, especially female youth, in working in groups and in building trust, as loans and financing are more easily available for youth organized in groups. Recommendations for Future Programs and Activities YP has succeeded in changing attitudes toward employment and behavior of participants. Recommendations for new activities building upon YP include the following:  Provide a sum of money to implementers to lend to participants, especially female participants, to start their businesses. 39 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report  Offer hard-skills training to a larger proportion of participants with some kind of a give-back-to￾community provision to prevent migration of youth to greener pastures.  Establish MOUs between the government and private sector that incentivize the private sector to offer practical training to the participants and to hire locally.  In collaboration with the Bureau of Women’s and Children’s Affairs, develop programs to incentivize hiring of female employees by the public and private sector and ensure existing programs and regulations are implemented.  Initiate/include an awareness campaign and community-based training on equal opportunities for female participants and on encouraging female participation in nontraditional employment.  Enhance the capacity of the government to support creation of local job opportunities through systems strengthening and by providing exposure to regional and international best practices. 40 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report ANNEXES Contents of this section: (i) Statement of work for the YCS evaluation (ii) YP results framework (iii) Summary of the YCS baseline report (iv) Summary of the YCS midline report (v) List of study questions and crosswalk from questions to indicators to sources of data (vi) List of people consulted and interviewed and FGDs conducted (vii) Sources of data and a list of documents reviewed (viii) Data collection instruments a. Endline survey b. Financial diary questionnaire c. Guide for FGDs – Youth Participants d. Interview Guide - Partners e. Interview Guide - Employers f. Guide for FGDS – Parents g. Guide for KIIs – YP Facilitators h. Guide for KIIs – Microfinance Institutions (ix) Select additional data tables, including regression data tables (x) Study team a. Study team disclosure and conflicts of interest b. Study team profiles (xi) Post Evaluation Action Plan 41 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report ANNEX I: YCS STATEMENT OF WORK INTRODUCTION USAID/Ethiopia developed this Statement of Work for an independent performance evaluation of the USAID/Ethiopia’s Building the Potential of Youth. The findings of this evaluation will inform activity mid￾term course corrections and future youth workforce development programs in Ethiopia. USAID’s Building the Potential of Youth is a five-year activity implemented by Save the Children International in partnership with Education Development Center, Inc. (EDC), HUNDEE-Oromo Grassroots Development Initiative (HUNDEE), and Professional Alliance for Development in Ethiopia (PADet), Relief Society of Tigray (REST) and Facilitator for Change Ethiopia (FCE). The Activity’s information is summarized below. Activity Name USAID’s Building the Potential of Youth Cooperative Agreement AID-663-A-15-00006 Start Date – End Date Jan 1, 2015 – December 31,2019 Total Estimated Cost $ 17,326,954 Activity Funding DA, Higher Education Implementing Partner Save the Children International (SCI) Activity AOR Tahir Gero ACTIVITY OBJECTIVES AND IMPLEMENTATION APPROACH The purpose of the USAID’s Building the Potential of Youth activity is to provide Ethiopian youth (ages 15-29) in rural areas and towns with access to workforce development/livelihood support and resources, tailored to their specific needs and market demand, so that they can a) achieve increased income and b) strengthen skills, knowledge and social capital required to achieve economic self￾sufficiency over the longer-term. The Results Framework of the activity is included in the Annex section of this document. USAID’s Building the Potential of Youth proposed strategies and approaches respond to capacity needs of unemployed and underemployed youth, leverage existing youth development activities, and contribute to improving the quality and accountability of local training and business service providers by ensuring existing entities work in concert. ● By using targeted assessments, the USAID’s Building the Potential of Youth team will ensure training and service providers consider market relevance and target curricula and programs to emerging skills needs of youth participants. ● By focusing on skills building, will offer tailored technical and life skills trainings to youth to create more viable livelihood prospects. ● By expanding approaches and offerings in Work-Based Learning, including employer visits, 42 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report organized job-shadowing, short-term employment, and internships, both learners and employers benefit from training and supervisory support, leading to workforce-ready and employable youth with practical experience. ● By implementing “Hubs,” USAID’s Building the Potential of Youth will build Youth Service Provider Networks and provide a cost-effective approach to enhance support service access for remote communities and disadvantaged groups. SCOPE OF WORK Purpose & Overview The purpose of this task is to conduct an independent M&E activity for USAID’s Building the Potential of Youth activity. The activities performed under IR1 (“Rigorous evidence base developed to inform USAID and its development partners, including youth, GOE, and NGOs, on appropriate, effective, and sustainable market driven youth workforce development interventions”) are not subject to evaluation under this Task Order. The services to be provided under this activity include: ● Activity A: A Performance Evaluation including baseline (limited), midline and endline data collection, analysis and reporting. ● Activity B: A Performance Monitoring and Data Systems Review that will include an analysis of Save the Children (SCI) and its partners’ data collection, storage, analysis and reporting systems. ● Activity C: A Youth Cohort Study focused on a subset of youth participants and geographic implementation areas. Activity A: PERFORMANCE EVALUATION Evaluation Questions Table 1below outlines the evaluation questions to be addressed by the Performance Evaluation. The Table indicates whether the question is applicable to the baseline survey, midline or endline evaluation, likely sources of data and potential data collection methods. Wherever applicable/possible, the analysis for all the questions listed above should be presented by sex, age category [15-19, 20-24, and 25-29], socio-economic status, location, educational attainment, mother tongue and ethnicity. The information provided below is intended to assist evaluators in the development of a work/staffing plan and budgets. 43 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Table 1: PERFORMANCE EVALUATION QUESTIONS MATRIX Questions (in order of priority) Baseline/Mi dline/ Endline Suggested Data Sources Notes Geographic focus and illustrative sample/selection INDIVIDUAL-FOCUSED QUESTIONS 1 To what extent do youth participating in the Building the Potential of Youth have improved employment (including self-employment) status? How do these outcomes vary by participant characteristics and partner? Midline, Endline Secondary analysis of SCI and partners’ data from intake and exit participant surveys [SCI Youth Profile Data Collection Tool] Midline: Aggregate and analyze data on all participants, or a sub￾sample of participants. Endline: Aggregate and analyze data on all participants, or a sub￾sample of participants. 2 To what extent do youth participating in the Building the Potential of Youth have increased income? How do these outcomes vary by participant characteristics and partner? Midline, Endline Secondary analysis of SCI and partners’ data from intake and exit participant surveys [SCI Youth Profile Data Collection Tool] Midline: Aggregate and analyze data on all participants, or a sub￾sample of participants. Endline: Aggregate and analyze data on all participants, or a sub￾sample of participants. 3 What are the outcomes, [measured in income, productive assets, and debt level and employment quality] for youth receiving microfinance loans facilitated by the Building the Potential of Youth program, how do these outcomes vary by participant characteristics and partner? Midline Endline Primary data collection with microfinance recipients. Secondary analysis of SCI and partners’ data (to the extent available). Midline: data collection from initial 46 recipients of microfinance in Tigray. AND assume a sample of 80 recipients across 6 Regions and 18 Woredas for primary data collection and analysis. Endline: assume a sample of 80 recipients across 6 Regions and 18 Woredas for primary data collection and analysis. 4 What percentage of Work Ready Now (WRN)! completers engage in quality work-based learning opportunities facilitated by SC and its partners? How do these outcomes vary by participant characteristics and partner? Midline, Endline Secondary analysis of SCI and partners’ administrative data to determine overall percent. Plus, sample￾based primary data collection from youth participants to determine quality of opportunities. Midline: assume a sample of 60 recipients across 4 Regions and 8 Woredas for primary data collection and analysis. Endline: assume a sample of 80 recipients across 6 Regions and 18 Woredas for primary data collection and analysis. 44 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report 5 What percentage of internships/apprenticeships transition into paid employment? How do these outcomes vary by participant characteristics and partner? Midline, Endline Secondary analysis of SCI and partners’ administrative data to determine overall percent. Plus, sample￾based primary data collection from youth participants to validate and supplement data. Midline: assume a sample of 60 recipients across 4 Regions and 8 Woredas for primary data collection and analysis. Endline: assume a sample of 80 recipients across 6 Regions and 18 Woredas for primary data collection and analysis. 6 To what degree has the Building the Potential of Youth program improved the functional literacy and numeracy of youth participants? Endline Secondary analysis of SCI and partners’ administrative data. 7 How effectively has the Building the Potential of Youth program produced gender-equitable outcomes for youth participants? Endline Analysis of secondary and primary data. SYSTEM-FOCUSED QUESTIONS 8 To what extent have the “YES Hubs” implemented by the Building the Potential of Youth program increased access to quality employment-related services (e.g., job facilitation, business development, entrepreneurial training, connection to resources) for participating youth? Midline Endline Sample-based primary data collection from YES Hub stakeholders, including youth facilitators, youth participants, partner staff. Midline: assume a sample of 16 “YES Hubs” (stratified by type of Hub), across 4 Regions and 8 Woredas for primary data collection and analysis. Endline assume a sample of 36 Yes Hubs (stratified by type of Hub) across 6 Regions and 18 Woredas for primary data collection and analysis. 9 For Baseline: 9b1.What is the status of youth focused training in the TTIs in terms of availability, quality and relevance for the needs of young men and women? 9b2.What are the major challenges of local training centers/ institutes in target Woredas to integrate and deliver market driven trainings? For Midline and Endline: 9e1.To what extent have funds provided to Training Institutes increased the availability, quality and relevance (in terms of employment/income) of training for youth? Baseline/Mid line, Endline Primary data collection (including baseline) from Training Institute staff and employment partners, and youth participants. Secondary analysis of SCI and Training Institute administrative records, as available. Baseline: assume a stratified sample of 18 Training Institutions across Amhara, Tigray, Oromia, Somali, Afar and SNNPR regions (Afar and SNNPR baselines may need to take place at midline due to TI selection process timing). Midline: assume a stratified sample of 18 Training Institutions across Amhara, Tigray, Oromia and Somali regions. 45 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Endline: assume a stratified sample of 18 Training Institutions across Amhara, Tigray, Oromia, Somali, Afar and SNNPR regions. 10 To what extent and in what manner are public-private partnerships providing quality training and/or employment opportunities for participants in the Building the Potential of Youth program? Endline Primary data collection with a sample of employment partners. Secondary analysis of SC and partners’ partnership records. Endline: assume a purposeful stratified sample of 36 partnerships across 6 Regions and 18 Woredas for primary data collection and analysis. 11 What is the operational and service delivery capacity of the Central YES Hub? How sustainable (operationally and financially) is the Central YES Hub likely to be post-USAID support? Endline Primary data collection focused on Central YES Hub, utilizing organizational capacity and sustainability assessment frameworks. 12 Is the Building the Potential of Youth training model (Master Trainers training youth trainers who then train participants) achieving the stated learning objectives of the WRN! curriculum? Midline Primary data collection including knowledge assessments, classroom observations during WRN! training. Plus, secondary analysis of SC and partners’ data. Midline: assume a sample of 20 youth trainers and 80 youth participants across (4) partners (Amhara, Tigray, Oromia and Somali regions). Evaluation design should support analysis of implementation fidelity and efficacy for each cascade. 13 What have been the most significant successes and challenges for the design, implementation and M&E (treated separately) for the Building the Potential of Youth program? Endline Summary analysis of secondary and primary data collected. Midline Performance Evaluation In addition to addressing the questions outlined in the Evaluation Question matrix above (those indicated as part of the midline), the Contractor must analyze performance monitoring data as well as quarterly and annual reports to identify priority areas of interest for the midline data collection/analysis. Priority areas of interest may include: ● Geographic areas where activity monitoring data indicates under or over performance vis-a-vis targets. ● Local implementing/training partners reporting participant outcomes of interest such as low/high 46 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report retention rates, low/high performance on skills assessments, low/high job placement rates, low/high improvements in employment/income status. Final selection of midline priority areas of interest and midline design will be conducted in collaboration with USAID and SCI. As part of the Midline Evaluation, the Contractor will conduct an in-country workshop with USAID, SCI and its three local partners, government officials and other stakeholders (at the direction of USAID) to review the draft report and key findings. The workshop outcomes will include: A summary of key success factors to date with action plan to expand successes. ● A prioritized list of program/implementation corrective actions with associated action plan and timeline. ● A prioritized list of monitoring/data collection corrective actions with associated action plan and timeline. ENDLINE PERFORMANCE EVALUATION The Contractor will conduct an in-country workshop with USAID, SCI and its partners, government officials and other stakeholders (at the direction of USAID) within 4 months of endline data collection to review the draft report and key findings. The workshop outcomes will include: 1. A summary of key lessons learned regarding design and implementation of USAID’s Building the Potential of Youth. 2. A prioritized list of research/evaluation questions for future youth workforce development activities in Ethiopia. 3. A prioritized list of policy recommendations to improve employment and economic outcomes for sub population of youth as needed. ACTIVITY B: PERFORMANCE MONITORING AND DATA SYSTEMS REVIEW The Contractor for the Ethiopia Performance Monitoring and Evaluation Service (EPMES) contract, who will be managing this activity, hereinafter “Contractor,” is required to conduct a Performance Monitoring and Data Systems Review to initiate the activities under this activity. The objectives of the Performance Monitoring and Data Systems Review include: ● Determine the extent to which SCI and its partners’ performance monitoring and data systems are capturing reliable data aligned with the Building the Potential of Youth PMEP. ● Provide recommendations for improving the quality, utilization, reliability and scope of the data being collected by SCI and its partners, in support of the requirements of the PMEP. ● Implement a follow-up review to determine the extent to which the recommendations have been implemented. It is anticipated that this activity will include detailed reviews of measurement strategies, instruments, data collection, aggregation, and analysis process, data storage and reporting processes. 47 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report The Contractor should include a report detailing the recommendations. Additionally, the Contractor must plan and facilitate an in-country workshop with SCI, its three local partners and USAID to review the recommendations and identify priorities and process for integrating recommendations into the Building the Potential of Youth monitoring and data systems. The information obtained during the Performance Monitoring and Data Systems Review (in terms of availability and quality of implementing partner data) should be utilized to finalize the Performance Evaluation and Youth Cohort Study designs. ACTIVITY C: YOUTH COHORT STUDY USAID desires a Youth Cohort Study to deepen its understanding of participant outcomes. The overarching purpose of the Youth Cohort Study is to determine the extent to which the Building the Potential of Youth program has improved the employment status and/or income level of a subset of youth participating in the program. Specifically, the Youth Cohort Study will address the following questions: 1. To what extent has the income and employment status of study participants changed between enrollment in the Building the Potential of Youth program and twelve months following enrollment? Does increased income led to improved health outcomes among study participants? 2. To what extent has participation in the Building the Potential of Youth program improved the work readiness skills of study participants? 3. To what extent has participation in the Building the Potential of Youth program improved knowledge of and access to employment/entrepreneurial resources for study participants? 4. To what extent has participation in the Building the Potential of Youth program improved, functional literacy and numeracy skills of study participants? 5. How do the outcomes addressed in the questions above vary by participant characteristics (e.g., sex, age, ethnic/linguistic group, socio-economic status, marital status, and parenting status) and service delivery partner? The Contractor is expected to collect three or four rounds of data (depending on budget) on a sample of youth (panel design) across four Regions (Oromia, Amhara, Tigray and Somali). Data collection will take place in 10-15 Woredas across the four Regions. The Woreda selection will be led by the Contractor in consultation with SCI and USAID/Ethiopia and should be stratified across productive and pastoral Woredas. The Contractor should able to propose innovative methods for collecting reliable data on income and employment over time (such as financial diaries) for a panel of youth with potentially high mobility. For budgeting and planning purposes, the Contractor should assume a sample of 90-100 youth participants in each region, for a total approximate sample of 360-400 youth participants. The Contractor will develop a sample design that balances precision, efficiency and cost, in consultation with SCI and USAID/Ethiopia. Strengths and limitations of the sampling approach shall be described in the Design Report. 48 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report The Contractor should propose a design for the Youth Cohort Study, utilizing the illustrative timeline below as a guide: Key Evaluation Principles USAID’s Evaluation Policy encourages independent external evaluation to increase accountability for results, refine ongoing activities, and improve future designs. Transparency: Extensive documentation of evaluation design, sample and selection approach, data collection methods, instruments and other processes will be required so that USAID and other audiences can determine the internal and external validity of the evaluation. This understanding is critical for appropriate use of evaluation findings. Leveraging Best Practice: The Contractor is expected to utilize best practices when developing the evaluation design. The Evaluation Design must reflect proven methods for sampling, selection, data collection and analysis. All instruments utilized in the performance evaluation and youth cohort study should be based on tested measurements for youth programming. Consultative Design Process: The Contractor should plan for a highly consultative process with USAID, SCI and the government representatives to develop study designs and instruments under this activity. EVALUATION DESIGN AND METHODOLOGY The methodology proposed must comply with USAID’s Evaluation Policy (http://www.usaid.gov/evaluation). A mixed method approach utilizing quantitative and qualitative Baseline data collection/cohort intake Program intake Post-WRN! /other training assessment 4-8 weeks Data collection focused on employment, income, access to resources 3 months Endline data collection focused on employment, income, access to resources 12 months 49 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report methods shall be included to answer the evaluation questions. To extent possible, data should be disaggregated by sex, age, geographic location, socioeconomic status, ethnic/linguistic group, etc. Data collection methods proposed by the contractor should be linked to the specific baseline/evaluation question(s). The suggested methodology should include, but is not limited to: ● Survey of beneficiaries. ● Pre-and post-assessments of literacy/numeracy and work readiness skills of youth study participants. ● Key Informant Interviews/Surveys with employers, technical vocational education and training (TVET) instructors, technical training institute (TTI) leaders, etc. ● Focus Group Discussions with youth participants. ● Observation of participant training, youth engagement groups, skills-building classes/workshops, work-based learning activities, etc. ● Key Informant Interviews/Surveys with private sector partners. ● Existing Data Review of SC and implementing partners’ baseline data, assessments, and tracking records of program participants. EXISTING INFORMATION SOURCES Existing data and program information will be made available to the contractor by USAID, SCIs and its partners. A list of potential documents for the contractor to review is presented below. 1. Activity original and amended description (if any) 2. Activity Work Plan 3. Activity reports 4. Activity M&E Plan 5. Survey of youth pre-& post (after six months) participating in the Building the Potential of Youth, collected by SC and its partners. 6. Employer surveys 7. Out-of-school literacy (OSL) and numeracy assessments, implemented by SC and its partners. 8. Pre & post-workforce readiness assessments, implemented by SC and its partners. 9. Technical Training Institute (TTI) records 10. Agreement documents between Building the Potential of Youth partners and employers 11. Tracking records of program participants including internships, class/workshop attendance, etc. Analysis of previously collected data and data collected by the contractor must be disaggregated and compared across groups as specified in the baseline/evaluation question(s). The contractor will determine the acceptable level of margin of error depending on the statistical measure used. In the report(s), the contractor must mention inherent limitations in the data collected and analyzed. 50 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report GEOGRAPHIC SCOPE OF PRIMARY DATA COLLECTION The following section outlines the geographic scope of the anticipated primary data collection to be executed under this activity. Secondary data analysis performed under this Task Order will cover the entire geographic scope of the Building Potential of Youth program. Baseline Data Collection The primary baseline data collection focused on Question 9 of the Evaluation Matrix (targeting the Training Institutions) will take place in intervention regions. The sample should be stratified, at a minimum, across type of Training Institution, productive/pastoral Woreda designation. See Evaluation Question Matrix for additional details. Midline Data Collection The Midline Evaluation will include primary data collection in intervention regions. In each of the regions, data collection will take place in sample Woredas. The Woreda selection will be led by the Contractor in consultation with SCI and USAID/Ethiopia and should be stratified across productive and pastoral Woredas. Youth Cohort Study The Youth Cohort Study will include primary data collection in intervention regions. In each of the regions, data collection will take place in sample Woredas. The Woreda selection will be led by the Contractor in consultation with SCI and USAID/Ethiopia and should be stratified across productive and pastoral Woredas. Endline Data Collection The Endline Evaluation will include primary data collection in intervention regions. In each of the regions, data collection will take place in sample Woredas. The Woreda selection will be led by the Contractor in consultation with SCI and USAID/Ethiopia and should be stratified across productive and pastoral Woredas. ANTICIPATED TIMELINE The Table below outlines the anticipated timing for each of the major activities under this Task Order. The final timeline proposed by the Contractor should maximize data collection/field time efficiencies to the extent possible. Activity Anticipated Timing (in-country work) M&E Review July/August/Sept 2016 Baseline for Question #9 July/August/Sept 2016 51 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Midline Feb/Mar/Apr 2017 Youth Cohort Study Feb/Mar/Apr 2017 Endline May/Jun/Jul 2019 DELIVERABLES The contractor is responsible for preparing and submitting the following deliverables: Activity A: Performance Evaluation Deliverables: 1. Performance Evaluation Design Report (EDR) (draft and final): The EDR should articulate how the design will address all questions in the Evaluation Question Matrix and should include a detailed methodology, sample design, data collection, management and analysis plan, data collection tools and timeline. The Performance Evaluation EDR should cover the baseline, midline and endline. The draft evaluation report will be submitted by the contractor to the USAID’s Program Office. USAID will provide comments within 10 business days. 2. Performance Evaluation Work Plan: Prior to each round of data collection, the Contractor shall schedule an in-briefing with USAID Thirty days in advance of this meeting, the Contractor shall provide a detailed work plan (draft and final) that includes a list of in-country consultations, a schedule for the field work, and a list of the team members with contact details while in-country. 3. Baseline Study Report focused on Question #9 in the Evaluation Question Matrix (changes in Training Institute capacity and function). The Contractor will collect primary data to answer baseline questions in the performance evaluation questions matrix (see # 9b1 and 9b2 in Table 1). In addition, the Contractor will review the baseline/background information collected for each youth enrolled by the implementing partner and analyze the data collected both from primary and secondary sources and finally produce a baseline survey report. USAID will review and approve the Baseline Study Report., not exceed 20 pages of main text. 4. Midline Performance Evaluation Report (draft and final): The contractor shall prepare a descriptive and analytical report presenting the main findings and suggesting appropriate options and recommendations. Not to exceed 20 pages excluding Annexes. 5. Midline Performance Evaluation Executive Summary and Slide Deck (draft and final): The contractor shall prepare a 2-3-page Executive Summary and 10-15 slide presentation of key midline evaluation findings. 6. Facilitated in-country workshop with SCI, its partners, USAID and other stakeholders to review the findings of the Midline Evaluation. 7. Summary notes from in-country workshop. 8. Endline Performance Evaluation Report (draft and final). The contractor shall prepare descriptive and analytical report presenting the main findings and suggesting appropriate options and recommendations. Not to exceed 20 pages excluding Annexes. 9. Endline Performance Evaluation Executive Summary and Slide Deck (draft and final): The contractor shall prepare a 2-3-page Executive Summary and 10-15 slide presentation of key endline evaluation findings. 52 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Activity B: Performance Monitoring and Data Review Deliverables: 1. Performance Monitoring and Data Review Design Report (draft and final). The Design Report should articulate how the Contractor will approach the review and should include a detailed methodology, data collection, management and analysis plan, any data collection tools, and timeline. Additionally, the Design Report should include requests for information/data/staff time from SCI and USAID. 2. Performance Monitoring and Data Review Work Plan: Prior to in-country field work, the Contractor must schedule an in-briefing with USAID. Thirty days in advance of this meeting, the Contractor shall provide a detailed work plan (draft and final) that includes a list of in-country consultations, a schedule for the field work, and a list of the team members with contact details while in-country. 3. Performance Monitoring and Data Review Study Report (draft and final) that includes prioritized recommendations for USAID and SC. Not to exceed 15 pages excluding Annexes. 4. Facilitated in-country dissemination workshop with SCI, its partners, USAID, GoE representatives (Ministry of Youth and Sports, Ministry of Education and Ministry of Agriculture and Natural Resources) and other stakeholders to review the findings of the Performance Monitoring and Data Review Study. 5. Summary notes from in-country workshop. 6. Documented Follow-up Review Report: 4-6 months after the workshop to assess the degree to which priority recommendations have been implemented, based on interviews and consultations with stakeholders. Youth Cohort Study Deliverables: 1. Youth Cohort Study Design Report (draft and final): The Study Design Report should articulate how the design will address all 5 study questions and should include a detailed methodology, sample design, data collection, management and analysis plan, data collections tools and timeline. USAID will review and approve the Youth Cohort Study Design Report. 2. Youth Cohort Study Work Plan: Prior to each round of data collection, the Contractor shall schedule an in-briefing with USAID. 30 days in advance of this meeting, the Contractor shall provide a detailed work plan (draft and final) that includes a list of in-country consultations, a schedule for the field work, and a list of the team members with contact details while in-country. 3. Summary Report (5-7 pages) for each of the four data collection cycles (draft and final), including analysis of key data and findings. 4. Draft Youth Cohort Study Report following final data collection. 5. Facilitated in-country workshop with SCI, its partners, USAID and other stakeholders to review the findings of the Youth Cohort Study. 6. Summary notes from in-country workshop. 7. Final Youth Cohort Study Report (not to exceed 30 pages excluding Annexes) following in-country workshop. 8. Youth Cohort Study Report Executive Summary and Slide Deck (draft and final): The contractor shall prepare a 3-5-page Executive Summary and 10-15 slide presentation of key Youth Cohort Study Report findings. 53 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report TEAM COMPOSITION The contractor is expected to have the appropriate competencies in the team to fully execute scope of this Task Order. The team should have expertise and significant experience in the following areas (not in order of priority): 1. Complex team and program management, including management of local sub-contractors. 2. Evaluation design: including sample design, mixed method designs, data analysis and instrument development. 3. Assessment and use of existing data sources to be utilized for secondary analysis. 4. Measurement strategies associated with youth workforce development programs. 5. Data collection planning and implementation in complex and geographically diverse settings. 6. Presentation of evaluation findings to a broad range of stakeholders, including government officials, to optimize uptake of recommendations. 7. Effective collaboration and coordination with implementing partners and other local stakeholders during evaluation design and implementation. 8. Knowledge of USAID requirements with respect to data sharing, treatment of PII, ethical considerations/IRB. 9. Excellent written and oral communication in English and Amharic. 10. In-depth knowledge of the Ethiopian context and development programming in Ethiopia. 11. At least one Key Personnel must have demonstrable experience in integrating and using gender sensitive approaches in M&E work. 12. Experience in gender sensitive monitoring and evaluation methods 13. Knowledge of local languages Oromo, Tigrinya, Somali, Afar and/or other local languages among team members would be preferred. The Contractor must propose a staffing plan that includes team leader and other experts and fully supports the requirements of this activity. The proposed staffing plan must explicitly address how the plan complies with the expertise and experience requirements cited above and how the team members will work together. Additionally, the staffing plan must describe what each team member's’ primary responsibility will be and their anticipated LOE for the duration of each task stipulated in this SOW. The Contractor must also make use of services of a local subcontractor to support the work under this activity (e.g., instrument translation/validation, data collection, logistics for field work). No evaluation team members shall have been directly involved in the implementation of the Building the Potential of Youth program. A statement of potential bias or conflict of interest (or lack thereof) is required of each team member. USAID MANAGEMENT The Contractor will identify and hire the evaluation team, pending the Contracting Officer’s Representatives (COR’s) and relevant technical office’s concurrence, assist in facilitating the work plan, 54 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report and arrange meetings with key stakeholders identified prior to the initiation of the fieldwork. The evaluation team will organize other meetings as identified during the course of the evaluation, in consultation with EPMES’s Contractor and USAID/Ethiopia. The Contractor is responsible for all logistical support required for the evaluation team, including arranging accommodation, security, office space, computers, Internet access, printing, communication, and transportation. The evaluation team will officially report to the Contractor, Social Impact. The Contractor is responsible for all direct coordination with the USAID/Ethiopia Program Office through the EPMES COR. From a technical management perspective, the evaluation team will work closely with Tahir Gero, USAID’s Building the Potential of Youth AOR in the Education and Youth Office. In order to maintain objectivity, all final decisions about the evaluation will be made by the Program Office. Additionally, the evaluation contractor will consult and coordinate with SCI and its partners during the design and implementation of this activity. LOGISTICS The Contractor will be responsible for all travel and logistics associated with conducting the evaluation. 55 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Annexes Annex 1: Results Framework SO: Unemployed and underemployed Ethiopian youth ages 15 - 29 in rural areas and towns attaining the skills, knowledge, and social capital that lead to increased income and long-term economic self-sufficiency will be achieved by improving access for Ethiopians to basic education, TVET, and practical work experiences and bringing together local stakeholders such as TVET institutions, private sector, youth, and government IR 1: Rigorous evidence base developed to inform USAID and its development partners, including youth, GOE, and NGOs, on appropriate, effective, and sustainable market driven youth workforce development interventions IR 1.1 Local context and beneficiary profile information base developed to inform the design of customized interventions that enable access to opportunities for youth skills development, decent work and increased income IR 1.2: Evidence base developed to enable program learning and course correction, and to inform future USAID and GOE investments in youth IR 2: Improved youth access to market relevant skills and experiences to support the transition to safe and viable employment IR 2.1: Improved access to quality and relevant basic life, work readiness and employability skills training, including literacy and numeracy learning IR 2.2: Improved access to quality career coaching and work-based learning (WBL) services IR 3: Coordinated delivery, among local institutions, of quality market driven services connecting undeserved youth, especially women, to employment and income opportunities IR 3.1: Increased capacity of local training and business service providers to integrate & deliver market driven employment IR 3.2: Existing and newly established networks of youth service provides and local industries are coordinated to connect youth to employment 56 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Annex 2: Geographic Coverage of the USAID’s Building the Potential of Youth Intervention The following Table outlines the 30 Woredas that will be targeted by USAID’s Building the Potential of Youth Table 1: Target Regions and Woredas PRODUCTIVE Amhara Tigray Oromia SNNP Somali Afar Woreda Woreda Woreda Woreda Woreda Woreda Danegela Ofla Limu-Bibilo Endegene Jawi Raya Alamata Kofele Merab Azernet Semin-Achefer Endamehoni Ada’a Yem Special Woreda Debube￾Achefer Welmera Bahir-Dar Ketma Zuria Sinana Dera Agarfa Liban Chiquala PASTORAL Yabello Esera Jijiga Chifra Dire Hammer Babile Dubti Gode Asayita Kelafo 57 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Annex 3: Targets by region and Phase Target by Phase Year 1 Year 2 Year 3 Year 4 Year 5 Total Phase 1 Woredas – 8 initial Amhara 600 600 600 600 600 3,000 Tigray 600 600 600 600 515 2,915 Somali 600 600 600 500 222 2,522 Oromia 600 600 600 600 600 3,000 Total 2,400 2,400 2,400 2,300 1,937 11,437 Phase 2 Woredas – additional 12 Amhara 600 600 600 600 2,400 SNNPR 900 900 900 900 3,600 Afar 600 600 600 600 2,400 Somali 300 300 300 300 1,200 Oromia 900 900 900 900 3,600 Total --- 3,300 3,300 3,300 3,300 13,200 Phase 3 Woredas – additional 10 Amhara 600 600 600 1,800 Tigray 300 300 300 900 SNNPR 600 600 600 1,800 Afar 300 300 300 900 Somali 300 300 300 900 Oromia 1200 1200 1200 3,600 Total --- --- 3,300 3,300 3,300 9,900 58 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Annex 4: Final List of Target Woreda and Intake POTENTIAL Program Yearly phase in operation Woredas No. Region POTENTIA L Enrolled Enrolled Enrolled Worda s Year 1 Year 2 Year 3 1 Amhara Debub Achefer X 2 Amhara Semin Achefer X 3 Amhara Bahir-Dar Ketma Zuria X 4 Amhara Dangela X 5 Amhara Jawi 6 Amhara Dera X X 7 Oromia Ada'a X 8 Oromia Liben Chulala X 9 Oromia Limu-Bilbilo X 10 Oromia Kofela X 11 Oromia Sinana X 12 Oromia Agarfa X 13 Oromia Welmera X 14 Oromia Yabello X 15 Oromia Dire X 16 SNNPR Endegegn X 17 SNNPR Mirab-Azernet X 18 SNNPR Esira X 19 SNNPR Yem Special Woreda X 20 SNNPR Hammer X 21 Tigray Raya Alamata X 22 Tigray Ofla X 23 Tigray Endemehoni X 24 Somali Jigjiga X 25 Somali Babile X 26 Somali Gode X 27 Somali Kelafo X 28 Afar Chifra X 29 Afar Dubti X 30 Afar Asayita X Total 8 12 10 59 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Annex 5: M&E and data collection activities under the Building the Potential of Youth cooperative agreement SC and its partners will be performing formative assessments at the Woreda level for a portion of the target Woredas. The assessments will include: ● Rapid youth assessment focused on knowledge, attitudes, practices and assets, with an emphasis on well-being ● Literacy and numeracy assessments of youth with the targeted profile ● Employment market assessment to identify employment opportunities, skills gaps and programed economic growth/employment opportunities ● Existence of youth service providers, their services, scale of reach, target youth population(s), and organizational capacity ● Existence of other youth-oriented programs (e.g., government, donor) The findings of these assessments will inform final activity design and implementation approach and identify required customizations at the Woreda and Kebele levels. These assessments will be performed on a rolling basis as the activity expands to additional geographic areas and are deliverables under IR1 of the Results Framework. SC sampling strategy for the assessments listed above is… ANNEX: Mapping of Intervention Areas ANNEX: Performance Monitoring Plan ANNEX: Implementation Timeline ANNEX: Targets (if applicable) ANNEX: Year 1 Annual Report ANNEX: Year 2, Q1 Report ANNEX: USAID Youth Workforce Development Research Agenda ANNEX: OTHER POTENTIAL reports??? Woreda Assessment Reports; Youth Profile Data; WRN Curricula Outline, YIA Curricula Outline, ANNEX: SC Youth Profile Data Collection Tool 60 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Annex 6: Report FINAL REPORT CONTENTS 1. Title Page 2. Table of Contents (including Table of Figures and Table of Charts, if needed) 3. List of Acronyms 4. Acknowledgements or Preface (optional) 5. Executive Summary (3-5 pages) The executive summary should succinctly capture the evaluation purpose and evaluation questions; program background; evaluation design, methods; and limitations; and the findings, conclusions, and recommendations. 6. Introductory Chapter a. A description of the program evaluated, including goals and objectives. b. Brief statement on purpose of the evaluation, including a list of the main evaluation questions. c. Brief statement on the methods used in the evaluation such as desk/document review, interviews, site visits, surveys, etc. d. Explanation of any limitations of the evaluation—especially with respect to the methodology (e.g., selection bias, recall bias, unobservable differences between comparator groups, etc.)—and how these limitations affect the findings. 7. Findings: This section should include findings relative to the evaluation questions. The information shall be organized so that each evaluation question is a sub-heading. 8. Conclusions: This section must answer the evaluation questions based upon the evidence provided through the Findings section. The information shall be organized so that each evaluation question is a sub-heading. 9. Recommendations: Based on the conclusions, this section must include actionable statements that can be implemented into the existing program or included into future program design. Recommendations are only valid when they specify who does what and relate to activities over which the USAID program has control. For example, recommendations describing government action is not valid, as USAID has no direct control over government actions. Alternatively, the recommendation may state how USAID resources may be leveraged to initiate change in government behavior and activities. It should also include recommended future objectives and types of specific activities based on lessons learned. The information shall be organized so that each evaluation question is a sub-heading. 10. Annex: The annexes to the final evaluation report should be submitted as separate documents—with appropriate labels in the document file name (e.g., Annex 1 – Evaluation SOW), and headers within the document itself—and may be aggregated in a single zipped folder. a. Evaluation Statement of Work b. Places visited; list of organizations and people interviewed, including contact details. c. Evaluation design and methodology. d. Copies of all tools such as survey instruments, questionnaires, discussions guides, checklists. e. Bibliography of critical background documents. 61 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report f. Meeting notes of all key meetings with stakeholders. g. “Statement of Differences” h. Evaluation Team CV’s Reporting Guidelines ● The format of the report shall be consistent with the USAID branding guidelines. ● The evaluation report should represent a thoughtful, well-researched and well￾organized effort to objectively evaluate what worked in the program over the given time period, what did not, and why. ● Evaluation reports shall address all evaluation questions included in the statement of work. ● The evaluation report should include the statement of work as an annex. All modifications to the statement of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline need to be agreed upon in writing by the Program Office. ● Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an annex in the final report. ● Evaluation findings will assess outcomes and impact on males and females, and data will be disaggregated by gender, age group, and geographic area wherever feasible. ● Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). ● Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative and/or qualitative evidence. ● Sources of information, including any peer-reviewed or grey literature, will be properly identified and listed in an annex. ● Recommendations will be supported by a specific set of findings. They will also be action-oriented, practical, and specific, with defined responsible parties for each action. 62 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report ANNEX II: YOUTH POTENTIAL ACTIVITY RESULTS FRAMEWORK SO: Unemployed and underemployed Ethiopian youth ages 15 - 29 in rural areas and towns attaining the skills, knowledge, and social capital that lead to increased income and long-term economic self-sufficiency will be achieved by improving access for Ethiopians to basic education, TVET, and practical work experiences and bringing together local stakeholders such as TVET institutions, private sector, youth, and government IR 1: Rigorous evidence base developed to inform USAID and its development partners, including youth, GOE, and NGOs, on appropriate, effective, and sustainable market driven youth workforce development interventions IR 1.1 Local context and beneficiary profile information base developed to inform the design of customized interventions that enable access to opportunities for youth skills development, decent work and increased income IR 1.2: Evidence base developed to enable program learning and course correction, and to inform future USAID and GOE investments in youth IR 2: Improved youth access to market relevant skills and experiences to support the transition to safe and viable employment IR 2.1: Improved access to quality and relevant basic life, work readiness and employability skills training, including literacy and numeracy learning IR 2.2: Improved access to quality career coaching and work-based learning (WBL) services IR 3: Coordinated delivery, among local institutions, of quality market driven services connecting undeserved youth, especially women, to employment and income opportunities IR 3.1: Increased capacity of local training and business service providers to integrate & deliver market driven employment IR 3.2: Existing and newly established networks of youth service provides and local industries are coordinated to connect youth to employment 63 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report ANNEX III: YCS BASELINE STUDY – SUMMARY United States Agency for International Development/Ethiopia (USAID/Ethiopia) has contracted with SI to implement the Ethiopia Performance Monitoring and Evaluation (EPMES) Activity. Under EPMES, USAID requested for an independent the performance evaluation of its Building the Potential of Youth (Youth Potential) Activity. This evaluation involves multiple approaches, which include: a baseline study; performance monitoring and data systems review and a multi-round cohort study of beneficiaries focused on a subset of youth participants and geographic implementation areas (henceforth referred to as the YCS. This report provides the baseline data for the YCS. The Youth Potential Activity is a five-year, USD 17.3 million USAID/Ethiopia Activity designed to provide youth (ages 15–29), especially girls and young women, in rural areas and towns with access to workforce development/livelihood support and resources, tailored to their specific needs and market demand, so that they can: (a) achieve increased income; and (b) strengthen skills, knowledge, and social capital required to achieve economic self-sufficiency over the longer term. Designed to reach approximately 34,537 unemployed or underemployed rural youth, the Activity is being implemented under a cooperative agreement between USAID and a group headed by SCI.35 Currently in its third year of implementation, the Activity is providing services in 30 Woredas in Afar, Amhara, Oromia, SNNPR, Somali, and Tigray regions of Ethiopia. The overall objective of the YCS is to evaluate whether youth who receive workforce development/livelihood support achieve increased income and strengthen skills, knowledge, and social capital required to achieve economic self-sufficiency over the longer term. The study will provide the stakeholders, USAID, the GoE, and the implementers of Youth Potential with information about the Activity and progress towards near-term outputs and outcomes. In addition, it will demonstrate long￾term contributions of Youth Potential initiatives on a cohort of youth. It will highlight major strengths and weaknesses of the Youth Potential’s initiatives to enable the stakeholders to make mid-course adjustments as necessary to the Activity as well as inform the design of similar programs in the future. Approach The YCS is following a subset of youth from Afar, Amhara, Oromia, and Tigray regions over a period of 12 months to better understand participant outcomes and the extent to which the Youth Potential Activity has improved the employment status and/or income level of youth participating in the program. It seeks to address the following broad questions36: 1. To what extent has the income and employment status of study participants changed between enrollment in the Youth Potential program and the twelve months following enrollment? Does increased income lead to improved health outcomes among study participants? 35 Cooperative Agreement # AID-663-A-15-00006; the implementation team includes Save the Children (SC) and its partners Education Development Center, Inc. (EDC), HUNDEE-Oromo Grassroots Development Initiative (HUNDEE), Professional Alliance for Development in Ethiopia (PADet), Relief Society of Tigray (REST) and Facilitator for Change Ethiopia (FCE). 36 From the EPMES statement of work 64 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report 2. To what extent has participation in the Youth Potential program improved the work-readiness skills of study participants? 3. To what extent has participation in the Youth Potential program improved knowledge of and access to employment/entrepreneurial resources for study participants? 4. To what extent has participation in the Youth Potential program improved functional literacy and numeracy skills of study participants? 5. How do the outcomes addressed in the questions above vary by participant characteristics (e.g., sex, age, ethnic/linguistic group, SES, marital status, parenting status) and by service delivery partner? This baseline study provides information on the status of each of the above among the study participants against which changes will be assessed and impact established. The focus of this study is to track changes on Activity beneficiaries included in the cohort and to document lessons learned for future similar Activity designs. Since Youth Potential covers a selection of eligible youth in specific locations/Kebeles within the Woreda and is implemented over an extended period, the study will employ a panel design to demonstrate whether the Youth Potential Activity caused changes in the expected outcomes or impacts. In a panel design, participants from whom data are collected over time serve as their own controls, since characteristics of an individual observed earlier can be controlled for when analyzing for changes in the expected outcomes. Baseline data collected on a select group of beneficiaries will help assess the effectiveness of the Activity as well as help focus the Activity on intended results by defining measures of the success of the Activity. To respond to the research questions listed above, data are being collected on several qualitative and quantitative indicators and sub-indicators that together will inform the results of the training programs as well as future possible modifications to the programs. To capture changes over the short and medium terms and attribute them to the Youth Potential Activity, data are being collected from a select group of participants (cohort) who have recently enrolled in the Activity at three points: (i) baseline, in the month of enrollment in the Activity (the current phase); (ii) mid-line, at six months from the point of enrollment; and (iii) endline, at twelve months post-enrollment in the Youth Potential Activity. This will be accompanied with periodic collection of income and expenditure data for the cohort. This baseline study was conducted during the period March to June 2017 using a mixture of qualitative and quantitative methods. Primary data were collected from a survey of 582 Youth Potential participants (out of a total expected sample of 670, response rate of 87%) from a purposive sample of 13 Woredas in the four regions of Afar, Amhara, Oromia, and Tigray.37 All Youth Potential participants from each of the selected Woredas who had enrolled in the program in the months of March and April 2017 were included in the sample. The survey instrument, designed in English, was translated into regional languages prior to administration. In parallel, qualitative data were collected by the study team through 29 focus groups with 277 participants (46% female) conducted in a purposive sub-sample of eight Woredas in the four regions (two Woredas in each region) and 54 KIIs and informational meetings held with stakeholders, including Government counterparts; TVET and TTI administrators; and Activity staff at the regional, Woreda, and Kebele levels. The Activity staff interviewed included the M&E officers, the Woreda officers, and the facilitators responsible for delivery of the training as well as coaching and 37 Thirteen percent of participants were either not traceable or chose not to respond. 65 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report arranging for the mentoring. In collaboration with the SC team, secondary data was collected through a review of Activity documents, websites, and administrative records. Main Findings The Youth Potential Activity, now in its third year of implementation, is offering participants a menu of five modular and integrated employability interventions38 that provide training on life skills, including skills designed to help participants prepare for work, search for employment opportunities, prepare their CVs and write applications, and improve their knowledge of resources available for finding employment as well as for self-employment. The participants are followed by facilitators who provide coaching39 and arrange for mentoring. All participants enrolled in the Activity start with a basic training of a set number of contact hours spread over 13–15 days, followed by modules selected on the basis of their personal development plan. The Activity offers training on a rolling basis, with new cohorts starting every four to six weeks. Each cohort has approximately 50 participants (two groups of 25 participants) from each of the active Woredas (for approximately 1,500 participants per cohort). Over the first two and a half years, the Activity trained about 12,000 participants. While both male and female participants, regardless of their literacy levels, are eligible to join the program, there are more male than female participants in the program, and the first few cohorts have focused on the literate candidates. The baseline data show that 43% of the 582 survey respondents were female, with the gender disparity being more significant in the Oromia regions (with 37% females). Over two thirds of the cohort were between 19–24 years of age, more than half have never been married, and most live with their parents. Overall, there is a vast disparity in the participants’ level of education ranging from no schooling all the way to completion of TVET or college. A clear majority of this group have completed some level of schooling, and over half have completed grade nine or higher (48% have completed grades 9-12, and 8% have completed TVET or college). There is also a disparity in the level of education of participants by region, especially in Afar where 19% of the participants had not attended any formal school and another 32% had completed only grade 5. 38 The training programs include Personal Youth Development, or Youth in Action and Work Ready Now! (WRN!), Be Your Own Boss, Work Ready Plus, and Aflateen. 39 Coaching refers to follow-on support provided to program participants by facilitators aimed at ensuring adequate use of the training. 66 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Demographic characteristics of participants, by region (in percentages) The first batch of Activity participants with lower levels of literacy has been recruited into the Activity in the March/April 2017 cohort. Participants are provided with a stipend/per diem of about US$1.3 per day (30 Birr) for attending the training, some of which is set aside for use by the participants as seed money to start their own enterprise.40 Income and Employment Status At baseline, 45% of the Youth Potential participants were unemployed and dependent on others for their sustenance, 41% were self-employed and the remaining 14% were employed either as daily laborers or engaged in short-term salaried jobs. Of those self-employed, two thirds were working in a self-owned business, and about a third were also looking for another job. More women than men were unemployed, and more women reported managing self-owned businesses. More male than female participants reported supporting family-owned activities and a very small proportion were engaged in group-owned businesses. Higher unemployment and lower self-employment rates were reported in Afar and Tigray compared with Amhara and Oromia regions. Most of the male participants reported being engaged in agriculture and livestock-related activities or daily labor, while the preferred fields for women were trading consumer products, providing services such as running a coffee/tea shop or a small restaurant, and agriculture and livestock-related activities. Lack of education and training, mismatch 40 The exchange rate used here and in the rest of this report is US$1= ETB 23.155 (as on June 16, 2017), as reported by EXCHANGE-RATES.org. 67 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report between training and job requirements, no work experience, and not enough jobs available were stated as the main reasons for unemployment or inability to work. For those engaged in self-employment, both male and female participants reported several hurdles in setting up their own business, including lack of land and working space, difficulty in obtaining working capital and financing, and licenses and market linkages. Current employment status of YCS respondents, by region (in percentages) Participants’ opinion of main reason for unemployment, by region (percent responding) Note: Numbers in the parenthesis in X-Axis represents number of unemployed respondents from each region. Out of total unemployed, 245 participants responded. Over 80 percent of focus group participants reported having little or no income. Further, 60 percent of participants lived with their families and worked in agriculture as sharecroppers or as daily labor, and thus could not definitively estimate their income or expenditures on themselves. For purposes of establishing a baseline, survey data was collected on household expenditures and assets owned by the individuals and their families. The largest expenditures reported were on investments in own business (26% of household expenditures), followed by house rent/lease (12%), support to other family members (12%), and food (10%). On an individual level, about 50% of all participants reported owning cattle and farm animals. Older participants reported owning assets such as land for farming and a house. Interestingly, 94% of male participants and 66% of female participants owned a mobile phone. While 67% of participants reported saving money for emergencies, participants in the focus groups conducted by the study team indicated that most incurred expenditure on health care only when they or their family 68 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report members were unwell. Most participants reported visiting the local health clinic/post or a nurse for most healthcare needs and going to the hospital in the city only in case of severe illness or emergency. Distribution of individuals reporting asset-holding, by region (in percentages) Work Readiness Skills Participants indicated a strong preference for self-employment over salaried employment, especially with the government as the salaries were rather low. Overall, they had a high opinion of their own readiness to work, rating themselves highly in terms of their interpersonal skills and self-confidence.41 Apart from the ability to write CVs or to access finance, land, etc., participants rated themselves as “good” or “neutral” on most items on a five-point scale ranging from very poor to very good. Male participants rated themselves higher (i.e., as being very good) compared to female participants. FGDs also indicated that participants felt they were ready for work and that the training would add the needed polish. Participants also expressed the hope that the training would help with full-time employment or business. Access to Employment/Entrepreneurial Resources Thirty-five percent of all survey respondents reported having tried to find employment in the past six months. Many employed or self-employed participants also reported looking for new jobs. While the most common avenues for looking for new opportunities were participation in job fairs, followed by asking relatives and friends for help, applying directly to employers and registering with public employment office, there are distinct gender differences in the preferred avenue for job search. Female participants reported being more likely to use avenues such as job fairs (51%), seeking the help of friends (29%), and registration with the public employment office (24%) (in that order), while male participants preferred seeking the assistance of friends (54%), followed by job fairs (47%), and direct applications (25%). Only 8% of male respondents reported responding to advertisements on the Internet or radio, an option not used by the females at all. Health Care Health care is a discretionary expenditure and one that is easily overlooked, except in an emergency. The hypothesis underlying the Youth Potential Activity is that with increased income, the participants in the Activity will spend more on their health and health-related expenditures, including family planning. 41 Note that for some of the baseline study participants (April cohort), WRN! training had started on April 1, 2017 while for others it started mid-Aril 2017. Data collection for all participants started in the second week of April 2017. Effectively, therefore, responses to questions related to work-readiness skills are spread over the duration of WRN! training. 69 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Thirty percent of all survey respondents reported being unwell and taking time off from work due to illness in the past six months. Most reported being unwell due to illnesses such as malaria, typhoid, allergic reactions, and a few reported serious illnesses such as breast and heart problems and kidney infections. Those who sought treatment reported going to a government hospital or a government health station/clinic. Forty-one percent of the respondents reported visiting a health care provider, between 1 and 3 times, over the past six months for themselves or someone in their family. Distribution of individuals reporting preferred provider for health care, by region (in percentages) Functional Literacy and Numeracy Skills While not a focus of the Youth Potential Activity, an expected benefit of enrolling in the Activity is improved levels of functional literacy and numeracy. Most participants, male and female, reported being able to read and write fluently, and most had more than just a functional level of literacy (the only exception was Chifra in Afar region). Over two thirds of the participants have completed grade 9 or higher (about 8% have completed TVET or college programs). Survey respondents were given simple tests of literacy and numeracy to assess their ability to read, write, and do basic arithmetic. Over two thirds of the participants demonstrated an ability to read and write with some fluency. Further, 97% of all participants could recognize numbers, and 92% were able do simple addition. Conclusions The YCS is following a subset of youth over a period of 12 months to enable better understanding of: (a) participant-level outcomes; and (b) the extent to which the Youth Potential Activity has improved the employment status and/or income level of a subset of youth participating in the project. The baseline study has collected qualitative and quantitative data on ten indicators for 582 participants in the month of enrollment in the program. This data serves as the benchmark against which the impact of the Activity will be assessed. Over the course of data collection, we noted that the Youth Potential Activity has gotten off to a slow start and has reached only a third of the target population at midpoint in the program. The team also experienced a need for continued communication and management of expectations for both participants and partners. Of the two major government stakeholders,42 the Youth Potential Activity has a strong relationship with the Ministry for Women and Children Affairs, which has oversight of the program; however, there is a need for strengthening the ties with the Ministry of Youth and Sports, which 42 At the time of award, the major government stakeholder was the Ministry (Bureau at the regional level) of Women, Children, and Youth Affairs. Since then, this has been reorganized into two entities: Ministry of Women and Children Affairs and the Ministry of Youth and Sports. 70 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report oversees government programming for youth. The focus group discussions revealed that participants had high expectations from the program. In addition to the training, they are expecting to be placed in an internship/WBL opportunity that would lead to a permanent job or be set up with enough financial and other resources for self-employment. This expectation must be managed to avoid a feeling of unfulfilled promises. In addition, the training is likely to generate a lot of demand for the few economic opportunities that are available in the Woredas, which has the potential of causing a lot of frustration among those who do not get their desired levels of employment and economic gains. Further, there is an expectation that successful participation in the Youth Potential Activity will result in higher incomes, which in turn would lead to gains in health outcomes and improved levels of functional literacy and numeracy. However, there is nothing in the training curriculum that directly addresses these topics. The training program does not have any training on health care or utilization of health services, nor does it have any activities that provide literacy or numeracy training. The difference between the theoretical construct and practical outcomes needs to be strategically managed. Challenges The team faced some challenges related to data collection. These include: (i) non-availability of data on enrollment of participants in the March/April cohort from SCI; (ii) difficulty in obtaining information on individual income and expenditures, since most participants live with their families; (iii) unavailability of simple easy-to-administer tests of literacy and numeracy relative to the more formal tests such as Early Grade Reading Assessment (EGRA) and Early Grade Math Assessment (EGMA); and (iv) despite all efforts to coordinate with the SC team, there were instances when the survey team was unable to find the facilitators and or the participants for conducting the surveys. Solutions for these challenges were found using alternate measures, for example, data from administrative reports was used to replace enrolment data, proxy data on ownership of items was used to determine income, and finally simple tests of literacy and numeracy were developed by the survey team, but the challenges remain. Similarly, the SC team has promised all help for the next round of data collection, but again the challenge remains. A separate challenge was to do with the context of the participants, we were unable to find out if there were training programs other than the Youth Potential Activity that were available to the participants in the selected Woredas. This is important insofar as it could compromise the impact of the interventions and causality of changes in the indicators after one year of enrollment. Recommendations This is a baseline study designed to provide a point of comparison for change resulting from the effects of participation in the Youth Potential Activity. However, during course of data collection a few points of note/recommendations for the implementing team are as follows. (i)The evaluation team noted that there is a need for strengthening the relationship of the Youth Potential team with the Ministry for Youth and Sports and to make the concerned officials more aware of the Youth Potential activities and initiatives. This will help in greater awareness of Youth Potential and provide Youth Potential participants with greater access to programs and resources managed by the Ministry. (ii) Participants have great expectations from the training program. Based on the FGDs, it appears that these expectations may be beyond the scope of Youth Potential program. There is a need for Youth Potential to discuss explicitly and upfront what the Activity can and cannot do. This will help set more realistic expectations on the part of the participants and reduce possible disappointment. (iii) As stated above, 71 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report the curriculum for Youth Potential does not address the issues of health or literacy and numeracy among the participants. There is a need for a discussion on the role of the Activity and its impact on the issues of health, literacy and numeracy on the participants between the implementers and USAID/Ethiopia. (iv) There is a need for further standardization and transparency in the selection process. While the selection criteria are very clear, the process itself could be improved upon. (v)There is a time lag between the start of the training and the report of the number of participants. In addition, there is a time lag between the participants dropping out of the training program and reporting by the field staff making it a problem for follow-up. In theory, the Youth Potential staff use electronic means for recording data, in practice this is not the case. Perhaps there is a need for better training on the use of electronic recording devices the Youth Potential field staff. (vi) finally, there is a concern regarding the increasing caseload of the facilitators. In order to ensure quality training and follow-up by the facilitators, SC and its partners should make it a priority to consider this and maintain a manageable caseload. 72 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report ANNEX IV: YCS MIDLINE STUDY – SUMMARY United States Agency for International Development/Ethiopia (USAID/Ethiopia) has contracted with SI to implement the Ethiopia Performance Monitoring and Evaluation (EPMES) Activity. Under EPMES, USAID requested an independent performance evaluation of its Building the Potential of Youth (Youth Potential) Activity. This evaluation involves multiple approaches, including a baseline study, performance monitoring and data systems review, and a multi-round cohort study of beneficiaries focused on a subset of youth participants and geographic implementation areas (henceforth referred to as the Youth Cohort Study). This report provides the midline data for the Youth Cohort Study (YCS). The Youth Potential Activity is a five-year, USD $17.3 million USAID/Ethiopia Activity designed to provide youth (ages 15–29), especially girls and young women, in rural areas and towns with access to workforce development/livelihood support and resources, and tailor to their specific needs and market demand, so that they can: (a) achieve increased income; and (b) strengthen skills, knowledge, and social capital required to achieve long-term economic self-sufficiency. Designed to reach approximately 34,537 unemployed or underemployed rural youth, the Activity is being implemented under a cooperative agreement between USAID and a group headed by Save the Children (SC) .43 Currently, in its fourth year of implementation, the Activity is providing services in 30 Woredas in Afar, Amhara, Oromia, Southern Nations, Nationalities, and Peoples' Region (SNNPR), Somali, and Tigray regions of Ethiopia. The overall objective of the YCS is to evaluate whether youth who receive workforce development/livelihood support achieve increased income and strengthened skills, knowledge, and social capital required to achieve economic self-sufficiency over the longer term. The study will provide the stakeholders, USAID, the Government of Ethiopia (GoE), and the implementers of Youth Potential with information about the Activity and progress towards near-term outputs and outcomes. In addition, it will demonstrate long-term contributions of Youth Potential initiatives on a cohort of youth and highlight major strengths and weaknesses of the Youth Potential’s initiatives to enable the stakeholders to make mid-course adjustments as necessary to the Activity as well as inform the design of similar programs in the future. Approach The YCS is following a subset of youth from Afar, Amhara, Oromia, and Tigray regions over a period of 12 months to better understand participant outcomes and the extent to which the Youth Potential Activity has improved the employment status and/or income level of youth participating in the Activity. It seeks to address the following broad questions44: 43 Cooperative Agreement # AID-663-A-15-00006; the implementation team includes Save the Children (SC) and its partners Education Development Center, Inc. (EDC), HUNDEE-Oromo Grassroots Development Initiative (HUNDEE), Professional Alliance for Development in Ethiopia (PADet), Relief Society of Tigray (REST) and Facilitator for Change Ethiopia (FCE). 44 From the EPMES statement of work 73 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report 1. To what extent has the income and employment status of study participants changed between enrollment in the Youth Potential Activity and the twelve months following enrollment? Does increased income lead to improved health outcomes among study participants? 2. To what extent has participation in the Youth Potential Activity improved the work-readiness skills of study participants? 3. To what extent has participation in the Youth Potential Activity improved knowledge of and access to employment/entrepreneurial resources for study participants? 4. To what extent has participation in the Youth Potential Activity improved functional literacy and numeracy skills of study participants? 5. How do the outcomes addressed in the questions above vary by participant characteristics (e.g., sex, age, ethnic/linguistic group, socio-economic status (SES), marital status, parenting status) and by service delivery partner? This midline study provides information on the status of each of the above among the study participants at six months’ post enrollment in the Youth Potential Activity. These changes will be compared to similar data collected at baseline to assess interim change and establish impact. The focus of this study is to track changes on Activity beneficiaries included in the cohort and to document lessons learned for future similar Activity designs. Since Youth Potential covers a selection of eligible youth in specific locations/Kebeles within the Woreda and is implemented over an extended period, the study employs a panel design to demonstrate whether the Youth Potential Activity led to expected changes in outcomes or impacts. In a panel design, participants from whom data are collected over time serve as their own controls, since characteristics of an individual observed earlier can be controlled for when analyzing for changes in the expected outcomes. Midline data collected on a select group of beneficiaries will help assess the effectiveness of the Activity as well as the interim outputs and outcomes that define the success of the Activity. To respond to the research questions listed above, data are being collected on several qualitative and quantitative indicators and sub-indicators that together will inform the results of the training programs as well as future possible modifications to the Activity. To capture changes over the short and medium terms and attribute them to the Youth Potential Activity, data are being collected from a select group of participants (cohort) who have recently enrolled in the Activity at three points: (i) baseline, in the month of enrollment in the Activity (completed March-June); (ii) mid-line, at six months from the point of enrollment (the current phase); and (iii) endline, at twelve months’ post-enrollment in the Youth Potential Activity. This will be accompanied with periodic (monthly financial diary interviews) collection of income and expenditure data for a small purposive sub-set of the cohort. The midline study was conducted during the period September to December 2017 using qualitative and quantitative methods. Primary data were collected from a survey of 582 Youth Potential participants from 13 Woredas in the four regions of Afar, Amhara, Oromia, and Tigray who completed the baseline survey. 45 Of those surveyed, 533 participants responded, representing a response rate of 92%.46 In 45 The baseline study was conducted during the period March to June 2017 using a mixture of qualitative and quantitative methods. Primary data were collected from a survey of A survey of 670 Youth Potential participants from 13 Woredas in the four regions of Afar, Amhara, Oromia, and Tigray yielded primary data on 582 participants (representing a response rate of 87%). 74 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report parallel, qualitative data were collected by the study team through 26 focus groups with 239 participants (50% female), 9 focus groups conducted with 50 parents and guardians of participants conducted in a purposive sub-sample of eight Woredas in the four regions (two Woredas in each region), and 48 KIIs and informational meetings held with stakeholders, including Government counterparts; Technical, Vocational, and Educational Training (TVET) and Technical Training Institutions (TTI) administrators and Activity staff at the national, regional, Woreda, and Kebele levels. The Activity staff interviewed included the M&E officers, the Woreda officers, and the facilitators responsible for delivery of the training as well as coaching and arranging for the mentoring. In collaboration with the SC team, secondary data was collected through a review of Activity documents, websites, and administrative records. Data from the midline study were analyzed together with the income and expenditure data collected periodically from approximately 80 members (20 per region) of the cohort. Main Findings As stated earlier, data were collected from 92% of the respondents; sixty-seven percent of the cohort is between 19 and 24 years of age, followed by 20% that is over 25 and 13% that is between 15 and 18 years of age. Within age groups, while the proportion of male and female participants is about the same in the over 25 years age group, there are more male participants than female in the 19-24 years age group (71% and 62%, respectively), and more females than males in the 15-18 age group (23% female, 6% male). Most (82%) of the respondents have completed at least grade five in school, with 57% having completed secondary school and beyond. Figure 1 presents these details. Only four cases across the four regions reported having a disability. Over half of the respondents (52%) reported that they were not married, and a similar proportion reported living with their parents. The distribution of respondents by sex and age group across regions for the midline study is consistent with the baseline. Figure 1 , Demographic characteristics of participants by region (%) 46 Eight percent of the respondents were either not available as they had moved out of the region for purposes of work (6%), due to marriage or for personal reasons (less than 1%), or chose to not respond (1.5%) 75 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Source of data: YCS Endline Survey Training Received Findings from the midline qualitative and quantitative data collection show that at a minimum, all youth from the March/April cohort completed Work Ready Now (WRN!) or Work Ready Plus (WR Plus) training components offered by the Youth Potential Activity. While all participants in Amhara, Oromia and Tigray attended the WRN! training, in Afar 56% of the participants attended WRN! and the remaining 44% attended the WR Plus training. Depending on the availability of the trainings, participants have also attended other follow-on trainings, such as Be your Own Boss (BYOB) and Positive Youth Development (PYD). A small proportion of the participants have also received training in technical skills provided by external providers under the Youth Potential Activity, (e.g., TVET, TTI or private providers). The most popular areas for training were agriculture and sheep fattening in Afar, Amhara and Oromia regions, followed by poultry farming and trading in Amhara and Oromia. In Tigray, however, fewer participants benefitted from this program and, for those that participated in it, the areas of choice were skills for hotel and tourism industry, and hairdressing. Some of the other options selected by participants included baking and running a bakery; carpentry, carving, or woodwork; mechanics and metal work; construction work (brick/stone laying, building, roads, etc.); running coffee and tea shops; and vegetable farming. A majority of those attending BYOB (81%) and technical training had completed grade 9 or higher levels of education. While training in agriculture and animal husbandry were popular with participants across all levels of education, there was a slight preference for those with lower levels of education to opt for training in skills such as tailoring, while those with higher levels of education selected training in bakery, marketing, tourism industry and hairdressing. To support the participants in the use of their training, they are provided with coaching from the facilitators and mentoring by experts in the field. While the mentoring program is just starting in all four of the regions under consideration and there have only been a few instances of participants engaging in job shadowing activities, the coaching is provided in individual or group settings on a regular basis. The coaching provides the participants with a support group as well as a person they can talk to about their issues and concerns. These sessions also serve the important purpose of holding the participants accountable for the implementation of plans they have developed, thereby helping them to become more organized, more productive, as well as to save more money. The Memorandums of Understanding 76 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report (MOU) between Youth Potential Activity and regional employers were also beginning to result in opportunities for the Youth Potential participants, increasing their work readiness. Employment Status The methodology used in this study allows the evaluation team to follow the Youth Potential participants who responded to the baseline survey through the midline and all the way to the endline survey. Prior to participating in the Youth Potential Activity, almost half of the participants were unemployed and dependent on others for their sustenance. Six months after participation in the WRN! training offered by Youth Potential, of the 533 Youth Potential beneficiaries that completed the survey, almost three quarters of the participants (74%) are employed, including 230 that remained employed from baseline and 165 newly employed in the six months since baseline. While there was some movement between those employed at baseline losing their jobs at midline (n=28), for the most part, the movement was in the direction of the unemployed- beneficiaries reporting they were now employed. Two thirds of all survey respondents were self-employed and only 9% were engaged in wage employment. The most significant gains in employment were seen in Afar and Tigray regions, with most of the survey respondents (90%) from the Tigray region reporting they were employed. While there were regional differences, more male (61%) than female (38%) survey respondents reported being employed, more males (52%) reported managing self-owned businesses compared to females (44%), and fewer males were unemployed as compared with females. More of those engaged in wage employment tended to have completed secondary school and beyond, with the self-employed equally spread across all levels of education. Those engaged in wage employment were working as security guards, Bajaj drivers, TVET assistants, schoolteachers, etc. For the self-employed, agriculture (growing crops) and livestock (sheep and goat rearing and fattening) remain the primary sources of employment, followed by trading. It is interesting to note that while 23% of the Activity participants were unemployed overall, an examination of the distribution by level of education revealed more of the college educated (45%) were unemployed. Survey respondents, especially those from Afar and Tigray, were mostly satisfied with their employment status. The training has helped bring about a change in attitude towards work among the beneficiaries of the Youth Potential Activity, with many preferring self-employment over wage employment. The training has helped them to value and respect their own work “no matter how lowly,” and to improve their work ethic. They now recognize that any work that helped them earn an income was employment. The community mapping exercise conducted as a part of the WRN! training (including WR Plus) helped the beneficiaries identify new business ideas that they could implement in their own communities. In addition, WRN! provided them with the skills needed to manage a business and handle customers. One recurrent theme heard in all focus groups across the four regions was that the training had made them realize the importance of “starting with something small, and to grow their Activity/business gradually,” resulting in many of them feeling encouraged to start their own business or to work more seriously on their existing work. Participants in the focus groups reported mixed feelings regarding work in the agriculture sector. Many of them reported that agriculture was a good opportunity and a good source of income, that “it’s the backbone of the Ethiopian economy” and that it gave them the opportunity to stay in their rural community. However, others felt that the work was too hard and challenging and that the constraints such as expensive inputs and frequent droughts made the work unattractive. Lack of access to finance, inadequate education and training, not enough jobs, and a mismatch between training and job 77 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report requirements were stated as the main reasons for unemployment. For those engaged in self￾employment, both male and female participants reported several hurdles in setting up their own business, including difficulty in obtaining working capital and financing, lack of land and working space. Figure 2 Current employment status of youth cohort study respondents at midline, by region Source of data: YCS Endline Survey Income Level and Expenditures Incurred Almost two thirds of the survey respondents (61%) reported an increase in income over the past six months, with the largest changes reported by participants in Tigray (Figure 3). The only exception to this were the participants with a college education, where only 45% of the participants reported their income had increased, with the majority reporting there was no change. More men than women reported that their income had increased over the past six months, a time line consistent with enrollment in the Youth Potential Activity and WRN! training. Even though most beneficiaries had increased income, the increase did not necessarily result in a change of life style. They said that because of the additional income they may spend a little more on entertainment or clothes as compared with six months ago, but in most cases, the additional funds went towards investment in business, payment of rent, purchase of food, support to other family members and putting money away for savings. As a majority of the participants are self-employed and not engaged in regular wage-based employment (with a salary), it is difficult to gather and verify income data. As such, data on expenditures incurred, investment in assets and the amount put away as savings was used as a proxy for change in level of income. A comparison of the expenditures incurred at midline with the baseline suggests that the beneficiaries are spending more overall and significantly more on investing in own business, supporting family members and education. The mean overall expenditure at midline (US$228 USD/5227 Birr) was significantly higher (p<.001) as compared with the expenditures at baseline (US$154 USD/3525 Birr) (see Table 9).47 Significantly more participants at midline reported purchasing mobile phones, motorcycles, land or animals as compared with baseline. Several also indicated that they were spending more on food and eating better. 47 Using exchange rate of $1= 22.9 Birr (the value used at baseline, to allow for meaningful comparisons) 78 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report As a result of the financial management skills learned through the WRN! program, the beneficiaries reported that they now spend money according to their plan and over half (56%) put away money in savings on a regular basis. The follow-on coaching as well as strategies such as participation in lottery￾based social groups have helped to reinforce the habit of saving money. An examination of the relationship between income and demographic characteristics and the employment status and the sector of employment suggest that male participants are almost 1.4 times more likely to report that their income had increased relative to female participants. Similarly, the odds of reporting higher income peak for those who are married (95%), have larger families (35%), and are in the 19-24 years’ age group. However, none of these by themselves are likely to make a significant difference. While an individual’s employment status (self-employed, wage employed, unemployed, etc.) did not have a significant effect on their level of income, an individual’s ability to obtain a loan is weakly related to their change in income. The regression analysis shows that the likelihood of receiving a higher income is more than double (2.19) when an individual is able to obtain loans. (see Annex 8, Table 3) Figure 3 Change in income over the past six months by region, sex and level of education (%) Source of data: YCS Endline Survey Change in Health Outcomes 79 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Health care is a discretionary expenditure and one that is easily overlooked, except in an emergency. Depending on the severity of the illness, most of the survey respondents reported visiting the local health clinic, a health provider or a nurse, and going to the hospital in the city in a case of severe illness or an emergency. A few also reported going to traditional healers or to “holy water.” Participants reported that after participation in the WRN! training, and because of the ongoing group and individual follow-on coaching, they had “learned to be more patient.” were “less quick to anger” and tried to put the customer’s perspective first. A clear majority of survey respondents (between 80% and 90%) agreed or strongly agreed that compared with six months ago, they were working harder, were generally happier, were less stressed, less anxious, less angry, and took better care of their health. A somewhat smaller proportion of the respondents agreed that they had fewer headaches (71%), ate better food, i.e., more vegetables and larger quantities (78%), or spent more money on food (61%). Several participants reported that they had decreased their consumption of alcohol, tobacco and “chat” as a result of the need to live within their budget as per the personal development plan developed because of the WRN! training, a finding corroborated by parents and guardians in a focus group. However, an equal number said that there was no change in their food habits, and that they were just saving more. Self-Perception of Work Readiness Skills Participants in focus groups indicated a strong preference for self-employment over salaried employment, especially with the government. Overall, they had a high opinion of their own readiness to work, rating themselves highly (good or very good) in terms of their communication skills, work habits and conduct, ability to build self-confidence, team work, ability to set goals and objectives, and saving habits. The respondents also rated themselves highly on skills needed for start-up of self-employment such as knowledge of financial resources, business start-up and management skills, obtaining licenses for a business, etc., rating themselves as “good” or “very good” on most items on a five-point scale ranging from very poor to very good. The only exceptions were the participants’ self-perceptions of job search skills, such as writing CVs, knowledge of job search resources and knowledge of ways and means of accessing resources (finance, land, inputs) on which the survey respondents did not rate themselves very highly (more of neutral and good ratings). Figure 4 presents the details. Figure 4 Self-ratings on measures of knowledge of resources (%) 80 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Source of data: YCS Endline Survey Participants reported that the training provided them with the skills, knowledge, and motivation to start their own business or improve their existing business, and the skills to improve their savings habits and better manage their time and finances. They went on to say that their personal development plans developed during the training had helped them articulate their goals and targets, and by following the plan, several participants have been able to expand the scope of their ongoing activities. In addition, the interpersonal skills and tools for anger and frustration management as well as the recognition of importance of time management have also helped the participants to grow. They reported having better time and financial management skills, and more confidence in their abilities to plan and manage their work. They were working longer hours and more days every week compared with six months ago, resulting in increased income and more productivity. Access to Employment/Entrepreneurial Resources Six months’ post enrollment in Youth Potential, while there has only been a 3 percentage points change (p<.05) in the number of those engaged in wage-based employment, the situation has changed dramatically for those unemployed and self-employed. The number of those unemployed has reduced significantly (change of 23 percentage points from 46% to 23%, p<.001). Forty four percent of all survey respondents (49% male and 36% female) reported having tried to find employment in the past six months. While this proportion is very similar to the baseline, the major difference is that over half of this group (53%) are currently self-employed compared to a quarter at baseline, and more of them are female. The most common avenues for looking for new opportunities were asking relatives and friends for help followed by participation in job fairs, registration with public employment offices and checking directly at worksites. Applying directly to employers was also common. Female participants reported being more likely to use avenues such as job fairs, seeking help from friends, and registration with the public employment office (in that order), while male participants preferred seeking the assistance of friends, followed by job fairs. Participants were also knowledgeable of the various requirements for self￾employment. Respondents across all regions, especially in Amhara, were aware of the need for accessing finance for creating self-employment as well as the minimum requirements for seeking a loan. The need to learn new skills ranked at par with the need to locate land, space and equipment, as well as seeking assistance of family, friends and colleagues. The facilitators supported the individuals and/or groups with completing the loan applications, getting the youth introductions to the Kebele and Woreda level government officials for letters of support, and in general, helped troubleshoot any process-related concerns. Overall, the Youth Potential participants feel more empowered in their search for employment and in creating self-employment, and report being able to get better and more complete information because of their training. Functional Literacy and Numeracy Skills While not a focus of the Youth Potential Activity, an expected benefit of enrolling in the Activity is improved levels of functional literacy and numeracy. At baseline, except for participants from Chifra in Afar region, most participants, male and female, reported being able to read and write fluently, and most 81 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report had more than just a functional level of literacy. These self-reports were confirmed using simple tests of literacy and numeracy. At six months’ post enrollment, most beneficiaries said that there was no change in their literacy and numeracy skills. Multiple young men and women across all four regions stated, “I knew how to read and write before joining the Potential training, and there has been no change in my ability.” At the same time, they also reported that the use of reading and writing skills during training and in the preparation of their personal development plans and business plans has reinforced these abilities. Participants also reported that managing their finances, developing budgets and monitoring their inputs and profits has also helped reinforce their ability to do simple addition and subtraction. They were also more aware of the need for literacy and numeracy and are making efforts to improve their own skills by making a conscious effort to read and write more, and were also looking to bring in someone with the necessary skills to help them with their accounts. Figure 5 contains the details by region. Figure 5 Level of functional literacy and numeracy by region (%) Source of data: YCS Endline Survey Next Steps This midline study will be followed by the endline study that will assess the changes on the outcomes of the participants at a point 12-months after enrollment in the Activity. Using the panel design (employed for both the baseline and midline), the study will use longitudinal data collected by following the cohort for a 12-month period. The key feature of such a panel design is that it will collect data on the same variables from the same sample at different points in time. While more complicated and difficult to carry out than simple cross-sectional studies, a panel offers a robust design to evaluate changes at the end of the study period and the contribution of the Youth Potential Activity to these changes. Challenges The team faced some challenges related to data collection as well as noted the challenges to the implementation and sustainability of the initiatives. 82 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Challenges related to data collection include: (i) non-availability of data from SC on participants in the March/April cohort included in the Youth Cohort Study, including data on participation in training components/programs, coaching and mentoring received, as well as any drop-outs from the training programs; (ii) difficulty in obtaining information on individual income and expenditures, since most participants live with their families; and (iii) difficulty in finding all survey respondents from the baseline study, despite best efforts. Solutions for these challenges were found using alternate measures, such as using data from administrative reports to replace enrollment data and data on trainings, using proxy data on ownership of items to determine income, building in the possibility of attrition into the sample size, and so on – but the challenges and limitations remain. Challenges related to implementation of the Youth Potential Activity include: (i) rapid staff turnover, which results in lack of continuity with post-training follow-up to support use of training; (ii) overburdening of facilitators with the provision of training and follow-up coaching, which affects the effectiveness of follow-up; (iii) difficulties faced by Youth Potential Activity with ensuring participants enrolled in technical training programs paid for by the Activity, complete the training programs and not drop out due to family commitments or lack of interest thereby wasting Activity resources ; (iv) managing partner and participant expectations regarding the training, which is important to ensure common understanding of the training; and (v) lack of wage employment and inability of participants to obtain land as well as funding/loans for self-employment, which could potentially defeat the purpose of the Activity. Recommendations This midline study is designed to provide an interim view of any changes from baseline to a point six months’ post participation in the Youth Potential Activity. However, during course of data collection a few recommendations that might result in better outcomes for the participants were noted. These include: (i) relax the savings and collateral requirements associated with obtaining microfinance loans for the Youth Potential participants to facilitate creation of self-employment; (ii) mobilize community resources (freeing up land; loan of equipment, farming tools, etc.) to help mitigate the paucity of land and available spaces for activities such as trading, opening a shop for provision of services, etc., and in the absence of availability of microfinance loans, mobilize alternative financing means such as village and savings loans, and borrowing from family and friends; (iii) increase involvement of parents and guardians in the training provided to the participants so that they are better informed and more willing to support their wards with collateral or, in the absence of external funding, the start-up capital for their wards; (iv) arrange for the provision of adequate coaching and mentoring from successful entrepreneurs in the community to all participants to better support the use of training and enhance the likelihood of success of the Youth Potential program; (v) take a closer look at the Tigray model of organizing Activity participants into groups and perhaps expand it to all regions to make it easier for participants to obtain start-up capital; (vi) explore new market opportunities across the agriculture value chains as well as the option of multiple sources of income (“mixed livelihoods”); (vii) take steps to reduce the level of implementing staff turnover; and (viii) establish linkages to ongoing government and donor funded programs to ensure use of all available resources. Several of these recommendations are systemic and may need to be addressed through joint efforts by all stakeholders, including the implementing partners, the government counterparts, the funding organization as well as the private sector. 83 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report ANNEX V: LIST OF RESEARCH QUESTIONS AND INDICATORS Research Questions – Detailed List Please note that the list below includes questions from the statement of work broken down into their subparts as well as additional questions posed by the evaluation team48. The questions added to the original list provided by the USAID/Ethiopia team are marked with an Asterisk (*). 1) *Description of the Activity (also responds to Question 6) a) *Who are the participants in the Activity? b) *What are the experiences/ training programs offered to the participants? c) *Are there any barriers to participation that are holding back either participants to these programs? (such as language, literacy-numeracy skills, SES, cultural barriers, safety concerns) d) *What proportion of the participants complete the training program? e) *While in the training program, how do the participants perform? f) *How is their performance monitored? g) *How are the participants tracked? h) *What proportion of the participants report using skills learned during the Youth Potential Activity? Questions from Statement of work: 2) To what extent has the income and employment status of study participants changed between enrollment in the Building the Potential of Youth Activity and twelve months following enrollment? Does increased income lead to improved health outcomes among study participants? a) Over the twelve months since joining the Activity, is there a change in the employment status of the participants? b) Type of job held? c) Is there a change in the income level of the participants? d) Is there increased spending on nutrition and preventive health care as a proxy for improved health outcomes (e.g., spending on fruits and vegetables, use of contraceptives, vaccinations for children, doctor visits for self or family) 3) To what extent has participation in the Building the Potential of Youth Activity improved the work readiness skills of study participants? a) Type of skills learned b) Use of skills learned 48 The additional questions posed by the evaluation team have been included to make the report more comprehensive and provide a context for the findings of the evaluation. They are not expected to extend the timeline or add to the cost of data collection. 84 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report 4) To what extent has participation in the Building the Potential of Youth Activity improved knowledge of and access to employment/entrepreneurial resources for study participants? a) Skills developed b) Resources accessed and used 5) To what extent has participation in the Building the Potential of Youth Activity improved functional literacy and numeracy skills of study participants? a) Level of education, any additional activities that provide literacy/numeracy training completed b) Self-reported proxy measures of literacy and numeracy skills, e.g., ability to read street signs, manuals, directions, simple math – addition, subtraction, etc. c) Performance on simple tests of functional literacy/numeracy as a proxy measure 6) How do the outcomes addressed in the questions above vary by participant characteristics (e.g., sex, age, ethnic/linguistic group, SES, marital status, parenting status) and by service delivery partner? a) Data from Q1 used with data on Q2-Q5 will provide data to be used to answer this question. Crosswalk of Study Questions to Indicators Research questions and sub questions Indicators49 Possible sources of data50 1) *Description of the Activity All questions in this section answered for (i) participants in Activity overall; and (ii) for participants in cohort study a) *Who are the participants in the Activity? - # of participants in Activity - # of participants in the cohort study - By gender, SES, Education, age group, marital status, ethnicity, Woreda and region - Review of Quarterly and Annual Activity reports; Youth Potential administrative records - For the cohort in the study, survey and focus groups b)*What are the experiences/ training activities offered to the participants? - List and type of activities offered - Curriculum for the Activity - #/percent of participants using coaching and mentoring - #/percent of participants using follow￾up activities 49 All data disaggregated by program, gender, Woreda, region, SES etc. See Q6. 50 Unless stated otherwise all data will be collected at Baseline, midline and endline. Only exception to this are focus groups with families of participants, these will be conducted only at midline and endline points. 85 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Research questions and sub questions Indicators49 Possible sources of data50 c) *Are there any barriers to participation in Youth potential Activity or education in general? (such as language, literacy-numeracy skills, SES, cultural barriers, safety concerns) - Barriers to participation reported by implementers - #/ percent of participants reporting problems/ barriers to participation by type of concerns (i.e. social, educational, cultural, political, economic) - Interviews with facilitators and Woreda/Kebele representatives d)*What proportion of the participants complete the training Activity? - #/ percent of participants who complete activities by type of Activity, Woreda and region - # percent of participants who do not complete program by type of program, Woreda and region - •# of participants using mentoring /coaching support services by Woreda and region e)*While in the training program, how do the participants perform academically? f) *How is their performance monitored? g) *How are the participants tracked? 2) To what extent has the income and employment status of study participants changed between enrollment in the Building the Potential of Youth Activity and twelve months following enrollment? Does increased income lead to improved health outcomes among study participants? a) Over the twelve months since joining the Activity, is there a change in the employment status of the participants? • Number (and percentage) of youth reporting new employment, new self￾employment, group employment or better employment as a result of participation in program; • Number (and percentage) of youth reporting internships, job-shadowing, or other paid and unpaid practical experience • Number of days of employment •Survey of Activity Beneficiaries • Focus groups with participants • Interviews with facilitators and Woreda/Kebele representatives • Focus groups with families b)Type of employment c)Is there a change in the income level of the participants • Number (and percentage) of youth beneficiaries reporting increased income; • Number (and percentage) of youth beneficiaries reporting increased assets; • Number (and percentage) of youth beneficiaries reporting increased financial contribution to larger family; d)Is there increased spending on nutrition and preventive health care as a proxy for improved health outcomes • Number (and percentage) of youth beneficiaries with increased spending on food – fruits and vegetables, health care for self or family, contraception, vaccinations for children 3) To what extent has participation in the Building the Potential of Youth Activity improved the work readiness skills of study participants? 86 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Research questions and sub questions Indicators49 Possible sources of data50 a) Type of skills learned b) Use of skills learned • Number of youth beneficiaries reporting improved skills; • Number of youth beneficiaries reporting use of skills; • List of skills learned • Number reporting improved literacy and numeracy skills as a result of participation in the Activity •Survey of Activity Beneficiaries •Focus groups with participants •Interviews with facilitators and Woreda/Kebele representatives 4) To what extent has participation in the Building the Potential of Youth Activity improved knowledge of and access to employment/entrepreneurial resources for study participants? a) Type of resources accessed b) Skills developed c) Resources accessed and used • Number of youth beneficiaries reporting access to improved knowledge of and access to employment resources; • Number of youth beneficiaries reporting access to improved knowledge of and access to resources for entrepreneurial work; •Survey of Activity Beneficiaries •Focus groups with participants •Interviews with facilitators and Woreda/Kebele representatives 5) To what extent has participation in the Building the Potential of Youth Activity improved functional literacy and numeracy skills of study participants? a) Level of education b) Self-reported proxy measures of literacy and numeracy skills, e.g., ability to read street signs, manuals, directions, simple math – addition, subtraction, etc. ● Number of persons with improved literacy skills as a result of POTENTIAL Activity; ● Number of persons with improved numeracy skills as a result of POTENTIAL Activity •Survey of Activity Beneficiaries •Focus groups with participants •Interviews with facilitators and Woreda/Kebele representatives 6) How do the outcomes addressed in the questions above vary by participant characteristics (e.g., sex, age, ethnic/linguistic group, SES, marital status, parenting status) and by service delivery partner? a) Data from Q1 on YCS sample will provide data to be used to answer this question. ● Gender (Male and Female); ● Age groups (15-19, 20-24, or 25-29); ● Ethnic/linguistic group (“Oromo”, “Amhara”, “Tigray”, or “Somali”); ● SES (Low level, middle level & high level) and ● Marital Status (single, Married or divorced) ...etc. •Survey of Activity Beneficiaries •Focus groups with participants •interviews with facilitators and Woreda/Kebele representatives 87 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report ANNEX VI: SOURCES OF DATA Both primary and secondary data were collected to inform the study and provide baseline data on the current status of program participants. Primary data collected from: ● Surveys with a sample of March/April 2017 cohort of participants of the Youth Potential Activity; ● KIIs with (list of people consulted is in Annex 5): (i) Ministry and Bureau officials at the regional, zonal and Woreda levels; (ii) Representatives of the selection committee from the Kebele, implementing partners, facilitators for Youth Potential Activity, and other program staff; and (iii) Other stakeholders such as TTI/TVET staff, representatives of the private sector/employers for the participants, Microfinance institutions. ● Focus groups conducted with a sample of eligible beneficiaries of the Youth Potential Activity and their parents and guardians Secondary data collected from: i. Documents reviewed: a. Youth Potential documents and administrative records: i. Program Summary ii. Activity original statement of work and any amendments iii. Monitoring and evaluation plan/PMP iv. Workplan for Year 3 v. Quarterly and Annual Reports Years 1, 2 and 3 vi. Quarterly Report Year 4 Q1 vii. MOUs between SC and three organizations viii. List of Youth Potential partners ix. Youth Potential data collection tools x. Youth profile from 12 Woredas of interest for YCS b. Program training materials: i. BYOB Training manual and facilitators guide ii. YiA resource manual and facilitators guide; and materials for module - myself iii. WRN Ethiopia trainer’s manuals and participant handbooks – modules M0 to M4 iv. Mentor’s guide c. Background articles and other relevant literature i. DHS Ethiopia 2011 - FR255 ii. Ethiopia_2014_EGRA_report_RTI_ (00000002) iii. LSMS Ethiopia ESS3 Community Questionnaire English iv. Other reports such as Program description on USAID website, The World Bank reports on current situation in Ethiopia, etc. cited in the body of the report 88 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report v. World Bank. 2017. Ethiopia - Country partnership framework for the period FY18 - FY22 (English). Washington, D.C. http://documents.worldbank.org/curated/en/202771504883944180/Ethiopia￾Country-partnership-framework-for-the-period-FY18-FY22 vi. UNESCO UIS Data for the Sustainable Development Goals. http://www.uis.unesco.org/Education/Documents/literacy-statistics-trends-1985- 2015.pdf accessed June 2018 vii. African Development Bank. 2004. Ethiopia – Country gender Profile - https://www.afdb.org/fileadmin/uploads/afdb/ Documents/Project-and￾Operations/ADBGenderEthiopia.pdf viii. Rubin, Deborah, Cristina Manfre and Kara Nichols Barrett. Promoting Gender Equitable Opportunities in Agricultural Value Chains. Washington, DC: U.S. Agency for International Development, 2009. In A Sample Gender Analysis by Catholic Relief Services – Abridged Version accessed at https://www.crs.org/sites/default/files/tools-research/a-sample-gender￾analysis.pdf 89 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report ANNEX VII: DATA COLLECTION DETAILS Distribution of Survey Respondents Survey respondents Baseline Midline Endline Region Male Female Total Male Female Total Male Female Total Afar 60 48 108 52 43 95 47 40 87 Amhara 107 93 200 93 79 172 72 61 133 Oromia 113 65 178 108 63 171 98 57 155 Tigray 52 44 96 51 44 95 44 39 83 Total 332 250 582 304 229 533 261 197 458 Focus Groups by category and region Baseline* Midline Endline #of Beneficia ry FGD # of FGD Participa nts #of Beneficia ry FGD # of FGD Participa nts # of Parent FGDs # of Parents #of Beneficia ry FGD # of FGD Participa nts # of Parent FGDs # of Parents Afar 8 76 8 80 2 9 4 36 2 4 Amhara 6 61 6 40 2 8 10 65 5 33 Oromia 10 97 8 84 3 20 7 74 5 45 Tigray 4 22 4 35 2 13 2 20 2 18 Total 28 256 26 239 9 50 23 195 14 100 *FGDs with parents and guardians were conducted only at midline and endline 90 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report KIIs conducted by category and region Baseline Midline* Endline Category of Key Informants** Afar Amhara Oromia Tigray Total Afar Amhara Oromia Tigray Total Addis Afar Amhara Oromia Tigray Total Total Potential Staff 3 9 6 2 4 24 3 5 8 5 4 22 5 7 8 6 4 30 76 TVET/TTI/Pol ytechnic 0 1 2 1 2 6 0 2 3 3 2 10 0 2 3 2 3 10 26 Microfinance 0 0 0 0 0 0 0 0 1 3 2 6 0 0 1 2 2 5 11 Employers 0 1 0 0 0 1 0 2 1 0 2 5 0 2 1 2 1 6 12 Gov’t Stakeholders* 0 7 5 5 6 23 0 1 1 2 0 5 4 6 6 21 46 Total 3 18 13 8 12 54 0 10 13 11 11 45 5 16 17 18 16 72 171 *Interviews with government stakeholders were conducted at midline only if there was a change in the official or if a new agency was included ***Many of the same individuals were interviewed at baseline, midline and endline – hence while 171 KIIs were conducted, the number of individuals interviewed is less Source of data: YCS Endline qualitative data List of People Consulted at endline Note: As many of the same people were interviewed at baseline, midline and endline, only the list from endline is presented here. Region Woreda Kebele Name Position Organization/ Office Implementer Contact Information Addis Ababa N/A N/A Olaf Erz, Cop Youth Potential Project/Sci SCI olaf.erz@saveth echildren.org Addis Ababa N/A N/A Dr. Dereje Tadese Regional Manager HUNDEE HUNDEE 913105582 Addis Ababa N/A N/A Wondwosen Wogayehu Regional MEAL Manager HUNDEE HUNDEE 913139996 Addis Ababa N/A N/A Habtamu Sewnet Program Manager PADet PADet 911759564 Addis Ababa N/A N/A Amanuel Mekonen Executive Director PADet PADet 911115095 Afar Semera Semera Afika Ali Teachers Dev't Program Head Region Education Office SCI +251- 974108129 Afar Assayita Assayita Fatuma Siraj Youth Facilitator SCI SCI 935444098 Addis Ababa Ababa 91 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Region Woreda Kebele Name Position Organization/ Office Implementer Contact Information Afar Assayita Assayita Zufan Belayneh Woreda Officer SCI SCI 912195690 Afar Chifera Chifera Hiwot Endrias Potential Woreda Officer SCI SCI +251- 910781118 Afar Semera N/A Osman Taha Owner Afar Technology And Science College N/A 921548024 Afar Semera N/A Mohamed Ahmed Director Community Svc. Directorate University Of Semera N/A 913282832 Afar Semera N/A Mohamed Ambesa Director Labour And Social Affairs Office N/A +251911915808 Afar Semera N/A Hamid Baw Senior Officer Labour And Social Affairs Office N/A +251911915808 Afar Semera N/A Abdu Hassan Training Coordinator Afar TVET N/A 910717984 Afar Semera N/A Mahar Ali Commissioner Afar Youth And Sport Office N/A 911897210 Afar Semera N/A Omer Mohamed Officer Afar Youth And Sport Office N/A 912960546 Afar Chifera N/A Fetene Damte Youth Facilitator SCI SCI 913453758 Afar Chifera N/A Osman Mohamed Youth Facilitator SCI SCI 911088093 Afar Semera N/A Dawud Mohammed Potential Region Manager SCI SCI +251- 931203011 Afar Semera N/A Seid Mohamed Adadale Poly Technique College Adadale Polytechnic College N/A 912815854 Afar Assayita N/A Ali Habib Youth Focal Person Youth And Sport Office N/A 920638796 Afar chifra N/A Awol Endris Woreda Officer Trade And Industry Office N/A 922894150 Afar chifra N/A Mohamed Hassen Woreda Youth And Sport Process Head Youth And Sport Office N/A 913821871 Amhara Bahir Dar Bahir Dar Feleke Wube Ploy Technic College Dean Bahir Dar Ploy Technic College PADet +251- 918012408 Amhara Bahir Dar Bahir Dar Dagarege Yazachew Region Potential Coordinator PADet PADet dogiyazachew@ gmail.com;09181 92 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Region Woreda Kebele Name Position Organization/ Office Implementer Contact Information 64818 Amhara Bahir Dar Bahir Dar Getachew Habtom Woreda Officer PADet PADet 918081360 Amhara Bahir Dar Bahir Dar Gashaw Yalew Employer GYM N/A 918715364 Amhara Jawi Jawi Genetu Andarge Youth Facilitator PADet PADet 926750945 Amhara Jawi Jawi Asheshim Wasim Youth Team Leader Youth And Sport Office N/A 921573912 Amhara D/achefer D/achefer Negalegn Tegegne Team Leader SME N/A 918819622 Amhara D/achefer D/achefer Mekunant Arega Branch Manager ACSI N/A 918126015 Amhara D/achefer D/achefer Tesfaye Gashaye Project Officer PADet PADet 940199032 Amhara D/achefer D/achefer Esaynesh Getu Youth Facilitator PADet PADet 918562925 Amhara Jawi Jawi Abiye Dejene Office Head Technical And Vocational Enterprise Office N/A 913296016 Amhara Jawi Jawi Amlaku Atalele Youth Facilitator PADet PADet 921282968 Amhara Jawi Jawi Minyichil Tarekegn Youth Facilitator PADet PADet 922556345 Amhara Jawi Jawi Zelalem Asmamaw Acting Woreda Head Women &Children Affairs Office N/A Amhara Jawi Jawi_01 Gizachew Mulu Office Head Jawi Microfinance PADet 929465920 Amhara South Achefer - Negese Zelalem Woreda Officer PADet PADet 913288030 Amhara South Achefer - Dires Ayana Deputy DEA Education And Training Durbete TVET PADet 918010456 Oromia Wolmera Holeta Zewde Beysa Youth Expert Woreda Youth And Sport Office HUNDEE 913938173 Oromia Wolmera Holeta Abdi Lemesa Planning Officer Woreda Youth And Sport Office HUNDEE 915961443 Oromia Wolmera Holeta Legesse Dinbesa Team Leader Woreda Youth And Sport Office HUNDEE 913033864 Oromia Wolmera Holeta Getachew Jewu Youth Capacity Building Officer Woreda Youth And Sport Office HUNDEE 912059807 Oromia Wolmera Holeta Kuma Adesa Expert Women And Children Affairs Office HUNDEE 913841372 93 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Region Woreda Kebele Name Position Organization/ Office Implementer Contact Information Oromia Wolmera Holeta Meskerem Anbesa Deputy Head Women And Children Affairs Office HUNDEE 910999020 Oromia Wolmera Holeta Wokense Eliyas Expert Women And Children Affairs Office HUNDEE 917073156 Oromia Wolmera Holeta Kumsa Gonfere Core Owner Processor Women And Children Affairs Office HUNDEE 918758101 Oromia Wolmera Holeta Kidist Mesfine Head Of The Office Women And Children Affairs Office HUNDEE 911353198 Oromia Wolmera Holeta Solomon Zleke Employer Microfinance Institutions HUNDEE 911560833 Oromia Wolmera Holeta Gelan Eficha Youth Facilitator HUNDEE HUNDEE Hika Oromia Wolmera Holeta Abdisa Negare Junior Expert HUNDEE HUNDEE 910855454 Oromia Wolmera Holeta Ebisa Beyene Woreda Officer HUNDEE HUNDEE 920406650 Oromia Wolmera Holeta Mergia Negasa Vice Dean Holleta Poly Tech College HUNDEE 912121944 Oromia Wolmera Holeta Adugna Bedada Senior Expert Urban Job Creation And Food Security HUNDEE 947527213 Oromia Yabelo Yabelo Jarso Borruw Deputy Office Head Youth And Sport Office HUNDEE 932188699 Oromia Yabelo Yabelo Lalisa Gurumu Woreda Officer HUNDEE HUNDEE 947763711 Oromia Yabelo Yabelo Deyna Oshe Branch Manager Microfinance Institutions HUNDEE 932720063 Oromia Yabelo Yabelo Zeyitu Kemal Office Head Women And Children Affairs Office HUNDEE 932577313 Oromia Yabelo Yabelo Chala Wario Employee HUNDEE HUNDEE 925593151 Oromia Yabelo Yabelo Dida Dubie Employee Water Resource And Irrigation Development HUNDEE 905004824 Oromia Yabelo Yabelo Chernet Bekele Vice Dean TVET HUNDEE 912750252 Oromia Yabelo Yabelo Mohamed Haro Expert Labour & Social Affairs HUNDEE 966822836 Oromia Yabelo Yabelo Abdinaser Shereka Officer Labour & Social Affairs HUNDEE 916459840 Oromia Yabelo Yabelo Kumye Kedir Planning & Budgeting Officer Labour & Social Affairs HUNDEE 912766796 94 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Region Woreda Kebele Name Position Organization/ Office Implementer Contact Information Oromia Yabelo Yabelo Kasech Beysa Team Leader Water And Irrigation Development HUNDEE 910140811 Tigray Bureau Of Agriculture And Natural Resources Tigray Technical Vocational Training Center – Tilahun Yigzaw Tigray Maichew Polytechnic College Tigray Bureau Of Labour And Social Affairs – Tigray Region – Education And Training Expert Tigray Office Of Labour And Social Affairs – Endamahoni Woreda Tigray Youth Facilitator – Mekele Tigray Agriculture And Natural Resources – Rural Job Opportunity Expert – Mekele 95 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report ANNEX VIII: DATA COLLECTION INSTRUMENTS Instruments included are: ● The survey instruments ● Guides for focus group with Youth Potential participants and their parents and guardians ● Guides for interviews and informational meeting with: o o implementing partners for the Youth Potential Activity, including facilitators and Woreda officers key stakeholders including officials from the Ministry and Bureaus involved with the program at the regional and Woreda levels 96 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Endline Survey Youth Cohort Survey Questionnaire Survey Information to be completed by Enumerator: Name of enumerator ________________________ Name of Supervisor: Interview date: __________________________ Starting time: _____________________ Location: Region: ____________________ Woreda: _________________ Kebele: ___________________ General Identification 1. ID Number_______________________________________ (fill in by SART) 2. Name of Participant (with grandfather’s name): __________________________________ 3. Participant’s Current address: Personal Tel. (mobile): ______________________, Residence: ___________________ E-mail (if available) or Facebook (if available): __________________________________ Region: ____________________ Woreda: _________________ Kebele: ___________________ 4. Contact information of close relative/neighbor/friends (in addition to contact address): Name: __________________________ ______________________________ Address፡ Region: ____________________ Woreda: Tel: _________________ Kebele: _______________ 5. Age of Participant: ________ 6. Sex: ☐ Male ☐ Female 7. Ethnicity and Language a. Ethnicity: 1 Afar 2 Amhara 3 Guragie 4 Nuwer 5 Oromo 6 Sidamo 7 Somali 8 Tigray 9 Welaita 10 Other _______ b. Language 1. Amarigna 2. Oromigna 3. Tigrigna 4. Afarigna 5. Other___________ 8. Disability: (check all that apply) ☐ ☐ ☐ ☐ ☐ ☐ No disability 1. Yes 2.No Deaf or hard of hearing 1. Yes 2.No Blind (one or both eyes) 1. Yes 2.No Physical handicap 1. Yes 2.No Cognitive disability (mental disability) 1. Yes 2.No Other 1. Yes 2.No 9. Current marital status of participant (selec one): 1. Never married 2. Married 3. Living together 4. Separated/Divorce 5. Widowed 10. Living status: 1. Living with parents 2. Living independently 11. Household Size: _____________ (people living in the same house permanently) 12. Number of Participant’s Children: ________ Total number (zero if none), ______ Girls ______ Boys 97 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Education status and Functional Literacy/Numeracy 13. Functional literacy (use a letter card, choose one response) 1. Can’t read & write 2. Can read & write with difficulty 3. Can read & write with some fluency 4. Can read & write very easily 14. Functional numeracy (use a number card) 1. Recognize numbers 1. Yes 2.No 2. Add simple numbers 1. Yes 2.No 3. Subtract simple numbers 1. Yes 2.No 4. Multiply simple numbers 1. Yes 2.No 5. Divide simple numbers 1. Yes 2.No 15. Education Status (select one response) 1. No school 2. Only traditional/non-formal school 3. Grade level or number of years in school ___________ (enter a number from 1- 12; includes grade completed or dropped-out) 4. Completed TVET (drop-outs captured under grade level) 5. Completed College/ University 16. Are you still in school? 1. Yes 2. No (skip to 18) 17. If yes, what are you studying? ____________________ (skip to 19) 18. If no, what was the main reason for stopping your education? (select one) 1. Finished course/ graduated 2. Failed examinations 3. Did not qualify for the entrance exam for TVET, College/University Did not enjoy schooling 4. Wanted to start working 5. To get married 6. Parents did not want you to continue schooling Economic reasons (could not afford/too poor/needed to earn money to support family) Other _______________________ 19. Education Status of mother (select one response) 1. No school Only traditional/non-formal school 2. Grade level or number of years in school ___________ (enter a number from 1- 12; includes grade completed or dropped-out) 3. Completed TVET (drop-outs captured under grade level) 4. Completed College/ University 5. Don’t Know 20. Education Status of father (select one response) 1. No school 2. Only traditional/non-formal school 3. Grade level or number of years in school ___________ (enter a number from 1- 12; includes grade completed or dropped-out) 4. Completed TVET (drop-outs captured under grade level) Completed College/ University Don’t Know 98 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Employment Status 21. What is your current employment (income earning) status? Primary job In school or training Paid/wage/paid-in kind employed (Skip to Q23) Self-employed Self-employed but looking for a job Unemployed (dependent on support from others) (Skip to Q 25) Unemployed but not seeking work for other reasons (example disabilities) (skip to 28) Other _____________ 22. If self-employed, what kind of self-employment are you engaged in: 1. Self-owned (skip to Q22C) 2. Family owned (skip to Q22d) 3. Group owned, a. If group owned, how many people are in the group? __________ (enter number) b. When did you join the group? 1. A year or more 2. Between six months to a year ago, 3. Less than six months ago c. When did you become self-employed? _________________ 1. A year or more ago, 2. Between six months to a year ago, 3. Less than six months ago, d. When did you join the family owned business or activity? ________________ 1. A year or more ago, 2. Between six months to a year ago, 3. Less than six months ago, 23. Which of the following sectors do you work in (for wage/salary employment or self-employment)? Note to interviewers - For female participants ask – Aside from your own housework, if you work for payment in-cash or kind which of the following sectors do you work in (for wage/salary employment or self-employment)? This is my main job (select one) This is my secondary job (select one) 1 Agricultural + Livestock activities 2 Trading agricultural products (buying or selling agricultural products) 3 Trading consumer products/items – small shop/kiosk 4 Carpentry, carving, or woodwork, 5. Mechanics, metal work 6 Construction work (brick/stone laying, building, etc.) 7 Daily labourer 8 Electrical work 9 Computer technology and administration 10 Tailoring (making clothes or textiles) 11 Transport service 12 Hairdressing/Barber 13 Providing services (such as coffee/tea, small restaurant, shoe shine, repair and maintenance, etc.) 14 Hotel industry 99 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report This is my main job (select one) This is my secondary job (select one) 15 Education 16 Work for family 17 Government 18 NGO 19 Others (specify) 20 Don’t have secondary job xxxxxxxxxx If response to q23=agriculture, as main job or secondary job ask 23a, else go to 23b 23a. Please tell us: if you are engaged in the following agricultural activities Auto furnish from midline data - engaged in Auto furnish from midline data - engaged in activity six months ago 23a.a. Engaged in activity now? [Endlline] 1=Yes 2=No (Skip to 23a.c.) 23a.b. Scope of activity compared to six months ago 1= expanded, 2=same, 3=decreased, 23a.c. Will you be engaged in activity six months from now? 1=Yes 2=No 23a.1 I am growing a single crop in a year 23a. 2 I am growing two or more crops in a year 23a. 3 I am fattening animals 23a. 4 I am rearing animals 23a. 5 I am doing poultry farming 23a. 6 I am doing dairy production 23b. Is the main job you mentioned in question 23 above self-employment or wage employment? 1= Self-employment individual /group 2=self-employment family 3= wage employment 23c Is the secondary job you mentioned in question 23 above self-employment or wage employment? 1= Self-employment individual/group 2=self-employment family 3= wage employment 24. To what extent are you satisfied with your current main job? 1. Satisfied 2. Mostly Satisfied 3. Neutral 4. Mostly Unsatisfied 5. Unsatisfied 25. Have you tried to find employment/new employment in the past six months? (For those unemployed ask the question as tried to find employment; for those employed ask the question as tried to find new employment) 1. Yes 2=No (skip to Q 29) 100 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report 26. If looking for a job ask the questions below 27. How long have you been available for work and actively looking for a job? (Answer should be dependent on both conditions) 1. Less than a week 2. 1-4 weeks 3. 1-2 months 4. 3-6 months 5. 6 months-1 year 6. More than 1 year 28. Which of the following sectors would you wish to work in (for wage/salary employment or self￾employment)? Note to interviewers - For female participants ask – Aside from your own housework, if you work for payment in-cash or kind which of the following sectors do you wish to work in (for wage/salary employment or self-employment)? This is my main job (select one) This is my secondary job (select one) 1 Agricultural + Livestock activities 2 Trading agricultural products (buying or selling agricultural products) 3 Trading consumer products/items – small shop/kiosk 4 Carpentry, carving, or woodwork, 5 Mechanics, metal work 6 Construction work (brick/stone laying, building, etc.) 7 Daily labourer 8 Electrical work 9 Computer technology and administration 10 Tailoring (making clothes or textiles) 11 Transport service 12 Hairdressing/Barber 13 Providing services (such as coffee/tea, small restaurant, shoe shine, repair and maintenance, etc.) 14 Hotel industry 15 Education 16 Work for family 17 Government 18 NGO 19 Others (specify) 28a. Would you want your main job to be self-employment or wage employment? 1= Self-employment individual/group 2= self-employment family 3= wage employment 28b. Would you want your secondary job to be self-employment or wage employment? 1= Self-employment individual/group 2 = self-employment family 3 = wage employment 4 =Do not want a secondary job 101 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report If unemployed ask Q29, else go to Q30፡ 29. If unemployed, in your opinion what are the main three reasons that you are not employed (wage/self)? (please put in the first three) Reason one Reason two Reason three 1 No education 2 Unsuitable vocational education 3 No suitable training opportunities 4 Mismatch between education requirements and that required by the position available 5 No work experience 6 Not enough jobs available 7 Considered too young 8 Discriminatory prejudices (for example, disability, religion, race, appearance, etc.) 9 Low wages in available jobs 10 Poor working conditions in available jobs 11 Don't have the right personal or family connections 12 Lack of access to finance 13 Lack of land for agriculture 14 Lack of workspace to start a business 15 No response 16 Other Specify: 30. Have you ever refused a job that was offered to you? 1. Yes 2. No (skip to 32) 31. Why did you refuse? (select the main reason) 1. Wages offered were too low 2. Work was not interesting 3. Location was not convenient 4. Work would not match my level of qualifications 5. Work would require too few hours 6. Work would require too many hours 7. Waiting for a better job offer 8. The contract length was too short 9. Saw no possibilities for advancement 10. Other 32. Have you tried to find employment/new employment in the past six months? 1. Yes 2. No (skip to Q32b) 102 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report 32a. How have you tried to find employment/new employment in the past six months? Probe and Check all that apply 1. Yes 2. No 32a.1 Through education/training institution 32a.2 By Attending job fairs 32a.3 Registration at a public employment office 32a.4 Registration at a private employment office 32a.5 Direct application for employers, participation in a competition 32a.6 Checking at worksites, farms, factory gates, markets or other assembly places 32a.7 Placing and answering newspaper advertisements 32a.8 Answering advertisements on the internet, etc. 32a.9 Seeking assistance of friends, relatives, colleagues, unions, etc. 32a.10 Other 32b. If you were to create self-employment what kinds of information would you need? Probe and Check all that apply 1. Yes 2. No 32b.1 I am not looking to create self-employment 32b.2 Seek assistance of friends, relatives, colleagues, unions, etc. 32b.3 Look for land, building, machinery, equipment to establish own enterprise 32b.4 Arrange for financial resources 32b.5 Apply for permits, licenses 32b.6 Learn new skills 32b.7 Other__________________________________________ Participation in Youth Potential Program 33. When did you enroll/start in the Youth Potential program? (EC. mm/year) (EC. mm/year) Month ___________________ Year ________________________ 34. What Youth Potential program activities have you attended to-date? (check all that apply) (Please read out this full question and then depending on which ones the respondent selects go to follow-on question below) 34.1 Work Ready Now (WRN) training (Please know this program is for people who can read) 1. Yes 2. No 34.2 Work Ready Plus (WRPlus) training (Please know this program is for people who cannot read) 1. Yes 2. No 34.3 Be Your Own Boss (BYOB) 1. Yes 2. No 34.4 Positive Youth Development/Youth in Action (YiA) training 1. Yes 2. No 103 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report 34.5 Technical training at TVET or Farmers Training Center or other training institution 1. Yes 2. No 34.6 Work Based Learning (apprenticeship, experience, exposure etc…) 1. Yes 2. No 34.7 Received coaching and advisory services 1. Yes 2. No 34.8 Linkage to source of finance, government/community support/inputs/advise, mentors, etc. 1. Yes 2. No 34.9 Mentoring from a successful business person/expert in your area of work 1. Yes 2. No 34.10. Youth Economic and Social group 1. Yes 2. No 34.11 Other, specify ______________________________ 1. Yes 2. No 34 a. Did you develop a PDP (Personal Development Plan)? 1. Yes 2. No (Skip to 34f) 34b. How long is the PDP for? 1. 3 months, 2. 6 months, 3. 12 months4. Don’t know 34c. Are you implementing the PDP? 1. Yes 2. No (Skip to 34f) 34d. How often do you review the PDP? 1. Never, (Skip to 34f 2. Eevery three months, 3. Every six months, 4. Once a year 34e.With whom do you review the PDP (multiple response is possible)? 1. facilitator/coach, 2. By myself, 3. With my parents/guardians 34f. If “Yes” in “received coaching and advisory services” at Q34.7, How often do you receive coaching from the facilitator? 1. At least once a month 2. Once in two months 3. Once in six months 104 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report 34f.a In your opinion is the coaching provided by Youth Potential useful 1. Yes 2. No (skip to Q34G) 34f.b. If yes what is it useful for (Check all that apply) 1. It helps me resolve my problems 1. Yes 2. No 2. It is an opportunity to learn from others 1. Yes 2. No 3. I receive guidance on how to find employment 1. Yes 2. No 4. My coach helped me write a cv 1. Yes 2. No 5. My coach helped me write an application letter 1. Yes 2. No 6. My coach helped me connect with a loan institution 1. Yes 2. No 7. My coach helped me to identify a self-employment activity 1. Yes 2. No 8. My coach helped me develop a business plan 1. Yes 2. No 9. My coach helped me form a business group 1. Yes 2. No 10. It is only a good social experience and a chance to meet friends 1. Yes 2. No 11. Other ____________________ 1. Yes 2. No 34G. If “Yes” at “Mentoring from a successful business person/expert in your area of work” at Q34.9 Tell us about the mentoring experience? (Check all that apply) I have learned 1. To do better in my business/work 1. Yes 2. No 2. New techniques for work 1. Yes 2. No 3. To better manage my accounts 1. Yes 2. No 4. New direction for my work/focus on a different area 1. Yes 2. No 5. Other specify____________________ 1. Yes 2. No 34H If “Yes” in “Work Based Learning” at Q34.6, What type of Work-Based Learning (WBL) support did you receive? (Check all that apply) 1. Job shadowing (observing someone doing the job) 1. Yes 2. No 2. On the job training 1. Yes 2. No 3. Apprenticeship (hands-on work at a place) 1. Yes 2. No 4. Internship 1. Yes 2. No 5. Other __________________ 1. Yes 2. No 34H.a Who decided what type of Work-Based Learning (WBL) support you should receive? (Multiple response is possible) 1. Facilitator/coach, 2. By myself, 3. With my parents/guardians 105 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report 34H.b For how long did you participate in the Work-Based Learning (WBL) activity(ies)? _ 1. less than a week 2. 1-2 weeks 3. Less than a month 4. 1 to 3 months, 3. 35. Are you currently enrolled in any other non-formal training program (like the Youth Potential program)? 1. Yes 2. No (skip to 35a) 35a. If yes, what training program are you attending? _____________________________ 35b. In the past six months you were enrolled in any training program paid for by the Youth Potential activity)? 1. Yes 2. No (skip to 36) 35c. If yes, what training program are you attending? Check all that apply 1. Yes 2. No 35b.1 Agriculture (farming, horticulture, irrigation) 35b.2 Animal fattening, animal husbandry 35b.3 Poultry farming 35b.4 Marketing 35b.5 Bakery 35b.6 Carpentry, carving, or woodwork, 35b.7 Mechanics, metal work 35b.8 Construction work (brick/stone laying, building, roads, etc.) 35b.9 Hotel and tourism 35b.10 Electrical work 35b.11 ICT, Computer technology and administration 35b.12 Tailoring (making clothes or textiles) 35b.13 Transport services, driving, 35b.14 Car/vehicle repair shop, tyre repair shop 35b.15 Hairdressing/Barber 35b.16 Providing services (such as coffee/tea, small restaurant, shoeshine, repair and maintenance, etc.) 35b.17 Other specify 36. Have you previously been enrolled in any non-formal training program? 1. Yes 2. No (skip to 37) If yes, what training program did you attend? ______________________ 37. Please rate yourself on the following skills (Work Readiness skills) 106 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Skills Very Good Good Neutral Poor Very Poor 37.1 Interpersonal communication 37.2 Work habits and conduct 37.3 Job search application skill, writing CV 37.4 Knowledge of resources of where to look for jobs 37.5 Business management/improvement skills 37.6 Skills to start-up a business 37.7 Knowledge of resources for obtaining loans, seed money, etc. 37.8 Access resources (finance, land, inputs) 37.9 Access markets 37.10 Knowledge of how to obtain license for business 37.11 Saving habits 37.12 Setting objectives/goals and preparing personal development plan 37.13 Improved social bond and team work 37.14 Build self-confidence 37.Oth Other, specify____________________________ 37a. Please tell us the degree to which you agree or disagree with the following statements As compared with a year ago Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree 37a.1 I feel more confident of my abilities 37a.2 I know how to develop a business plan 37a.3 I know how to calculate profit/loss 37a.4 I spend money according to a plan 37a.5 My business has expanded 37a.6 I have better marketing skills 37a.7 I am able to interact better with customers 37a.8 My business has more customers 37a.9 I work more days of the week 37a.10 I have better time management skills 37a.11 I am more patient 37a.12 I get into fewer fights and arguments 37a.13 I am more persistent and don’t give up as easily 37a.14 I feel more capable to solve problems when they arise 37a.15 I am more respectful of all types of jobs 37a.16 I am more willing to try different kinds of jobs 37a.17 I am more motivated to work 37b. in the past six months, have you applied for a loan 1= Yes 2= No (skip to 38) 37c. Did you receive the loan? 1= Yes 107 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report 2= No (skip to 37e ወደ 37f) 37d. Was it a group loan or individual loan 1=Group loan 2=Individual loan 37e. What was the amount? Birr___________________ (fill the amount and skip to question 38) 37f. Reason for not obtaining a loan Read the list to the respondent Check all that apply 1. Yes 2. No 37f.1 No collateral 37f.2 No agreement within group 37f.3 Did not meet minimum funds requirement 37f.4 I was applying for an individual loan but only group loans were available 37f.5 Forms were too difficult to complete Income and assets 38. What assets do you have at this time? Do you or your family own: Owned by you (check all that apply) 1. Yes 2. No Owned by family (check all that apply) 1. Yes 2. No 38a.1 House 38a.2 Radio/Tape recorder/ MP3 player 38a.3 Television 38a.4 Mobile Telephones 38a.5 Non-mobile telephone 38a.6 Own bicycle 38a.7 Animal-drawn cart 38a.8 Motor cycle/scooter 38a.9 Car/truck 38a.10 Land for farming, grazing, etc. 38a.11 Animals 38a.Oth1 Other 1_________________________________ _________________________________ _________________________________ 38a.Oth2 Other 2 38a.Oth3 Other 3 39. How much land do you own? a. _______________________hectare (enter value by converting from local measure to hectare, enter zero if the respondent has no land) 40. What type and how many animals do you have? Animal Number owned (please write number, zero for none) a. Milk cows, oxen or bulls? b. Goat c. Sheep 108 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report d. Camel e. Horses, donkeys, or mules? f. Poultry, chicken, hens, g. Bee hives h. Fish i. Other 1_______________ _____________ Write number and description j. Other 2 40a. Please tell us about your income level. Compared with six months ago, has your income level: 1. Increased 2. Decreased 3. Stayed the same 41. In the past six months, how have you used your income? Please estimate how much of your total income did you spend on each of these categories? Check all the categories that you spend money on 1. Yes 2. No If yes, ask how much was spent in the past six months 1. Yes 2. No Q41.1.1 Rent/lease Q41.1.2 Taxation Q41.1.3 Transportation Q41.1.4 Buy food and related goods Q41.1.5 Pay for electricity, water, Q41.1.6 Clothing Q41.1.7 Buy household goods Q41.1.8 Education / school fees Q41.1.9 Health care related expenditures e.g. health insurance, provider fees, medicine, etc. Q41.1.10 Contraceptives and family planning Q41.1.11 Support to other family members Q41.1.12 Invest in business or other assets (including purchase of goats, sheep other animals) Q41.1.13 Savings Q41.1. Oth1 Other 1 (specify) 1___________ _ __________ ___________ ___________ Q41.1. Oth 2 Other 2 (specify) 2 Q41.1. Oth 3 Other 3 (specify) 3 Q41.1. Oth 4 Other 4 (specify) 4 42. Do you currently have savings? 1= Yes 2= No (skip to 45) (if no skip to Q45, also please confirm they are consistent with Q41) 42a. If yes, how much do you have? ___________ 43. How often do you put aside money to save? (only one answer allowed) 1. Never 2. A few times year/seasonal 109 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report 3. Every month 4. Every week 5. Every day 44. Where do you keep your savings? (check all that apply) Q44.1 At home in a safe place 1.Yes 2. No Q44.2. In a bank/microfinance institution 1.Yes 2. No Q44.3 With someone (friend/relative/other) I trust 1. Yes 2. No Q44.4 With an individual who will pay me an interest on my savings 1.Yes 2. No Q44.5 Oth.Other________________________ 1.Yes 2. No 45. If you do not save why do you not save? (check all that apply) Q45.1 Have no money to save 1.Yes 2. No Q45.2 Spend on clothes, shoes, etc. 1.Yes 2. No Q45.3 Pay off loans 1.Yes 2. No Q45.4 Use money for investment (including animals) 1.Yes 2. No Q45.5 Not willing to save 1.Yes 2. No Q45.6 Oth. Other _______________ 1.Yes 2. No Health Related Questions 46. Have you had any illness in the past six months? 1= Yes 2= No (skip to 47) 46a. What was your illness? ________________________________ 46b. Did you take time off work due to illness? ---------------(number of days) 46c. How often have you visited a health care provider in the past six months? ----------------- (number of times) 47. How often have you visited a health care provider in the past six? -----------------(number of times) 48. Where did you seek advice or treatment the last time you were sick? (multiple response allowed) Q48.1 Govt. Hospital 1. Yes 2. No Q48.2 Govt. Health Center 1. Yes 2. No Q48.4 Govt. Health Post/ Health Extension Worker 1. Yes 2. No Q48.5 NGO 1. Yes 2. No Q48.7 Private Hospital 1. Yes 2. No Q48.8 Private Clinic 1. Yes 2. No Q48.9 Drug Vendor 1. Yes 2. No Q48.10 Shop 1. Yes 2. No Q48.11 Traditional Healer 1. Yes 2. No Q48.Oth Other Specify ___________ 1. Yes 2. No 49. If you have not been sick in the past six months but had to seek healthcare, where would you go? (multiple response allowed) Q49.1 Govt. Health Center 1. Yes 2. No Q49.2 Govt. Health Center 1. Yes 2. No Q49.4 Govt. Health Post/ Health Extension Worker 1. Yes 2. No Q49.5 NGO 1. Yes 2. No 110 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Q48.7 Private Hospital 1. Yes 2. No Q49.8 Private Clinic 1. Yes 2. No Q49.9 Drug Vendor 1. Yes 2. No Q40.10 Shop 1. Yes 2. No Q49.11 Traditional Healer 1. Yes 2. No Q49.Oth Other Specify ___________ 1. Yes 2. No 50. Did you obtain any of the following from your health care provider in the past six months? (Note to interviewer – if uncomfortable answering skip item and move to next) Q50.1 Medicines for illness Q50.2 Male contraceptives Q50.3 Female contraceptives Q50.4 Advice on nutrition for yourself and your family 51. To what extent do you agree with the following statements As compared with six months ago Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree Q51.1 My health has improved Q51.2 I am less stressed Q51.3 I am less anxious Q51.4 I get fewer headaches Q51.5 I am less angry Q51.6 I eat better food Q51.7 I spend more money on food Q51.8 I take better care of my health Q51.9 I am happier Thank you for your time today and we may be contacting you for any future interviews. Name of interviewer: __________________________ Date: _______________Ending time: __________ 111 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Youth Cohort Study Financial Diary Questionnaire Financial Diary Survey Information to be completed by Enumerator: Name of enumerator ____________________________ Interview date: __________________________ Starting time: _____________________ Location: Region: ____________________ Woreda: _________________ Kebele:___________ 52. ID Number (please use the baseline survey ID number) _______________________________________ 53. Name of Participant (with grandfather’s name): __________________________________ PART I: General Identification (To be completed only the first time) Notes for interviewers: (Prefill data from the baseline survey. Confirm responses to questions below for the first time only, do not collect after that as the data is not going to change. For age, even if there is a change of up to one year in age do not collect data) 54. Age of Participant: ________ 55. Sex: ☐ Male ☐ Female 56. Ethnicity and Language a. Ethnicity: ☐ Afar ☐ Amhara ☐Guragie ☐Nuwer ☐ Oromo ☐Sidamo ☐ Somali ☐ Tigray ☐Welaita ☐ Other __________________ b. Language Amarigna ☐, Oromigna ☐, Tigrigna ☐Afarigna ☐ Guragigna ☐ Sidamigna ☐ Somaligna ☐ Welaitigna ☐ Other ☐ 57. Disability: (check all that apply) ☐ No disability ☐ Deaf or hard of hearing ☐ Blind (one or both eyes) ☐ Physical handicap ☐ Cognitive disability (mental disability)☐ Other 58. What is the interviewee’s highest educational attainment? (includes grade completed or dropped￾out) 01 = No schooling completed 02 = Only traditional/non-formal school 03 = Highest grade completed _________ 04 = Completed TVET (drop-outs captured under grade level) 05 = Completed College/ University 06 = Other specify: 112 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report PART II: General Identification (Use data from the baseline survey; check for any changes every time) 59. Participant’s Current address: ______________________________________________________ Region: ____________________ Woreda: _________________ Kebele: ___________________ Personal Tel. (mobile): ______________________, Residence: ___________________ E-mail (if available) or Facebook (if available): __________________________________ 60. Contact information of close relative/neighbor/friends (in addition to contact address): Name__________________________ Tel: ______________________________ Address __________________________________________________________ Region: ____________________ Woreda: _________________ Kebele: _______________ 61. Current marital status of participant (select one): ☐Never married ☐Married ☐Living together (but not married) ☐Separated/Divorce ☐Widowed 62. Living status: _________Living with parents _____________Living independently 63. Household Size: _____________ (people living in the same house permanently, including the respondent) 64. Number of Participant’s Children: Total number (enter zero if none) _____, Girls_____ Boys PART III: Employment Status and Income (collect first time, confirm for any changes after that) 65. What are you doing now? (current status) (note to interviewer: let the respondent tell you what they are doing and code response into the categories below. current status (0=not 1= yes) Did you work this in the last month? i. Work for wage/salary with an employer ii. Have a job, but temporarily absent from work iii. Self-employed - self owned iv. Self-employed – family owned v. Self-employed – group owned vi. Work as an unpaid family member (work for family gain) vii. Engaged in home duties (including child care) [skip to Q18] viii. In school or engaged in training [skip to Q18] ix. Unemployed but actively looking for work x. Unemployed but not seeking work for other reasons (e.g. disabilities) [skip to Q18] xi. Other _____________ 66. Which of the following sectors do you work in (for wage/salary or self-employment)? Note to interviewers – For female participants ask – Aside from your own housework, if you work for payment in-cash or kind or are self-employed which of the following sectors do you work in? Sector of employment Primary employment (select one) Secondary employment (select one) Agricultural + Livestock activities Trading agricultural products (buying or selling agricultural products) Trading consumer products/items – small shop/kiosk Carpentry, carving, or woodwork, Mechanics, metal work Construction work (brick/stone laying, building, etc.) Daily laborer 113 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Sector of employment Primary employment (select one) Secondary employment (select one) Electrical work Computer technology and administration Tailoring (making clothes or textiles) Transport service Hairdressing/Barber Hotel industry Education Work for family Government NGO Providing services (such as coffee/tea, small restaurant, shoe shine, repair and maintenance, etc.) Don’t have secondary job Others (specify) ________ 67. For how many days did you work in the past month and how much did you earn? Note to interviewers –  For those working in a salaried job or for wages ask for amount of wages earned;  for those paid in kind please ask the respondent to estimate the monetary value of the in-kind payment  for the self-employed please ask for the net income (profit earned after deducting for costs of good/services; not for total(gross) amount earned Primary employment Secondary employment a. Number of days worked in the month b. Income (wages/net profit) earned in the past month 68. If you did not work in the past month, please tell us the reason for why you did not work? (Allow multiple responses) Primary employment Secondary employment Own illness, injury Family member’s illness, injury Holiday, vacation Personal, family responsibilities Education leave or training (outside the place of work) Starting work in the future Strike Temporary lay-off (with formal job attachment) Lock-out (at work) Slack period, off season Reduction in economic activity Temporary disorganization, suspension of work (bad weather, mechanical, electrical breakdown, shortage of raw materials, fuel, etc.) Other involuntary (economic) reason 69. Have you borrowed money in the past month? 114 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report 0= No (skip to Q12) 1= Yes a. Where did you borrow from? 1= Micro Finance Institution □ □ □ □ Yes □ □ □ □ No 2= Bank Yes No 3= Friend/ Family/ other relative Yes No 4= other (specify) Yes No b. Why did you borrow the money? (0=No, 1= yes) 1=Starting own business □ □ □ □ □ □ □ Yes □ □ □ □ □ □ □ No 2=Investing in existing business Yes No 3=Buy assets (animals, land, house, bicycle, fan, mobile phone, etc.) Yes No 4=To pay for household expenses Yes No 5=To pay for my own healthcare (treatment due to my own illness) Yes No 6= To pay for a family member’s healthcare (treatment due to family member’s illness) Yes No 7=other _______ Yes No PART IV: EXPENDITURE 70. In the past month, how have you used your income? Please estimate how much of your total income did you spend on each of these categories? Note to interviewers: For each of the other categories please enter the name of the expenditure Check all the categories that you spent money on in the past month (0=No, 1= yes) If yes, ask how much was spent in the past month (enter Amount) a. Gave money to my family for household expenses b. Rent/lease c. Taxation d. Transportation e. Buy food and related goods f. Pay for electricity, water, g. Clothing h. Buy household goods i. Education / school fees j. Paid loan k. Health insurance l. Other Health care related expenditures m. Contraceptives and family planning n. Support to other family members (cash or in-kind) o. Invest in business or other assets (for business purposes) p. Purchase of Mobile phone and airtime q. Purchase of assets such as radio, TV, bicycle, r. Purchase of animals (assets) such as goats, sheep, cows, other animals s. Savings 115 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Check all the categories that you spent money on in the past month (0=No, 1= yes) If yes, ask how much was spent in the past month (enter Amount) t. Other 1 (specify) ______________________ u. Other 2 (specify) v. Other 3 (specify) w. Other 4 (specify) 71. Were you or your family sick in the last month? 1= Yes, I was sick, 2= Yes, my family member was/were sick 3= Both I and my family member/s were sick 4= None of us were sick (Skip to end) 72. Where did you seek advice or treatment? (multiple response allowed) For You (0=No, 1=yes) For Your Family (0=no, 1=yes) a. Govt. Hospital b. Govt. Health Center c. Govt. Health Station/Clinic d. Govt. Health Post/ Health Extension Worker e. NGO f. Health Facility g. Private Hospital h. Private Clinic i. Drug Vendor j. Shop k. Traditional Healer l. Other Specify __________________________ 73. How much did you spend on the treatment? (we want to know the overall Health care related expenditures and if possible the breakdown by categories below) For You (amount) For Your Family (amount)  Total (overall)  Payment to doctor/clinic/traditional healer/other provider  Medicine, etc.  Other 116 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Guide for FGDs - Youth General Introduction and Informed Consent Welcome everyone! My name is _________________________________ from Social Impact, an evaluation company from America. We are conducting a study to assess the effect of the Youth Potential program from selected Woredas in four regions. For this, we are following a cohort of over 400 participants for 12 months. We started the first round in April-May, and we are following up with you now six months later. This group discussion is part of cohort study of the USAID/Youth Potential program, a program to improve youth employment opportunities in Ethiopia. This study will not determine if the program will continue, nor is it to evaluate your performance. It is only to learn about your expectations /experience with the program, use of the training received, the resources available and any challenges. We are very grateful for your time; this discussion should take about an hour and a half. The information you provide to us here is confidential and will not be shared with anyone outside. Please know that we will not share your name with anyone outside the research team, or quote you in our report without your permission, so we also hope that you will provide honest and accurate answers. For the note taker: Location: Region: _____________ Woreda: ______________Kebele: ______________ Focus Group date: __________________________ Starting time: _____________________ Number of males: Number of females: Pass out a sheet of paper to gather information on: Participant name, age, gender, Kebele, level of education, participant signature Focus group guide: 1. Before we begin, kindly; introduce yourself…. 2. Tell us about yourself (5 minutes of ice breaker – show of hands and count) a. How many are married? b. How many live with their parents? Independently? c. How many of you have had technical/vocational training in Potential program? d. How many have had internships, volunteer work, job shadowing opportunities? e. By show of hands, how many of you are working in: ● Wage employment? ● How many are self-employed? In which field? (see reference list below) ● For how many is this the family business? ● How many are in school? ● How many are unemployed? f. How many of you think agriculture is a good opportunity for youth? g. How many of you prefer to be self-employed? Why? h. How many prefer to have wage employment? Why? 117 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report ● Reference list of fields of work ● Agricultural + livestock activities ● Trading agricultural products (buying or selling agricultural products) ● Trading consumer products/items – small shop/kiosk ● Carpentry, carving, or woodwork ● Mechanics, metal work ● Construction work (brick/stone laying, building, etc.) ● Daily laborer ● Electrical work ● Computer technology and administration ● Tailoring (making clothes or textiles) ● Transport service ● Hairdressing/barber ● Providing services (such as coffee/tea, small restaurant, shop) ● Education ● Government ● NGO ● Others (specify) 3. Changes in employment status over the past six months a. How many of you have more than one type of work? If yes in what areas? (Mixed livelihoods – for example, helping family farm and making & selling soap) b. How did the Potential program help you? - Get a job? (Write a CV, find job postings, interview skills) - Start their business? (Give them resources to start their businesses (loan, materials, etc.)?) c. Do you think you could have secured your job or started your business without participating in Potential? d. For those of you who already had a business before joining the program, how has your participation in the program affected your business (increase income, expand services/products) e. For those of you who already had wage-based employment before joining the program, how has your participation in the program affected your status (better employment, more income) 4. For those of you who are not employed what have been your biggest challenges in finding employment/starting a business? 5. Relevance of experience with the Youth Potential program a. What did you learn from the Potential training? (For example, gained new hard skills; gained new soft skills; access to finance; access to materials; improved literacy and numeracy; improved perception of self/more confidence) i. What did you like the most? ii. What did you like the least? iii. Please describe your apprenticeship or work-based experience, if you had any. iv. Please describe your mentoring/coaching support, if you had any. b. What new skills are you using as the result of the program? c. Thinking back to five months ago, do you have better access to employment/ entrepreneurship information? d. Beyond job-related support, how do you think your life has or has not changed since participating in Potential? 118 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report e. What did you learn from the Potential training? (For example, gained new hard skills; gained new soft skills; access to finance; access to materials; improved literacy and numeracy; 6. Income-related questions – Has your income level changed since you participated in the WRN! training? Are you spending money differently now? a. How much of your income do you spend on your family? If you had a steady income, how would you spend the money? b. How about food – fruits and vegetables? c. Health care services, like visits to the doctor, ● vaccinations for kids, ● use of contraceptives, etc. d. Do you think your health has improved compared to five months ago? Why? 7. Changes in functional literacy and numeracy a. Has your reading and writing skill improved in the past 4 to 5 months? If yes, was that due to your participation in Youth Potential and if so how? b. Has your addition and multiplication skill improved in the past 4 to 5 months? If yes, was that due to your participation in the Youth Potential and if so how? 8. What additional support do you think you need from Potential to help you find employment, either in wage employment or self-employment? 9. Is there anything else you think we should know about youth and employment in this region? THANK YOU 119 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Interview Guide - Partners General Introduction and Informed Consent Hello! My name is _________________________________ from Social Impact, an evaluation company from America. We are conducting a study to assess the effect of the Youth Potential program, a program to improve youth employment opportunities in Ethiopia. For this, we are following a cohort of over 400 youth participants from selected Woredas in four regions for 12 months. We started the first round in April-May, and we are following up with the youth now six months later. We would like to hear your opinion of the program and its beneficiaries. We are very grateful for your time; this discussion should take about forty minutes. The information you provide to us is confidential and will not be shared with anyone outside. Please know that we will not share your name with anyone outside the research team, or quote you in our report without your permission, so we also hope that you will provide honest and accurate answers. For the note taker: Participant name: ________________________ Company Name: ________________________ Location: Region: _____________ Woreda: ______________Kebele: ______________ Interview date: __________________________ Starting time: _____________________ 1. We would like to learn more about your organization. a. What sector is your company based in? b. In a couple of words, what product/service does your organization provide? c. When was the organization established d. How many people do you employ? (employ less than 10 employees? In between 10 and 50? More than 50?) 2. In what way are you partnering with POTENTIAL Activity? a. What is your motivation for collaboration? b. What kinds of support does your organization provide to the project? c. Are there any differences in the types of support provided by your organization for young men versus young women? ● Are you providing apprenticeships or other work-based learning opportunities to youth from Potential program? Have you hired any of them after their apprenticeship? 3. If you already have youth from Potential program at your organization (either as apprenticeships, WBL, or employment), please describe how the experience has been for your organization so far. -Probe: In what ways are youth from Potential different than other youth at your organization? 4. Would you hire future Potential training graduates? Why or why not? 5. Is there any other feedback you think it is important to share with us about the Potential program? THANK YOU 120 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Interview Guide - Employers General Introduction and Informed Consent Hello! My name is _________________________________ from Social Impact, an evaluation company from America. We are conducting a study to assess the effect of the Youth Potential program, a program to improve youth employment opportunities in Ethiopia. For this, we are following a cohort of over 400 youth participants from selected Woredas in four regions for 12 months. We started the first round in April-May, and we are following up with the youth now six months later. We would like to hear your opinion of the program and its beneficiaries. We are very grateful for your time; this discussion should take about forty minutes. The information you provide to us is confidential and will not be shared with anyone outside. Please know that we will not share your name with anyone outside the research team, or quote you in our report without your permission, so we also hope that you will provide honest and accurate answers. For the note taker: Participant name: ________________________ Company Name: ________________________ Location: Region: _____________ Woreda: ______________Kebele: ______________ Interview date: __________________________ Starting time: _____________________ 1. We would like to learn more about your company. a. What sector is your company based in? b. In a couple of words, what product/service does your company provide? c. When did you establish this company? d. How many people do you employ? (employ less than 10 employees? In between 10 and 50? More than 50?) 2. Do you know an Activity or a project called Youth potential? If yes, how was it known to you? a. Are you collaborating with them? in what way? b. Are you providing apprenticeships or other work-based learning opportunities to youth from Potential program? 3. What is your motivation for providing apprenticeship and/or work-based learning opportunities to youth? a. Have you hired any of them after their apprenticeship? 4. What is your hiring perspective for the next year? -Probe: Is your sector in general growing? Are there likely to be new jobs available for young people? 5. How do you typically fill vacancies/recruit for entry-level jobs? - Probes – how do they advertise jobs? How common is word of mouth? Etc.) - Probe: Which entry-level jobs are the most difficult to fill? Why? 6. Please describe the types of jobs available for young people in entry-level positions at your enterprise. -Probe: Are there any differences in the types of jobs available for young men versus young women? 7. Which skills are most important for entry-level positions (probe on hard and soft skills)? 8. If you already have youth from Potential program at your company (either as apprenticeships, WBL, or employment), please describe how the experience has been for your company so far. -Probe: In what ways are youth from Potential different than other youth at your company? 9. Would you hire future Potential training graduates? Why or why not? 10. Is there any other feedback you think it is important to share with us about the Potential program? THANK YOU 121 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Guide for FGDs – Parents/Guardians General Introduction and Informed Consent Welcome everyone! My name is _________________________________ from Social Impact, an evaluation company from America. We are conducting a study to assess the effect of the Youth Potential program, a program to improve youth employment opportunities in Ethiopia. For this, we are following a cohort of over 400 youth participants from selected Woredas in four regions for 12 months. We started the first round in April-May, and we are following up with the youth now six months later. This group discussion will not determine if the program will continue, nor is it to evaluate your children’s performance. It is only to learn about your children’s expectations /experience with the program, the resources available and any challenges. We want to get the first￾hand perspectives of parents/guardians of Youth Potential Activity participants. We are very grateful for your time; this discussion should take about an hour. The information you provide to us here is confidential and will not be shared with anyone outside. Please know that we will not share your name with anyone outside the research team, or quote you in our report without your permission, so we also hope that you will provide honest and accurate answers. For the note taker: Location: Region: _____________ Woreda: ______________Kebele: ______________ Focus Group date: __________________________ Starting time: _____________________ Number of males: Number of females: Pass out a sheet of paper to gather information on: Participant name, gender, Kebele, participant signature 1. Before we begin, kindly; introduce yourself…. 2. Tell us about yourself (5 minutes of ice breaker – show of hands and count) a. How many of you are parents of participants in POTENTIAL program vs guardians? b. How many children do you have? c. How many of them live at home? d. How many of you own a house? e. How many of you own land for agriculture? If so, do your children help you with farming? f. How many of you have your own business? If so, do your children help you with your family business? g. How many of you have a child (children) who are excited to be participating in Potential program? 3. What do you know about the Potential program? a. How did you hear about the Youth Potential program? b. Did you encourage your child to join Potential? Why or why not? c. What is the reputation of Youth Potential program in your community? 4. What type of skills is Potential helping your child gain, if any? - Probe with examples if needed: -Agriculture -Self-employment (open shops, find a business idea, customer service, how to calculate profit, marketing, etc.) -Accessing financing -Soft skills (confidence, job applications, interview skills, communication, being proactive, etc.) 122 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report -Technical/vocational skills (hairdressing, tailoring, bakery, welding, mechanics, construction, driving, etc.) 5. Have you noticed any changes in your child since he or she joined Potential? (For example, has more confidence, saves more money, is more responsible, has new hard skills, etc.) 6. What kinds of work would you like to see for your children? (Probe on agriculture, off-farm self-employment opportunities) a. What are the main barriers for youth to start their own business in your community? b. What are the main barriers for youth to obtain wage employment? c. Do you think there are enough jobs for youth in your community? 7. How is Potential impacting your child in terms of employment opportunities? (Probe on both starting new businesses and gaining wage employment.) a. How do you think Potential has, or has not, helped your child overcome these barriers to start their own business? b. How do you think Potential has or has not helped your child overcome these barriers to get employed? 8. If your household had more income, what type of health-related expenses would you spend more money on? (For example, preventive care, vaccinations, reproductive health, etc.) - Probe – has your child’s employment made any difference to how much you spent or she/he spends? 9. Is there any other feedback you think it is important to share with us about the Potential program? THANK YOU 123 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Guide for KII – YP Facilitators and Woreda Officers General Introduction and Informed Consent Welcome! My name is _________________________________ from Social Impact, an evaluation company from America. We are conducting a study to assess the effect of the Youth Potential program from selected Woredas in four regions. For this we are following a cohort of over 400 participants for 12 months. We started the first round in April-May, and we are following up now six months later. This KII is part of cohort study of the USAID/Youth Potential project. This interview is to learn about your thoughts on the Youth Potential project. We are very grateful for your time; this interview should take about half an hour. The information you provide to us here is confidential and will not be shared with anyone outside. Please know that all information will be combined together for report, we will not quote you in our report without your permission, so we hope that you will provide honest and accurate answers. Location: Region: _____________ Woreda: __________Kebele: ______________ Office_____________ Position of the participant_____________________ (Ask Q1 and Q2 only for new facilitators – don’t need to repeat if we interviewed facilitator at the baseline.) 1. What is your role on the project? How long have you been involved with it? 2. Can you describe the project activities? (Note: for new facilitators ask questions below in general and for the rest ask as below) Thinking of the cohort from March/April 2017: 3. How are youth doing in Potential training program over the past 6 months? a. What proportion of youth are completing the soft skills training? (PYD, WRN, etc.) The business training (BYOB)? b. What are the main reasons youth drop-out of the training? What have you found to be most helpful in overcoming the drop-out challenges? c. What are lessons learned in working with youth to prepare their personal development plans? What about the business idea plans? d. What is the experience with illiterate youth? Is there any support for literacy/numeracy? e. What proportion of youth receive hard skills training? f. What are the most common topics for hard skills training? What additional hard skills topics are youth requesting training? (e.g. driving, construction, baking, etc.?) g. How many youth have opportunities for work based learning? Which are most common – internships, job shadowing, volunteering, other? h. What are challenges in youth experiences with work-based learning? What about challenges with employers? 4. Support for youth in self-employment/starting their own businesses a. How are youth identifying viable business ideas in their communities? b. How are youth identifying agriculture opportunities in their communities? (e.g. vegetable production, fattening, chicken production, etc.) c. How many youth receive coaching/mentoring? On average, how many sessions do they receive? What has been working well with coaching/mentoring, what has been challenging? d. How are youth getting support to get start-up finance for their business idea? How are they accessing the materials and equipment they need? What about land for agriculture opportunities? e. Are youth showing interest in starting group business with their peers? 124 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report 5. Support for youth in finding wage employment a. What is the experience so far of youth finding jobs/wage employment after training? In what sectors and types of occupations are they finding jobs? b. What sort of job placement support is Potential providing? What are the main challenges in being able to offer this support? c. What is the experience in working with employers for the work-based learning, and job placement? 6. What are the challenges in tracking and following up on trained beneficiaries? What have you found to work well in tracking the youth? a. How many youth in your cohort migrated to a new town/region to find more employment opportunities? 7. Are youth sharing any of their experiences and challenges related to health problems and health outcomes? 8. What is your experience with Woreda government partners and other stakeholders (TVET, TTIs, credit association/microfinance, etc.)? 9. Is there anything else you would like to share with us about how youth are doing in income and employment status, or general challenges/lessons learned? THANK YOU 125 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Guide for KII – Microfinance Providers General Introduction and Informed Consent Welcome! My name is _________________________________ from Social Impact, an evaluation company from America. We are conducting a study to assess the effect of the Youth Potential program from selected Woredas in four regions. For this we will follow a cohort of over 400 participants for 12 months. This KII is part of cohort study of the USAID/Youth Potential project. This interview is to learn the firsthand perspective of microfinance providers. We are very grateful for your time; this interview should take about half an hour. The information you provide to us here is confidential and will not be shared with anyone outside. Please know that all information will be combined together for report, we will not quote you in our report without your permission, so we hope that you will provide honest and accurate answers. For the note taker: Name of Participant:_________________________ Name of Institution: ____________________ Location: Region: _____________ Woreda: ______________Kebele: ______________ Interview date: __________________________ Starting time: _____________________ 1. What is your role at the microfinance institution? How long have you been working here? 2. How did you start partnering with the Potential program? 3. What share of your overall clients are young people? 4. What types of financial products and services do you offer to young people – e.g. loans, savings accounts, etc.? Are these the same financial products available to Potential youth, or do they have access to different products? (Probe – do they have any group products for youth available?) 5. What types of businesses do Potential youth who apply for your loans have? What is the quality of the business plans they share with you? 6. What is the average size of loans you give to Potential youth, and what is the range of the loan size? 7. What do youth typically present as collateral when they apply for loans? 8. In what ways, if any, are youth from Potential different than other youth who apply for loans at your microfinance institution? (Probe – Is there any difference in repayment?) THANK YOU 126 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report ANNEX IX: DATA TABLES AND GRAPHS Additional Data tables and Graphs not included in the body of the report, Regression Analysis and Expenditures by Category 127 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Table X1. Distribution of YCS survey respondents: at baseline. midline and endline (%) Distribution of survey respondents Baseline N=582 Midline N=533 Endline N=458 Variable Percent Percent Percent Sex Female 43% 43% 43% Male 57% 57% 57% Region Afar 19% 18% 19% Amhara 34% 32% 29% Oromia 31% 32% 34% Tigray 16% 18% 18% Age Group 15 to 18 14% 13% 12% 19 to 24 67% 67% 66% 25 to 29/30 19% 20% 21% Level of education No School/Traditional 7% 6% 5% Grade1 to 5 15% 12% 11% Grade 6 to 8 24% 25% 24% Grade 9 to 12 48% 48% 50% Completed 5% 7% 8% College/University 3% 2% 2% Marital status Never married 58% 52% 48% Married 35% 38% 47% Separated/Divorce/Widowed 7% 11% 5% Living (Residential) status: Living with parents 79% 57% 57% Functional Literacy and Numeracy Can’t read & write 9% 11% 7% Can read & write with difficulty 9% 8% 7% Can read & write with fluency 83% 82% 86% Functional numeracy - Recognize numbers 97% 94% 96% Functional numeracy - Add simple numbers 92% 90% 91% Employment Status Paid/Wage/Paid-In Kind Employed 7% 9% 9% Self-Employed 41% 65% 60% Unemployed (includes the small number in school) 52% 26% 32% 128 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Table X2. Distribution of subgroup of YCS survey respondents that responded to the monthly financial diary interviews Financial diary interview respondents N=83 Variable Percent Sex Female 57% Male 43% Region Afar 16% Amhara 27% Oromia 28% Tigray 29% Age group 15 to 18 11% 19 to 24 73% 25-29 16% Level of education No School/Traditional 0% Grade1 to 5 0% Grade 6 to 8 18% Grade 9 to 12 73% Completed TVET 8% College/University 2% Marital status Never married 65% Married 28% Separated/Divorce/Widowed 2% 129 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Table X3. Change in Employment Status from Baseline to Midline to Endline (n=458) From Baseline to midline Midline Change from BL to ML Baseline In school or training Paid/wage /paid-in kin Self - employed Unemployed Unemployed (not seeking job) Baseline Total Baseline Total Midline Total In school or training 0 1 6 13 1 21 21 9 Paid/wage/paid￾in kin 0 11 15 3 0 29 29 43 Self-employed 1 9 164 15 3 192 192 296 Unemployed 8 21 105 69 4 207 207 102 Unemployed (not seeking job) 0 1 6 2 0 9 9 8 Total 9 43 296 102 8 458 458 458 From midline to endline Endline Change from ML to EL Midline In school or training Paid/wa ge/paid￾in kin Self￾employed Unemployed Unemployed (not seeking job) Midline Total Midline Total Endline Total In school or training 3 0 2 4 0 9 9 13 Paid/wage/paid￾in kin 1 15 20 7 0 43 43 39 Self-employed 5 18 216 57 0 296 296 277 Unemployed (dependent 4 6 36 54 2 102 102 126 Unemployed but not se 0 0 3 4 1 8 8 3 Total 13 39 277 126 3 458 458 458 *Note: This table presents the dynamic process of employment status to shed light on change in employment status and mobility from baseline to midline to endline. The rows represent the participants’ employment status at baseline while the columns show their changed status at midline, similarly from midline to endline. For example, the table suggests that of the 458 respondents at endline, 207respondents were unemployed at Baseline and 69 of these remained so in midline. Among the remaining 21 became wage employed and 105 became self-employed. On contrary, 15 respondents who were self-employed became unemployed during midline, Similarly, we can track changes in status from midline to endline . 130 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Table X4. Changes in sector of employment of YCS respondents: baseline vs midline (%) Broad Categories Detailed Categories Baseline n=225 Midline n=230 Endline n=316 Difference BL-EL Endline n=316 Male Female Agriculture Trading Agricultural + Livestock activities 45.6% 46.1% 50% -4% 58% 36% Agriculture Trading Manufacturing & construction Manufacturing & construction Trading agricultural products (buying or selling) 8.8% 11.8% 7% 2% 7% 6% Trading consumer products/items 11.4% 12.3% 11% 0% 5% 23% Manufacturing & construction Manufacturing & construction Daily Labor Carpentry, carving, or woodwork 1.3% 0.4% 1% 0% 1% 1% Mechanics, metal work 1.3% 1.8% 2% -1% 2% 1% Construction work (brick/stone laying) 1.3% 1.3% 2% -1% 3% 1% Daily Labor Services Daily laborer 9.6% 4.4% 7% 3% 7% 5% Daily Labor Services Professional Professional Electrical work 0% 0% 0% Computer tech and administration 0.9% 0.9% 1% 0% 1% 1% Tailoring (making clothes or textiles) 2.2% 1.8% 1% 1% 1% 1% Transport Service 3.1% 3.1% 3% 0% 4% Hairdressing/Barber 0.9% 1.3% 3% -2% 1% 5% Providing Services (such as coffee/tea, restaurant, etc.) 8.8% 7.5% 4% 5% 2% 9% Hotel industry 0% 0% 0% Professional Professional Other Education 0.4% 0.4% 1% -1% 1% Government 0.4% 0.9% 3% -3% 2% 5% NGO 0.4% 1.3% 1% -1% 0% 1% Other 3.5% 4.8% 3% 1% 2% 5% Source of data: YCS Baseline, Midline and Endline Surveys 131 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Table X5. Profile of survey respondents by combined employment status at baseline, midline and endline Constant employment (Employed at BL, ML and EL) Employed post training (Unemployed at BL, employed at ML and EL) Insecure employment (Unemployed at BL, employed at either ML or EL) Chronic unemployed (Unemployed at BL, ML and EL) Total n=176 n=95 n=122 n=65 n=458 Variable Percent Percent Percent Percent Percent Sex Female 32 39 50 65 43 Male 68 61 50 35 57 Region Afar 6 21 23 43 19 Amhara 45 24 16 17 29 Oromia 38 17 42 32 34 Tigray 11 38 19 8 18 Age Group 15 to 18 6 15 15 23 12 19 to 24 57 74 72 69 66 25 to 29/30 37 12 13 8 21 Level of education No School/Traditional 5 7 5 5 5 Grade1 to 5 11 9 12 9 11 Grade 6 to 8 28 21 20 22 24 Grade 9 to 12 47 42 55 58 50 Completed TVET 7 15 7 3 8 College/University 2 5 1 3 2 Living (Residential) status: Living with parents 38 64 63 86 57 Living independently 63 36 37 14 43 132 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Determinants of Income – Regression Tables Table X6. Did income increase in the last six months? (Q40a=1--> increased) (1) (2) VARIABLES Between Baseline and midline Between midline and endline Respondent's gender (Male=1) 1.42 1.23 Respondent's age group [Reference: age group from 19-24] Age group: 15 to 18 1.30 0.79 Age group: 25 to 30 0.80 0.62* Household size: [Reference group: 3-5] Household size: 1-2 1.04 1.16 Household size: 5-10 1.11 1.33 Household size: 10-15 1.71 0.69 Marital status [Reference: Never married] Married 0.79 1.37 Others including widowed, separated etc. 0.85 1.18 Level of education [No education] Grade1 to 5 1.10 0.68 Grade 6 to 8 1.09 0.53 Grade 9 to 12 1.19 0.41* Completed TVET 1.26 0.71 College/ University 0.78 0.30 Self-employment status [Reference: All others] Self-employed not looking for job 4.3*** 3.5*** Self-employed looking for job 3.1*** 3.8*** Sectoral employment [Reference: Agriculture] Sectoral employment: Trading 1.96** 0.62 Sectoral employment: Manufacture 3.07* 3.73* Sectoral employment: Daily lab 2.62* 1.03 Sectoral employment: Service 2.2** 0.70 Location: Regions [Reference: Tigray] Afar 0.79 0.23*** Amhara 0.27*** 0.36*** Oromia 0.40** 0.41*** Constant 0.86 2.35 Observations *** p<0.01, ** p<0.05, * p<0.1 Source of data: YCS Endline Survey 133 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Table X7. Fairlie decomposition for determinants of income Groups Components Coef. Std. Err. z P>z [95% Conf. Interval] Gender Respondent's gender (Male=1) 0.008 0.003 2.760 0.006 0.002 0.014 Log of total expenditure Log of total expenditure 0.092 0.016 5.920 0.000 0.062 0.123 Age group [(15 to 18); (19 to 24); (25 to 30)] 0.002 0.002 1.010 0.315 -0.002 0.005 Groups of household size [(1-2) (3-5), (5-10) (10-15)] 0.000 0.001 0.300 0.763 -0.002 0.003 Marital status [Unmarried, married, others] 0.002 0.002 1.040 0.297 -0.002 0.006 Level of education [grades: (1-5), (6-8), (9-12), TVET, College & above] -0.003 0.002 -1.790 0.073 -0.006 0.000 Regions [Amhara, Afar, Oromia and Tigray] -0.012 0.005 -2.170 0.030 -0.022 -0.001 Panel II Total Number of obs 916 N of obs in endline 458 N of obs in baseline 458 Probability of being employed in endline 0.69 Probability of being employed in baseline 0.49 Difference 0.20 Total explained 0.09 Source of data: YCS Endline Survey 134 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Figure X1. Distribution of monthly expenditures (financial diary) for a select sample 135 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Table X8. Self-perception of use of skills Categories Midline Endline Difference p-Value I feel more confident of my abilities 4.47 4.27 -0.19 <0.000001 I know how to develop a business plan 3.83 3.63 -0.19 <0.002370 I know how to calculate profit/loss 4.15 4.10 -0.05 0.394600 I spend money according to a plan 4.24 4.14 -0.10 <0.032195 My business has expanded 3.18 3.38 0.20 <0.003636 I have better marketing skills 3.40 3.59 0.20 <0.003645 I am able to interact better with customers 3.49 3.60 0.11 0.123664 My business has more customers 3.18 3.35 0.16 0.017485 I work more days of the week 3.69 3.69 0.00 1.000000 I have better time management skills 3.96 3.94 -0.02 0.777643 I am more patient 4.29 4.27 -0.02 0.635600 I get into fewer fights and arguments 3.69 3.71 0.02 0.758884 I am more persistent and don’t give up as easily 4.25 4.24 0.00 0.916419 I feel more capable to solve problems when they arise 4.29 4.25 -0.04 0.346004 I am more respectful of all types of jobs 4.45 4.39 -0.05 0.173518 I am more willing to try different kinds of jobs 4.48 4.38 -0.10 0.023098 *Please note question was only asked at midline and endline, not at baseline Source of data: YCS Midline and Endline Survey 136 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report ANNEX X: Study Team The data collection Team Baseline Deepika Chawla (TL) Worku Ambelu Hika Alemu Ermias Assefa Basab Dasgupta Francis Okello Midline Deepika Chawla (TL) Worku Ambelu Hika Alemu Ermias Assefa Basab Dasgupta Estera Barbarasa Endline Deepika Chawla (TL) Worku Ambelu Hika Alemu Ermias Assefa Basab Dasgupta Jeffrey Tines Supported by: Team of translators – local citizens and university students – with fluency in English and local languages for qualitative data collection Data collectors from SART for the quantitative survey 137 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Study Team Disclosures of Conflict of Interest 138 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report 139 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report 140 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report 141 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report 142 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report 143 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report 144 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Study Team Profiles Dr. Deepika Chawla, Team Leader, brings over 15 years of experience in project management and evaluation (including impact assessments) of education, workforce development and health programs in the US as well as in several countries around the world. In the course of her career she has worked in and supported projects in Africa (Ethiopia, Liberia, Malawi, Morocco, Nigeria, and Sudan), Eastern Europe and South Asia (India, Afghanistan, Bangladesh). She has successfully managed several multi￾million-dollar USAID funded programs. Dr. Chawla has designed and implemented several quantitative and qualitative evaluations of education and health programs, as well as training and capacity building programs in the United States and overseas. She has also designed and supported the monitoring and evaluation of several USAID funded projects overseas to study the impact and effectiveness of these programs. These evaluations have included the conduct of paper-pencil and web-based surveys, site visits, focus groups and interviews, as well as reviews of administrative data. Proficient in qualitative and quantitative data analysis, Dr. Chawla has led research studies on child welfare programs, trafficking in persons, labor force, etc., in Eastern Europe. In Asia, she has conducted an analysis of factors affecting participation of girls in education in rural areas of India and identified the principal determinants that act as barriers to continued participation and retention in the education cycle. In another study in India, she analyzed the factors affecting academic achievement for urban boys and girls and the barriers to girls’ achievement in school. In Bangladesh, she analyzed the data from the Demographic Health Survey to identify the demand and utilization of contraception among urban and rural women. She is currently analyzing the factors affecting reading outcomes for children in South Asia. Dr. Chawla has developed National Education Accounts, a rapid assessment tool for mapping expenditure flows in the education sector according to the sources-to-financing agents-to-uses framework, which is increasingly being used by policy makers and donors interested in improving education outcomes at the state and national level. She has worked with in-country teams to build capacity for the conduct of future NEAs. In addition to successfully managing multimillion-dollar programs, she has been actively involved in new business development and knowledge management for the Education for Development division of Creative Associates and prior to that at Abt Associates. Dr. Chawla holds an Ed.D. in education administration, training and policy studies, and a Ph.D. in psychology with a focus on psychometrics and education assessment. Mr. Ermias Assefa Eshetu, Local Expert, brings over 10 years of monitoring and evaluation and development experience to the team. Mr. Eshetu’s has experience in qualitative and quantitative research, data collection, analysis, and management, conducting community discussions, recruiting research participants, preparing planning, budgeting, and M&E manuals, and in preparing questionnaires using statistical packages. Mr. Eshetu also has extensive experience developing and utilizing socio￾economic and demographic surveys, such as Behavioral Surveillance Survey (BSS); Demographic Health survey (DHS); Agricultural samples; Food security; small, medium, and large-scale industries and construction; educational, health, nutrition, and others. 145 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report As a member of the study team, Mr. Eshetu was responsible for: participating in the development of data collection instruments; training data collectors and supervisors; following the performance of a data collection firm; ensuring that study outputs are of high quality and useful; identifying a sampling frame for primary qualitative and/or quantitative data collection during all phases of fieldwork; conducting data collection, including interviews and FGDs with stakeholders and households; compiling project monitoring and government data by liaising with implementing, funding, and government stakeholders, and; participating in data analysis, presentation of findings, and report writing. Mr. Eshetu holds an M.A. in Measurement & Evaluation from Addis Ababa University, an M.Sc. in Economics (Specialized Agricultural Economics) from Haromaya University, and a B.Sc. in Information Systems from Addis Ababa University. Dr. Jeffrey Tines, International Expert (endline), brings over 30 years of experience in vocational skills education. In the course of his career, he has worked on projects in Latin American, Asia, Europe, the Middle East, and Africa. He has worked extensively in the construction sector where he has been involved in providing technical as well as managerial support in the following areas – competency-based curricula development, teacher training, apprenticeship, internship and mentoring, and employment￾related services. Dr. Tines’ expansive experience includes work with ILO, DFID, USAID, UNESCO, GIZ and CIDA projects which have focused on vocational training and employment creation. He has managed teams from as small as three members to as large as 45, with budgets ranging from five to 80 million dollars, and undertaken assignments from dedicated consultancy work to high scale projects that aim to reach a large target group. Dr. Tines is comfortable working at the macro, meso, and micro levels to build ownership and contribute to systemic change. At the macro level, he has worked with ministries of education and vocational training and ministries of labor around the world to improve planning and monitoring and evaluation capacities; developed and conducted market analysis in agricultural and non-agricultural markets understanding opportunities for sector growth and labour market needs, market determinant factors, value chains and regulatory and control mechanisms; and strengthened and developed TVET – Management Information Systems (MIS) – including enabling MIS systems to properly respond to data collection and analysis needs of the informal and formal sectors. Dr. Tines has worked with key stakeholders to develop inclusive public private partnerships to enhance the quality and relevance of vocational training courses and improve national planning and policy setting. Using data collected from qualitative and quantitative research methodologies, he has developed white papers and strategy papers to promote inclusive vocational education that meet the needs of the labor market and increase equitable access for rural and urban youth, youth with lower educational attainment and out of school youth, and male and female youth. At the meso level, Dr. Tines has worked to assess and review the technical, managerial, and organizational capacity of TVET institutes, working alongside them to put improvement and development plans in place and achieve higher performance through following strategic development steps. He recently facilitated a Vocational Training Center Leadership Training Workshop for UNESCO in Afghanistan where more than 90 participants from all TVET institutes were trained on how to improve their leadership skills as well as development TVET improvement plans. At the micro level, Dr. Tines has worked with TVET institutions and stakeholders to develop and conduct market information assessments; agree occupational competency profiles; and design and develop courses and curricula that promote the dual VET system of practical and theoretical training. 146 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report He has conducted barrier analyses to determine missing competencies among youth and designed life skills courses to complement technical courses and meet requirements. Dr. Tines has worked with representatives of local and central government, private sector employers, vocational training providers, regional employment offices, chambers of commerce and business associations, and offices for and members of communities to identify skills gaps and ensure inclusion of all youth regardless of educational attainment, gender, or ethnicity. In his working career, Dr. Tines has served at various times as a member of senior management teams, chief technical advisor; project director; team leader, senior advisor, and chief of party. He is adept at ensuring that he builds capacity among his team to ensure that they can gradually take over responsibilities and management roles. He has a track record of successful coordination between different stakeholders in TVET from the public and private realms at all levels and has been successful in working with partners to leverage opportunities from other activities and programs. He has worked in teams comprised of single implementing partners and of consortia, and is experienced in managing diverse styles and work modalities in order to get the job done. He has established and implemented robust monitoring and evaluation systems to ensure that targets and objectives are met. Dr. Tines holds a Ph.D. in Vocational and Technical Education, Special Education from the University of Illinois and a Master’s Degree in International Administration and Management, in addition to a B.Sc. and an Ed.M. in Special Education. Ms. Estera Barbarasa, International Expert (midline), brings over 10 years of international workforce development, youth employment, labor markets, and monitoring and evaluation experience to the team. She is experienced in conducting mixed methods research, designing data collection instruments, facilitating focus groups, conducting KIIs, writing technical reports, and conducting youth employment and labor market evaluations. She is experienced in program design, project implementation, and evaluation with IDB, USAID, and World Bank. Ms. Barbarasa served as lead researcher of a thematic evaluation of the Inter-American Development Bank (IDB) Multilateral Investment Fund (MIF's) youth employment and entrepreneurship projects in Latin America and the Caribbean, evaluating 20 projects in eight countries to assess youths’ labor market outcomes. She also provided technical assistance in overseeing the design and completion of a national labor market assessment in El Salvador identifying jobs and skills trends, and she co-authored a report for the World Bank on “New and Promising Approaches in Youth Employment Programs,” which assessed youth employment design across 19 projects in 15 countries (including sub-Saharan Africa); highlighted innovations in skills training, youth self-employment in rural contexts, and youth and agricultural value chains; and analyzed M&E methods to track programs’ impact on job and income outcome metrics for youth. Ms. Barbarasa has worked in El Salvador, Ethiopia, India, Liberia, Malawi, and Morocco. She holds an M.P.P. in economic development from Harvard Kennedy School, and a B.A. in political science from University of Chicago. Mr. Worku Ambelu, Evaluation Methods Specialist for the USAID/Ethiopia Performance Monitoring and Evaluation Services Activity implemented by Social Impact, brings 20 years of development work expertise to the team. Mr. Ambelu possesses a wealth of knowledge in project 147 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report monitoring and evaluation, planning, statistics, assessments and research activities gained from different organizations. Presently, as Evaluation Methods Specialist, he is responsible for coordinating all the evaluation activities and providing oversight on monitoring and learning activities. Mr. Ambelu coordinated and provided tool development, data analysis and report review support to the Graduation with Resilience to Achieve Sustainable Development (GRAD) final evaluation, Community Managed Disaster Risk Reduction (CMDRR) final evaluation, Ethiopia Gender Analysis and Youth Potential Baseline Survey. Before joining Social Impact, Mr. Ambelu was an M&E Expert for the World Bank/VNG International Consulting firm on a program called Promoting Basic Services - Ethiopia Social Accountability Program 2 (ESAP2), the objective of which was to capacitate citizens and citizen groups to hold accountable service providers and government officials for the basic services they provide in Agriculture, Education, Health, Water and Sanitation, and Rural Road sectors. Mr. Ambelu monitored if implementing partners had equipped citizens, service providers and government officials with the necessary capacity such as assessing the aforementioned basic services against the service standards, and Mr. Ambelu assessed if proper tool utilization in identifying service issues were being utilized (this includes the application of a community score card, citizen report card, participatory planning and budgeting, and public expenditure and tracking survey tools). FGDs were intensively used as ongoing evaluation tools to identify the implementation status of projects by different implementing partners and to identify opportunities and challenges. Prior to that, Mr. Ambelu was working with World Learning Inc. to lead overall M&E issues for the School Community Partnership Serving OVC affected by HIV/AIDS in Primary Schools project. Mr. Ambelu spearheaded a wide variety of tasks from conducting annual and baseline surveys, developing database systems, producing survey/research reports and training staff on M&E, data management, indicator development. Mr. Ambelu assumed the Monitoring and Evaluation Specialist and Database Manger positions for the USAID-Community Government Partnership program that was implemented by World Learning and other partners to enhance quality and equity in primary education. He worked as the Monitoring and Evaluation Officer for the health PEPFAR project called CASCADE (Communities and Schools for Children Affected by HIV/AIDS). Before re-joining World Learning in 2012, Mr. Ambelu was at the Population Council as a Research Coordinator working on Poverty, Gender, Youth, HIV/AIDS and Reproductive Health programs. His role in the organization included facilitating baseline, endline and impact evaluation surveys/research protocols, coordinating data collection and data entry, preparing data entry templates, developing sampling methodology, sampling respondents, producing data tables, interpreting results, producing reports, and disseminating findings. The youth survey, coordinated by Mr. Ambelu, focused on a range of youth issues such as Education, Livelihoods, Access and Utilization of Services, Participation and Parent￾Child Relationships, Female Genital Mutilation and Male Circumcision, Alcohol and Chat/Khat, etc. This included both qualitative and quantitative data analysis and report writing. He also contributed to the production of service statistics and routine monitoring reports. Most importantly, Mr. Ambelu’s research findings informed policy makers and project implementers to take different actions. 148 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Mr. Ambelu has a strong statistical background receiving his M.Sc. and B.Sc. degrees in the field of Statistics from the Addis Ababa University and is also well trained in different statistical and qualitative software. Mr. Hika Alemu, Monitoring, Evaluation, Accountability, and Learning Specialist for the USAID/Ethiopia Performance Monitoring and Evaluation Services Activity, brings over 12 years of experience in development as a Performance MEL Adviser, Senior Program Officer, M&E Specialist, M&E Officer, Project Coordinator and Facilitator and Trainer for three USAID funded international NGOs; and two USAID MEL contractors with strong experience in education as well. Mr. Alemu has worked in Ethiopia with international development firms and agencies such as Social Impact Inc., Population Council Inc, Management Systems International (MSI), EngenderHealth Inc., and Food for the Hungry (FH). As a Performance, Monitoring Advisor for Social Impact, she has been supporting the USAID/Ethiopia Mission and its Implementing Partners (IPs) in terms of strengthening its Monitoring, Evaluation and Learning System and was responsible for the management of the USAID/Ethiopia Mission and its IPs MLE system and capacity development. During her work at Population Council Inc., Mr. Alemu revised and designed the M&E system, developed the project’s log frame and produced project proposals to donors. As a manager, she exhibited excellency in clearly articulating expectations and holding staff accountable for results, and for providing support and input where needed. During her work at MSI, Mr. Alemu was extensively and actively involved in developing, refining, and documenting the USAID/Ethiopia PMP/M&E Plan and project proposal to ensure that USAID and government requirements were met, and that PMP/M&E indicators informed decision￾making. As M&E Officer for EngenderHealth Inc., Mr. Alemu oversaw the overall MEL of four local partners at 31 project sites. She facilitated different trainings for partners and for EngenderHealth Inc. staff on basic monitoring tools and their utilization. Mr. Alemu has completed training in MEAL, basic research methodologies, and HIV/AIDS prevention, and possesses good knowledge of various software packages for both data entry and processing, such as MS Access, MS Excel, SPSS, STATA, EventSmart and Epinfo. Mr. Alemu holds an M.A. in Development Economics from Unity University, a B.A. in Economics from Alpha University College, and a B.Sc. in Biology from Addis Ababa University. Dr. Basab Dasgupta, Senior Technical Expert, possesses more than 18 years of research experience and 13 years of experience designing, implementing and managing evaluations, assessments and surveys within an international development context. Currently as one of the SI’s Technical Directors in Impact Evaluation team, Dr. Dasgupta is participating in several performance and impact evaluations as Senior Technical Expert and leading projects as Principal Investigator in Social Impact. For example, Basab, as Project Director, is leading Early Grade Reading Program in Haiti (USAID funded), and Girls Education Challenges in Ghana (DFID funded). He is also leading the impact evaluation component of the Young African Leadership Program in 14 East African countries. As Senior Technical Expert, Basab is also working in several projects such as, 149 Ethiopia Performance Monitoring and Evaluation Service Youth Cohort Study – Final Report Evaluation of ‘Honduras Threshold Program (MCC funded)’; and Impact Evaluation of ‘Lesotho Metlong Dam and Water Supply (MCC funded)’. Prior to joining SI, Dr. Dasgupta worked as a Senior Consultant/Economist for the World Bank for more than 11 years. As an economic researcher and evaluation specialist in the World Bank Water global practice, he conducted (i) quantitative and qualitative analysis to assess the impact of institutional bottlenecks in the water sector on access to improved water, improved sanitation, poverty and human development outcomes, (ii) water resource management and functionality of water schemes and points, (iii) informal water markets in Nigeria, Palestine and Tanzania, and (v) prepared policy notes by stock taking of impact and performance evaluation in the water and sanitation sector. In the Economic Policy Unit, South Asia region of the World Bank, he designed the strategy and methodology for evaluation of the Impact of the Telangana Rural Inclusive Growth Project in India. As a core team member, Dr. Dasgupta prepared scoping paper for M&E, and designed the evaluation methodology to evaluate the impact of Nutrition-Water-Sanitation linkages under the Rural Inclusive Livelihood Project for Andhra Pradesh and Telengana states in India. In his roles at the World Bank Sustainable Development Network, Dr. Dasgupta worked on Economic Sector Work on infrastructure development in LAC region, evaluated a housing subsidy program in South Africa and worked on inclusive growth and housing investment in Sub Saharan Africa, Turkey and Tunisia. Dr. Dasgupta contributed to several poverty assessment reports, analyzed wage and employment trends in India, global price shocks on rural poor in Bangladesh, shortcomings in the functioning of labor markets and governments in Europe and Central Asia, poverty in the European Union, and the financial crisis across five countries such as Turkey, Montenegro, Bulgaria, Armenia and Romania. He also spearheaded evaluations of the social assistance program on work disincentives in Armenia, support for poor and disadvantaged area program in Indonesia, and mediation and a community legal empowerment program in Indonesia. He led an analysis of the quality of education and health services in Indonesia and developed an analytical model for a cost-benefit analysis to evaluate alternative infrastructural amenities in Brazil. Dr. Dasgupta has coauthored several articles in peer reviewed journals, World Bank Working papers series and contributed to Book Chapters. During his stay in India, Dr. Dasgupta worked as Consultant for IFPRI for 2 years and as Research Associate in Indian Institute of Management, Ahmedabad for more than 3 years. Dr. Dasgupta received his Ph.D. in Economics from the University of Connecticut and M.Phil. in Economics from Jawaharlal Nehru University. He is fluent in English, Bengali and Hindi. U.S. Agency for International Development Entoto Street PO Box 1014 Addis Ababa, Ethiopia ANNEX XI: POST-EVALUATION ACTION PLAN This Post-Evaluation Action Plan is developed to create an opportunity to openly discuss evaluation findings, conclusions, and recommendations within the Mission, in collaboration with MEL technical working group and Evaluation Contractor, and plan for use/ adoption of findings. The below two tables are adapted from the USAID Post-Evaluation Plan templates for this exercise, and the following are the principles (The P6) the Mission needs to follow to execute the Post-Evaluation Action Plan. PRINCIPLE ONE: After an evaluation is completed, the COR or the Program Office at large and the Evaluation Contractor (as appropriate) shall seek the overall impression of the relevant Office/s within the Mission about the quality of the evaluation. PRINCIPLE TWO: After the evaluation report is approved, the evaluation COR or the Program Office at large and the Evaluation Contractor (as appropriate) shall sit together with the relevant office/s within the Mission to identify relevant/doable and non-doable recommendations (this is a completion to Table 1). PRINCIPLE THREE: The evaluation COR or the Program Office at large and the Evaluation Contractor (as appropriate) shall discuss with the relevant Office/s within the Mission to plan how evaluation findings/recommendations shall be used, the responsible person to follow up on that, and the timeline for adoption (this is a partial completion to Table 2.) PRINCIPLE FOUR: Actions completed under PRINCIPLES ONE, TWO, and THREE shall be part of the Annexes of the Final Evaluation Report by the Evaluation Contractor. PRINCIPLE FIVE: Tables 1 and 2 shall be shared, for instance as google sheet, with the wider Mission community to promote transparency and learning. PRINCIPLE SIX: The evaluation COR or the Program Office at large and the Evaluation Contractor (as appropriate) shall keep track of the plan and update the status on a quarterly basis by following up on the actions with the technical team (this is a completion to Table 2). Table ____: Post evaluation Management Evaluation Title: Youth Cohort Study Evaluation Completion Date: December 2018 Overall Impression about the Evaluation: Evaluation Recommendation USAID Response: Accepted / Partially Accepted / Rejected If not accepted or partially accepted, provide more information why it was not ACCEPTED or PARTIALLY ACCEPTED General Recommendations for YP Activity Strengthen coaching and mentoring.  Hire more facilitators.  Rebalance workload and responsibilities of facilitators.  Identify and train more mentors.  Educate participants and prospective mentors to maximize benefits to the participant.  Publicly recognize and honor facilitators and mentors so that they are considered critically important in guiding beneficiaries in achieving their employment and income targets.   Increase the involvement of parents and guardians to ensure post-training support for participants.  Educate parents on how to enhance the support they can provide to their wards.  Periodically share information about YP and the progress of their wards.  Involve parents in designing the essential elements of training so that there is ownership of training, leading to parental support for youth after training.   Diversify access to new market opportunities for youth.  Maintain and regularly update an inventory of market opportunities in   both traditional and new areas, such as in the agriculture supply chain space, and share widely with beneficiaries, enabling them to avail the opportunities .  Establish linkages with ongoing government and donor -funded programs and activities and help beneficiaries get greater access to nontraditional opportunities .  Establish linkages with the private sector to help beneficiaries get access to training and internship opportunities . Ensure personal development plans developed by the beneficiaries are viable .  Review the personal development plans with an eye to implementation .  To the extent possible, include a cost -benefit analysis of the proposed plan to ensure there is full information available to the beneficiary .   Provide greater support and training on the usefulness of mentoring .  Educate parents and communities to support mentoring by community members .  Educate youth on the advantages of mentoring by those from within the community .  Support mentors with training on how to provide mentoring .   Help beneficiaries obtain financing .  Educate beneficiaries about the various public and private sources and schemes of financing, eligibility requirements, and how and where to apply .  Continue advocacy for public and private financing programs and an easing of the collateral requirements for YP participants .    Provide specialized training and counseling to beneficiaries to avail group loan opportunities . Strengthen job placement services .  Collect and share information about more and better job opportunities, map participants’ skills with jobs, and advertise the availability of skills to potential employers .  Strengthen relationship s with private and government employers about current and future job opportunities, and align the training provided by YP .   Strengthen linkages with government and private programs and donor - funded initiatives that promote employment .  Identify government programs that promote self- and wage employment ; strengthen linkages with relevant public officials ; and make connections for beneficiaries, especially females, to access these initiatives .  Identify donor -funded initiatives that promote self- and wage employment and strengthen linkages with relevant staff running these initiatives to help YP participants find better wage or self-employment opportunities at the end of their training .  Identify private companies and strengthen linkages with these companies and their appointed staff to help YP participants find better self- or wage employment opportunities at these companies at the end of their training .   Continue evaluating YP. One year is a very short time to study YP’s long -term effect s. The evaluation team recommends the following :  Carry out a follow -up study of the YCS cohort to understand long -term outcomes .  Increase focus on the mentorship component in the YP final evaluation .  Since YP is beginning to change its approach to coaching from life -of￾project coaching to coaching limited to six months, include a comparative study of participants under the two models in the final evaluation .  Evaluate components such as YES groups that are still in a nascent stage . Specific Recommendations for Female Participants ( While all the above recommendations apply for both male and female participants, some specific recommendations for female participants are listed below. ) Improve access for female s to new market opportunities .  Establish linkages with ongoing government and donor -funded programs and activities and help beneficiaries get greater access to nontraditional opportunities, especially for females .  Establish linkages with the private sector to help female beneficiaries obtain access to training and internship opportunities .   Foster a culture supportive of female employment in general, and in nontraditional roles in particular .  Educate parents on gender equality and the importance of encouraging girls to explore nontraditional avenues of work .  Work with the Bureau of Women’s and Children’s Affairs to ensure training programs support female youth .    Foster community support for female youth in nontraditional employment/roles . Train on group work as a means of improving access to loans/financing .  As loans and financing are more readily available for youth organized in groups, there is a need to trai n all participants on working in groups and in building trust. This is particularly important for female youth.   Recommendations for Future Programs and Activities Future programs and activities YP has succeeded in changing participants’ behavior and attitudes toward employmen t. Although many of the above recommendations will carry over, recommendations for new activities building upon YP include the following :  Set aside a dedicated sum for the implementers to use as a revolving loan or start -up financing for all participants, with special consideration for female participants .  Offer hard -skills training to a larger proportion of participants with a give - back -to -community provision to prevent migration of youth to greener pastures .  Establish memoranda of understanding (MOUs ) between the government and private sector that incentivize the private sector to offer practical training to participants and to hire locally .  In collaboration with the Bureau of Wom e n’s and Children’s Affairs, develop programs to incentivize hiring of female employees by the public and private sector and to ensure existing programs and regulations are implemented.  Initiate/include an awareness campaign and community-based training on equal opportunities for female participants and to encourage female participation in nontraditional employment.  Enhance the capacity of the government to support creation of local job opportunities through systems strengthening and by providing exposure to regional and international best practices. Table 21: Post-evaluation action plan Evaluation Title: Youth Cohort Study Evaluation Completion Date: December 2018 Recommendations fully or partially accepted Adoption plan Follow-up USAID Focal Office USAID Focal Person and Title How will the Office use/adopt the recommendation? When is the expected time for use/adoption? Status of Actions (as of date) Any remarks 1. 2. 3. 4. 5. 6. 7.