February 2018 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by the Ethiopia Performance Monitoring and Evaluation Service (EPMES) of Social Impact, Inc., the USAID/Ethiopia Contractor. EVALUATION Building the Potential of Youth Activity Youth Cohort Study – Midline Report EVALUATION Building the Potential of Youth Activity Youth Cohort Study – Midline Report A cohort study to measure the results of an Activity that empowered youth through workforce development and livelihood support February 2018 Evaluation Mechanism Number: AID-663-C-16-00010-EPMES Ethiopia Performance Monitoring and Evaluation Service (EPMES) Activity for USAID/Ethiopia Cover photo by Social Impact, Inc. ABSTRACT Ethiopia’s workforce is young, with 29% of the population aged between 15-29 years. Employment options for the youth are limited, and the majority engages in unpaid work or are underemployed, particularly those with low levels of literacy and women. This multi-round Youth Cohort Study, conducted as a part of the evaluation of the Building the Potential of Youth (Youth Potential) Activity under the broader Ethiopia Performance Monitoring and Evaluation Service (EPMES) Activity, measures Activity outcomes over a twelve-month period from a subset of beneficiaries of the Youth Potential Activity from four Ethiopian regions. Methods involve qualitative and quantitative approaches to answer questions on the extent of changes in participants’ income, employment work-readiness skills, knowledge and access to employment/entrepreneurial resources, and functional literacy and numeracy skills. The study assesses the status of beneficiaries of the Youth Potential Activity at six months post￾enrollment in the Activity. Using a panel design, the study follows individuals surveyed at baseline. A total of 533 Youth Potential Activity participants (92% of those surveyed at baseline) completed the survey. In addition, 35 focus group discussions (FGDs) and 48 key informant interviews (KIIs) and informational meetings were conducted. Findings show that after participating in the Youth Potential Activity, 31% of the respondents (165 individuals) gained new employment in the six-month period since the baseline. Of those employed, a majority (88%) were self-employed, with agriculture and trading as the primary occupations. According to data gathered through self-reporting, Youth Potential training has led to a change in attitudes towards work and self-employment among the participants and has resulted in greater productivity and higher income. It has also helped participants generally feel less stressed and less anxious, increased their confidence in their abilities, strengthened their work-readiness skills, and enhanced the general awareness of available resources for those seeking employment in Ethiopia. All in all, short-term results of the Youth Potential Activities have been positive. iv Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline ACKNOWLEDGEMENTS The Youth Cohort Midline Study is the result of the cooperation of several organizations and individuals, and the team would like to take this opportunity to thank them all. We would like to start by expressing our appreciation of USAID/Ethiopia and the implementing team for Building the Potential of Youth (Youth Potential) Activity for helping facilitate the data collection. As with the baseline study, Save the Children (SCI) and its partners implementing the Youth Potential Activity enabled the study team to access the participants as well as the implementing team on the ground and conduct meetings with the government officials at the regional, woreda, and the kebele levels. Our thanks to the focus group participants, the beneficiaries of the Activity and their guardians, the Technical, Vocational, and Educational Training (TVET) personnel and the local entrepreneurs that are providing training to the Activity participants, and all those who took the time out of their busy schedules to sit down with us for interviews and informational meetings and shared their thoughts and opinions about the Youth Potential Activity. The Social Impact, Inc. (SI) team in Ethiopia and at the headquarters office has been a tremendous support, stepping in as needed with solutions to any and every problem. The team of surveyors from the Sub-Saharan Africa Research Center (SART) has proved to be invaluable in collecting quality data. The members of the study team worked together as a whole and their knowledge of the Ethiopian context and the various technical insights they provided have made the data collected that much better and more robust. Finally, we would like to acknowledge the drivers of the team vehicles who, despite the very rough terrain, drove us patiently to various corners of Amhara, Afar, Oromia, and Tigray regions. Without the help and valuable contribution of each group of people mentioned here, and several whom we may have missed, it would not have been possible to complete this exercise and the writing of this report. On behalf of the EPMES YCS team Deepika Chawla (Team Leader) EPMES YCS Team SI staff Consultants Worku Ambelu Ermias Assefa Hika Alemu Estera Barbarasa Basab Dasgupta Deepika Chawla v Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline CONTENTS Abstract......................................................................................................................................... iii Acknowledgements...................................................................................................................... iv Contents......................................................................................................................................... v Lists of Tables and Figures ..........................................................................................................vii Acronyms.....................................................................................................................................viii Geographic Coverage .................................................................................................................. ix Executive Summary...................................................................................................................... x Introduction................................................................................................................................... 1 Description of USAID’s Youth Potential Activity ................................................................................................. 2 Description of this Evaluation .................................................................................................................................... 3 Organization of the Report ........................................................................................................................................ 3 Study Objectives and Approach .................................................................................................. 4 Activity Description...................................................................................................................................................... 4 Study Objectives............................................................................................................................................................ 4 Research Questions...................................................................................................................................................... 4 Relevance of the Study................................................................................................................................................. 4 Study Approach ............................................................................................................................................................. 5 Limitations of the Study............................................................................................................................................... 7 Main Findings ................................................................................................................................. 9 Training, Coaching, and Mentoring......................................................................................................................... 12 Employment Status of Study Participants.............................................................................................................. 16 Income, Expenditures And Assets .......................................................................................................................... 26 Change in Health Outcomes.................................................................................................................................... 32 Self-Perception of Work Readiness Skills ............................................................................................................. 33 Knowledge of and Access to Employment/Entrepreneurial Resources......................................................... 36 Functional Literacy and Numeracy Skills............................................................................................................... 39 Conclusions.................................................................................................................................. 41 Challenges..................................................................................................................................................................... 45 Recommendations....................................................................................................................................................... 46 Annexes........................................................................................................................................ 49 Annex I: Youth Potential Activity Results Framework ........................................................... 50 Annex II: YCS Statement of Work ........................................................................................... 51 Annex III: Baseline Study – Executive Summary ..................................................................... 70 Annex IV: List of Research Questions and Indicators ............................................................. 78 Annex V: List of People Interviewed and Informational Meetings......................................... 82 Annex VI: Sources of Data ......................................................................................................... 84 Annex VII: Data Collection Instruments .................................................................................. 85 vi Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Annex VIII: Data Tables and Graphs....................................................................................... 108 Annex IX: Study Team Disclosures of Conflict of Interest................................................... 113 Annex X: Study Team Profiles ................................................................................................ 119 vii Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline LISTS OF TABLES AND FIGURES Table 1 Distribution of survey & focus group participants and KIIs, by region.................................................. 7 Table 2 Distribution of respondents by sex and region (%) ................................................................................... 9 Table 3 Change in employment status between midline and baseline................................................................ 17 Table 4 Change in employment status of respondents: baseline versus midline (%, n=533)........................ 18 Table 5 Changes in employment status of YCS respondents: baseline vs midline (%)................................... 20 Table 6 Examples of self-employment opportunities available per key informants, by region ..................... 21 Table 7 Youth engagement in agriculture and livestock activities (self-reported responses; %) ................. 22 Table 8 Youth opinions on whether agriculture is a good opportunity for young people (youth FGDs) . 24 Table 9 Distribution of those reporting incurring expenditures by category of expenditures.................... 28 Table 10 Distribution of individuals reporting asset-holding and comparison with baseline (%)................. 29 Table 11 Change in health status and eating habits (self-reports by participants............................................ 33 Table 12 Participants’ self-perception of Work Readiness Skills......................................................................... 35 Table 13 Changes in skills and behavior post participation in Youth Potential (self-perception, %)........... 36 Table 14 Change in level of functional literacy and numeracy, from baseline .................................................. 40 Figure 1 Demographic characteristics of participants by region (%)................................................................... xii Figure 2 Current employment status of youth cohort study respondents at midline, by region ...............xiv Figure 3 Change in income over the past six months by region, sex and level of education (%)................ xv Figure 4 Self-ratings on measures of knowledge of resources (%)..................................................................... xvi Figure 5 Level of functional literacy and numeracy by region (%).................................................................... xviii Figure 6 Demographic characteristics of midline survey participants, by region (%)...................................... 10 Figure 7 Distribution of participants, by region and level of education (%)...................................................... 11 Figure 8 Distribution of participants, by region and level of literacy and numeracy (%)................................ 12 Figure 9 Participation in training programs, by region and level of education (%)........................................... 14 Figure 10 Current employment status of YCS respondents at midline, by region.......................................... 18 Figure 11 Change in sector of employment at midline compared to baseline (n=230) ................................. 19 Figure 12 Level of satisfaction with current job, by region (%)............................................................................ 25 Figure 13 Participants’ perceptions of main reason for unemployment, by region (%).................................. 26 Figure 14 Change in income over the past six months by region and sex (%)................................................. 27 Figure 15 Average monthly expenditure for select group of participants across regions............................. 30 Figure 16 Provider where health care sought (self-reports by participants) (%)............................................. 32 Figure 17 Self-ratings on measures of work readiness........................................................................................... 34 Figure 18 Self-ratings on measures of knowledge of resources........................................................................... 34 Figure 19 Avenues for finding employment/new employment, by region (%).................................................. 37 Figure 20 Information required for creating self-employment, by region (%).................................................. 37 Figure 21 Level of functional literacy and numeracy, by region, sex and age (%)............................................ 39 viii Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline ACRONYMS BYOB Be Your Own Boss COR Contracting Officer’s Representative EDC Education Development Center EPMES Ethiopia Performance Monitoring and Evaluation Service FCE Facilitator for Change Ethiopia FGD Focus Group Discussion GDP Gross domestic product GoE Government of Ethiopia GTP Growth and Transformation Plan HUNDEE HUNDEE-Oromo Grassroots Development Initiative ICT Information and Communications Technology KII Key Informant Interview MOU Memorandum of Understanding NGO Non-governmental organization PADet Professional Alliance for Development in Ethiopia PYD Positive Youth Development REST Relief Society of Tigray SART Sub Saharan Africa Research Center SCI Save the Children SES Socio-Economic Status SI Social Impact, Inc. SNNPR Southern Nations, Nationalities, and Peoples' Region TTI Technical Training Institutions TVET Technical, Vocational, and Educational Training USAID United States Agency for International Development USD United States Dollar WBL Work-based learning WRN! Work Ready Now! WR Plus Work Ready Plus YCS Youth Cohort Study YES Youth Economic Strengthening YIA Youth in Action program ix Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline GEOGRAPHIC COVERAGE *Please note that in the SNNPR region, the Youth Potential Activity is no longer active in Hammer and Esirano areas; activities are now being implemented in Hawassa, Zuria and Dawro areas instead. The map above shows both the old and the new woredas. x Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline EXECUTIVE SUMMARY United States Agency for International Development/Ethiopia (USAID/Ethiopia) has contracted with SI to implement the Ethiopia Performance Monitoring and Evaluation (EPMES) Activity. Under EPMES, USAID requested an independent performance evaluation of its Building the Potential of Youth (Youth Potential) Activity. This evaluation involves multiple approaches, including a baseline study, performance monitoring and data systems review, and a multi-round cohort study of beneficiaries focused on a subset of youth participants and geographic implementation areas (henceforth referred to as the Youth Cohort Study). This report provides the midline data for the Youth Cohort Study (YCS). The Youth Potential Activity is a five-year, USD $17.3 million USAID/Ethiopia Activity designed to provide youth (ages 15–29), especially girls and young women, in rural areas and towns with access to workforce development/livelihood support and resources, and tailor to their specific needs and market demand, so that they can: (a) achieve increased income; and (b) strengthen skills, knowledge, and social capital required to achieve long-term economic self-sufficiency. Designed to reach approximately 34,537 unemployed or underemployed rural youth, the Activity is being implemented under a cooperative agreement between USAID and a group headed by Save the Children (SCI).1 Currently, in its fourth year of implementation, the Activity is providing services in 30 woredas in Afar, Amhara, Oromia, Southern Nations, Nationalities, and Peoples' Region (SNNPR), Somali, and Tigray regions of Ethiopia. The overall objective of the YCS is to evaluate whether youth who receive workforce development/livelihood support achieve increased income and strengthened skills, knowledge, and social capital required to achieve economic self-sufficiency over the longer term. The study will provide the stakeholders, USAID, the Government of Ethiopia (GoE), and the implementers of Youth Potential with information about the Activity and progress towards near-term outputs and outcomes. In addition, it will demonstrate long-term contributions of Youth Potential initiatives on a cohort of youth and highlight major strengths and weaknesses of the Youth Potential’s initiatives to enable the stakeholders to make mid-course adjustments as necessary to the Activity as well as inform the design of similar programs in the future. Approach The YCS is following a subset of youth from Afar, Amhara, Oromia, and Tigray regions over a period of 12 months to better understand participant outcomes and the extent to which the Youth Potential Activity has improved the employment status and/or income level of youth participating in the Activity. It seeks to address the following broad questions2: 1. To what extent has the income and employment status of study participants changed between enrollment in the Youth Potential Activity and the twelve months following enrollment? Does increased income lead to improved health outcomes among study participants? 2. To what extent has participation in the Youth Potential Activity improved the work-readiness skills of study participants? 3. To what extent has participation in the Youth Potential Activity improved knowledge of and access to employment/entrepreneurial resources for study participants? 4. To what extent has participation in the Youth Potential Activity improved functional literacy and numeracy skills of study participants? 1 Cooperative Agreement # AID-663-A-15-00006; the implementation team includes Save the Children (SCI) and its partners Education Development Center, Inc. (EDC), HUNDEE-Oromo Grassroots Development Initiative (HUNDEE), Professional Alliance for Development in Ethiopia (PADet), Relief Society of Tigray (REST) and Facilitator for Change Ethiopia (FCE). 2 From the EPMES statement of work xi Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline 5. How do the outcomes addressed in the questions above vary by participant characteristics (e.g., sex, age, ethnic/linguistic group, socio-economic status (SES), marital status, parenting status) and by service delivery partner? This midline study provides information on the status of each of the above among the study participants at six months’ post enrollment in the Youth Potential Activity. These changes will be compared to similar data collected at baseline to assess interim change and establish impact. The focus of this study is to track changes on Activity beneficiaries included in the cohort and to document lessons learned for future similar Activity designs. Since Youth Potential covers a selection of eligible youth in specific locations/kebeles within the woreda and is implemented over an extended period, the study employs a panel design to demonstrate whether the Youth Potential Activity led to expected changes in outcomes or impacts. In a panel design, participants from whom data are collected over time serve as their own controls, since characteristics of an individual observed earlier can be controlled for when analyzing for changes in the expected outcomes. Midline data collected on a select group of beneficiaries will help assess the effectiveness of the Activity as well as the interim outputs and outcomes that define the success of the Activity. To respond to the research questions listed above, data are being collected on several qualitative and quantitative indicators and sub-indicators that together will inform the results of the training programs as well as future possible modifications to the Activity. To capture changes over the short and medium terms and attribute them to the Youth Potential Activity, data are being collected from a select group of participants (cohort) who have recently enrolled in the Activity at three points: (i) baseline, in the month of enrollment in the Activity (completed March-June); (ii) mid-line, at six months from the point of enrollment (the current phase); and (iii) end-line, at twelve months’ post-enrollment in the Youth Potential Activity. This will be accompanied with periodic (monthly) collection of income and expenditure data for a small purposive sub-set of the cohort. The midline study was conducted during the period September to December 2017 using qualitative and quantitative methods. Primary data were collected from a survey of 582 Youth Potential participants from 13 woredas in the four regions of Afar, Amhara, Oromia, and Tigray who completed the baseline survey. 3 Of those surveyed, 533 participants responded, representing a response rate of 92%.4 In parallel, qualitative data were collected by the study team through 26 focus groups with 239 participants (50% female), 9 focus groups conducted with 50 parents and guardians of participants conducted in a purposive sub-sample of eight woredas in the four regions (two woredas in each region), and 48 KIIs and informational meetings held with stakeholders, including Government counterparts; Technical, Vocational, and Educational Training (TVET) and Technical Training Institutions (TTI) administrators and Activity staff at the national, regional, woreda, and kebele levels. The Activity staff interviewed included the M&E officers, the woreda officers, and the facilitators responsible for delivery of the training as well as coaching and arranging for the mentoring. In collaboration with the SCI team, secondary data was collected through a review of Activity documents, websites, and administrative records. Data from the midline study were analyzed together with the income and expenditure data collected periodically from approximately 80 members (20 per region) of the cohort. 3 The baseline study was conducted during the period March to June 2017 using a mixture of qualitative and quantitative methods. Primary data were collected from a survey of A survey of 670 Youth Potential participants from 13 woredas in the four regions of Afar, Amhara, Oromia, and Tigray yielded primary data on 582 participants (representing a response rate of 87%). 4 Eight percent of the respondents were either not available as they had moved out of the region for purposes of work (6%), due to marriage or for personal reasons (less than 1%), or chose to not respond (1.5%) xii Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Main Findings As stated earlier, data were collected from 92% of the respondents; sixty-seven percent of the cohort is between 19 and 24 years of age, followed by 20% that is over 25 and 13% that is between 15 and 18 years of age. Within age groups, while the proportion of male and female participants is about the same in the over 25 years age group, there are more male participants than female in the 19-24 years age group (71% and 62%, respectively), and more females than males in the 15-18 age group (23% female, 6% male). Most (82%) of the respondents have completed at least grade five in school, with 57% having completed secondary school and beyond. Figure 1 presents these details. Only four cases across the four regions reported having a disability. Over half of the respondents (52%) reported that they were not married and a similar proportion reported living with their parents. The distribution of respondents by sex and age group across regions for the midline study is consistent with the baseline. Figure 1 Demographic characteristics of participants by region (%) Source of data: YCS Midline Survey Training Received Findings from the midline qualitative and quantitative data collection show that at a minimum, all youth from the March/April cohort completed Work Ready Now (WRN!) or Work Ready Plus (WR Plus) training components offered by the Youth Potential Activity. While all participants in Amhara, Oromia and Tigray attended the WRN! training, in Afar 56% of the participants attended WRN!, and the remaining 44% attended the WR Plus training. Depending on the availability of the trainings, participants have also attended other follow-on trainings, such as Be your Own Boss (BYOB) and Positive Youth Development (PYD). A small proportion of the participants have also received training in technical skills provided by external providers under the Youth Potential Activity, (e.g., TVET, TTI or private providers). The most popular areas for training were agriculture and sheep fattening in Afar, Amhara and Oromia regions, followed by poultry farming and trading in Amhara and Oromia. In Tigray, however, fewer participants benefitted from this program and, for those that participated in it, the areas xiii Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline of choice were skills for hotel and tourism industry, and hairdressing. Some of the other options selected by participants included baking and running a bakery; carpentry, carving, or woodwork; mechanics and metal work; construction work (brick/stone laying, building, roads, etc.); running coffee and tea shops; and vegetable farming. A majority of those attending BYOB (81%) and technical training had completed grade 9 or higher levels of education. While training in agriculture and animal husbandry were popular with participants across all levels of education, there was a slight preference for those with lower levels of education to opt for training in skills such as tailoring, while those with higher levels of education selected training in bakery, marketing, tourism industry and hairdressing. To support the participants in the use of their training, they are provided with coaching from the facilitators and mentoring by experts in the field. While the mentoring program is just starting in all four of the regions under consideration and there have only been a few instances of participants engaging in job shadowing activities, the coaching is provided in individual or group settings on a regular basis. The coaching provides the participants with a support group as well as a person they can talk to about their issues and concerns. These sessions also serve the important purpose of holding the participants accountable for the implementation of plans they have developed, thereby helping them to become more organized, more productive, as well as to save more money. The Memorandums of Understanding (MOU) between Youth Potential Activity and regional employers were also beginning to result in opportunities for the Youth Potential participants, increasing their work readiness. Employment Status The methodology used in this study allows the evaluation team to follow the Youth Potential participants who responded to the baseline survey through the midline and all the way to the end-line survey. Prior to participating in the Youth Potential Activity, almost half of the participants were unemployed and dependent on others for their sustenance. Six months after participation in the WRN! training offered by Youth Potential, of the 533 Youth Potential beneficiaries that completed the survey, almost three quarters of the participants (74%) are employed, including 230 that remained employed from baseline and 165 newly employed in the six months since baseline. While there was some movement between those employed at baseline losing their jobs at midline (n=28), for the most part, the movement was in the direction of the unemployed- beneficiaries reporting they were now employed. Two thirds of all survey respondents were self-employed and only 9% were engaged in wage employment. The most significant gains in employment were seen in Afar and Tigray regions, with most of the survey respondents (90%) from the Tigray region reporting they were employed. While there were regional differences, more male (61%) than female (38%) survey respondents reported being employed, more males (52%) reported managing self-owned businesses compared to females (44%), and fewer males were unemployed as compared with females. More of those engaged in wage employment tended to have completed secondary school and beyond, with the self-employed equally spread across all levels of education. Those engaged in wage employment were working as security guards, Bajaj drivers, TVET assistants, schoolteachers, etc. For the self-employed, agriculture (growing crops) and livestock (sheep and goat rearing and fattening) remain the primary sources of employment, followed by trading. It is interesting to note that while 23% of the Activity participants were unemployed overall, an examination of the distribution by level of education revealed more of the college educated (45%) were unemployed. Survey respondents, especially those from Afar and Tigray, were mostly satisfied with their employment status. The training has helped bring about a change in attitude towards work among the beneficiaries of the Youth Potential Activity, with many preferring self-employment over wage employment. The training has helped them to value and respect their own work “no matter how lowly,” and to improve their work ethic. They now recognize that any work that helped them earn an income was employment. The community mapping exercise conducted as a part of the WRN! training (including WR Plus) helped the beneficiaries identify new business ideas that they could implement in their own communities. In xiv Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline addition, WRN! provided them with the skills needed to manage a business and handle customers. One recurrent theme heard in all focus groups across the four regions was that the training had made them realize the importance of “starting with something small, and to grow their Activity/business gradually,” resulting in many of them feeling encouraged to start their own business or to work more seriously on their existing work. Participants in the focus groups reported mixed feelings regarding work in the agriculture sector. Many of them reported that agriculture was a good opportunity and a good source of income, that “it’s the backbone of the Ethiopian economy” and that it gave them the opportunity to stay in their rural community. However, others felt that the work was too hard and challenging and that the constraints such as expensive inputs and frequent droughts made the work unattractive. Lack of access to finance, inadequate education and training, not enough jobs, and a mismatch between training and job requirements were stated as the main reasons for unemployment. For those engaged in self￾employment, both male and female participants reported several hurdles in setting up their own business, including difficulty in obtaining working capital and financing, lack of land and working space. Figure 2 Current employment status of youth cohort study respondents at midline, by region Source of data: YCS Midline Survey Income Level and Expenditures Incurred Almost two thirds of the survey respondents (61%) reported an increase in income over the past six months, with the largest changes reported by participants in Tigray (Figure 3). The only exception to this were the participants with a college education, where only 45% of the participants reported their income had increased, with the majority reporting there was no change. More men than women reported that their income had increased over the past six months, a time line consistent with enrollment in the Youth Potential Activity and WRN! training. Even though most beneficiaries had increased income, the increase did not necessarily result in a change of life style. They said that because of the additional income they may spend a little more on entertainment or clothes as compared with six months ago, but in most cases, the additional funds went towards investment in business, payment of rent, purchase of food, support to other family members and putting money away for savings. As a majority of the participants are self-employed and not engaged in regular wage-based employment (with a salary), it is difficult to gather and verify income data. As such, data on expenditures incurred, investment in assets and the amount put away as savings was used as a proxy for change in level of income. A comparison of the expenditures incurred at midline with the baseline suggests that the beneficiaries are spending more overall and significantly more on investing in own business, supporting family members and education. The mean overall expenditure at midline ($228 USD/5227 Birr) was significantly higher (p<.001) as compared with the expenditures at baseline ($154 USD/3525 Birr) (see xv Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Table 9).5 Significantly more participants at midline reported purchasing mobile phones, motorcycles, land or animals as compared with baseline. Several also indicated that they were spending more on food and eating better. As a result of the financial management skills learned through the WRN! program, the beneficiaries reported that they now spend money according to their plan and over half (56%) put away money in savings on a regular basis. The follow-on coaching as well as strategies such as participation in lottery￾based social groups have helped to reinforce the habit of saving money. An examination of the relationship between income and demographic characteristics and the employment status and the sector of employment suggest that male participants are almost 1.4 times more likely to report that their income had increased relative to female participants. Similarly, the odds of reporting higher income peak for those who are married (95%), have larger families (35%), and are in the 19-24 years’ age group. However, none of these by themselves are likely to make a significant difference. While an individual’s employment status (self-employed, wage employed, unemployed, etc.) did not have a significant effect on their level of income, an individual’s ability to obtain a loan is weakly related to their change in income. The regression analysis shows that the likelihood of receiving a higher income is more than double (2.19) when an individual is able to obtain loans. (see Annex 8, Table 3) Figure 3 Change in income over the past six months by region, sex and level of education (%) Source of data: YCS Midline Survey 5 Using exchange rate of $1= 22.9 Birr (the value used at baseline, to allow for meaningful comparisons) xvi Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Change in Health Outcomes Health care is a discretionary expenditure and one that is easily overlooked, except in an emergency. Depending on the severity of the illness, most of the survey respondents reported visiting the local health clinic, a health provider or a nurse, and going to the hospital in the city in a case of severe illness or an emergency. A few also reported going to traditional healers or to “holy water.” Participants reported that after participation in the WRN! training, and because of the ongoing group and individual follow-on coaching, they had “learned to be more patient.” were “less quick to anger” and tried to put the customer’s perspective first. A clear majority of survey respondents (between 80% and 90%) agreed or strongly agreed that compared with six months ago, they were working harder, were generally happier, were less stressed, less anxious, less angry, and took better care of their health. A somewhat smaller proportion of the respondents agreed that they had fewer headaches (71%), ate better food, i.e., more vegetables and larger quantities (78%), or spent more money on food (61%). Several participants reported that they had decreased their consumption of alcohol, tobacco and “chat” as a result of the need to live within their budget as per the personal development plan developed because of the WRN! training, a finding corroborated by parents and guardians in a focus group. However, an equal number said that there was no change in their food habits, and that they were just saving more. Self-Perception of Work Readiness Skills Participants in focus groups indicated a strong preference for self-employment over salaried employment, especially with the government. Overall, they had a high opinion of their own readiness to work, rating themselves highly (good or very good) in terms of their communication skills, work habits and conduct, ability to build self-confidence, team work, ability to set goals and objectives, and saving habits. The respondents also rated themselves highly on skills needed for start-up of self-employment such as knowledge of financial resources, business start-up and management skills, obtaining licenses for a business, etc., rating themselves as “good” or “very good” on most items on a five-point scale ranging from very poor to very good. The only exceptions were the participants’ self-perceptions of job search skills, such as writing CVs, knowledge of job search resources and knowledge of ways and means of accessing resources (finance, land, inputs) on which the survey respondents did not rate themselves very highly (more of neutral and good ratings). Figure 4 presents the details. Figure 4 Self-ratings on measures of knowledge of resources (%) Source of data: YCS Midline Survey xvii Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Participants reported that the training provided them with the skills, knowledge, and motivation to start their own business or improve their existing business, and the skills to improve their savings habits and better manage their time and finances. They went on to say that their personal development plans developed during the training had helped them articulate their goals and targets, and by following the plan, several participants have been able to expand the scope of their ongoing activities. In addition, the interpersonal skills and tools for anger and frustration management as well as the recognition of importance of time management have also helped the participants to grow. They reported having better time and financial management skills, and more confidence in their abilities to plan and manage their work. They were working longer hours and more days every week compared with six months ago, resulting in increased income and more productivity. Access to Employment/Entrepreneurial Resources Six months’ post enrollment in Youth Potential, while there has only been a 3 percentage points change (p<.05) in the number of those engaged in wage-based employment, the situation has changed dramatically for those unemployed and self-employed. The number of those unemployed has reduced significantly (change of 23 percentage points from 46% to 23%, p<.001). Forty four percent of all survey respondents (49% male and 36% female) reported having tried to find employment in the past six months. While this proportion is very similar to the baseline, the major difference is that over half of this group (53%) are currently self-employed compared to a quarter at baseline, and more of them are female. The most common avenues for looking for new opportunities were asking relatives and friends for help followed by participation in job fairs, registration with public employment offices and checking directly at worksites. Applying directly to employers was also common. Female participants reported being more likely to use avenues such as job fairs, seeking help from friends, and registration with the public employment office (in that order), while male participants preferred seeking the assistance of friends, followed by job fairs. Participants were also knowledgeable of the various requirements for self￾employment. Respondents across all regions, especially in Amhara, were aware of the need for accessing finance for creating self-employment as well as the minimum requirements for seeking a loan. The need to learn new skills ranked at par with the need to locate land, space and equipment, as well as seeking assistance of family, friends and colleagues. The facilitators supported the individuals and/or groups with completing the loan applications, getting the youth introductions to the kebele and woreda level government officials for letters of support, and in general, helped troubleshoot any process-related concerns. Overall, the Youth Potential participants feel more empowered in their search for employment and in creating self-employment, and report being able to get better and more complete information because of their training. Functional Literacy and Numeracy Skills While not a focus of the Youth Potential Activity, an expected benefit of enrolling in the Activity is improved levels of functional literacy and numeracy. At baseline, except for participants from Chifra in Afar region, most participants, male and female, reported being able to read and write fluently, and most had more than just a functional level of literacy. These self-reports were confirmed using simple tests of literacy and numeracy. At six months’ post enrollment, most beneficiaries said that there was no change in their literacy and numeracy skills. Multiple young men and women across all four regions stated, “I knew how to read and write before joining the Potential training, and there has been no change in my ability.” At the same time, they also reported that the use of reading and writing skills during training and in the preparation of their personal development plans and business plans has reinforced these abilities. Participants also reported that managing their finances, developing budgets and monitoring their inputs and profits has also helped reinforce their ability to do simple addition and subtraction. They were also more aware of the need for literacy and numeracy and are making efforts to improve their own skills by making a conscious effort to read and write more, and were also looking to bring in xviii Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline someone with the necessary skills to help them with their accounts. Figure 5 contains the details by region. Figure 5 Level of functional literacy and numeracy by region (%) Source of data: YCS Midline Survey Next Steps This midline study will be followed by the end-line study that will assess the changes on the outcomes of the participants at a point 12-months after enrollment in the Activity. Using the panel design (employed for both the baseline and midline), the study will use longitudinal data collected by following the cohort for a 12-month period. The key feature of such a panel design is that it will collect data on the same variables from the same sample at different points in time. While more complicated and difficult to carry out than simple cross-sectional studies, a panel offers a robust design to evaluate changes at the end of the study period and the contribution of the Youth Potential Activity to these changes. Challenges The team faced some challenges related to data collection as well as noted the challenges to the implementation and sustainability of the initiatives. Challenges related to data collection include: (i) non-availability of data from SCI on participants in the March/April cohort included in the Youth Cohort Study, including data on participation in training components/programs, coaching and mentoring received, as well as any drop-outs from the training programs; (ii) difficulty in obtaining information on individual income and expenditures, since most participants live with their families; and (iii) difficulty in finding all survey respondents from the baseline study, despite best efforts. Solutions for these challenges were found using alternate measures, such as using data from administrative reports to replace enrollment data and data on trainings, using proxy data on ownership of items to determine income, building in the possibility of attrition into the sample size, and so on – but the challenges and limitations remain. Challenges related to implementation of the Youth Potential Activity include: (i) rapid staff turnover, which results in lack of continuity with post-training follow-up to support use of training; (ii) overburdening of facilitators with the provision of training and follow-up coaching, which affects the effectiveness of follow-up; (iii) difficulties faced by Youth Potential Activity with ensuring participants enrolled in technical training programs paid for by the Activity, complete the training programs and not xix Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline drop out due to family commitments or lack of interest thereby wasting Activity resources ; (iv) managing partner and participant expectations regarding the training, which is important to ensure common understanding of the training; and (v) lack of wage employment and inability of participants to obtain land as well as funding/loans for self-employment, which could potentially defeat the purpose of the Activity. Recommendations This midline study is designed to provide an interim view of any changes from baseline to a point six months’ post participation in the Youth Potential Activity. However, during course of data collection a few recommendations that might result in better outcomes for the participants were noted. These include: (i) relax the savings and collateral requirements associated with obtaining microfinance loans for the Youth Potential participants to facilitate creation of self-employment; (ii) mobilize community resources (freeing up land; loan of equipment, farming tools, etc.) to help mitigate the paucity of land and available spaces for activities such as trading, opening a shop for provision of services, etc., and in the absence of availability of microfinance loans, mobilize alternative financing means such as village and savings loans, and borrowing from family and friends; (iii) increase involvement of parents and guardians in the training provided to the participants so that they are better informed and more willing to support their wards with collateral or, in the absence of external funding, the start-up capital for their wards; (iv) arrange for the provision of adequate coaching and mentoring from successful entrepreneurs in the community to all participants to better support the use of training and enhance the likelihood of success of the Youth Potential program; (v) take a closer look at the Tigray model of organizing Activity participants into groups and perhaps expand it to all regions to make it easier for participants to obtain start-up capital; (vi) explore new market opportunities across the agriculture value chains as well as the option of multiple sources of income (“mixed livelihoods”); (vii) take steps to reduce the level of implementing staff turnover; and (viii) establish linkages to ongoing government and donor funded programs to ensure use of all available resources. Several of these recommendations are systemic and may need to be addressed through joint efforts by all stakeholders, including the implementing partners, the government counterparts, the funding organization as well as the private sector. 1 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline INTRODUCTION Ethiopia, a large, landlocked, diverse country, is home to sub-Saharan Africa’s second largest population—about 94 million people—the vast majority of whom live in rural areas.6 Ethiopia is the 11th poorest country in the world by income per person, with an estimated annual per capita income of $590 in 2015 (World Bank).7 Roughly 30% of Ethiopians live below the poverty line of $1.25 a day and are vulnerable to food insecurity.8 Despite regular cycles of drought in several parts of the country, however, Ethiopia recorded a Gross Domestic Product (GDP) growth rate of 10.5% in 2015 and is one of the fastest-growing economies in the world. In addition, Ethiopia has made tremendous development gains in education, health, and food security over the past decade. To further advance the country’s progress and support continued growth of the economy and poverty reduction, the GoE has committed itself to a five-year Growth and Transformation Plan (GTP), which includes sustainably improving rural livelihoods and national food security. The first five-year plan (GTP I) was implemented from 2010/11 to 2014/15, a period that saw a steady increase in GDP (which grew at 10.5% in 2015), falling rates of unemployment, and transition of the labor force from agriculture to construction and services. Building upon the success of GTP I, GoE is currently implementing GTP 2, covering the period 2015/16 to 2019/20, and is mobilizing internal and external resources through bilateral and multilateral cooperation. Sustaining the growth rates realized during GTP 1 and ensuring that the fruits of the growth reach the poor and vulnerable sections of the society, however, require that all segments of the population are included in the growth strategies. Ethiopia faces many challenges in this regard. The workforce in Ethiopia is young, and approximately 29% of the population of Ethiopia is between 15 and 29 years of age.9 However, literacy rates among the youth are low (47% for women and 63% for men) and are even lower if the entire adult population over 15 years of age is included (28% for women, 49% for men).10 Unsurprisingly, therefore, many of these youths are either limited to unpaid work or are underemployed. The situation is worse for women, one in five of whom is economically inactive. In addition, gender stereotypes and assumptions prevail, where men and women have complementary but distinct roles and tasks in both agriculture and livestock production and marketing, for example men are considered the main breadwinners, farmers and laborers, and women’s farming work is considered informal and part of the private sphere. Women and girls are also responsible for all domestic and reproductive duties, which restricts their capabilities to improve their SES. These gender biases also extend to the control of finances in the home, with men having more control and benefits, accessing nutrition (women eat after men and eat what remains) and health care, and employment opportunities available to women.1112 6 https://www.usaid.gov/ethiopia accessed May 2017 7 http://www.worldbank.org/en/country/ethiopia/overview 8 Ref: USAID ibid 9 http://www.youthpolicy.org/national/Ethiopia_2004_National_Youth_Policy.pdf 10 http://www.uis.unesco.org/Education/Documents/literacy-statistics-trends-1985-2015.pdf 11 African Development Bank. 2004. Ethiopia – Country gender Profile - https://www.afdb.org/fileadmin/uploads/afdb/ Documents/Project-and-Operations/ADBGenderEthiopia.pdf 12 Rubin, Deborah, Cristina Manfre and Kara Nichols Barrett. Promoting Gender Equitable Opportunities in Agricultural Value Chains. Washington, DC: U.S. Agency for International Development, 2009. In A Sample Gender Analysis by Catholic Relief Services – Abridged Version accessed at https://www.crs.org/sites/default/files/tools-research/a-sample-gender-analysis.pdf 2 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline DESCRIPTION OF USAID’S YOUTH POTENTIAL ACTIVITY In keeping with its commitment to the GoE to support its efforts to provide health, education, and economic growth opportunities to its people, the United States Agency for International Development (USAID) is funding the Building the Potential of Youth Activity (henceforth referred to as the Youth Potential Activity), with the goal of “helping youth increase their assets and incomes through employment or better employment for them to achieve economic self-sufficiency, while building the capacity of local institutions to continue this work in the long term.”13 The support aims at providing rural youth with literacy, numeracy, and life skills training alongside vocational and entrepreneurship development activities through partnerships with training, small business, and micro-finance institutions. USAID/Ethiopia’s Youth Potential Activity is a five-year,14 USD $17.3 million Activity designed to provide youth (ages 15–29), especially girls and young women, in rural areas and towns with access to workforce development/livelihood support and resources, tailored to their specific needs and market demand, so that they can: (a) achieve increased income; and (b) strengthen skills, knowledge, and social capital required to achieve economic self-sufficiency over the longer term. It is being implemented under a cooperative agreement15 between USAID and a group headed by SCI including Education Development Center, Inc. (EDC), HUNDEE-Oromo Grassroots Development Initiative (HUNDEE), Professional Alliance for Development in Ethiopia (PADet), Relief Society of Tigray (REST) and Facilitator for Change Ethiopia (FCE). It was awarded on January 1, 2015 and is currently in its fourth year of implementation. The Youth Potential Activity has three objectives: (see Annex 1for the Activity results framework): • IR1: Rigorous evidence base developed to inform USAID and its development partners, including youth, GoE and non-governmental organizations (NGO), on appropriate, effective, and sustainable market-driven youth workforce development interventions. • IR 2: Improved youth access to market-driven skills and experiences to support the transition to safe and viable employment. • IR3: Coordinated delivery among local institutions of quality market-driven services connecting underserved youth, especially women, to employment and income opportunities. Designed to reach 34,53716 unemployed or underemployed rural youth in the 15–29 age group and focusing on the most vulnerable sections of the society, particularly women, and those transitioning out of pastoralism, it will achieve these objectives by: • Ensuring that training and service providers consider market relevance and target curricula and programs to meeting the skills needs of youth participants17 • Offering tailored technical and life skills trainings to youth to create more viable livelihood prospects • Expanding approaches and offerings in work-based learning (WBL), including employer visits, organized job shadowing, short-term employment, and internships//WBL opportunities, that will lead to workforce-ready and employable youth with practical experience • Building Youth Service Provider Networks and providing a cost-effective approach to enhance support service access for remote communities and disadvantaged groups 13 Program description as found on USAID/Ethiopia webpage - Ref: https://www.usaid.gov/ethiopia/education 14 Period of implementation January 1, 2015 – December 31, 2019 15 Cooperative Agreement # AID-663-A-15-00006 16 Planned target for life of the Activity 17 The terms “participants” and “beneficiaries” are used interchangeably in this report. 3 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline The Activity is being implemented in six regions of Ethiopia (Afar, Amhara, Oromia, SNNPR, Somali and Tigray) with some very difficult terrain and under very harsh conditions. While the regions (and woredas within the regions) are selected in collaboration with USAID, the Ministry, and other stakeholders, the kebeles from where the participants are drawn are nominated by the woreda administration. As a result, some of the places where the Activity is being implemented have only dirt roads (or no roads). A map showing the location of current activities is placed in the section titled Geographic Coverage. As of the end of June 2017, the Activity had provided life skills training, coaching, and mentoring to over 15,343 youth.18 DESCRIPTION OF THIS EVALUATION United States Agency for International Development/Ethiopia (USAID/Ethiopia) has contracted with SI to implement the EPMES Activity. Under EPMES, USAID requested for an independent performance evaluation of its Youth Potential Activity. This evaluation involves multiple approaches, which include: a baseline study; performance monitoring and data systems review; and a multi-round cohort study of beneficiaries focused on a subset of youth participants and geographic implementation areas (henceforth referred to as the YCS). Annex 2 has the details. The YCS is a multi-round cohort study of beneficiaries of the Youth Potential Activity. Focused on a subset of youth participants and geographic implementation areas, the study is designed to inform the impact of the Youth Potential Activity on the employment status and/or income level over a period of time for this subset of youth as well as to draw some lessons for strengthening the performance of the Activity. Data will be collected at three points – baseline (completed March –June 2017), midline (ongoing) and end-line (planned from March to June 2018). This report presents the results of a midline analysis for the YCS. ORGANIZATION OF THE REPORT The rest of this document is organized as follows. Section 2 discusses the objectives and the approach used for the conduct of the study. Section 3 contains the major findings from the midline study in terms of program implementation and the status of the Youth Potential Activity participants to inform each of the study questions, and Section 4 presents the conclusions and recommendations. This document also has ten annexes, which respectively contain: (i) the Youth Potential Activity results framework; (ii) the statement of work, (iii) summary of the baseline report; (iv) a list of study questions and draft crosswalk from questions to indicators to sources of data; (v) a list of people consulted and interviewed and focus groups conducted; (vi) sources of data and a list of documents reviewed; (vii) data collection instruments; (viii) regression data tables; (ix) study team disclosure and conflicts of interest; and (x) has the study team profiles. 18 Youth Potential – Quarterly report – PY3 Quarter 3 4 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline STUDY OBJECTIVES AND APPROACH ACTIVITY DESCRIPTION As described elsewhere in this report, the YCS is conducted under the aegis of the USAID/Ethiopia’s EPMES Activity – a contract between USAID/Ethiopia and SI.19 The YCS is one of several approaches used to conduct an independent performance evaluation of USAID/Ethiopia’s Youth Potential Activity. (Annex 1) STUDY OBJECTIVES The overall objective of the YCS is to evaluate whether youth who receive workforce development/livelihood support achieve increased income and strengthen skills, knowledge, and social capital required to achieve economic self-sufficiency over the longer term. The premise underlying the Youth Potential Activity is that participation of youth in training programs that build capacity and work￾readiness skills will result in the youth having better skills, including skills in functional literacy and numeracy, that make them better prepared to find employment or engage in self-employment, leading to higher income and health outcomes (see Activity Results Framework in Annex 2). This principle has shaped the design of this study. RESEARCH QUESTIONS Specifically, the YCS addresses the following broad questions as identified by USAID in the statement of work (See Annexes 1 and 3): 1. To what extent has the income and employment status of study participants changed between enrollment in the Youth Potential Activity and the twelve months following enrollment? Does increased income lead to improved health outcomes among study participants? 2. To what extent has participation in the Youth Potential Activity improved the work-readiness skills of study participants? 3. To what extent has participation in the Youth Potential Activity improved knowledge of and access to employment/entrepreneurial resources for study participants? 4. To what extent has participation in the Youth Potential Activity improved functional literacy and numeracy skills of study participants? 5. How do the outcomes addressed in the questions above vary by participant characteristics (e.g., sex, age, ethnic/linguistic group, SES, marital status, parenting status) and by service delivery partner? While all the questions above20 will be answered by the last (third) round of the study, this second round of the study provides information on the status for each of the above questions after six months of the baseline. RELEVANCE OF THE STUDY The YCS, upon completion, will provide a deep understanding of participant outcomes as a result of the capacity building training (henceforth referred to as training) provided by the Youth Potential Activity. The findings will inform the stakeholders, especially USAID, and the implementers of the Activity with information about the Activity as well as progress towards near-term outputs and outcomes and 19 Evaluation Mechanism Number: #AID-663-C-16-00010-EPMES, USAID/Ethiopia contract with Social Impact 20 Questions from the EPMES statement of work. 5 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline demonstrate longer-term contributions of Youth Potential Activity initiatives on a cohort of youth. The study will highlight major strengths and weaknesses of the Youth Potential Activity and provide recommendations for mid-term modifications as necessary as well as future design of similar programs. STUDY APPROACH This study is designed to respond to each of the broad questions above, gather evidence on the extent to which participation in the Youth Potential Activity has improved the employment status and/or income level of the participants or had an impact on their work-readiness skills, awareness of and access to employment and entrepreneurial resources, health outcomes, and functional literacy and numeracy. This is done by following the progress for a cohort of Activity beneficiaries, over a period of twelve months, using a variety of qualitative and quantitative methods. The study design is described below. The focus of the YCS overall is to track changes in outcomes for beneficiaries of the Youth Potential Activity included in the cohort over a period of time (twelve months) and to document lessons learned for future similar Activity designs. The hypothesis underlying the Youth Potential Activity is that participation of youth in activities that build capacity and work-readiness skills will result in their having better skills, including in functional literacy and numeracy, which make them better prepared to find employment or engage in self-employment, leading to higher income and health outcomes. Since the Youth Potential Activity covers most eligible youth in the woreda and is implemented over an extended time, the study will employ a panel design to demonstrate the contributions of the Activity. In a panel design, participants from whom data are collected from over time serve as their own controls, since characteristics of an individual observed earlier can be controlled for when analyzing for changes in the outcomes. Baseline data collected on a select group of beneficiaries will help in assessing effectiveness of the Activity; by defining measures of the success of the Activity, it will help focus the Activity on intended results. The YCS follows a cohort of participants from the time it enrolls in the Youth Potential training Activity to a time twelve months from the date of enrollment. To assess the changes and the contribution of the Youth Potential Activity, data are being collected from a select group of participants (cohort) who enrolled in the training program in March/April 2017, at three points: (i) baseline, in the month of enrollment in the training (collected in April 2017); (ii) mid-line, at six months from the point of enrollment (current phase); and (iii) end-line, at twelve months’ post-enrollment in the Youth Potential Activity. This is accompanied by periodic collection of income and expenditure data (using monthly interviews) for a purposive subset of the cohort. To the extent possible, to ensure comparability, data at mid-line and end line will be collected using the same instruments as used for the baseline study. Additional questions will be added, as needed, to assess the contributions of the Youth Potential activities. Family members of the trainees were also interviewed at mid-line and will be examined in the end line study, providing another perspective on the outcome of the Activity on the lives of the youths and their roles in communities. Data collected at midline was compared with that at baseline to determine any change. Similarly, data collected at endline will be compared with that from baseline and midline l to respond to each of the study questions and assess the changes and the contributions of the Activity. The assessment methodology employed follows the logic of the Activity and examines all initiatives within this component to determine progress. A mixture of qualitative and quantitative methods was employed for the collection of primary and secondary data. Primary data was collected through: (i) survey of Activity participants; (ii) focus groups with a subgroup of participants; (iii) focus groups with the guardians of the participants; (iv) KIIs conducted with stakeholders and Activity implementers; and (v) monthly interviews to monitor status of employment and use of income from a sub-group of approximately 80 study participants from the March/April 2017 cohort. Secondary data was collected from a review of documents, including programmatic reports and databases already in place for Youth 6 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Potential Activity, and was triangulated against the primary data to obtain a more complete picture. Field visits to a sample of woredas in the Amhara, Oromia, Tigray, and Afar regions of Ethiopia were conducted to collect data from participants on their perspectives on the Activity, their current situation, and any changes that can be considered as a contribution of the Activity.21 Baseline Study The baseline study was conducted during the period March to June 2017. Six hundred seventy Youth Potential participants from 13 woredas in the four regions of Afar, Amhara, Oromia, and Tigray were selected and yielded primary data on 582 participants (representing a response rate of 87 %).22 The data collected provided the following information on the participants: 1. Demographic characteristics 2. Income and employment status at the time of enrolling in the Activity as well as usage of health services 3. Self-perception of work-readiness skills 4. Knowledge of and access to employment/entrepreneurial resources The data were disaggregated by participant characteristics, including sex, age, marital status, and region (which could also be used as a proxy for the implementers)23. In the four regions selected for the study, Youth Potential Activity is implemented by a different partner, under general oversight from SCI. Activities in Amhara region are implemented by Professional Alliance for Development in Ethiopia (PADet), in Afar by SCI; in REST and in Oromia by HUNDEE. As such the findings from the regions can also be thought of as a proxy for implementing partners. The executive summary from the baseline report is included in Annex 3. Midline Study Data for the midline study were collected from several primary and secondary sources, including survey of participants, focus groups, KIIs, and review of documents, websites, and administrative records conducted over a period of six weeks spread over September and December 2017. Data were also collected from monthly interviews conducted between June and September 2017 with a subgroup of participants to monitor employment and income expenditure data. Primary data for the midline study were collected from 533 of the 582 respondents in the baseline, representing a response rate of 92%.24 (See Table 1) The remaining 8% were either not available as they had moved out of the region for purposes of work (6%), due to marriage or for personal reasons (less than 1%), or chose to not respond (1.5%). 21 Regions, as identified in the statement of work, were Amhara, Oromia, Tigray, and Somali regions. However, due to instability in the Somali region, in consultation with USAID and Save the Children, it was substituted with the Afar region, which has very similar characteristics (pastoral region where the Youth Potential Activity is being implemented by Save the Children itself). 22 The regions and woredas were selected in collaboration with USAID and Save the Children. The regions were stratified by pastoral/productive areas and the implementing partners. Stability in the region was an additional consideration used for the selection of region. The woredas were selected based on the following criteria: ongoing training program at the time of baseline data collection, pastoral/productive areas and the program year in which activities were first implemented in the woreda. 23Save the Children in addition to its role as a leading partner is responsible for direct implementation of the Activity in Afar, Professional Alliance for Development is responsible for the implementation of the Activity in Amhara, HUNDEE is responsible for implementation in Oromia and Relief Society of Tigray is responsible for activities in Tigray. Therefore, the findings of outputs and outcomes for each region can be considered to be a proxy for outcomes by the partners. 24 All participants from each of the selected woredas who had completed the training or were enrolled in the Youth Potential Activity in the months of March and April 2017 were included in the sample for the baseline study. 7 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Table 1 Distribution of survey & focus group participants and KIIs, by region Survey Respondents Focus Group Participants KIIs Baseline Midline Youth Potential Participants Parents/ Guardians Stakeholders Region Total Male Female Total Male Female Total Total Total Afar 108 52 43 95 20 20 40 9 10 Amhara 200 93 79 172 46 34 80 8 13 Oromia 178 108 63 171 33 51 84 13 11 Tigray 96 51 44 95 21 14 35 20 11 Total 582 304 229 533 120 119 239 50 45* *Three additional interviews were held with Activity implementers in Addis, bringing the total number of KIIs conducted to 48. Source of data: YCS Midline Survey Data were collected through surveys administered by trained interviewers, who used handheld electronic devices to ensure rapid and accurate data collection.25 The survey instrument for the baseline study was reviewed and additional questions were added to assess the effect of the Youth Potential Activity on the participants. The survey, revised in English, was translated into the regional languages prior to administration. In parallel, qualitative data were collected through 35 focus groups with 289 Activity participants and their parents and guardians (26 focus groups were conducted with Activity participants and 9 with their parents), and 48 KIIs across the four regions. Informational meetings with stakeholders, including a select group of Government counterparts; employers; microfinance officials; TVET and TTI administrators; employers; WBL providers and Youth Potential staff at the regional, woreda, and kebele levels were also held. The focus groups were conducted in a purposive sub-sample of 7 woredas in the 4 regions (two woredas in each region; except in Tigray) and the Youth Potential staff included the regional managers, monitoring and evaluation (M&E) officers, woreda officers, and the facilitators responsible for delivery of the training as well as coaching and arranging for the mentoring. Interviews were also held with the SCI staff responsible for oversight and implementation of the Youth Potential Activity. Annex 5 contains a list of people interviewed, Annex 6 lists the sources of data, and Annex 7 contains the survey instrument as well as the focus group and interview guides used. LIMITATIONS OF THE STUDY 1. The ability of the enumerators to locate the participants is compromised by the fact that the youthful composition of the participants, some of which are from pastoral areas, represents a highly mobile population. Difficulties in contacting and tracking these respondents have affected the survey response rates, despite a cushion built into the baseline sample size to allow for attrition. 2. Getting participants’ cooperation with regular monthly completion of the financial diary has not been straightforward. In the absence of incentives, corrective actions have been taken to streamline the process and reduce respondent burden and the situation has improved, but is not fully resolved. 3. As noted during the baseline survey, access to the internal databases used by Youth Potential is important to track participants for updated information on the number and type of trainings attended by the participants and frequency of participant contacts with the facilitators. In the absence of this information, we are unable to corroborate participants’ reports on number and 25 The surveys data collection was completed by SART on the EPMES team with oversight from Social Impact. 8 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline type of trainings with the data maintained by the Youth Potential. This information would also help compare data from the YCS participants with all participants in the Activity. 4. The study is following a purposive random sample of Youth Potential Activity participants from four regions. All participants from the March/April cohort were included in the study; however, as the internal database used by the implementing was not available, no effort was made to establish the representativeness of the sample. As such the findings from the study are limited to the study cohort, and cannot be extrapolated to the larger population of recipients of the Youth Potential Activity. 5. The assessment of the contribution of the Youth Potential Activity on the individuals is based on self-reports and information obtained from families and other key respondents. This can introduce self-reported bias, which cannot be addressed or mitigated. 6. Data on improvements in literacy and numeracy as well as health outcomes is based on proxy measures. These topics are not directly addressed by the curriculum taught by the Youth Potential Activity and as such, the outcomes are expected to be tangential benefits of the Activity. 9 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline MAIN FINDINGS The findings from the midline data collection for the YCS are organized by the questions the study seeks to answer and are presented below in order. For the sake of completeness, a brief description of the study participants is also included. Accordingly, the first subsection provides a brief description of the demographic characteristics of the YCS participants. This is followed by a description of the participants’ income and employment status, and utilization of health care six months after enrolling in the Youth Potential Activity; self-perceptions of their work-readiness skills; and participants’ knowledge of and access to employment/entrepreneurial resources. To the extent possible, all findings are discussed by participant characteristics, such as sex, age, marital status, and by region (as a proxy for implementing partner).2627 Where available, comparisons are drawn with the baseline data to determine the interim changes and outcomes for the Youth Potential Activity participants.28 Demographic Characteristics of Survey Participants As noted earlier, 92% of the YCS participants from the baseline study completed the midline survey. The distribution of respondents across sex and regions for the midline study is consistent with the baseline. As Table 2 shows, 57% and 43% of respondents of both midline and baseline study are male and female respectively. The regional distributions are also quite consistent, with 18% of all respondents of the midline survey coming from Afar compared to 19% for the baseline survey. Similar percentages for Amhara, Oromia and Tigray for the midline survey are 32%, 32% and 18% respectively, compared to 34%, 31% and 16% respectively for the baseline survey.29 Table 2 Distribution of respondents by sex and region (%) Baseline (N=582) Midline (N=533) Male Female Total Male Female Total Percentage Percentage Afar 18 19 19 17 19 18 Amhara 32 37 34 31 34 32 Oromia 34 26 31 36 28 32 Tigray 16 18 16 17 19 18 Total 57 43 100 57 43 100 Source of data: YCS Midline Survey 26 The statement of work also called for disaggregation by ethnicity, which was dropped following discussions with USAID at the time of study design. 27 In the four regions selected for the study, Youth Potential Activity is implemented by a different partner, under general oversight from SCI. Activities in Amhara region are implemented by Professional Alliance for Development in Ethiopia (PADet), in Afar by Save the Children (SCI); in Tigray by Relief Society of Tigray (REST) and in Oromia by HUNDEE-Oromo Grassroots Development Initiative (HUNDEE). As each of the implementing partners is responsible for implementation of Youth Potential Activity in each of the regions selected for this study, the findings of outputs and outcomes for each region can be considered to be a proxy for outcomes by the partners. 28 Youth Cohort Study, Baseline Evaluation Report for Building the Potential of Youth Activity. August 2017. Submitted to USAID by Social Impact Inc. under the Ethiopia Performance Monitoring and Evaluation Service (EPMES) Activity of USAID/Ethiopia. Evaluation mechanism number: #AID-663-C-16-00010-EPMES. 29 All 49 respondents from baseline (8%) that did not complete the midline survey were from the Oromia region. Fifty-nine percent of them were male and 41% female; 61% were in the 19-24 age group, 12% in the 15-18 age group, and 16% over 25 years of age. Ninety percent of them could read and write easily and almost all could recognize numbers and do simple addition. At baseline, 76% of these 49 were employed. 10 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Sixty-seven percent of the cohort is between 19 and 24 years of age, followed by 20% over 25 and 13% that is between 15 and 18 years of age. Within age groups, while the proportion of male and female participants is about the same in the over 25 years age group, there are more male participants than female in the 19-24 years age group (71% and 62%, respectively), and more females than males in the 15- 18 age group (23% female, 6% male). Only four cases across the four regions reported having a disability. Figure 6 Demographic characteristics of midline survey participants, by region (%) Source of data: YCS Midline Survey Most participants belong to the region where they are currently living (between 95% and 100%) and almost all speak the language of the region. Fifty-two percent of all midline survey participants have never been married, followed by 38% who are married. More women participating in Youth Potential are married, divorced, or widowed than the male participants. The proportion of married participants varies across regions, with most married participants were from Afar (55%), followed by Oromia (43%), Amhara (33%) and Tigray (21%). Overall, 57% of all participants live with their parents, of which the proportion of men living with parents (61%) is higher than women (53%).30 Sixty seven percent of those married reported having children. Figure 6 presents distribution of survey respondents by sex and age group by region. Similar figures showing distribution by marital status and living status by regions are included in Annex VIII. 30 In Ethiopia, it is culturally expected that young men and women will live with their families until they marry and sometimes even beyond. 11 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Figure 7 Distribution of participants, by region and level of education (%) Source of data: YCS Midline Survey Ninety-four percent of all participants have attended school, and 86% of this group had completed grade six or higher. Education patterns are very similar for both young men and women, with 49% male and 53% females completing high school (grades 9-12). Approximately a tenth of the total sample reported completing TVET and college (10% for males and 9% for females). However, while 14% of the sample reported completing less than grade 5, the patterns for male and female participants were very different. A larger proportion of females reported attending no school (8%) compared to men (4%), but a smaller proportion reported less than grade 5 schooling (10%) relative to men (16%). At the regional level, Afar had a largest proportion of participants with no formal education (23%), while Tigray had the largest proportion of participants that had completed high school and beyond (76%). The levels of literacy and numeracy31 among participants also vary by region and appear to be in line with their levels of education. While most participants (both men and women) can read and write very easily (74%) and recognize numbers (94%), of those with lower levels of literacy and numeracy, there were more female (22%) than male (15%) respondents with lower levels of literacy and lower levels of numeracy (14% female and 7% male could not add numbers). Figures 7 and 8 present the distribution of education levels, functional literacy and numeracy by region. 31 The participants’ level of literacy and numeracy were assessed using basic tests of these skills during the survey data collection. 12 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Figure 8 Distribution of participants, by region and level of literacy and numeracy (%) Source of data: YCS Midline Survey Most of the participants reported stopping their education due to inability to pass examinations (34%) or cited economic reasons leading to inability to afford continuing education (23%). Female participants reported marriage as a major reason for dropping out of school (20%), a point also made by participants in focus groups. TRAINING, COACHING, AND MENTORING The Youth Potential Activity provides a range of trainings designed to provide participants with work readiness skills, including skills designed to assist participants prepare for self-employment (identifying business ideas, customer service, managing their own business, calculating profit, etc.), prepare for wage employment (searching for employment opportunities, preparing CVs and writing applications, etc.) and improve participants’ knowledge of resources available for self-employment as well as finding employment. These trainings include: • PYD, or Youth in Action (YIA) training32, designed to prepare participants for the next phase by encouraging the youth to reflect on themselves and their objectives and goals in life and to explore the resources available in their communities. • WRN! and WR Plus training to provide participants with soft skills in interpersonal communication, work habits, and conduct; and develop their leadership skills. The WR Plus program is targeted towards participants with no/low levels of literacy. At the end of this training the participants are expected to develop a personal development plan that guides all follow-up trainings and activities. • BYOB training, designed to provide trainees that wish to start their own business with skills on developing business and work plans, basic bookkeeping, assistance with loan applications, etc. • Aflateen, training designed to provide foundational social and financial education to youth in the 15-18 years’ age group to empower them to lead responsible lives and be agents of change. • Additional one-off trainings (e.g., training on driving) based on demand from the participants. SCI also draws upon its organizational networks to create internship/WBL opportunities for 32 This training was adapted from Save the Children’s Youth in Action program’s modules “Myself” and “My Community”, and thereafter adjusted and translated for Potential. 13 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline participants that range from 5 to 25 days in duration and help participants practice their basic training. • Support for a few participants to attend technical/vocational training programs at various public and private TVETs, TTIs or at private institutions (trainings include tailoring, hairdressing, secretarial courses, use of computers, etc.)33 The training program is designed to provide all participants enrolling in the Youth Potential Activity to receive a basic training (YIA and WRN! or WR Plus) with a set number of contact hours spread over 13–15 days, followed by modules selected based on their personal development plan and community mapping. All training is provided in a co-ed setting, with both male and female participants receiving the same information. The participants are followed by youth facilitators (henceforth referred to as facilitators) who provide one-on-one and group-based coaching and arrange for mentoring to participants until the participants are fully established and decide they no longer need the coaching or the mentoring. Participation in training programs Findings from the midline qualitative and quantitative data collection show that at a minimum all youth from the March/April cohort completed WRN! or WR Plus components of the training program. All participants in Amhara, Oromia and Tigray attended the WRN! training. In Afar, however, 56% of the participants attended WRN!, while the remaining 44% attended the WR Plus program. This finding is consistent with the levels of education reported by the participants from these regions. The PYD/YIA training and the BYOB training have not been offered consistently across all four regions, and survey data reflect this difference between the regions. As Figure 9 shows, none of the participants in Afar had attended the BYOB training, while only 41% of the respondents attended BYOB training in Tigray. Similarly, participants from only Oromia and Amhara regions attended the PYD/YiA training. Slightly more males than females (27% and 21% respectively) had attended the BYOB training. A majority (82%) of those attending BYOB training had completed grade 9 or higher. Interviews with Youth Potential staff indicated that BYOB training is offered only to those participants who have decided to set up their own business, and that in some regions the implementation of the BYOB training had been delayed by a few months due to budget constraints and the need to meet their targets on number of youth reached by Youth Potential34. 33 As per SCI, only 7% of all trainees are offered hard-skills training at TTIs/TVETs and allied institutions. 34 As explained by the Activity staff, due to the need to meet targets for youth reached, a choice was made to divert resources to training new participants and postpone BYOB training for existing participants to the next quarter/financial year. 14 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Figure 9 Participation in training programs, by region and level of education (%) Source of data: YCS Midline Survey Approximately, a third of the participants, more females than males, from each of the regions (except Tigray where only 15% attended a training) reported that they have benefitted from WBL opportunities and scholarships from Youth Potential to attend trainings provided by external providers (e.g., TVET, TTI or private providers). Over half of the participants in this program had completed grade 9 or higher. The most popular areas for training were agriculture and sheep fattening in Afar, Amhara and Oromia regions, followed by poultry farming and trading in Amhara and Oromia. In Tigray, however, fewer participants benefited from this program and the areas of choice were skills for hotel and tourism industry, and hairdressing. Some of the other options selected by participants included baking and running a bakery; carpentry, carving, or woodwork; mechanics and metal work; construction work (brick/stone laying, building, roads, etc.); running coffee and tea shops; and vegetable farming. While training in agriculture and animal fattening were selected by all participants, there were some differences by sex in the type of training selected by the participants, with more males opting for tailoring, bakery and construction work and more females attending training in hotel and tourism industry as well as Information and Communications Technology (ICT) and computer skills. Similarly, while training in agriculture and animal husbandry as well as provision of coffee and tea services were popular with participants across all levels of education, there was a slight preference for those with lower levels of education to opt for training in skills such as tailoring, while those with higher levels of education selected training in bakery, marketing, tourism industry, hairdressing and ICT. Data from the focus groups with participants and KIIs conducted in the regions support these findings. According to the Youth Potential staff, training in hard and soft skills through TVETs, TTIs or with private providers (tailoring, hair dressing, etc.) is fee-based (where tuition fee is paid by Youth Potential 15 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline to the provider on behalf of the participants) and has been provided only to a few beneficiaries from each region.35 The beneficiaries for this initiative are selected based on their personal development plan, evidence of their continuing interest and capability, as well as the resources available. For many of the beneficiaries, especially from Amhara and Afar, the budget as well as the fact that the cohort started in March/April (middle of the academic year) has been problematic and further training through TTIs or TVET has been delayed to the next financial year (for Youth Potential). WBL activities usually three-days in duration and half day job shadowing activities were more common. Participants in focus groups indicated that these programs had been very beneficial and they had learnt from observing “a business similar to [their] own.” One success story, reported by regional Youth Potential staff comes from Oromia where a youth shadowed a model farmer in the community to learn from his experience. He started from scratch, and today owns apple trees, does animal fattening, and is a poultry and dairy farmer. He was an inspiration for the youth to try to follow in his footsteps. A similar story was reported by a female FGD participant in Amhara, where following a job-shadowing visit with a trader in a city, the petty trader has been able to expand her own inventory leading to better income. Coaching and mentoring Coaching, defined as follow-up of training and assisting participants use their training, is provided by the facilitators in one-on-one and group coaching sessions and through information exchange with other programs, such as the SCI Youth Economic Strengthening (YES) program. FGDs with participants as well as interviews with facilitators indicated that in three regions, Amhara, Tigray and Afar (except in Asaiyta where males reported they were not meeting for coaching), facilitators organize group-coaching sessions on a monthly basis following the WRN! training. Not all participants attend every month, even though the sessions are open to all. Focus group participants and the coaches reported that the benefits of coaching included group solutions to problems; a sense of community through regular meetings; supporting youth maintain hope and not give up over obstacles; accountability for youth to continue saving and search for employment; and support for loan applications where microfinance was available. In addition to group-coaching, the youth in these three regions are also provided with more informal one-on-one coaching where the coaches/facilitators interact with the youth in the market place, at their place of work, by telephone, etc., to ensure the youth are staying with their personal development plan and to offer advice as needed. Youth in the Oromia region have limited or no coaching. In a few cases, the facilitators call the youth by phone to follow-up with them or pay informal visits, but this is not organized on a consistent basis. One young person in Yabelo woreda in Oromia commented, “the training was good but we are left with only the training.” Parents in the Oromia region also said that there was little or no follow-up support for their children after the training, a point also made by parents in Amhara region. Youth across all focus groups expressed the need to get coaching/mentoring from others with business and agriculture experience on specialized topics of farming and livestock management (e.g. use of fertilizers, livestock feed, what type of cow to purchase, etc.). According to the Youth Potential staff the facilitators along with the woreda officer have a demanding workload. They are responsible for providing WRN! training, coaching to participants, and arranging for mentoring. The main challenges to conducting coaching, as per the facilitators and their supervisors, are distance, rough terrain and the growing number of participants. In some cases, facilitators/coaches take up to four hours to get to hard-to-reach kebeles by walking or using public transport. With the Youth Potential strategy to keep coaching open to all beneficiaries of the Activity until they find a job or are settled in self-employment, the number of participants each facilitator is responsible for is growing, 35 As per the SCI staff, as per the Activity plans only 7% of the total cohort are entitled for hard skills training. 16 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline making it difficult for the facilitators to do full justice to all their trainees. They do not always have sufficient time for following-up with individual coaching. A related challenge is that given the frequency with which the coaches meet with the participants for group coaching, there is a need for new materials to keep the attention and interest of the participants. The situation is further worsened in Yabelo in Oromia, where following a turnover in the woreda officer, the two facilitators have been covering as many as 11 kebeles over the past months. There was no mentoring support provided for youth in any of the regions because mentoring program is just getting started36. While coaching is delivered by WRN! facilitators and is more general in focus, mentorship is designed to be delivered by community leaders/successful entrepreneurs and focused on business support. The first training of mentors has just been completed in Tigray, and they are expected to begin providing mentorship of the Youth Potential beneficiaries starting in November 2017. Participation in internships/work-based training Youth Potential has enlisted the support of various stakeholders for internship/WBL opportunities for the beneficiaries. This includes government agencies (such as Children and Women Affairs offices, Micro and Small-Scale Enterprise agency, municipalities, TVETs, and others) and non-governmental organizations, such as micro-finance and private organizations. Youth Potential is working with the Bureau of Education in Afar to provide interns for training as teachers for alternate education programs. Seventy-five youth with the required basic qualifications have already been hired and it is expected that a similar number will be hired in the next school year. Youth Potential is also working with ongoing activities and programs in the region, such as the Sugar factory in Amhara and the planned laundry center at the TVET in Amhara to see how resources can be leveraged to support the Youth Potential participants. The Activity has established agreements and MOUs with several of these organizations. The agreement with the sugar factory, for instance, is that the Youth Potential Activity will provide capacity building for the management staff in exchange for internships/WBL opportunities for Youth Potential beneficiaries. With TVET, the agreement is that Youth Potential will provide the machines for the center and TVET will provide hands-on experience to the Youth Potential participants. However, as per key respondents both from the partner organizations and from the implementers, the process for procurement37 was very slow and the partners were beginning to lose patience. In sum, opportunities for internships/WBL in the four regions are few and far between. EMPLOYMENT STATUS OF STUDY PARTICIPANTS Employment Status This study is following a cohort of 582 Youth Potential participants from the time they enrolled in the Activity. At baseline, 258 of these respondents were employed and the remaining 324 were either unemployed or enrolled in school. In the six months between baseline and midline, 165 of these 324 (51%) found employment, leaving 159 (49%) unemployed at midline. At the same time, 28 of those that were employed at baseline lost their employment, leaving only 230 of the employed baseline respondents employed also at midline. The total number of Youth Potential participants employed at midline was thus 395, and those who were unemployed at midline was 187. Of the 187, 49 participants 36 As per SCI the first round of training for mentors was organized in Activity Year 2; however, the evaluation did not find any evidence of mentoring in the cohort being followed. Despite asking a direct question to this effect, neither the survey respondents nor participants in focus groups mentioned receiving mentoring. 37 The process for procurement refers to the time taken to arrive at an agreement with the partners on the equipment to be purchased, and the SCI process of obtaining quotes and approvals as well as the final placement and delivery of orders. 17 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline did not respond to the midline survey, yielding data on 138 unemployed.38 Table 3 below summarizes the employment status of 533 Youth Potential participants at midline compared to baseline. Table 3 Change in employment status between midline and baseline Midline Baseline Total* Change from Baseline to Midline Baseline Unemployed Agri. Trading Manuf. /Const. Daily labor Service Professional Baseline Total Midline Total Change ML-BL Unemployed (including those in school) 110 83 31 5 14 25 7 275 275 138 -137 Agriculture 14 84 11 0 1 7 1 118 118 188 70 Trading 4 4 35 0 0 6 1 50 50 86 36 Manufacturing/ Construction 0 2 0 4 1 2 0 9 9 13 4 Daily Labor 3 8 1 2 8 3 0 25 25 24 -1 Service 5 6 8 2 0 28 2 51 51 71 20 Professional 2 1 0 0 0 0 2 5 5 13 8 Midline Total* 138 188 86 13 24 71 13 533 533 533 0 *Note: This table presents the dynamic process of employment status to shed light on change in employment status and inter-sectoral mobility from baseline to midline. The rows represent the participants’ employment status at baseline while the columns show their changed status at midline. For example, the table suggests that 110 respondents were unemployed at Baseline and remained so in midline. Among the remaining 165 unemployed in baseline, 83 found jobs in agriculture, 31 found jobs in trading, 5 in manufacturing, 14 in daily labor jobs and 25 in service sectors. On contrary, 14 respondents who were employed in agriculture during baseline are now unemployed during midline, 84 of the remaining 104 are still in agriculture while 7 moved to service sector and one each to daily labor jobs and professional sector. Source of data: YCS Midline Survey Analysis of the survey data of 533 midline respondents shows that six months after participating in the Youth Potential training, 65% were self-employed, 9% were working in wage or in-kind employment or employed as daily laborers, 2% were continuing with education or training programs, and the remaining 24% were unemployed and dependent on others for their sustenance. Of those self-employed, 74% were working in a self-owned business, 15% supported family-owned activities, and 11% were engaged in group-owned businesses. More males (79%) than female (66%) survey respondents reported being employed, and fewer males were unemployed (18%) compared to females (29%). More of those engaged in wage employment tended to have completed secondary school and beyond, with the self-employed equally spread across all levels of education. Of those self-employed, while an equal number of male and female respondents reported being engaged in self-owned employment, a slightly larger proportion of male respondents were engaged in family owned work as compared with females (16% and 13% respectively) and more females were engaged in group activities as compared with male respondents (10% and 13% respectively). A larger proportion of respondents from Afar region was unemployed (45%) compared to Oromia (23%), Amhara (17%) and Tigray (9%). Tigray had the largest proportion of self-employed (80%) followed by Amhara (71%), Oromia (60%) and Afar (49%). Of those self-employed 38 Of the 49 that did not respond to the midline survey, 70% were self-employed and 6% wage employed; Fifty-six percent of the self-employed were looking for a job at baseline. Of those employed (self- or wage), 43% were employed in agriculture and livestock activities, 16% in trading, 4% daily labor and the remaining in other employments, such as transport service, education, government, small restaurants, providing coffee/tea, etc. Fifty-five percent of the participants reported looking for jobs in the previous six months (all data from the baseline). 18 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline at midline, 40% were looking for employment or new employment, with a small proportion (16%) looking for wage employment. Overall, 49% of the survey participants reported looking for employment at midline, significantly lower than the proportion seeking employment at baseline (58%, p<.01). Figure 10 has the distribution of employment status of the survey respondents at midline. Figure 10 Current employment status of YCS respondents at midline, by region Source of data: YCS Midline Survey There is a significant change in the employment status of the survey respondents from baseline to midline (Table 4). The proportion of youth reporting being unemployed decreased significantly between baseline and midline (from 47% to 23%, p<.01), while the proportion of those reporting self-employment increased significantly during the same period (41% to 65%, p<.01). It is also interesting to note that while 23% of the Activity participants were unemployed overall, an examination of the distribution by level of education revealed more of the college educated (45%) were unemployed. The most dramatic decrease in unemployment is found in the Tigray and Afar regions, which also experienced the greatest increase in employment, especially for the self-employed. In addition, of those that reported being unemployed at baseline across the four regions, half (50%) were now, at midline, self-employed and 10% were engaged in wage/daily employment. A similar change was noted within the category of self￾employment. Since participation in the training, 76% of those that were engaged in family-based employment are now engaged in self-owned activities. Furthermore, 30% of those reporting being employed also reported holding a second job, mostly in agriculture and livestock, and trading. Table 4 Change in employment status of respondents: baseline versus midline (%, n=533) Employment Status Baseline Midline Difference p-value In school or training 4.90% 2.10% -2.81% p<.01 Paid/wage/paid-In kind employed 6.00% 8.80% 2.81% p<.05 Self-employed 29.30% 39.20% 9.94% p<.01 Self-employed but looking for a job 12.00% 25.70% 13.70% p<.01 Unemployed (dependent on support from others) 46.00% 22.70% -23.26% p<.01 Unemployed but not seeking work for other reasons 0.90% 1.10% 0.19% - Other 0.90% 0.40% -0.56% - Source of data: YCS Midline Survey 19 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Of the 230 Youth Potential participants who were employed at both baseline and midline, several changed jobs in the intervening period of six months. As Figure 11 shows, the largest change was seen in the percent employed as daily labor, which fell from 9.6% at baseline to 4.3% at midline. The largest gains at midline were recorded in trading, up from 20% at baseline to 23.9% at midline, and those engaged in professional work, which saw an increase from 1% at baseline to 3% at midline. Figure 11 Change in sector of employment at midline compared to baseline (n=230) Source of data: YCS Midline Survey Table 5 presents the change between baseline and midline in terms of detailed categories of employment. The most notable change is in the category of daily labor, which saw a significant drop at midline (4.4%) relative to baseline (9.6%). Other categories that recorded a drop were carpentry (from 1.3% at baseline to 0.4% at midline), tailoring (from 2.2% at baseline to 1.8% at midline) and providing services, such as tea/coffee (from 8.8% at baseline to 7.5% at midline), but none of these changes is significant. 20 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Table 5 Changes in employment status of YCS respondents: baseline vs midline (%) Broad Categories Detailed Categories Midline* n=230 Baseline* n=230 Difference* T-Value p-value Agriculture Agricultural + Livestock activities 46.1% 45.6% 0.4% 0.16 0.88 Trading Trading agricultural products (buying or selling) 11.8% 8.8% 3.1% 1.40 0.16 Trading consumer products/items 12.3% 11.4% 0.9% 0.47 0.64 Manufacturing & construction Carpentry, carving, or woodwork 0.4% 1.3% -0.9% -1.00 0.32 Mechanics, metal work 1.8% 1.3% 0.4% 1.00 0.32 Construction work (brick/stone laying) 1.3% 1.3% 0.0% 0.00 1.00 Daily Labor Daily laborer 4.4% 9.6% -5.3% -3.05 0.00 Services Electrical work Computer tech and administration 0.9% 0.9% 0.0% 0.00 1.00 Tailoring (making clothes or textiles) 1.8% 2.2% -0.4% -0.58 0.56 Transport Service 3.1% 3.1% 0.0% 0.00 1.00 Hairdressing/Barber 1.3% 0.9% 0.4% 1.00 0.32 Providing Services (such as coffee/tea, restaurant, etc.) 7.5% 8.8% -1.3% -0.90 0.37 Hotel industry Professional Education 0.4% 0.4% 0.0% 0.00 1.00 Government 0.9% 0.4% 0.4% 0.58 0.56 NGO 1.3% 0.4% 0.9% 1.42 0.16 Other 4.8% 3.5% 1.3% 0.73 0.47 Total 100 100 *all numbers rounded to the nearest whole number Source of data: YCS Midline Survey Focus group participants engaged in wage employment reported that they were working in a variety of positions, such as security guards, domestic workers, Bajaj drivers, TVET assistance instructor, storekeeper, and factory labor jobs, among others. According to KIIs conducted in the Afar region, a large group of youth (from a different cohort) are working as school teachers in Afar public schools. For those engaged in self-employment, the main activities were agriculture, growing one to two crops, sheep and goat farming and livestock fattening. Many female participants reported running shops for petty trading and coffee and tea. A small number also reported Chat trading, small restaurants, running a tailoring shop, etc. In Amhara region, one focus group participant also reported growing sugarcane as a part of a group-based self-employment. (see Table 6) 21 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Table 6 Examples of self-employment opportunities available per key informants, by region Region Self-employment activities Afar • Crop production (maize, tomato, onion, pepper, potato) • Livestock: animal fattening (shoat), goat trading, hay production animal feed • Petty trading • Selling Injure, selling chat, selling tea/coffee Amhara • Crop production (millet, peas, maize, nuts, onion, sugarcane, potato) • Input suppliers: growing/selling seedlings; selling plastic bags for seedlings • Livestock: animal fattening (sheep, cattle); sheep rearing; poultry production • Selling tea & coffee; small restaurants; retail trading; photography • Charcoal production & trading Oromia • Crop production (bananas, sugarcane, maize, teff, potato) • Livestock: cattle fattening, animal breeding (shoats, cattle), poultry • Restaurant, tea house, beauty/hair salon, petty trading (trading cereal, maize), retail shop, jewelry/small ornaments Tigray • Crop production (teff and vegetables) • Livestock: sheep rearing, poultry production, animal fattening (cattle, sheep), dairy production • Crop trading, petty trading • Hair dressing; barber shop; small shop (soft drinks, tea); restaurant; Bajaj driving; making/selling handicrafts/embroidery Source of data: YCS Midline Survey Among survey respondents working in agriculture and livestock activities as their primary or secondary job, 56% said they are growing two or more crops, 50% are rearing animals, 33% are fattening animals, 33% are poultry farming, and 16% are in dairy production. Data from focus groups with participants confirmed these findings. Participants in focus groups reported that in terms of agriculture Activity, youth are mainly focused on crop production with very few engaged in activities associated with the agriculture value chains.39 Only a handful of participants from the Amhara region reported working as input suppliers, growing and selling seedlings. Most of the respondents, especially in Afar and Tigray, also indicated that the scope of the Activity had expanded as compared with six months ago, a time line that is concurrent with their enrollment into the Youth Potential Activity. Youth from the Tigray region reported a sharp increase over the last six months in all agriculture and livestock activities except dairy production. In the other three regions, while there was an increase in the proportion of youth growing two or more crops in the last six months, other activities remained the same or showed a slight decrease (See Table 7). 39 Activities associated with agriculture crop value chains include distributers, input suppliers, marketing, post-harvest, or transport, etc. 22 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Table 7 Youth engagement in agriculture and livestock activities (self-reported responses; %) Engaged in Activity Afar (n=36) Amhara (n=65) Oromia (n=69) Tigray (n=55) All Regions (n=225) I Am Growing a Single Crop in a Year Six Months Ago 33% 9% 30% 2% 18% Now 25% 9% 29% 5% 17% I Am Growing Two Or More Crops in a Year Six Months Ago 31% 75% 54% 33% 51% Now 42% 75% 59% 40% 56% I Am Fattening Animals Six Months Ago 75% 34% 32% 2% 32% Now 67% 35% 17% 29% 33% I Am Rearing Animals Six Months Ago 81% 68% 39% 7% 46% Now 75% 49% 36% 53% 50% I Am Doing Poultry Farming Six Months Ago 11% 43% 52% 4% 31% Now 3% 46% 48% 20% 33% I Am Doing Dairy Production Six Months Ago 6% 26% 25% 0% 16% Now 6% 23% 26% 0% 16% Source of data: YCS Midline Survey Gender differences in employment: Both quantitative and qualitative findings showed gender-based differences between fields of work. Of those employed (n=395), more males were working in agriculture and livestock (61%) than females (25%). More females (30%) were active in trading agricultural and consumer products than men (10%). Female focus group participants said that their husbands typically plow and cultivate, and the women support the household farming work (Amhara). In terms of providing services, females concentrated in coffee/tea, small restaurants, retail shops, hairdressing, while men worked in tailoring and transportation (e.g. Bajaj driving). Baseline data showed 14% males and 5% females working as daily laborers; six months later, the percent of males engaged in daily labor fell to 6%, while the proportion of females in daily labor increased to 6%. These findings are in line with the strongly held gender stereotypes and assumptions prevalent in most of Ethiopia where men are considered the main breadwinners, farmers and laborers, and women and girls are responsible for managing the home including domestic work and child rearing. As mentioned before, these informal gender biases also limit women’s participation in economic activities40.41 However, focus groups with the female participants revealed that following participation in Youth Potential Activity, the female participants are encouraged to explore options in fields such as daily labor, growing cash crops (growing sugar cane), and expanding their petty trading business. In addition, while making decisions in concert with their families, several female participants reported having expanded what they were doing before (particularly for those engaged in animal farming and coffee/tea shops) or branching out into new activities. While few and far between, instances of such activities are increasing. However, both the participants and key informants reported that the prevalent biases and stereotypes continue to have an impact on the women’s ability to obtain loans and employment in areas not considered to be the typical female domain; an observation also supported by KIIs. Change in attitude towards self-employment: As already noted, 65% of study participants are self￾employed six months following participation in Youth Potential Activity. Data from FGDs supports these findings, with the focus group participants indicating a strong preference for self-employment over wage employment. Many focus group participants explained that the WRN! (or WR Plus, as applicable) training assisted them start their own business and “be their own boss.” This was especially true for 40 African Development Bank. 2004; Ibid. 41 Rubin, et al, 2009. Ibid 23 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline those who were formerly unemployed. According to the participants, their attitude towards self￾employment has changed because of the training. They have begun to appreciate the fact that wage employment was not the only kind of employment available to them, and that any work that led them to earn an income was employment. The community mapping exercise conducted as a part of the WRN! training helped them identify new business ideas that they could implement. The training also helped them to value and respect their own work “no matter how lowly” (youth FGD, Amhara), and to improve their work ethic. In addition, it provided them with the skills needed to manage a business and handle customers. One recurrent theme heard in all focus groups across the four regions was that the training had made them realize the importance of “starting with something small, and to grow their Activity/business gradually,” resulting in many of them feeling encouraged to start their own business or to work more seriously on their existing work. Comments like “I used to work two days a week, now I work all five days” and “I take better care of my sheep and goats [so they can be sold for a better price]” from focus group participants in Afar and Amhara support these findings. Focus group participants reported that they often chose self-employment over wage employment as self-employment was considered to be economically more profitable in their local economies.42 They reported that they could earn anywhere from 650-900 Birr/month in factory wage jobs (in unskilled positions) compared to 1,300-1,500 Birr/month income (on average) in self-employment, a finding also corroborated by KIIs in the regions (youth FGDs and KII with Potential staff, Amhara and Tigray). They appreciated independence, flexibility, and not being dependent on others for job stability and income (FGDs, all regions) One young person stated that “it’s good if we are supported through self￾employment rather than waiting to get a job from the government; we can be more productive by ourselves” (FGD, Amhara). Another reported from Oromia, “working in my own business will assist me to give job opportunities to others,” noting the job creation potential of self-employment. Participants from Afar and Amhara also reported that the coaches/facilitators had provided them with training on how to grow more/better crops and had arranged for an expert in agriculture to visit their farms. Attitudes towards agriculture: Participants in the focus groups had mixed opinions regarding work in the agriculture sector. Many of them felt it was a good opportunity and a good source of income, that “it’s the backbone of the Ethiopian economy” and that it gave them the opportunity to stay in their rural community (youth FGD, Amhara, Afar, Tigray.) One young person from Amhara said, “I was thinking of going somewhere else for a job, but realized I can stay and make an income here.” Most parents also said that engaging youth in agriculture kept them close to home (parent FGDs, all regions). Some disagreed, arguing that it was too challenging and that the constraints made the work unattractive: ploughing was difficult, weeding was challenging, inputs such as fertilizer and seed were expensive, drought was more common, and the fertility of the land is decreasing (youth FGD, Amhara and Oromia). Table 8 below presents their statements. 42 A recent randomized control trial in Ethiopia came to a similar conclusion. The study found that the individuals in the control group with factory jobs did not have higher weekly wages than the comparison group individuals engaged in entrepreneurship; and when constraints to self-employment were removed, employment seekers preferred self-employment to industrial opportunities. See Blattman, Christopher and Dercon, Stefan, The Impacts of Industrial and Entrepreneurial Work on Income and Health: Experimental Evidence from Ethiopia (June 22, 2017). 24 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Table 8 Youth opinions on whether agriculture is a good opportunity for young people (youth FGDs) Agriculture is a good opportunity Agriculture is not a good opportunity ● “Agriculture is a good source of income if you work hard and if land is available.” (All regions) ● “Agriculture is very good as it is the backbone of the economy.” (Oromia) ● “People do not always buy tea/coffee when they have less income, so it is good to also farm.” (Amhara) ● “Through modern farming, it’s possible to produce more than once a year. If we could produce two or three times per year, I would consider agriculture a good opportunity.” (Oromia) ● “If you used mechanized farming, it would be a good opportunity for young people. “(Amhara) ● “Instead of migrating, agriculture keeps young people around in their community.” (Amhara, Afar, Tigray) ● “It is better to rear/herd animals than to farm. The (farming) work is tiresome.” (Amhara) ● “It’s better to do animal fattening because in three months you can get good income, rather than waiting one year for the crop income.” (Amhara) ● “Fertilizer and seed are expensive.” (All regions) ● “Youth’s ability to plough the land is weak.” (Oromia) ● “Agriculture was the job of our ancestors, and they were not changed in their livelihoods. Before there was better rain distribution, but now there is no rain at all. The fertility of the land is decreasing, animal breeding is decreased due to loss of grass and drought.” (Oromia) ● “We are educated and should not depend on agriculture.” (Oromia) Source of data: YCS Midline Survey Notably, some participants made conditional statements maintaining that agriculture was a good opportunity if modern practices and mechanization were applied to make farming more productive and less labor-intensive (youth FGD, Amhara).43 Additionally, one distinction the participants made was that they preferred livestock activities to farming because they believed it was more profitable and less labor￾intensive (youth FGD, Amhara, Oromia) For example, they said that they could buy an animal for 2,000 Birr ($87.34 USD) and sell it for up to 6,000 Birr ($262 USD) after fattening, thereby getting an income in three months rather than waiting one year for harvest (youth FGD, Amhara, Oromia). Attitude towards wage employment: As noted, 9% of survey respondents were working in wage employment (including daily laborers), a small, but statistically significant, increase from the 6% reported at baseline. Those engaged in wage employment reported liking the stability and the fact that wage￾employment was a more reliable source of income (FGD Amhara). They also said that in wage employment opportunities they could grow professionally and contribute to their country (FGD Oromia). While a large proportion of focus group participants expressed preference for self￾employment to wage employment, several also reported a lack of wage-employment opportunities in their area and that the available jobs did not pay well. For example, one young man reported that he was employed at a factory in Tigray for 900 Birr ($33.33 USD) a month, and he quit after a few months because the pay was too low. However, KIIs with employers, TVET providers, and other stakeholders in Afar and Tigray revealed that there were untapped wage employment opportunities such as construction work, electricity distribution, cable installation, factory work (in Afar), and government jobs that were available in each region but may not be attractive to youth due to the low wages. These included factories (sugar, garments/textiles, cement, flower farms, bottled water, beer, and particle boards) across the four regions that hire youth in unskilled positions such as laborers, security guards, cleaners, and drivers, as well as for skilled technical positions44. Similarly, there are untapped opportunities in the agriculture 43 This is consistent with literature findings that young people find agriculture more attractive if linked to technology and modernized farming ref: “Youth Engagement in Agriculture Value Chains across Feed the Future: Synthesis Report,” September 2016. See: http://www.acdivoca.org/wp-content/uploads/2016/10/LEO-Youth-Engagment-Ag-VCs-Across-FTF.pdf - 44 Please note that not all types of employment opportunities exist in all the regions. 25 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline sector as well. For example, the Holeta agriculture TVET officials in Oromia said there are value chain opportunities in agriculture with opportunities for employment as distributers, input suppliers, marketing, post-harvest, or transport of produce. In Oromia, the “milk shed” of Ethiopia, opportunities also exist in dairy production and milk productions, animal science, fruits, vegetables and flower farming. The TVET would be willing to partner with Potential in supporting youth to better connect to these value chains. Additionally, the Raya Brewery in Tigray has contracts with smallholder farmers in the community to grow and supply barley, and the brewery pays them above market prices. The brewery expressed interest to source from Potential youth farmers in the future. Likewise, a representative from the sugar factory in Afar said that they currently source sugarcane from farmer cooperatives in the community, and this could also be a potential market linkage for youth farmers in Afar. Determinants of Employment and Unemployment Satisfaction with current job: Survey data shows that 67% of employed respondents (395 youth) were mostly satisfied or satisfied with their current employment (significantly different from baseline, p<.001). More male than female respondents and more of the younger respondents (age 15-18) reported being satisfied or mostly satisfied with their current job. (Figure 12 and Annex VIII have the details). At the same time, however, 48% of survey respondents said they had been trying to find employment/new employment during the past six months. Of these, 63% had been searching for a suitable position for 1-6 months, and 21% had been looking for longer. Figure 12 Level of satisfaction with current job, by region (%) Source of data: YCS Midline Survey Reasons for unemployment: When asked for their perceptions of the reasons for unemployment or inability to find employment of their choice, survey respondents listed lack of access to finance (20%), and lack of space for setting up a business (7%), not enough jobs available (18%), and a lack/mismatch of education and skills (lack of education 13%, unsuitable vocational education 9%, and a mismatch of skills 7%), and no work experience (4%)as the main reasons for their inability to find employment. Other reasons (14%) reported were lack of land for agriculture, low wages and poor conditions in jobs available, discrimination and young age. Male and female participants stated similar reasons for unemployment in survey responses, though there some regional differences in the participants listing of reasons for unemployment. 26 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Figure 13 Participants’ perceptions of main reason for unemployment, by region (%) Source of data: YCS Midline Survey Data from the KIIs as well as the focus groups supported the above findings. For instance, the Afar Education Bureau said they were hiring more than 300 teachers with Grade 10 education and Afar language skills. However, most of the Youth Potential study participants have less than a Grade 10 level of education (73% of the Afar cohort) and so are not eligible for these positions. Similarly, in Oromia, participants reported that the cement factory in the region had promised to hire 200 people from the nearby communities, including themselves, but that had not materialized and there was a perception that the low level of education of Youth Potential participants was a major barrier (FGD Oromia). Both male and female participants in focus groups reported that the main hurdles to starting their own business included lack of access to finance/working capital; lack of land and/or working space; and lack of start-up materials (equipment, tools, etc.). One young man in Amhara expressed, “Getting capital and a place of work, the two are very critical. We know what to do if we get these two things.” Parents and guardians of the participants also noted the lack of land and finance as major hindrances for the youth. In the words of one parent, “It would be good if they [youth] could be employed in agriculture, but there is not enough land. How can I give land for five of my children?” (FGD Tigray). It is interesting to note that only a small proportion of survey respondents from Afar listed lack of access to finance as a barrier, even though there were no microfinance institutions in the region. However, while Tigray had the greatest availability of microfinance loans and supportive policies, survey respondents from Tigray listed it as the main barrier to employment (see section on Knowledge and Access to Resources). INCOME, EXPENDITURES AND ASSETS Income Level of Study Participants Sixty one percent of the survey respondents reported an increase in income over the past six months, with the largest average changes reported by participants in Tigray. More men than women reported that their income had increased over the past six months, a time line consistent with enrollment in the Youth Potential Activity and WRN! training (Figure 14). The only exception to this were the participants with a college education, where only 45% of the participants reported their income had increased, with the majority reporting there was no change. A small proportion also indicated a decline in income, but the reasons for the decline were not very clear. An examination of the survey data further suggests that the largest proportion of those reporting an increase in income (77%) were engaged in self-employment. 27 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline These results are consistent with the findings from the focus group as well as the survey data indicating a major expansion of self-employment in Tigray, especially for the male participants. As most of the participants are self-employed and not engaged in regular wage based employment (with a salary), it is difficult to gather and verify income data. As such, data on expenditures incurred on day￾to-day expenditures as well as investment in assets and the amount of savings was used as a proxy for change in level of income. Each of these is discussed in turn below. Figure 14 Change in income over the past six months by region and sex (%) Source of data: YCS Midline Survey Expenditures incurred Youth Potential participants were asked about how they spent their money.45 Many of the focus group participants reported that the increase in income did not result in a change of lifestyle for them. They said that because of the additional income they may spend a little more on entertainment or clothes as compared with six months ago, but in most cases the additional funds went towards building assets such as a house, purchasing land or animals and putting away money as savings. Several, especially women, also indicated that they were spending more on food and eating better. A few women reported, “we now eat more vegetables”, or “we eat in our own plates”, or simply, “we eat more.” These findings 45 As mentioned above in the absence of verifiable income data, spending on day-to-day expenditures and assets was used as a proxy for change in income. 28 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline regarding changing food consumption could be a function of the season or given the prevailing social norms, where women eat less, be a positive outcome of larger expendable income. Data from the survey of Youth Potential beneficiaries across the four regions supported these findings. Given that over half of the respondents continue to live with their families (57%; 60% male and 53% female), it is important to look at expenditures incurred for the family and assets owned by the individuals themselves as well as their families, as in most cases it is difficult for the youth to separate out the expenditures incurred by them versus those by family. As there is a broad range of amount spent on each category of expenditures, we have used median values to present the distribution. Overall expenditures by category are also presented for reference. All expenditures are presented in USD46 (see Annex 8, Table 5) Expectedly, the largest expenditures reported were on investment in own business, followed by those on house rent/lease, food, support to other family members and putting money away for savings. A comparison of the expenditures incurred at midline with the baseline suggests that the beneficiaries are spending more overall and significantly more on investing in own business, supporting family members and education. As Table 9 shows, the mean overall expenditure at midline ($228 USD/5227 Birr) was significantly higher (p<.001) as compared with the expenditures at baseline ($154 USD/3525 Birr). Table 9 Distribution of those reporting incurring expenditures by category of expenditures n =533 Baseline Midline Difference T-value P-value. Rent/lease 19.5% 12.6% 6.9% 3.73 0.00 Taxation 9.2% 9.0% 0.2% 0.12 0.91 Transportation 80.7% 83.7% -3.0% -1.35 0.18 Buy food and related goods 57.8% 47.5% 10.3% 4.13 0.00 Pay for electricity, water 15.2% 16.9% -1.7% -0.98 0.33 Clothing 82.0% 88.2% -6.2% -3.05 0.00 Buy household goods 19.5% 33.4% -13.9% -5.74 0.00 Education / school fees 4.9% 4.9% 0.0% 0.00 1.00 Health care related expenditure 29.1% 22.1% 6.9% 2.90 0.00 Contraceptives and family care 2.1% 3.2% -1.1% -1.28 0.20 Support to other family 13.5% 13.3% 0.2% 0.09 0.93 Invest in business or other 18.4% 8.6% 9.8% 5.12 0.00 Savings 55.9% 43.2% 12.8% 4.89 0.00 Source of data: YCS Midline Survey Assets Youth Potential beneficiaries are using their income to build assets and invest in their own business especially in purchasing animals. Significantly more survey respondents reported owning animals, land, mobile phones, televisions and motor bikes/scooter at midline as compared with the baseline. A mobile telephone was the most reported asset, followed by animals and land for grazing and farming. This was true regardless of youth participants’ sex, age group, or region. Seventy-three percent of the respondents reported that their families owned a house, while only about 18% reported owning a house 46 Even though the current exchange rate is 1 USD=27.06 Birr, the rate used here is 1 USD=22.90 Birr, the rate prevalent in April 2017 to allow for meaningful comparison between baseline and midline 29 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline at the individual level (down by two percentage points from baseline).47 It is also important to note that since the participants live with their families they also have access to the assets owned by the family and may be contributing to their acquisition. Table 10 has the details. Table 10 Distribution of individuals reporting asset-holding and comparison with baseline (%) Assets owned Owned by family (N=533) Owned by individual (N=533) Midline Baseline Difference T- value p-value House 73% 18.2% 20.3% -2.1% -1.166 0.244 Radio 32% 18.0% 20.5% -2.4% -1.335 0.183 TV 13% 5.4% 4.1% 1.3% 1.151 0.250 Mobile 60% 86.9% 82.9% 3.9% 3.088 0.002 Non-mobile Phone 1% 0.0% 0.0% 0.0% -- -- Bicycle 2% 1.9% 2.3% -0.4% -0.577 0.564 Animal Drawn cart 4% 1.1% 2.1% -0.9% -1.213 0.226 Motor Cycle 2% 1.5% 0.6% 0.9% 2.245 0.025 Car/Truck 1% 0.4% 0.2% 0.2% 0.577 0.564 Land for farming 66% 24.0% 18.8% 5.3% 2.591 0.010 Animals 73% 57.2% 48.2% 9.0% 3.558 0.000 Other (sewing machine, bee -hive, Bajaj vehicle, refrigerator, etc.) 1% 1.7% -- -- -- -- Source of data: YCS Midline Survey Savings As described above, all respondents were asked how they spent their money. Regardless of whether they were employed or unemployed, 56% of the participants (55% male, 57% female) reported putting away money in savings over the past six months, significantly (p<.01) higher than at baseline. However, a detailed examination of the data reveal very different patterns regionally. Almost all respondents from Tigray region (90%) and two thirds from Amhara region (71%) reported that they had put away money as savings, while only about half of the respondents from Oromia (49%) and less than a tenth from Afar (8%) put away money. However, this does not mean that the rest have no savings. As several focus group participants commented – instead of putting away money as savings, they were investing their earnings into their business, buying animals especially shoats (sheep and goats), land or larger tea and coffee shops to expand the scope of the business. Afar is a pastoral region where the main Activity for all participants is animal rearing and fattening, so investment into animals is understandable. Across the four regions, the range of amount saved was large, ranging from 50 Birr ($2.18 USD) to thirty thousand Birr, ($1,310 USD; 12% of those saving reported saving between $218 USD to $1310 USD), with participants from Oromia and Tigray reporting higher amounts saved. Over half of those that save put away money a few times a year/seasonally (54%) and about a third do so every month (35%). While a small proportion (17%) reported keeping their savings at home, the majority put it in a bank or microfinance institution (83%). Of those that reported not saving, 83% said that they had no money to save and the rest reported investing it in animals, returning loans or spending on things like clothes and shoes. Focus group data support the above findings. 47 Based on a comparison of only those responding to both baseline and midline surveys (n=533). An examination of the data showed that the change in homeownership is due to a change in the individuals marital status or living arrangements (living independently to living with parents). However, the change is not statistically significant. 30 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline FGDs discussions and KIIs revealed that youth are using various strategies for saving. Some youth in Afar were members of village savings groups started by elders, and youth in Tigray were members of various savings group, including a lottery-based social group. Youth FGD participants said that the follow-on coaching by Youth Potential facilitators helped keep them accountable because they were asked monthly if they were saving and if so, how much. FGD participants also attributed their willingness to work on savings related activities to the training received through WRN!. For instance, one daily laborer in Amhara said that “before I used to spend what I earned in the same day, but now I have learned to save and spend according to a plan.” Youth FGD participants also said that they were using their savings to invest in their business to grow the business. For example, they were saving to buy water pumps (parent and youth FGD, Amhara). In Oromia, youth had more limited ability to save regularly because they were not generating enough income. In some cases, youth stopped saving because their income decreased. As one young man from Yabelo said, “we learned about saving but we don’t have what to save.” Change in Spending Over Time Data was collected from a small subgroup of participants (n=80; approximately 20 per region) on a monthly basis to examine in detail changes in patterns of household expenditure as well as changes in utilization of health care. Figure 15 presents the average monthly expenditure for this subgroup of Youth Potential participants over four months from June to September. (See Annex VIII for a detailed distribution). Figure 15 Average monthly expenditure for select group of participants across regions Note: Region average is calculated based on relative shares of each region in total number of respondents interviewed. *Omitted the outlier for Afar (for expenditure of 45,000 Birr/$1965USD in June) Also in Afar, several respondents reported very high expenditures ranging from 4,700 to 13,965 Birr (205-610 USD) due to medical expenditures for illness in the family Source of data: YCS Midline Survey The data indicates that while expenditures move up and down, there is a general peak in expenditures by the participants in the month of August followed by a slowdown in September. This peak, noted in all regions and for both men and women, could be due to any number of reasons, including the fact that this was the month preceding a major Ethiopian holiday, a time when almost everyone in Ethiopia purchases new clothes and travels. A detailed examination of the monthly data revealed that the monthly expenditures incurred by men were larger than those incurred by women for the same period, and that while women spent more on running the household, men reported larger investments in business and purchase of animals. These findings are as expected based on the cultural norms prevalent in Ethiopia, where men are more likely to spend money on major household expenditures as well as 31 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline expenditures for purchase of agricultural products such seeds and tools, while women (unless they are the heads of households) are more likely to incur expenditures on food for the family, clothes, and the like48. Determinants of Income The relationships between youth income and demographic characteristics and the employment status and the sector of employment were examined. To identify which of the observed characteristics of the March/April 2017 cohort significantly affect the likelihood of reported increase in income, the study used a logistic model to estimate the effect of these observed characteristic of interest (the independent variable) on respondents’ acknowledgement of increase in income. The results are presented and interpreted in terms of the odds ratios, defined as the likelihood of income increase for each of these observed characteristics as compared to an internal reference group. Demographic characteristics such as sex of the participants, age group, marital status, and household size, self-reported employment status and sector of employment were included to capture their employment status to control for their where they were employed. Locational heterogeneity was controlled by creating dummy variables for region. Annex 8 has the results. Findings suggest that male participants are almost 1.4 times (or 40%) more likely to report that their income had increased relative to female participants. Similarly, the odds of reporting higher income peak for those who are married (95%, compared to those not married), have larger families (35%) and are in the 19-24 years’ age group (compared with those in the 15-18 or 25+ age groups). However, none of these by themselves are likely to make a significant difference. While an individual’s employment status (self-employed, wage employed, unemployed, etc.,) did not have a significant effect on their level of income, an individual’s ability to obtain a loan is weakly related to their change in income.49 To analyze the effect of the YCS on change in employment between the baseline and midline, Fairlie decomposition method was used to identify and quantify the change that occurred due to respondent level characteristics, such as their gender, education, experience, marital status, and their locational heterogeneity, and the change due to the intervention.50 The explained portion of the gap between baseline and midline represents the changes due to respondent level characteristics while unexplained portion of the gap between baseline and midline indicates the contribution of the intervention, assuming that there are no effects of unobserved factors such as macroeconomic changes during this period. The findings suggest that there was a 27 percent (from probability of 47 percent to 74 percent) change in the likelihood of the participants being employed from baseline to midline, of which only 6 percent could be explained by respondent level characteristics such as gender, income and whether they received loans. (see panel II of table 3, in Annex 8). There was no significant effect of location or region on the likelihood of employment, as indicated by the level of significance of the regional dummies. Hence, the remaining 20 percent change in the likelihood of employment between mid and baseline can be attributed to the intervention (Youth Potential). This is also supported by qualitative data from the focus groups with participants and their parents and guardians. Youth reported a change in attitude towards work in general, and self-employment in particular, resulting in stronger work ethic, greater pride in 48 Rubin et al 2009; African Development Bank 2004. Ibid 49 Please note that the comparisons were drawn using Tigray, where the loans are easier to obtain, as a reference group. 50 Fairlie decomposition method is a widely used technique used for identifying and quantifying the separate contributions of group differences in measurable characteristics, such as education, experience, marital status, and geographical location, to racial and gender gaps in outcomes for binary variables, such as employment, college attendance, or teenage pregnancy, Here (in this report) it is used to identify changes that occurred in employment— a binary outcome (employed=1 and unemployed=0). Ref: Fairlie, R. W (Nov. 2003) An Extension of the Blinder-Oaxaca Decomposition Technique to Logit and Probit Models http://www.econ.yale.edu/growth_pdf/cdp873.pdf 32 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline their products, better customer service, improved farming and livestock practices, and hence greater productivity. CHANGE IN HEALTH OUTCOMES One hypothesis51 underlying the Youth Potential Activity is that increased income will lead to improved health outcomes among study participants, with the participants spending more on their health and health-related expenditures, including better nutrition, preventive care and family planning.52 Survey respondents reported that they spend an average of $17 per month (median expenditure $11) per month on health care related expenditures and an average of $2 on contraceptives and family planning. Focus group participants reported that they spend money on health care as needed, but did not plan for it or have a contingency fund. They incurred expenditure on health care only when they themselves or their family members were unwell. Depending on the severity of the illness, most reported visiting the local health clinic, a health provider or a nurse, and going to the hospital in the city in a case of severe illness or an emergency. A few also reported going to traditional healers or to “holy water”. Figure 16 Provider where health care sought (self-reports by participants) (%) Source of data: YCS Midline Survey Only 33% of the survey respondents had been unwell over the last six months and about half of them (49.7%) had taken time off (1-7 days) work to recuperate. Malaria, typhoid, abdominal aches and pains as well as common cough and cold were the most commonly cited illnesses. Approximately two-thirds of the respondents (64%) had visited a health care provider for themselves while a little over a quarter (28%) had taken a family member to a provider (health care worker, nurse or a doctor). Figure 16 presents the distribution of preferred health care providers by region. Those who sought treatment reported going to a government hospital, health center or a government health post. Sixty four percent of the respondents reported visiting a health care provider, between 1-2 times over the past six months, for themselves or while 28% reported visiting them for someone in their families. A very small number (3%, n=16) of the respondents reported going to a traditional healer. Almost half of the respondents reported receiving medicines (45%), female contraceptives (10%), and 51 Responds to the second part of the research question - To what extent has the income and employment status of study participants changed between enrollment in the Youth Potential Activity and the twelve months following enrollment? Does increased income lead to improved health outcomes among study participants? 52 A review of Activity documents and conversations with Activity staff indicated that the Youth Potential Activity does not directly address the topic of health care; if it comes up at all, it is in the context of workplace safety. 33 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline advice on nutrition (28%) from the provider. Of those who had not needed health care, over half indicated that if necessary they would go to a government facility (53%). Participants in focus groups conducted by the evaluation team indicated that they did not notice any change in the number of days that they were ill for. However, some participants reported that after participation in the WRN! training and as a result of the ongoing group and individual follow-on coaching, they had “learned to be more patient”, they were “less quick to anger” and tried to put the customer’s perspective first. They were also working harder and this, combined with the patience and a general change in attitude, has had the general effect of making them feel less anxious and less stressed leading to fewer headaches and ill-health. Several of them also reported that they had decreased their consumption of alcohol, tobacco and chat as a result of the need to live within their budget as per the personal development plan (WRN!), a finding corroborated by parents and guardians in a focus group. They also reported, as cited elsewhere in this report, that they were eating more vegetables, such as cabbage, but fruits were not available to them. However, an equal number said that there was no change in their food habits, and that they were just saving more. A clear majority of survey respondents (between 80% and 90%) agreed or strongly agreed that as compared with six months ago, they were, in general, happier, their health had improved, they were less stressed, less anxious and less angry and they took better care of their health. A somewhat smaller proportion of the respondents agreed that they had fewer headaches (71%), ate better food (78%) or spent more money on food (61%). Table 11 has the details. Table 11 Change in health status and eating habits (self-reports by participants Compared with six months ago* Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree My health has improved 38% 46% 12% 3% 0% I am less stressed 31% 54% 7% 7% 1% I am less anxious 31% 53% 8% 7% 1% I get fewer headaches 24% 47% 16% 11% 2% I am less angry 28% 56% 8% 8% 2% I eat better food 24% 54% 13% 6% 3% I spend more money on food 14% 47% 20% 13% 5% I take better care of my health 35% 54% 8% 3% 0% I am happier 46% 41% 6% 6% 1% *Rows may not add to a 100% due to rounding. Source of data: YCS Midline Survey SELF-PERCEPTION OF WORK READINESS SKILLS Participants in Youth Potential, depending on their levels of functional literacy and numeracy, are enrolled in WRN! or WRN plus training components offered by Youth Potential, followed by other more advanced training programs. These programs provide the participants with training on soft skills for the workplace and increases their awareness of the resources available to them to support their search for employment/better employment including self-employment. 34 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Figure 17 Self-ratings on measures of work readiness Source of data: YCS Midline Survey To assess the impact of the Activity on work-readiness skills, participants were asked about their self￾perceptions of their work readiness as well as several of the more common skills needed to find employment. Overall, participants reported a high opinion of their own readiness to work, rating themselves highly (good or very good) in terms of their communication skills, work habits and conduct, ability to build self-confidence, team work, ability to set goals and objectives, and saving habits. Figures 17 and 18 have the details. Figure 18 Self-ratings on measures of knowledge of resources Source of data: YCS Midline Survey The respondents also rated themselves highly on skills needed for start-up of self-employment such as knowledge of financial resources, business start-up and management skills, obtaining licenses for a business, etc., rating themselves as “good” or “very good” on most items on a five-point scale ranging from very poor to very good The only exceptions were the participants self-perceptions of job search skills, such as writing CVs, knowledge of job search resources and knowledge of ways and means of accessing resources (finance, land, inputs) on which the survey respondents did not rate themselves very highly (more of neutral and good ratings). A comparison of the data showed that the participants rated themselves significantly higher in six of the fourteen skills relative to the baseline. Table 12 has the 35 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline details. Male participants rated themselves higher (i.e., as being very good) compared to female participants. Table 12 Participants’ self-perception of Work Readiness Skills Mean of self-perception ratings* (n=533) Midline Baseline Difference T- value p-value Interpersonal Communication 4.45 4.33 0.12 3.33 0.00 Work Habits 4.17 4.12 0.06 1.19 0.24 Job search application skills 2.71 2.58 0.13 2.15 0.03 Where to look for jobs 3.29 3.18 0.11 1.90 0.06 Business management 3.40 3.39 0.02 0.25 0.80 Skill to start up 3.50 3.56 -0.06 -1.11 0.27 Knowledge obtaining loans 3.53 3.50 0.03 0.40 0.69 Access to resources finance, land etc. 3.20 3.00 0.20 3.03 0.00 Access to market 3.51 3.53 -0.02 -0.34 0.73 How to obtain license for business 3.36 3.29 0.07 1.11 0.27 Savings habit 3.75 3.50 0.25 3.82 0.00 Setting objective/goals/personal development plan 3.91 3.84 0.07 1.51 0.13 Social bond/Team work 4.00 3.90 0.11 2.12 0.03 Building Self confidence 4.42 4.38 0.04 1.08 0.28 *Means computed by assigning the values of 5 to “very good” and 1 to a self-rating of “very poor”. Source of data: YCS Midline Survey Focus groups conducted with beneficiaries of Youth Potential from all four regions support these findings. During focus groups, the participants reported that the training gave them the skills, knowledge, and motivation to start their own business or improve their existing business; provided them with the skills to improve their savings habits and better manage their finances. They went on to say that the personal development plan developed by all the participants during the training has assisted them to articulate their goals and targets and by following the plan several of the participants have been able to expand the scope of their ongoing activities. In addition, the interpersonal skills and tools for anger and frustration management as well as the recognition of importance of time management have also helped the participants to grow. They reported working longer hours and more days every week, resulting in increased income and more productivity. Parents and guardians participating in focus groups also supported the beneficiaries’ self-perceptions. They reported that the youth showed positive changes after WRN! training including being better organized, managing their time better, not staying out late at night, taking care of their siblings, and not being idle and hopeless. One parent in Tigray commented, “Before the youth were unemployed, roaming about without a job, angry and with tension. Now, they are occupied, happy at home, sociable, and get along with the community.” Self-perceptions of these changes in skills as well as attitude and behavior were also reflected in the survey data. Most of the respondents agree or strongly agree that as compared with six months ago they have better time and financial management skills and are more confident of their abilities to plan and manage their work, whether it be self-employment, group employment or wage employment. They also agree that they are more patient and persistent and their business has grown. Table 13 has the details. It is also more interesting to note that a higher proportion of male respondents reported that they agree on most statements than the female respondents. 36 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Table 13 Changes in skills and behavior post participation in Youth Potential (self-perception, %) As compared with six months ago: Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree % (N=533) I feel more confident of my abilities 66 32 2 0 0 I know how to develop a business plan 18 68 12 1 1 I know how to calculate profit/loss 63 32 5 0 0 I spend according to plan 43 51 5 1 0 My business has expanded 34 52 14 1 0 I have better marketing skills 25 49 6 13 6 I am able to interact better 55 21 7 8 8 My business has more customers 16 59 5 8 12 I work more days of the week 51 28 13 7 1 I have better time management skills 52 37 11 1 0 I am more patient 25 59 12 4 0 I get into fewer fights and arguments 8 25 14 34 19 I am more persistent 18 66 14 2 0 Feel more capable 36 54 8 2 0 I am more respectful 59 37 4 0 0 I am willing to try different kinds of jobs 59 36 5 0 0 I am more motivated to work 82 11 6 1 0 Source of data: YCS Midline Survey KNOWLEDGE OF AND ACCESS TO EMPLOYMENT/ENTREPRENEURIAL RESOURCES As stated earlier in this report, there has been a significant change in the reported employment status of Youth Potential participants followed for this study. At baseline, the participants in this cohort had indicated that prior to enrolling in Youth Potential, 46% were unemployed, 41% were self-employed (either for themselves or their families) and 6% were engaged in wage employment. The rest either were in school or were not looking for work. Six months’ post enrollment, while there has only been a 3 percentage points change in the number of those engaged in wage-based employment, the situation has changed dramatically for those unemployed and self-employed. The number of those unemployed has reduced significantly (change of 24 percentage points from 46% to 23%,) since baseline, 10% of this group are now engaged in wage employment and over 50% are self-employed. Forty eight percent of all survey respondents (49% male and 36% female) reported having tried to find employment in the past six months. While this proportion is very similar to the baseline, the major difference is that over half of this group (53%) is currently self-employed compared to a quarter at baseline, and more of them are female. The most common avenues for looking for new opportunities were asking relatives and friend for help followed by participation in job fairs, registration with public employment offices and checking directly at worksites. Applying directly to employers was also common. Female participants reported being more likely to use avenues such as job fairs, seeking assistance from friends, and registration with the public employment office (in that order), while male participants preferred seeking the assistance of friends, followed by job fairs. Only 6% of male respondents reported responding to advertisements on webpages/the internet, radio, or in newspapers, 37 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline and 5% of the female respondents used the radio; none of the females used the internet to search for jobs. Similar differences were also noted regionally, with seeking assistance from friends and relatives being the preferred option in Tigray, and attending job fairs the preferred option in Amhara and Oromia, and registration with the public employment office in Afar (Figure 19). Figure 19 Avenues for finding employment/new employment, by region (%) Source of data: YCS Midline Survey Data from the survey of Youth Potential participants further provided evidence of the influence of the information provided by WRN!. Respondents across all regions, especially in Amhara, were aware of the need for arranging finance for creating self-employment (reported by 70%). The need to learn new skills (46%) ranked at par with the need to locate land, space and equipment (43%), as well as seeking assistance of family, friends and colleagues (43%). The need to obtain permits and licenses was also reported by a small proportion, a finding congruent with that reported elsewhere that the beneficiaries did not rank themselves very highly on obtaining license as a skill. Figure 20 has the details. Figure 20 Information required for creating self-employment, by region (%) Source of data: YCS Midline Survey 9% 1% 5% 8% 5% 41% 54% 46% 42% 50% 47% 10% 28% 12% 23% 5% 3% 15% 12% 9% 5% 1% 28% 9% 10% 9% 10% 12% 31% 13% 0% 12% 4% 2% 6% 0% 1% 3% 2% 2% 23% 73% 59% 79% 61% Afar (n=95) Amhara (n=172) Oromiya (n=171) Tigray (n=95) Regional Total (N=533) Avenues for Finding Employment (%) Education/Training Jobs fair Through Public Employment office Private employment office Direct application Checking at work sites News paper ads Internet ads Assistance from friends/relatives 34% 26% 57%60% 43% 11% 73% 36% 36%43% 38% 97% 70% 56%71% 16% 7%22% 1%12% 40%48% 42%57% 46% Afar (n=95) Amhara (n=172) Oromiya (n=171) Tigray (n=95) Regional Total (N=533) Information for Creating Self-Employment(%) Seek assistance from others Look for land, building etc. to establish enterprise Arrange for finance Apply for permit/licenses Learn new skills 38 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline These findings of mixed effects of the WRN! training is also reflected in the findings from the results from focus groups conducted with the beneficiaries. Focus group participants reported that the training has helped develop and enhance their soft skills and knowledge of the resources available to them. For instance, several focus group participants reported that as a result of the community mapping exercise, they are more aware of the opportunities for work and new business ideas as a well as the resources available to them within the community. Participants in the focus groups indicated that as a result of the training they are aware of the availability and process of obtaining individual and group loans, especially from microfinance institutions. Thanks to the WRN! training, they have the skills to develop a business plan to support their loan application as well as the support from the facilitators in preparing the application. The youth also reported that participation in WRN! assisted them in forming the initial groups that can be continued beyond the training for purposes of obtaining a group loan. However, while there are a few success stories, a majority of youth participants in the focus groups from Amhara, Afar, and Oromia indicated that they have not been able to obtain loans from financial institutions to start up their own business due to challenges such as lack of collateral and enough savings for individual loans. Several participants reported obtaining loans from their families has been the more common option. For group loans, they have not been successful due to a lack of agreement within the group as to the purpose of the planned business. These challenges are compounded by the limited availability of microfinance institutions across the regions. In Tigray, where microfinance loans were much more readily available with lower collateral or savings requirements, 22 out of the 35 youth focus participants have obtained loans ranging from 10,000 to 60,000 Birr ($435-2,620 USD). Survey data support these results. Less than a quarter of all survey participants reported that they had applied for a loan, however, within regions, the distribution was very different. In Afar, Amhara and Oromia less than one fifth (9%, 17% and 15% respectively) had applied for a loan, while in Tigray 71% of the respondents had applied for loans. Similarly, over 70% of those that applied for a loan in Tigray received the loan, in most cases (69%) for a group Activity. In the other three regions, only about a quarter of those that applied for a loan received it. In Afar and Oromia half of these loans were for group activities, while in Amhara they were individual loans. Similarly, the youth reported that the training provides them with the know-how of submitting the applications for licenses and permits needed for self-employment. The facilitators in all regions introduce the youth to the government officials at the kebele and woreda level. For example, in Tigray, officials from the woreda administration office, youth and sports office, microfinance office, women’s affairs office, and other offices meet with youth beneficiaries after the WRN! training to share information about the resources they have to offer such as microfinance, agriculture extensions services, and additional training; and as a result, youth in Tigray were more knowledgeable about kebele-level resources. While not as well organized as in other regions, the participants are aware of how and where to find information. As a focus group participant from Ahuri, Amhara, said “now we are getting better information because we have been trained in how to get information from the micro and small-scale enterprise agency – an expert from there provided us with information about entrepreneurship.” During interviews with several employers and private training providers the evaluation team learned that the employers while jobs are few and far between, the woreda officers and facilitators from Youth Potential are working with them to make them aware of the existence of this Activity and its trainees for future consideration. Some of them reported that based on their experience with the trainees that they have hired, they would be willing to consider more candidates from the Youth Potential Activity to fill future positions. 39 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline FUNCTIONAL LITERACY AND NUMERACY SKILLS While not a focus of the Youth Potential Activity, an expected benefit of enrolling in the training is improved levels of functional literacy and numeracy.53 At baseline, most participants, male and female, reported being able to read and write fluently, and most had more than just a functional level of literacy (the only exception was Chifra in Afar region). Analysis of survey data indicates that 70% of the participants have completed grade 8 or higher (about 8% have completed TVET or college programs). Survey respondents were again given simple tests of literacy and numeracy to assess their ability to read and write and do basic arithmetic at midline. As reported elsewhere in this report, almost three quarters (74%) of the participants demonstrated an ability to read and write with some fluency and almost all (94%) could recognize numbers and do basic addition (90%). However, there are some gender differences, in that more male participants could read fluently and while more females can recognize numbers, the proportion of females that demonstrate ability for higher order skills falls rapidly. Among the regions, participants from Afar had lower levels of literacy and numeracy compared to participants from other regions. Figure 21 has the details. Figure 21 Level of functional literacy and numeracy, by region, sex and age (%) Source of data: YCS Midline Survey It is interesting to note that there has been a significant change in the number of survey respondents reporting that they can read and write fluently, recognize numbers and do basic math. (Table 14) 53 The Youth Potential Activity description on the USAID website reads: “Rural youth will receive literacy, numeracy, and life skills training alongside vocational and entrepreneurship development activities through partnerships with training, small-business, and micro-finance institutions.” (ref: https://www.usaid.gov/ethiopa/education). However, review of Activity documents as well as interviews with Activity management show that the Activity is focused only on life skill training alongside vocational and entrepreneurship activities, it is not providing any training on literacy or numeracy. 40 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Table 14 Change in level of functional literacy and numeracy, from baseline Literacy and Numeracy Baseline Midline Difference p value.¹ Functional Literacy Mean Can’t read & write 0.08 0.11 0.03 <.001 Can read & write with difficulty 0.08 0.08 -0.01 Can read & write with some fluency 0.14 0.08 -0.06 <.001 Can read & write very easily 0.70 0.74 0.04 <.01 Functional Numeracy Recognize numbers 0.97 0.94 -0.03 <.01 Add simple numbers 0.92 0.90 -0.02 Subtract simple numbers 0.85 0.87 0.02 Multiply simple numbers 0.84 0.83 -0.01 Divide simple numbers 0.83 0.83 0.00 Source of data: YCS Midline Survey These findings are supported by the data collected through FGDs where most beneficiaries said that there was no change in their literacy and numeracy skills. Multiple young men and women across all four regions stated, “I knew how to read and write before joining the Potential training, and there has been no change in my ability.” At the same time, they also reported that the use of reading and writing skills during training and in the preparation of their personal development plans and business plans has reinforced these abilities. Participants also reported that managing their finances, developing budgets and monitoring their inputs and profits has also helped reinforce their ability to do simple addition and subtraction. For example, in Afar, female participants in a focus group reported that they have a group of businesses in farming and livestock activities, and they depended on one male secretary in their group who is skilled in literacy and numeracy to manage their business accounts and calculate the group’s income, expenditures, and profits. A female participant from Amhara who had less than Grade 10 education said she is using her math skills to count her hens and to register the eggs laid by the hens. In another instance, male participants from Afar reported using mobile phones to calculate the numbers for their business. Additionally, some participants reported an increased interest in reading more and greater ability to concentrate which in general is helping improve their ability to read and write. (Youth FGD, Amhara, Tigray) For example, one young female in Amhara who was a high school drop-out said that before she used to give up on reading and writing, but now she tries to read whatever she can get access to and tries to do more writing. 41 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline CONCLUSIONS The YCS is following a subset of youth over a period of 12 months to enable better understanding of: (a) participant-level outcomes; and (b) the extent to which the Youth Potential Activity has improved the employment status and/or income level of a subset of youth participating in the project. The overall purpose of this midline study was to collect qualitative and quantitative data on various indicators for a cohort of participants at six months after enrollment in the Youth Potential Activity in order to learn of their progress following the training. Data collected through a survey of participants and focus groups and KIIs conducted with the participants themselves, their parents and guardians, the implementation staff and other stakeholders appear to suggest that the Activity is well appreciated by the participants and, in the short-term (six months after enrollment), is beginning to have a positive effect on work readiness as well as employment and income status of the participants, regardless of their sex or region of residence. The rest of this section summarizes the findings presented for the midline study, provides a brief description of the training received by the participants and draws interim conclusions to respond to each of the evaluation questions on the effect of the Youth Potential Activity on participants’: (i) income and employment status; (ii) health outcomes; (iii) work readiness skills; (iv) knowledge of and access to employment/entrepreneurial resources; and (v) functional literacy and numeracy. The final section presents challenges to the program and the conduct of and recommendations on how activities could be better designed to improve the effectiveness of the program. Training received At a minimum, all study participants had completed WRN! or WR Plus training programs. While all participants in Amhara, Oromia and Tigray attended the WRN! training; in Afar, 56% of the participants attended WRN! and the remaining 44% the WR Plus program. Depending on the availability of the trainings, participants have also attended other follow-on trainings, such as BYOB and PYD54. A small proportion of the participants have also received training in technical skills provided by external providers under the Youth Potential Activity (e.g., TVET, TTI or private providers). The most popular areas for training were agriculture and sheep fattening in Afar, Amhara and Oromia regions, followed by poultry farming and trading in Amhara and Oromia. In Tigray, however, fewer participants benefitted from this program and the areas of choice were skills for hotel and tourism industry, and hairdressing. Some of the other options selected by participants included baking and running a bakery; carpentry, carving, or woodwork; mechanics and metal work; construction work (brick/stone laying, building, roads, etc.); running coffee and tea shops; and vegetable farming, with more males selecting skills such as construction, bakery and tailoring and more females selecting training in hairdressing and hotel and tourism sectors. To support the participants in the use of their training, they are provided with coaching from the facilitators and mentoring by experts in the field. While the mentoring program is just starting and there have only been a few instances of participants engaging in job shadowing activities in the four regions included in this study, coaching is provided in individual or group settings on a regular basis. The coaching provides the participants with a support group as well as a person they can talk to about their issues and concerns. These sessions also serve the important purpose of holding the participants accountable for the implementation of plans they have developed, thereby helping them to become more organized, more productive as well as to be able to save more. The MOUs between Youth 54 Per SCI, PYD has since become the starter piece of the training and is offered to all participants prior to the WRN component. 42 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Potential Activity and regional employers were also beginning to result in opportunities for the Youth Potential participants, increasing their work readiness. Employment Status Prior to participating in the Youth Potential Activity almost half of the participants were unemployed and dependent on others for their sustenance. More women than men reported that they were self￾employed, with two thirds reporting that they were managing self-owned businesses. More male than female participants reported supporting family-owned activities, and a very small proportion were engaged in group-owned businesses. Agriculture- and livestock-related activities, daily labor and trading consumer products were the preferred fields for men and women reported trading consumer products. Six months after participation in the WRN! training offered by Youth Potential, almost 74 percent of the 533 beneficiaries surveyed reported being employed, which included 230 that remained employed from baseline and 165 newly employed in the six months since baseline; less than 5% of the beneficiaries reported becoming unemployed since baseline. Two-thirds of all survey respondents were self-employed and only 9% were engaged in wage employment. The most significant gains in employment were seen in Afar and Tigray regions, 90 percent of the respondents from the Tigray region reported they were employed. While there were regional differences, more male (61%) than female (38%) survey respondents reported being employed, more males reported managing self-owned businesses (52%) compared to females (44%), and fewer males were unemployed as compared with females. Those engaged in wage employment were working as security guards, Bajaj drivers, TVET assistants, school teachers, etc. For the self-employed, agriculture and livestock remain the primary source of employment, followed by trading. Many of these differences are in keeping with the gender norms prevalent in Ethiopia and hence culturally acceptable. However, participation in the Youth Potential Activity has opened a space for the female participants to explore avenues of work, such as construction, that were traditionally not open to them and to recognize the need for assistance (for example someone to help with accounts) and arrange for support. Survey respondents, especially those from Afar and Tigray, were mostly satisfied with their employment status. The training has brought about a change in attitude towards work among the beneficiaries of the Youth Potential Activity, with many preferring self-employment over wage employment. The training has helped them to value and respect their own work “no matter how lowly,” and to improve their work ethic. They now recognize that any work that helped them earn an income was employment. The community mapping exercise conducted as a part of the WRN! training (including WR Plus) aided the beneficiaries in identifying new business ideas that they could implement in their own communities. In addition, WRN! provided them with the skills needed to manage a business and handle customers. One recurrent theme heard in all focus groups across the four regions was that the training had made them realize the importance of “starting with something small, and to grow their Activity/business gradually,” resulting in many of them feeling encouraged to start their own business or to work more seriously on their existing work. Participants in the focus groups had mixed opinions regarding work in the agriculture sector. Many of them felt it was a good opportunity and a good source of income, that “it’s the backbone of the Ethiopian economy” and that it gave them the opportunity to stay in their rural community. However, others felt that the work was too hard and challenging and that the constraints such as expensive inputs and frequent droughts made the work unattractive. Lack of education and training, not enough jobs, lack of finance, and a mismatch between training and job requirements were stated as the main reasons for unemployment. For those engaged in self-employment, both male and female participants reported several hurdles in setting up their own business, including difficulty in obtaining working capital and financing, lack of land and working space. 43 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Income and Expenditure Sixty one percent of survey respondents reported an increase in income over the past six months, with the largest changes reported by participants in Tigray. More men than women reported that their income had increased over the past six months, a time line consistent with enrollment in the Youth Potential Activity and WRN! training. Even though the majority of the beneficiaries had increased income, the increase did not result in a change of life style for them. They said that because of the additional income they may spend a little more on entertainment or clothes as compared with six months ago, but in most cases the additional funds went towards investment in business, payment of rent, purchase of food and savings. Significantly more participants reported purchasing mobile phones, motorcycles, land or animals as compared with baseline. Several also indicated that they were spending more on food and eating better. Because of the financial management skills learned through the WRN! training, the beneficiaries, both males and females, reported that they now spend money according to their plan and put away money in savings on a regular basis. The follow-on coaching as well as strategies such as participation in lottery￾based social groups have helped to reinforce the habit of saving money. Health expenditures Health care is a discretionary expenditure and one that is easily overlooked, except in an emergency. Depending on the severity of the illness, most reported visiting the local health clinic, a health provider or a nurse, and going to the hospital in the city in a case of severe illness or an emergency. A few also reported going to traditional healers or to “holy water”. Participants, both males and females from all four regions, reported that after participation in the WRN! training, and because of the ongoing group and individual follow-on coaching, they had “learned to be more patient”, were “less quick to anger” and tried to put the customer’s perspective first. A clear majority of survey respondents (between 80% and 90%) agreed or strongly agreed that compared with six months ago, they were working harder, were generally happier, were less stressed, less anxious and less angry and took better care of their health. A somewhat smaller proportion of the respondents agreed that they had fewer headaches (71%), ate better food, i.e., more vegetables and larger quantities (78%) or spent more money on food (61%). Several participants reported that they had decreased their consumption of alcohol, tobacco and “chat” as a result of the need to live within their budget as per the personal development plan developed because of the WRN! training, a finding corroborated by parents and guardians in a focus group. However, an equal number said that there was no change in their food habits, and that they were just saving more. Work-Readiness Skills Participants in focus groups indicated a strong preference for self-employment over salaried employment, especially with the government. Overall, they had a high opinion of their own readiness to work, rating themselves highly (good or very good) in terms of their communication skills, work habits and conduct, ability to build self-confidence, team work, ability to set goals and objectives, and saving habits. The respondents, more men than women, also rated themselves highly on skills needed for start￾up of self-employment such as knowledge of financial resources, business start-up and management skills, obtaining licenses for a business, etc., rating themselves as “good” or “very good” on most items on a five-point scale ranging from very poor to very good The only exceptions were the participants’ self-perceptions of job search skills, such as writing CVs, knowledge of job search resources and knowledge of ways and means of accessing resources (finance, land, inputs) on which the survey respondents did not rate themselves very highly (more of neutral and good ratings). Participants reported that the training gave them the skills, knowledge, and motivation to start their own business or improve their existing business provided them with the skills to improve their savings habits 44 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline and better manage their time and finances. They went on to say that the personal development plan developed by them during the training had helped them articulate their goals and targets, and by following the plan several participants have been able to expand the scope of their ongoing activities. In addition, the interpersonal skills and tools for anger and frustration management as well as the recognition of importance of time management have also helped the participants to grow. They reported having better time and financial management skills, and more confidence in their abilities to plan and manage their work. They were working longer hours and more days every week compared with six months ago, resulting in increased income and more productivity. Access to Employment/Entrepreneurial Resources Six months’ post enrollment, while there has only been a 3 percentage points change in the number of those engaged in wage-based employment, the situation has changed dramatically for those unemployed and self-employed. The number of those unemployed has reduced significantly (change of 23 percentage points from 46% to 23%.) Forty four percent of all survey respondents (49% male and 36% female) reported having tried to find employment in the past six months. While this proportion is very similar to the baseline, the major difference is that over half of this group (53%) are currently self-employed compared to a quarter at baseline, and more of them are female than at baseline. The most common avenues for looking for new opportunities were asking relatives and friend for assistance followed by participation in job fairs, registration with public employment offices and checking directly at worksites. Applying directly to employers was also common. Female participants reported being more likely to use avenues such as job fairs, seeking help from friends, and registration with the public employment office (in that order), while male participants preferred seeking the assistance of friends, followed by job fairs. Participants, both male and female, were also knowledgeable of the various requirements for self-employment. Respondents across all regions, especially in Amhara, were aware of the need for arranging finance for creating self-employment. The need to learn new skills ranked at par with the need to locate land, space and equipment, as well as seeking assistance of family, friends and colleagues. The facilitators supported the individuals and/or groups with completing the loan applications, getting the youth introductions to the kebele and woreda level government officials for letters of support, and in general assisted with troubleshooting any process related concerns. Overall, the Youth Potential participants reported feeling more empowered in their search for employment and in creating self-employment, and reported being able to get better and more complete information because of their training. Functional Literacy and Numeracy Skills While not a focus of the Youth Potential Activity, an expected benefit of enrolling in the Activity is improved levels of functional literacy and numeracy. At baseline, except for participants from Chifra in Afar region, most participants reported being able to read and write fluently, and most had more than just a functional level of literacy. These self-reports were confirmed using simple tests of literacy and numeracy. At six months’ post enrolment, a larger proportion of the survey respondents reported increased levels of fluency in reading and better numeracy skills. However, most beneficiaries participating in focus groups said that there was no change in their literacy and numeracy skills. Multiple young men and women across all four regions stated, “I knew how to read and write before joining the Potential training, and there has been no change in my ability.” At the same time, they also reported that the use of reading and writing skills during training and in the preparation of their personal development plans and business plans has reinforced these abilities. Participants also reported that managing their finances, developing budgets and monitoring their inputs and profits has helped reinforce their ability to do simple addition and subtraction. They were also more aware of the need for literacy and numeracy and were either making efforts to improve their own skills by making a conscious effort to read and write more, they were also looking to bring in someone with the necessary skills to help them with their accounts. 45 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Next Steps This midline study will be followed by the end-line study that will assess the changes on the outcomes of the participants at a point 12-months after enrollment in the Activity. Using the panel design employed both for the baseline and midline, the study will use longitudinal data collected by following the cohort for a 12-month period. The key feature of such a panel design is that it will collect data on the same variables from the same sample at different points in time. While more complicated and difficult to carry out than simple cross-sectional studies, a panel offers a robust design to evaluate changes at the end of the study period and the contribution of the Youth Potential Activity to these changes. CHALLENGES The challenges noted can be grouped into those related to the Youth Potential Activity and those related to the conduct of this study. They are discussed in turn. Challenges related to Youth Potential Activity: 1. Staff turnover: There is a very high level of staff turnover at the field level among the facilitators, woreda officers and even the regional managers. In theory, the person leaving provides detailed notes on the ongoing activities for their successor so that there is no break in services. In practice, however, the handover is not that smooth, the notes may or may not be there and often there is no overlap between the person leaving and their replacement leaving the participants in limbo. The relationship between the facilitators and the participants as well as the one-on-one coaching provided to the participants is jeopardized; the problem is compounded further if the program officer also changes. For example, in Amhara, the whole team – Program officer, M&E officer, and some of the facilitators – was new, having joined a week to three weeks prior to the evaluation visit and were still trying to catch-up. A similar situation was noted in Afar at the time of baseline data collection. Changes were noted at the Woreda level as well. While there were hand-over notes available, they varied in consistency and detail, and relationships have to be established anew with participants and stakeholders, slowing down all activities. 2. The Youth Potential Activity encourages participants to participate in coaching and mentoring until the participants feel settled enough in a new job or their self-employment to not need coaching any more. While a good thing from the perspective of the participants, it places an undue burden on the facilitators. With the ever-expanding pool of participants as the Activity expands to cover all eligible youth in all Kebeles in select woredas, the facilitators are stretched thin and as a result the quality of follow-up suffers. 3. The Youth Potential staff also indicated that based on experience with previous cohorts, it was a challenge to ensure that the participants that enroll in technical training programs provided by external providers and paid for by Youth Potential Activity complete their technical training. They reported that Youth Potential must pay for a certain number of slots to the providers in advance of the training, and if the participants do not attend, or are not serious about the training, it is a waste of resources. 4. Managing budget and partner expectations is a challenge. The partners have signed an MOU where the Youth Potential Activity has agreed to provide them with training or materials in exchange for internships/WBL opportunities for Youth Potential beneficiaries. Often the procurement process for these materials can be lengthy and slow, causing some frustration amongst the partners and slowing the availability of opportunities for the beneficiaries. 5. One of the mainstays of self-employment is the ability to obtain start-up funding. However, loans from banks and/or microfinance institutions are difficult to obtain. Participants in all regions, except Tigray where the loan requirements are lower, reported that they were unable to meet the requirements for collateral and savings and so did not get the loan. In Afar, 46 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline microfinance agencies do not exist at the time of writing this report and are planned for the near future. This was corroborated by the parents/guardians of the participants, the facilitators as well as the loan officers themselves. 6. A related challenge is the ability of the participants to organize themselves into groups. Loans are easier to obtain for group-based self-employment, as the group itself forms the collateral. However, participants engaged in group-based self-employment are few and far between as they are either unable to agree on the scope of work, or are unable to trust the group members. 7. Parents and guardians provide the collateral for loans, and in the absence of the loans from financing organization, put up the funds or the land to support their wards in self-employment. However, they are not directly involved with the Activity or have any first-hand information of what the youth are learning, making it difficult for them to support the initiatives. 8. Availability of employment opportunities in the region is a challenge. Wage-based opportunities that pay reasonably well are few and far between. On self-employment, there is a dearth of space (crates) for setting up a business (trading or providing services) and land for agriculture￾and livestock-related activities, the two mainstays of economy in these regions. 9. Youth are mostly focused on crop production and livestock activities. They are not working across agriculture crop value chains, where there are possible opportunities as distributors, input suppliers, marketing, and post-harvest for transporting the produce. 10. In addition, the youth are also not exploring the untapped market linkages. For example, the Holeta agriculture TVET officials in Oromia said there are value chain opportunities in dairy production and milk productions, animal science, fruits and vegetables, and flower farming; and that they would be interested to collaborate with Potential in supporting youth to better connect to these value chains. The area is called the “milk shade” and yet we did not come across any youth who were active in dairy farming from the study participants. Additionally, the Raya Brewery in Tigray has contracts with smallholder farmers in the community to grow and supply barley, and the brewery pays them above market prices. The brewery expressed interest to source from Potential youth farmers in the future. Likewise, a representative from the sugar factory in Afar said that they currently source sugarcane from farmer cooperatives in the community, and this could also be a potential market linkage for youth farmers in Afar. Challenges Related to the Conduct of the Study One challenge that the team faced in the conduct of the study pertained to data collection. While a detailed list is included in the section on limitations of the study (in the chapter on Study Approach), the more important ones are repeated here. These include: (i) non-availability of data on activities such as training programs attended and coaching and mentoring received by participants from the March/April cohort from SCI; (ii) difficulty in obtaining information on individual income and expenditures, since most participants live with their families; and (iii) difficulty in finding all survey respondents from the baseline study, despite best efforts. Solutions for these challenges were found using alternate measures, such as using data from administrative reports to replace enrollment data and data on trainings, using proxy data on ownership of items to determine income, building in the possibility of attrition into the sample size, and so on – but the challenges and limitations remain. Another challenge related to our inability to determine if there were training programs other than Youth Potential that were available to the participants in the selected woredas. This is important, in so far as the presence of other training programs could compromise the impact of the interventions and causality of changes in the indicators after one year of enrollment. RECOMMENDATIONS This midline study is designed to provide an interim view of any changes from baseline six months post participation in the Youth Potential Activity. However, during course of data collection a few points of 47 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline note/recommendations that might result in better outcomes for the participants were noted. Several of these recommendations are systemic and may need to be addressed through joint efforts by all stakeholders, including the implementing partners, government counterparts, funding organization as well as the private sector. The recommendations are placed below, followed by our assessment of specific actions by various partners. Overall Recommendations • Microfinance loans: Availability of funding is the most basic requirement for self-employment, both for start-up and for later expansion. However, except for beneficiaries in the Tigray region that has modified the requirements for collateral and savings for microfinance loans, it has been difficult for beneficiaries of the Youth Potential Activity to obtain a loan. There is a need for concerted efforts at all levels to examine the model followed in Tigray and make it easier for the beneficiaries to obtain loans. These efforts may include introduction of the Youth Potential beneficiaries to the loan officer at the kebele/woreda level, and a change in the requirements for collateral and savings through a dialogue between the government, Youth Potential Activity and the microfinance institutions. • Obtaining land for agriculture and space for self-employment activities: There is a paucity of land and available spaces for activities such as trading, opening a shop for provision of services, activities along the agriculture and livestock value-chains, etc., and a felt need for mobilizing community resources (for example freeing up land, loan of equipment, and farming tools, as seen in Lalibela, Amhara region.) At the local level, there is a need for increased support and advocacy for the beneficiaries by the facilitators and woreda officers. Similarly, at the woreda and regional levels there is a need for modifications to the policy to make it easier for the beneficiaries to access working space. • Greater involvement of parents and guardians: Parents and guardians of Youth Potential beneficiaries, in most cases, provide the collateral for loan applications, and in the absence of loans and other resources, provide the funds and/or land to help support the start-up efforts. However, many of them reported they would like greater involvement in the training program and perhaps a briefing from the implementers to learn more about the ongoing training and its possible benefits. • Coaching and mentoring: It was very clear from the FGDs discussions with the beneficiaries of the Youth Potential Activity that the individual and group level coaching was an important reason for the success of the training. Given the design of the Activity that allows for all beneficiaries to access coaching, there is a need to balance the workload of the facilitators so that they can remain effective. In addition, while the facilitators have a coaching manual, they are beginning to repeat topics of coaching and there is a need for new material to keep the interest of the participants engaged over the longer term. Mentoring is just starting, and there is a need to maximize coverage and training of participants to ensure they understand the meaning of mentoring and to prepare them to get the most out of the training. It is also necessary to bring in mentors to support the interests and the needs of the participants. • Assisting beneficiaries organize into groups: Microfinance loans are easier to obtain for group activities. However, the beneficiaries of the Youth Potential Activity have, for various reasons ranging from inability to agree on scope of work to lack of trust in the other members of the group, not been able to organize themselves into groups to maximize the opportunities for obtaining a loan. As a part of the WRN! training the Youth Potential Activity helps participants organize into groups. It would be worthwhile to explore if the model followed in Tigray region where the Youth Potential beneficiary groups have binding group constitution/rules, transparent accounting, and are provided with coaching in conflict management, can be incorporated into the Youth Potential training in other regions as well. 48 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline • There is a need for the beneficiaries of Youth Potential Activity to diversify and access new market opportunities. Most of those self-employed are engaged in agriculture and trading. However, very few of them are engaged in activities related to the agriculture supply chain (e.g., procurement of seeds, seedlings, and tools; marketing and transport of agricultural products; etc.), where there are good potential opportunities. While Youth Potential training helps beneficiaries to be aware of resources, it may be necessary for the coaches to spread the word about new opportunities in the supply chain space as well as with new industry as they become available. Coaches and Woreda officers could help establish linkages for youth with ongoing government and donor-funded programs and activities to assist the youth with loans and financial services, mentoring and skills training as well as value chain opportunities. • There is a perception among Youth Potential beneficiaries of a mismatch between job opportunities available in the region and their own qualifications. The coaches can support the youth by sharing information about job opportunities, map the requirements to the skills available and then helping the youth respond to these opportunities. In addition, the Youth Potential staff at the regional and woreda level must work with potential employers and help them become aware of the skill levels available locally so the employment opportunities can be aligned with the skills... • There is a need to support staff at the field level, reduce turnover, optimize logistics and provide transportation to enable staff to reach more of the participants to provide coaching. and to ensure continuity in the training and follow-up for the participants • Linkages must be established, where possible, with existing government programs to ensure that Youth Potential participants are aware of the resources available to them and how to access them. Resources for those engaged in agriculture and animal husbandry are available at the community and the Woreda level through existing government programs, for instance, agriculture extension workers can offer guidance to youth starting out in farming on better farming practices and strategies for better outputs. • Similarly, linkages must be established to other donor funded activities, especially those funded by USAID. This is especially important for those activities that focus on the agriculture sector and provide financial services and start-up capital for youth. Linkages to Feed the Future Activities will be useful to help the youth gain awareness of market-based value chain opportunities (production, inputs, post-harvesting, transportation, etc.) and perhaps a foot in the door with some of these activities. 49 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline ANNEXES 50 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline ANNEX I: YOUTH POTENTIAL ACTIVITY RESULTS FRAMEWORK SO: Unemployed and underemployed Ethiopian youth ages 15 - 29 in rural areas and towns attaining the skills, knowledge, and social capital that lead to increased income and long-term economic self-sufficiency will be achieved by improving access for Ethiopians to basic education, TVET, and practical work experiences and bringing together local stakeholders such as TVET institutions, private sector, youth, and government IR 1: Rigorous evidence base developed to inform USAID and its development partners, including youth, GOE, and NGOs, on appropriate, effective, and sustainable market driven youth workforce development interventions IR 1.1 Local context and beneficiary profile information base developed to inform the design of customized interventions that enable access to opportunities for youth skills development, decent work and increased income IR 1.2: Evidence base developed to enable program learning and course correction, and to inform future USAID and GOE investments in youth IR 2: Improved youth access to market relevant skills and experiences to support the transition to safe and viable employment IR 2.1: Improved access to quality and relevant basic life, work readiness and employability skills training, including literacy and numeracy learning IR 2.2: Improved access to quality career coaching and work-based learning (WBL) services IR 3: Coordinated delivery, among local institutions, of quality market driven services connecting undeserved youth, especially women, to employment and income opportunities IR 3.1: Increased capacity of local training and business service providers to integrate & deliver market driven employment IR 3.2: Existing and newly established networks of youth service provides and local industries are coordinated to connect youth to employment 51 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline ANNEX II: YCS STATEMENT OF WORK INTRODUCTION USAID/Ethiopia developed this Statement of Work for an independent performance evaluation of the USAID/Ethiopia’s Building the Potential of Youth. The findings of this evaluation will inform Activity mid￾term course corrections and future youth workforce development programs in Ethiopia. USAID’s Building the Potential of Youth is a five-year Activity implemented by SCI International in partnership with EDC, HUNDEE, and PADet, REST and FCE. The Activity’s information is summarized below. Activity Name USAID’s Building the Potential of Youth Cooperative Agreement AID-663-A-15-00006 Start Date – End Date Jan 1, 2015 – December 31,2019 Total Estimated Cost $ 17,326,954 Activity Funding DA, Higher Education Implementing Partner SCI Activity AOR Tahir Gero ACTIVITY OBJECTIVES AND IMPLEMENTATION APPROACH The purpose of the USAID’s Building the Potential of Youth Activity is to provide Ethiopian youth (ages 15-29) in rural areas and towns with access to workforce development/livelihood support and resources, tailored to their specific needs and market demand, so that they can a) achieve increased income and b) strengthen skills, knowledge and social capital required to achieve economic self￾sufficiency over the longer-term. The Results Framework of the Activity is included in the Annex section of this document. USAID’s Building the Potential of Youth proposed strategies and approaches respond to capacity needs of unemployed and underemployed youth, leverage existing youth development activities, and contribute to improving the quality and accountability of local training and business service providers by ensuring existing entities work in concert. ● By using targeted assessments, the USAID’s Building the Potential of Youth team will ensure training and service providers consider market relevance and target curricula and programs to emerging skills needs of youth participants. ● By focusing on skills building, will offer tailored technical and life skills trainings to youth to create more viable livelihood prospects. ● By expanding approaches and offerings in WBL, including employer visits, organized job￾shadowing, short-term employment, and internships/WBL opportunities both learners and employers benefit from training and supervisory support, leading to workforce-ready and employable youth with practical experience. ● By implementing “Hubs,” USAID’s Building the Potential of Youth will build Youth Service Provider Networks and provide a cost-effective approach to enhance support service access for remote communities and disadvantaged groups. SCOPE OF WORK 52 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Purpose & Overview The purpose of this task is to conduct an independent M&E Activity for USAID’s Building the Potential of Youth Activity. The activities performed under IR1 (“Rigorous evidence base developed to inform USAID and its development partners, including youth, GOE, and NGOs, on appropriate, effective, and sustainable market driven youth workforce development interventions”) are not subject to evaluation under this Task Order. The services to be provided under this Activity include: ● Activity A: A Performance Evaluation including baseline (limited), midline and endline data collection, analysis and reporting. ● Activity B: A Performance Monitoring and Data Systems Review that will include an analysis of SCI and its partners’ data collection, storage, analysis and reporting systems. ● Activity C: A YCS focused on a subset of youth participants and geographic implementation areas. Activity A: PERFORMANCE EVALUATION Evaluation Questions Table 1 below outlines the evaluation questions to be addressed by the Performance Evaluation. The Table indicates whether the question is applicable to the baseline survey, midline or endline evaluation, likely sources of data and potential data collection methods. Wherever applicable/possible, the analysis for all the questions listed above should be presented by sex, age category [15-19, 20-24, and 25-29], SES, location, educational attainment, mother tongue and ethnicity. The information provided below is intended to assist evaluators in the development of a work/staffing plan and budgets. 53 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Table 1: PERFORMANCE EVALUATION QUESTIONS MATRIX Questions (in order of priority) Baseline/Midline/ Endline Suggested Data Sources Notes Geographic focus and illustrative sample/selection INDIVIDUAL-FOCUSED QUESTIONS 1 To what extent do youth participating in the Building the Potential of Youth have improved employment (including self￾employment) status? How do these outcomes vary by participant characteristics and partner? Midline, Endline Secondary analysis of SCI and partners’ data from intake and exit participant surveys [SCI Youth Profile Data Collection Tool] Midline: Aggregate and analyze data on all participants, or a sub-sample of participants. Endline: Aggregate and analyze data on all participants, or a sub-sample of participants. 2 To what extent do youth participating in the Building the Potential of Youth have increased income? How do these outcomes vary by participant characteristics and partner? Midline, Endline Secondary analysis of SCI and partners’ data from intake and exit participant surveys [SCI Youth Profile Data Collection Tool] Midline: Aggregate and analyze data on all participants, or a sub-sample of participants. Endline: Aggregate and analyze data on all participants, or a sub-sample of participants. 3 What are the outcomes, [measured in income, productive assets, and debt level and employment quality] for youth receiving microfinance loans facilitated by the Building the Potential of Youth program, how do these outcomes vary by participant characteristics and partner? Midline Endline Primary data collection with microfinance recipients. Secondary analysis of SCI and partners’ data (to the extent available). Midline: data collection from initial 46 recipients of microfinance in Tigray. AND assume a sample of 80 recipients across 6 Regions and 18 Woredas for primary data collection and analysis. Endline: assume a sample of 80 recipients across 6 Regions and 18 Woredas for primary data collection and analysis. 4 What percentage of WRN! completers engage in quality WBL opportunities facilitated by SC and its partners? How do these outcomes vary by participant characteristics and partner? Midline, Endline Secondary analysis of SCI and partners’ administrative data to determine overall percent. Plus, sample￾based primary data collection from youth participants to determine quality of opportunities. Midline: assume a sample of 60 recipients across 4 Regions and 8 Woredas for primary data collection and analysis. Endline: assume a sample of 80 recipients across 6 Regions and 18 Woredas for primary data collection and analysis. 5 What percentage of internships/apprenticeships transition into paid employment? How do these outcomes vary by participant characteristics and partner? Midline, Endline Secondary analysis of SCI and partners’ administrative data to determine overall percent. Plus, sample￾based primary data collection from youth participants to validate and supplement data. Midline: assume a sample of 60 recipients across 4 Regions and 8 Woredas for primary data collection and analysis. Endline: assume a sample of 80 recipients across 6 Regions and 18 Woredas for primary data collection and analysis. 6 To what degree has the Building the Potential of Youth program improved the functional literacy and numeracy of youth participants? Endline Secondary analysis of SCI and partners’ administrative data. 7 How effectively has the Building the Potential of Youth program produced gender-equitable outcomes for youth participants? Endline Analysis of secondary and primary data. 54 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline SYSTEM-FOCUSED QUESTIONS 8 To what extent have the “YES Hubs” implemented by the Building the Potential of Youth program increased access to quality employment-related services (e.g., job facilitation, business development, entrepreneurial training, connection to resources) for participating youth? Midline Endline Sample-based primary data collection from YES Hub stakeholders, including youth facilitators, youth participants, partner staff. Midline: assume a sample of 16 “YES Hubs” (stratified by type of Hub), across 4 Regions and 8 Woredas for primary data collection and analysis. Endline assume a sample of 36 Yes Hubs (stratified by type of Hub) across 6 Regions and 18 Woredas for primary data collection and analysis. 9 For Baseline: 9b1.What is the status of youth focused training in the TTIs in terms of availability, quality and relevance for the needs of young men and women? 9b2.What are the major challenges of local training centers/ institutes in target Woredas to integrate and deliver market driven trainings? For Midline and Endline: 9e1.To what extent have funds provided to Training Institutes increased the availability, quality and relevance (in terms of employment/income) of training for youth? Baseline/Midline, Endline Primary data collection (including baseline) from Training Institute staff and employment partners, and youth participants. Secondary analysis of SCI and Training Institute administrative records, as available. Baseline: assume a stratified sample of 18 Training Institutions across Amhara, Tigray, Oromia, Somali, Afar and SNNPR regions (Afar and SNNPR baselines may need to take place at midline due to TI selection process timing). Midline: assume a stratified sample of 18 Training Institutions across Amhara, Tigray, Oromia and Somali regions. Endline: assume a stratified sample of 18 Training Institutions across Amhara, Tigray, Oromia, Somali, Afar and SNNPR regions. 10 To what extent and in what manner are public-private partnerships providing quality training and/or employment opportunities for participants in the Building the Potential of Youth program? Endline Primary data collection with a sample of employment partners. Secondary analysis of SC and partners’ partnership records. Endline: assume a purposeful stratified sample of 36 partnerships across 6 Regions and 18 Woredas for primary data collection and analysis. 11 What is the operational and service delivery capacity of the Central YES Hub? How sustainable (operationally and financially) is the Central YES Hub likely to be post-USAID support? Endline Primary data collection focused on Central YES Hub, utilizing organizational capacity and sustainability assessment frameworks. 12 Is the Building the Potential of Youth training model (Master Trainers training youth trainers who then train participants) achieving the stated learning objectives of the WRN! curriculum? Midline Primary data collection including knowledge assessments, classroom observations during WRN! training. Plus, secondary analysis of SC and partners’ data. Midline: assume a sample of 20 youth trainers and 80 youth participants across (4) partners (Amhara, Tigray, Oromia and Somali regions). 55 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Evaluation design should support analysis of implementation fidelity and efficacy for each cascade. 13 What have been the most significant successes and challenges for the design, implementation and M&E (treated separately) for the Building the Potential of Youth program? Endline Summary analysis of secondary and primary data collected. Midline Performance Evaluation In addition to addressing the questions outlined in the Evaluation Question matrix above (those indicated as part of the midline), the Contractor must analyze performance monitoring data as well as quarterly and annual reports to identify priority areas of interest for the midline data collection/analysis. Priority areas of interest may include: ● Geographic areas where Activity monitoring data indicates under or over performance vis-a-vis targets. ● Local implementing/training partners reporting participant outcomes of interest such as low/high retention rates, low/high performance on skills assessments, low/high job placement rates, low/high improvements in employment/income status. Final selection of midline priority areas of interest and midline design will be conducted in collaboration with USAID and SCI. As part of the Midline Evaluation, the Contractor will conduct an in-country workshop with USAID, SCI and its three local partners, government officials and other stakeholders (at the direction of USAID) to review the draft report and key findings. The workshop outcomes will include: A summary of key success factors to date with action plan to expand successes. ● A prioritized list of program/implementation corrective actions with associated action plan and timeline. ● A prioritized list of monitoring/data collection corrective actions with associated action plan and timeline. ENDLINE PERFORMANCE EVALUATION The Contractor will conduct an in-country workshop with USAID, SCI and its partners, government officials and other stakeholders (at the direction of USAID) within 4 months of endline data collection to review the draft report and key findings. The workshop outcomes will include: 1. A summary of key lessons learned regarding design and implementation of USAID’s Building the Potential of Youth. 2. A prioritized list of research/evaluation questions for future youth workforce development activities in Ethiopia. 3. A prioritized list of policy recommendations to improve employment and economic outcomes for sub population of youth as needed. ACTIVITY B: PERFORMANCE MONITORING AND DATA SYSTEMS REVIEW The Contractor for the EPMES contract, who will be managing this Activity, hereinafter “Contractor,” is required to conduct a Performance Monitoring and Data Systems Review to initiate the activities under this Activity. The objectives of the Performance Monitoring and Data Systems Review include: 56 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline ● Determine the extent to which SCI and its partners’ performance monitoring and data systems are capturing reliable data aligned with the Building the Potential of Youth PMEP. ● Provide recommendations for improving the quality, utilization, reliability and scope of the data being collected by SCI and its partners, in support of the requirements of the PMEP. ● Implement a follow-up review to determine the extent to which the recommendations have been implemented. It is anticipated that this Activity will include detailed reviews of measurement strategies, instruments, data collection, aggregation, and analysis process, data storage and reporting processes. The Contractor should include a report detailing the recommendations. Additionally, the Contractor must plan and facilitate an in-country workshop with SCI, its three local partners and USAID to review the recommendations and identify priorities and process for integrating recommendations into the Building the Potential of Youth monitoring and data systems. The information obtained during the Performance Monitoring and Data Systems Review (in terms of availability and quality of implementing partner data) should be utilized to finalize the Performance Evaluation and YCS designs. ACTIVITY C: YCS USAID desires a YCS to deepen its understanding of participant outcomes. The overarching purpose of the YCS is to determine the extent to which the Building the Potential of Youth program has improved the employment status and/or income level of a subset of youth participating in the program. Specifically, the YCS will address the following questions: 1. To what extent has the income and employment status of study participants changed between enrollment in the Building the Potential of Youth program and twelve months following enrollment? Does increased income led to improved health outcomes among study participants? 2. To what extent has participation in the Building the Potential of Youth program improved the work readiness skills of study participants? 3. To what extent has participation in the Building the Potential of Youth program improved knowledge of and access to employment/entrepreneurial resources for study participants? 4. To what extent has participation in the Building the Potential of Youth program improved, functional literacy and numeracy skills of study participants? 5. How do the outcomes addressed in the questions above vary by participant characteristics (e.g., sex, age, ethnic/linguistic group, SES, marital status, and parenting status) and service delivery partner? The Contractor is expected to collect three or four rounds of data (depending on budget) on a sample of youth (panel design) across four Regions (Oromia, Amhara, Tigray and Somali). Data collection will take place in 10-15 Woredas across the four Regions. The Woreda selection will be led by the Contractor in consultation with SCI and USAID/Ethiopia and should be stratified across productive and pastoral Woredas. The Contractor should able to propose innovative methods for collecting reliable data on income and employment over time (such as financial diaries) for a panel of youth with potentially high mobility. For budgeting and planning purposes, the Contractor should assume a sample of 90-100 youth participants in each region, for a total approximate sample of 360-400 youth participants. The Contractor will develop a sample design that balances precision, efficiency and cost, in consultation with 57 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline SCI and USAID/Ethiopia. Strengths and limitations of the sampling approach shall be described in the Design Report. The Contractor should propose a design for the YCS, utilizing the illustrative timeline below as a guide: Key Evaluation Principles USAID’s Evaluation Policy encourages independent external evaluation to increase accountability for results, refine ongoing activities, and improve future designs. Transparency: Extensive documentation of evaluation design, sample and selection approach, data collection methods, instruments and other processes will be required so that USAID and other audiences can determine the internal and external validity of the evaluation. This understanding is critical for appropriate use of evaluation findings. Leveraging Best Practice: The Contractor is expected to utilize best practices when developing the evaluation design. The Evaluation Design must reflect proven methods for sampling, selection, data collection and analysis. All instruments utilized in the performance evaluation and YCS should be based on tested measurements for youth programming. Consultative Design Process: The Contractor should plan for a highly consultative process with USAID, SCI and the government representatives to develop study designs and instruments under this Activity. EVALUATION DESIGN AND METHODOLOGY The methodology proposed must comply with USAID’s Evaluation Policy (http://www.usaid.gov/evaluation). A mixed method approach utilizing quantitative and qualitative methods shall be included to answer the evaluation questions. To extent possible, data should be disaggregated by sex, age, geographic location, SES, ethnic/linguistic group, etc. Data collection methods proposed by the contractor should be linked to the specific baseline/evaluation question(s). The suggested methodology should include, but is not limited to: ● Survey of beneficiaries. Baseline data collection/cohort intake Program intake Post-WRN! /other training assessment 4-8 weeks Data collection focused on employment, income, access to resources 3 months Endline data collection focused on employment, income, access to resources 12 months 58 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline ● Pre-and post-assessments of literacy/numeracy and work readiness skills of youth study participants. ● KIIs/Surveys with employers, technical vocational education and training (TVET) instructors, TTI leaders, etc. ● FGDs with youth participants. ● Observation of participant training, youth engagement groups, skills-building classes/workshops, WBL activities, etc. ● KIIs/Surveys with private sector partners. ● Existing Data Review of SC and implementing partners’ baseline data, assessments, and tracking records of program participants. EXISTING INFORMATION SOURCES Existing data and program information will be made available to the contractor by USAID, SCIs and its partners. A list of potential documents for the contractor to review is presented below. 1. Activity original and amended description (if any) 2. Activity Work Plan 3. Activity reports 4. Activity M&E Plan 5. Survey of youth pre-& post (after six months) participating in the Building the Potential of Youth, collected by SC and its partners. 6. Employer surveys 7. Out-of-school literacy (OSL) and numeracy assessments, implemented by SC and its partners. 8. Pre-& post- workforce readiness assessments, implemented by SC and its partners. 9. TTI records 10. Agreement documents between Building the Potential of Youth partners and employers 11. Tracking records of program participants including internships/WBL opportunities, class/workshop attendance, etc. Analysis of previously collected data and data collected by the contractor must be disaggregated and compared across groups as specified in the baseline/evaluation question(s). The contractor will determine the acceptable level of margin of error depending on the statistical measure used. In the report(s), the contractor must mention inherent limitations in the data collected and analyzed. GEOGRAPHIC SCOPE OF PRIMARY DATA COLLECTION The following section outlines the geographic scope of the anticipated primary data collection to be executed under this Activity. Secondary data analysis performed under this Task Order will cover the entire geographic scope of the Building Potential of Youth program. Baseline Data Collection The primary baseline data collection focused on Question 9 of the Evaluation Matrix (targeting the Training Institutions) will take place in intervention regions. The sample should be stratified, at a minimum, across type of Training Institution, productive/pastoral Woreda designation. See Evaluation Question Matrix for additional details. Midline Data Collection 59 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline The Midline Evaluation will include primary data collection in intervention regions. In each of the regions, data collection will take place in sample Woredas. The Woreda selection will be led by the Contractor in consultation with SCI and USAID/Ethiopia and should be stratified across productive and pastoral Woredas. YCS The YCS will include primary data collection in intervention regions. In each of the regions, data collection will take place in sample Woredas. The Woreda selection will be led by the Contractor in consultation with SCI and USAID/Ethiopia and should be stratified across productive and pastoral Woredas. Endline Data Collection The Endline Evaluation will include primary data collection in intervention regions. In each of the regions, data collection will take place in sample Woredas. The Woreda selection will be led by the Contractor in consultation with SCI and USAID/Ethiopia and should be stratified across productive and pastoral Woredas. ANTICIPATED TIMELINE The Table below outlines the anticipated timing for each of the major activities under this Task Order. The final timeline proposed by the Contractor should maximize data collection/field time efficiencies to the extent possible. Activity Anticipated Timing (in-country work) M&E Review July/August/Sept 2016 Baseline for Question #9 July/August/Sept 2016 Midline Feb/Mar/Apr 2017 YCS Feb/Mar/Apr 2017 Endline May/Jun/Jul 2019 DELIVERABLES The contractor is responsible for preparing and submitting the following deliverables: Activity A: Performance Evaluation Deliverables: 1. Performance Evaluation Design Report (EDR) (draft and final): The EDR should articulate how the design will address all questions in the Evaluation Question Matrix and should include a detailed methodology, sample design, data collection, management and analysis plan, data collection tools and timeline. The Performance Evaluation EDR should cover the baseline, midline and endline. The draft evaluation report will be submitted by the contractor to the USAID’s Program Office. USAID will provide comments within 10 business days. 2. Performance Evaluation Work Plan: Prior to each round of data collection, the Contractor shall schedule an in-briefing with USAID Thirty days in advance of this meeting, the Contractor shall provide a detailed work plan (draft and final) that includes a list of in-country consultations, a schedule for the field work, and a list of the team members with contact details while in-country. 60 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline 3. Baseline Study Report focused on Question #9 in the Evaluation Question Matrix (changes in Training Institute capacity and function). The Contractor will collect primary data to answer baseline questions in the performance evaluation questions matrix (see # 9b1 and 9b2 in Table 1). In addition, the Contractor will review the baseline/background information collected for each youth enrolled by the implementing partner and analyze the data collected both from primary and secondary sources and finally produce a baseline survey report. USAID will review and approve the Baseline Study Report., not exceed 20 pages of main text. 4. Midline Performance Evaluation Report (draft and final): The contractor shall prepare a descriptive and analytical report presenting the main findings and suggesting appropriate options and recommendations. Not to exceed 20 pages excluding Annexes. 5. Midline Performance Evaluation Executive Summary and Slide Deck (draft and final): The contractor shall prepare a 2-3-page Executive Summary and 10-15 slide presentation of key midline evaluation findings. 6. Facilitated in-country workshop with SCI, its partners, USAID and other stakeholders to review the findings of the Midline Evaluation. 7. Summary notes from in-country workshop. 8. Endline Performance Evaluation Report (draft and final). The contractor shall prepare descriptive and analytical report presenting the main findings and suggesting appropriate options and recommendations. Not to exceed 20 pages excluding Annexes. 9. Endline Performance Evaluation Executive Summary and Slide Deck (draft and final): The contractor shall prepare a 2-3-page Executive Summary and 10-15 slide presentation of key endline evaluation findings. Activity B: Performance Monitoring and Data Review Deliverables: 1. Performance Monitoring and Data Review Design Report (draft and final). The Design Report should articulate how the Contractor will approach the review and should include a detailed methodology, data collection, management and analysis plan, any data collection tools, and timeline. Additionally, the Design Report should include requests for information/data/staff time from SCI and USAID. 2. Performance Monitoring and Data Review Work Plan: Prior to in-country field work, the Contractor must schedule an in-briefing with USAID. Thirty days in advance of this meeting, the Contractor shall provide a detailed work plan (draft and final) that includes a list of in-country consultations, a schedule for the field work, and a list of the team members with contact details while in-country. 3. Performance Monitoring and Data Review Study Report (draft and final) that includes prioritized recommendations for USAID and SC. Not to exceed 15 pages excluding Annexes. 4. Facilitated in-country dissemination workshop with SCI, its partners, USAID, GoE representatives (Ministry of Youth and Sports, Ministry of Education and Ministry of Agriculture and Natural Resources) and other stakeholders to review the findings of the Performance Monitoring and Data Review Study. 5. Summary notes from in-country workshop. 6. Documented Follow-up Review Report: 4-6 months after the workshop to assess the degree to which priority recommendations have been implemented, based on interviews and consultations with stakeholders. YCS Deliverables: 1. YCS Design Report (draft and final): The Study Design Report should articulate how the design will address all 5 study questions and should include a detailed methodology, sample design, data 61 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline collection, management and analysis plan, data collections tools and timeline. USAID will review and approve the YCS Design Report. 2. YCS Work Plan: Prior to each round of data collection, the Contractor shall schedule an in￾briefing with USAID. 30 days in advance of this meeting, the Contractor shall provide a detailed work plan (draft and final) that includes a list of in-country consultations, a schedule for the field work, and a list of the team members with contact details while in-country. 3. Summary Report (5-7 pages) for each of the four data collection cycles (draft and final), including analysis of key data and findings. 4. Draft YCS Report following final data collection. 5. Facilitated in-country workshop with SCI, its partners, USAID and other stakeholders to review the findings of the YCS. 6. Summary notes from in-country workshop. 7. Final YCS Report (not to exceed 30 pages excluding Annexes) following in-country workshop. 8. YCS Report Executive Summary and Slide Deck (draft and final): The contractor shall prepare a 3-5-page Executive Summary and 10-15 slide presentation of key YCS Report findings. TEAM COMPOSITION The contractor is expected to have the appropriate competencies in the team to fully execute scope of this Task Order. The team should have expertise and significant experience in the following areas (not in order of priority): 1. Complex team and program management, including management of local sub￾contractors. 2. Evaluation design: including sample design, mixed method designs, data analysis and instrument development. 3. Assessment and use of existing data sources to be utilized for secondary analysis. 4. Measurement strategies associated with youth workforce development programs. 5. Data collection planning and implementation in complex and geographically diverse settings. 6. Presentation of evaluation findings to a broad range of stakeholders, including government officials, to optimize uptake of recommendations. 7. Effective collaboration and coordination with implementing partners and other local stakeholders during evaluation design and implementation. 8. Knowledge of USAID requirements with respect to data sharing, treatment of PII, ethical considerations/International Review Board. 9. Excellent written and oral communication in English and Amharic. 10. In-depth knowledge of the Ethiopian context and development programming in Ethiopia. 11. At least one Key Personnel must have demonstrable experience in integrating and using gender sensitive approaches in M&E work. 12. Experience in gender sensitive monitoring and evaluation methods 13. Knowledge of local languages Oromo, Tigrinya, Somali, Afar and/or other local languages among team members would be preferred. The Contractor must propose a staffing plan that includes team leader and other experts and fully supports the requirements of this Activity. The proposed staffing plan must explicitly address how the plan complies with the expertise and experience requirements cited above and how the team members will work together. Additionally, the staffing plan must describe what each team member's’ primary responsibility will be and their anticipated LOE for the duration of each task stipulated in this SOW. 62 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline The Contractor must also make use of services of a local subcontractor to support the work under this Activity (e.g., instrument translation/validation, data collection, logistics for field work). No evaluation team members shall have been directly involved in the implementation of the Building the Potential of Youth program. A statement of potential bias or conflict of interest (or lack thereof) is required of each team member. USAID MANAGEMENT The Contractor will identify and hire the evaluation team, pending the Contracting Officer’s Representatives (COR’s) and relevant technical office’s concurrence, assist in facilitating the work plan, and arrange meetings with key stakeholders identified prior to the initiation of the fieldwork. The evaluation team will organize other meetings as identified during the course of the evaluation, in consultation with EPMES’s Contractor and USAID/Ethiopia. The Contractor is responsible for all logistical support required for the evaluation team, including arranging accommodation, security, office space, computers, Internet access, printing, communication, and transportation. The evaluation team will officially report to the Contractor, SI. The Contractor is responsible for all direct coordination with the USAID/Ethiopia Program Office through the EPMES COR. From a technical management perspective, the evaluation team will work closely with Tahir Gero, USAID’s Building the Potential of Youth AOR in the Education and Youth Office. In order to maintain objectivity, all final decisions about the evaluation will be made by the Program Office. Additionally, the evaluation contractor will consult and coordinate with SCI and its partners during the design and implementation of this Activity. LOGISTICS The Contractor will be responsible for all travel and logistics associated with conducting the evaluation. 63 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Annexes Annex 1: Results Framework SO: Unemployed and underemployed Ethiopian youth ages 15 - 29 in rural areas and towns attaining the skills, knowledge, and social capital that lead to increased income and long-term economic self-sufficiency will be achieved by improving access for Ethiopians to basic education, TVET, and practical work experiences and bringing together local stakeholders such as TVET institutions, private sector, youth, and government IR 1: Rigorous evidence base developed to inform USAID and its development partners, including youth, GOE, and NGOs, on appropriate, effective, and sustainable market driven youth workforce development interventions IR 1.1 Local context and beneficiary profile information base developed to inform the design of customized interventions that enable access to opportunities for youth skills development, decent work and increased income IR 1.2: Evidence base developed to enable program learning and course correction, and to inform future USAID and GOE investments in youth IR 2: Improved youth access to market relevant skills and experiences to support the transition to safe and viable employment IR 2.1: Improved access to quality and relevant basic life, work readiness and employability skills training, including literacy and numeracy learning IR 2.2: Improved access to quality career coaching and work-based learning (WBL) services IR 3: Coordinated delivery, among local institutions, of quality market driven s ervices connecting undeserved youth, especially women, to employment and income opportunities IR 3.1: Increased capacity of local training and business service providers to integrate & deliver market driven employment IR 3.2: Existing and newly established networks of youth service provides and local industries are coordinated to connect youth to employment 64 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Annex 2: Geographic Coverage of the USAID’s Building the Potential of Youth Intervention The following Table outlines the 30 Woredas that will be targeted by USAID’s Building the Potential of Youth Table 1: Target Regions and Woredas PRODUCTIVE Amhara Tigray Oromia SNNPR Somali Afar Woreda Danegela Jawi Semin-Achefer Debube-Achefer Bahir-Dar Ketma Zuria Dera Woreda Ofla Raya Alamata Endamehoni Woreda Limu-Bibilo Kofele Ada’a Welmera Sinana Agarfa Liban Chiquala Woreda Endegene Merab Azernet Yem Special Woreda Woreda Woreda PASTORAL Yabello Dire Esera Hammer Jijiga Babile Gode Kelafo Chifra Dubti Asayita 65 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Annex 3: Targets by region and Phase Target by Phase Year 1 Year 2 Year 3 Year 4 Year 5 Total Phase 1 Woredas – 8 initial Amhara 600 600 600 600 600 3,000 Tigray 600 600 600 600 515 2,915 Somali 600 600 600 500 222 2,522 Oromia 600 600 600 600 600 3,000 Total 2,400 2,400 2,400 2,300 1,937 11,437 Phase 2 Woredas – additional 12 Amhara 600 600 600 600 2,400 SNNPR 900 900 900 900 3,600 Afar 600 600 600 600 2,400 Somali 300 300 300 300 1,200 Oromia 900 900 900 900 3,600 Total --- 3,300 3,300 3,300 3,300 13,200 Phase 3 Woredas – additional 10 Amhara 600 600 600 1,800 Tigray 300 300 300 900 SNNPR 600 600 600 1,800 Afar 300 300 300 900 Somali 300 300 300 900 Oromia 1200 1200 1200 3,600 Total --- --- 3,300 3,300 3,300 9,900 66 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Annex 4: Final List of Target Woreda and Intake POTENTIAL Program Yearly phase in operation Woredas No. Region POTENTIA L Worda s Enrolled Year 1 Enrolled Year 2 Enrolled Year 3 1 Amhara Debub Achefer X 2 Amhara Semin Achefer X 3 Amhara Bahir-Dar Ketma Zuria X 4 Amhara Dangela X 5 Amhara Jawi 6 Amhara Dera X X 7 Oromia Ada'a X 8 Oromia Liben Chulala X 9 Oromia Limu-Bilbilo X 10 Oromia Kofela X 11 Oromia Sinana X 12 Oromia Agarfa X 13 Oromia Welmera X 14 Oromia Yabello X 15 Oromia Dire X 16 SNNPR Endegegn X 17 SNNPR Mirab-Azernet X 18 SNNPR Esira X 19 SNNPR Yem Special Woreda X 20 SNNPR Hammer X 21 Tigray Raya Alamata X 22 Tigray Ofla X 23 Tigray Endemehoni X 24 Somali Jigjiga X 25 Somali Babile X 26 Somali Gode X 27 Somali Kelafo X 28 Afar Chifra X 29 Afar Dubti X 30 Afar Asayita X Total 8 12 10 67 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Annex 5: M&E and data collection activities under the Building the Potential of Youth cooperative agreement SC and its partners will be performing formative assessments at the Woreda level for a portion of the target Woredas. The assessments will include: ● Rapid youth assessment focused on knowledge, attitudes, practices and assets, with an emphasis on well-being ● Literacy and numeracy assessments of youth with the targeted profile ● Employment market assessment to identify employment opportunities, skills gaps and programed economic growth/employment opportunities ● Existence of youth service providers, their services, scale of reach, target youth population(s), and organizational capacity ● Existence of other youth-oriented programs (e.g., government, donor) The findings of these assessments will inform final Activity design and implementation approach and identify required customizations at the Woreda and Kebele levels. These assessments will be performed on a rolling basis as the Activity expands to additional geographic areas and are deliverables under IR1 of the Results Framework. SC sampling strategy for the assessments listed above is… ANNEX: Mapping of Intervention Areas ANNEX: Performance Monitoring Plan ANNEX: Implementation Timeline ANNEX: Targets (if applicable) ANNEX: Year 1 Annual Report ANNEX: Year 2, Q1 Report ANNEX: USAID Youth Workforce Development Research Agenda ANNEX: OTHER POTENTIAL reports??? Woreda Assessment Reports; Youth Profile Data; WRN Curricula Outline, YIA Curricula Outline, ANNEX: SC Youth Profile Data Collection Tool 68 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline ANNEX 6: Report FINAL REPORT CONTENTS 1. Title Page 2. Table of Contents (including Table of Figures and Table of Charts, if needed) 3. List of Acronyms 4. Acknowledgements or Preface (optional) 5. Executive Summary (3-5 pages) The executive summary should succinctly capture the evaluation purpose and evaluation questions; program background; evaluation design, methods; and limitations; and the findings, conclusions, and recommendations. 6. Introductory Chapter a. A description of the program evaluated, including goals and objectives. b. Brief statement on purpose of the evaluation, including a list of the main evaluation questions. c. Brief statement on the methods used in the evaluation such as desk/document review, interviews, site visits, surveys, etc. d. Explanation of any limitations of the evaluation—especially with respect to the methodology (e.g., selection bias, recall bias, unobservable differences between comparator groups, etc.)— and how these limitations affect the findings. 7. Findings: This section should include findings relative to the evaluation questions. The information shall be organized so that each evaluation question is a sub-heading. 8. Conclusions: This section must answer the evaluation questions based upon the evidence provided through the Findings section. The information shall be organized so that each evaluation question is a sub-heading. 9. Recommendations: Based on the conclusions, this section must include actionable statements that can be implemented into the existing program or included into future program design. Recommendations are only valid when they specify who does what, and relate to activities over which the USAID program has control. For example, recommendations describing government action is not valid, as USAID has no direct control over government actions. Alternatively, the recommendation may state how USAID resources may be leveraged to initiate change in government behavior and activities. It should also include recommended future objectives and types of specific activities based on lessons learned. The information shall be organized so that each evaluation question is a sub-heading. 10. Annex: The annexes to the final evaluation report should be submitted as separate documents— with appropriate labels in the document file name (e.g., Annex 1 – Evaluation SOW), and headers within the document itself—and may be aggregated in a single zipped folder. a. Evaluation Statement of Work b. Places visited; list of organizations and people interviewed, including contact details. c. Evaluation design and methodology. d. Copies of all tools such as survey instruments, questionnaires, discussions guides, checklists. e. Bibliography of critical background documents. f. Meeting notes of all key meetings with stakeholders. g. “Statement of Differences” h. Evaluation Team CV’s 1. Reporting Guidelines 2. The format of the report shall be consistent with the USAID branding guidelines. 3. The evaluation report should represent a thoughtful, well-researched and well- organized effort to objectively evaluate what worked in the program over the given time period, what did not, and why. 4. Evaluation reports shall address all evaluation questions included in the statement of work. 5. The evaluation report should include the statement of work as an annex. All modifications to the statement of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline need to be agreed upon in writing by the Program Office. 6. Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an annex in the final report. 69 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline 7. Evaluation findings will assess outcomes and impact on males and females, and data will be disaggregated by gender, age group, and geographic area wherever feasible. 8. Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). 9. Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative and/or qualitative evidence. 10. Sources of information, including any peer-reviewed or grey literature, will be properly identified and listed in an annex. 11. Recommendations will be supported by a specific set of findings. They will also be action-oriented, practical, and specific, with defined responsible parties for each action. 70 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline ANNEX III: BASELINE STUDY – EXECUTIVE SUMMARY United States Agency for International Development/Ethiopia (USAID/Ethiopia) has contracted with SI to implement the Ethiopia Performance Monitoring and Evaluation (EPMES) Activity. Under EPMES, USAID requested for an independent the performance evaluation of its Building the Potential of Youth (Youth Potential) Activity. This evaluation involves multiple approaches, which include: a baseline study; performance monitoring and data systems review and a multi-round cohort study of beneficiaries focused on a subset of youth participants and geographic implementation areas (henceforth referred to as the YCS. This report provides the baseline data for the YCS. The Youth Potential Activity is a five-year, USD 17.3 million USAID/Ethiopia Activity designed to provide youth (ages 15–29), especially girls and young women, in rural areas and towns with access to workforce development/livelihood support and resources, tailored to their specific needs and market demand, so that they can: (a) achieve increased income; and (b) strengthen skills, knowledge, and social capital required to achieve economic self-sufficiency over the longer term. Designed to reach approximately 34,537 unemployed or underemployed rural youth, the Activity is being implemented under a cooperative agreement between USAID and a group headed by SCI.55 Currently in its third year of implementation, the Activity is providing services in 30 woredas in Afar, Amhara, Oromia, SNNPR, Somali, and Tigray regions of Ethiopia. The overall objective of the YCS is to evaluate whether youth who receive workforce development/livelihood support achieve increased income and strengthen skills, knowledge, and social capital required to achieve economic self-sufficiency over the longer term. The study will provide the stakeholders, USAID, the GoE, and the implementers of Youth Potential with information about the Activity and progress towards near-term outputs and outcomes. In addition, it will demonstrate long￾term contributions of Youth Potential initiatives on a cohort of youth. It will highlight major strengths and weaknesses of the Youth Potential’s initiatives to enable the stakeholders to make mid-course adjustments as necessary to the Activity as well as inform the design of similar programs in the future. Approach The YCS is following a subset of youth from Afar, Amhara, Oromia, and Tigray regions over a period of 12 months to better understand participant outcomes and the extent to which the Youth Potential Activity has improved the employment status and/or income level of youth participating in the program. It seeks to address the following broad questions56: 1. To what extent has the income and employment status of study participants changed between enrollment in the Youth Potential program and the twelve months following enrollment? Does increased income lead to improved health outcomes among study participants? 2. To what extent has participation in the Youth Potential program improved the work-readiness skills of study participants? 3. To what extent has participation in the Youth Potential program improved knowledge of and access to employment/entrepreneurial resources for study participants? 55 Cooperative Agreement # AID-663-A-15-00006; the implementation team includes Save the Children (SCI) and its partners Education Development Center, Inc. (EDC), HUNDEE-Oromo Grassroots Development Initiative (HUNDEE), Professional Alliance for Development in Ethiopia (PADet), Relief Society of Tigray (REST) and Facilitator for Change Ethiopia (FCE). 56 From the EPMES statement of work 71 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline 4. To what extent has participation in the Youth Potential program improved functional literacy and numeracy skills of study participants? 5. How do the outcomes addressed in the questions above vary by participant characteristics (e.g., sex, age, ethnic/linguistic group, SES, marital status, parenting status) and by service delivery partner? This baseline study provides information on the status of each of the above among the study participants against which changes will be assessed and impact established. The focus of this study is to track changes on Activity beneficiaries included in the cohort and to document lessons learned for future similar Activity designs. Since Youth Potential covers a selection of eligible youth in specific locations/kebeles within the Woreda and is implemented over an extended period, the study will employ a panel design to demonstrate whether the Youth Potential Activity caused changes in the expected outcomes or impacts. In a panel design, participants from whom data are collected over time serve as their own controls, since characteristics of an individual observed earlier can be controlled for when analyzing for changes in the expected outcomes. Baseline data collected on a select group of beneficiaries will help assess the effectiveness of the Activity as well as help focus the Activity on intended results by defining measures of the success of the Activity. To respond to the research questions listed above, data are being collected on several qualitative and quantitative indicators and sub-indicators that together will inform the results of the training programs as well as future possible modifications to the programs. To capture changes over the short and medium terms and attribute them to the Youth Potential Activity, data are being collected from a select group of participants (cohort) who have recently enrolled in the Activity at three points: (i) baseline, in the month of enrollment in the Activity (the current phase); (ii) mid-line, at six months from the point of enrollment; and (iii) end-line, at twelve months post-enrollment in the Youth Potential Activity. This will be accompanied with periodic collection of income and expenditure data for the cohort. This baseline study was conducted during the period March to June 2017 using a mixture of qualitative and quantitative methods. Primary data were collected from a survey of 582 Youth Potential participants (out of a total expected sample of 670, response rate of 87%) from a purposive sample of 13 woredas in the four regions of Afar, Amhara, Oromia, and Tigray.57 All Youth Potential participants from each of the selected woredas who had enrolled in the program in the months of March and April 2017 were included in the sample. The survey instrument, designed in English, was translated into regional languages prior to administration. In parallel, qualitative data were collected by the study team through 29 focus groups with 277 participants (46% female) conducted in a purposive sub-sample of eight woredas in the four regions (two woredas in each region) and 54 KIIs and informational meetings held with stakeholders, including Government counterparts; TVET and TTI administrators; and Activity staff at the regional, woreda, and kebele levels. The Activity staff interviewed included the M&E officers, the woreda officers, and the facilitators responsible for delivery of the training as well as coaching and arranging for the mentoring. In collaboration with the SCI team, secondary data was collected through a review of Activity documents, websites, and administrative records. Main Findings The Youth Potential Activity, now in its third year of implementation, is offering participants a menu of five modular and integrated employability interventions58 that provide training on life skills, including skills designed to help participants prepare for work, search for employment opportunities, prepare their CVs and write applications, and improve their knowledge of resources available for finding 57 Thirteen percent of participants were either not traceable or chose not to respond. 58 The training programs include Personal Youth Development, or Youth in Action and Work Ready Now! (WRN!), Be Your Own Boss, Work Ready Plus, and Aflateen. 72 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline employment as well as for self-employment. The participants are followed by facilitators who provide coaching59 and arrange for mentoring. All participants enrolled in the Activity start with a basic training of a set number of contact hours spread over 13–15 days, followed by modules selected on the basis of their personal development plan. The Activity offers training on a rolling basis, with new cohorts starting every four to six weeks. Each cohort has approximately 50 participants (two groups of 25 participants) from each of the active woredas (for approximately 1,500 participants per cohort). Over the first two and a half years, the Activity trained about 12,000 participants. While both male and female participants, regardless of their literacy levels, are eligible to join the program, there are more male than female participants in the program, and the first few cohorts have focused on the literate candidates. The baseline data show that 43% of the 582 survey respondents were female, with the gender disparity being more significant in the Oromia regions (with 37% females). Over two thirds of the cohort were between 19–24 years of age, more than half have never been married, and most live with their parents. Overall, there is a vast disparity in the participants’ level of education ranging from no schooling all the way to completion of TVET or college. A clear majority of this group have completed some level of schooling, and over half have completed grade nine or higher (48% have completed grades 9-12, and 8% have completed TVET or college). There is also a disparity in the level of education of participants by region, especially in Afar where 19% of the participants had not attended any formal school and another 32% had completed only grade 5. Demographic characteristics of participants, by region (in percentages) The first batch of Activity participants with lower levels of literacy has been recruited into the Activity in 59 Coaching refers to follow-on support provided to program participants by facilitators aimed at ensuring adequate use of the training. 73 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline the March/April 2017 cohorts. Participants are provided with a stipend/per diem of about US$1.3 per day (30 Birr) for attending the training, some of which is set aside for use by the participants as seed money to start their own enterprise.60 Income and Employment Status At baseline, 45% of the Youth Potential participants were unemployed and dependent on others for their sustenance, 41% were self-employed and the remaining 14% were employed either as daily laborers or engaged in short-term salaried jobs. Of those self-employed, two thirds were working in a self-owned business, and about a third were also looking for another job. More women than men were unemployed, and more women reported managing self-owned businesses. More male than female participants reported supporting family-owned activities and a very small proportion were engaged in group-owned businesses. Higher unemployment and lower self-employment rates were reported in Afar and Tigray compared with Amhara and Oromia regions. Most of the male participants reported being engaged in agriculture and livestock-related activities or daily labor, while the preferred fields for women were trading consumer products, providing services such as running a coffee/tea shop or a small restaurant, and agriculture and livestock-related activities. Lack of education and training, mismatch between training and job requirements, no work experience, and not enough jobs available were stated as the main reasons for unemployment or inability to work. For those engaged in self-employment, both male and female participants reported several hurdles in setting up their own business, including lack of land and working space, difficulty in obtaining working capital and financing, and licenses and market linkages. Current employment status of YCS respondents, by region (in percentages) 60 The exchange rate used here and in the rest of this report is US$1= ETB 23.155 (as on June 16, 2017), as reported by EXCHANGE-RATES.org. 74 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Participants’ opinion of main reason for unemployment, by region (percent responding) Note: Numbers in the parenthesis in X-Axis represents number of unemployed respondents from each region. Out of total unemployed, 245 participants responded. Over 80 percent of focus group participants reported having little or no income. Further, 60 percent of participants lived with their families and worked in agriculture as sharecroppers or as daily labor, and thus could not definitively estimate their income or expenditures on themselves. For purposes of establishing a baseline, survey data was collected on household expenditures and assets owned by the individuals and their families. The largest expenditures reported were on investments in own business (26% of household expenditures), followed by house rent/lease (12%), support to other family members (12%), and food (10%). On an individual level, about 50% of all participants reported owning cattle and farm animals. Older participants reported owning assets such as land for farming and a house. Interestingly, 94% of male participants and 66% of female participants owned a mobile phone. While 67% of participants reported saving money for emergencies, participants in the focus groups conducted by the study team indicated that most incurred expenditure on health care only when they or their family members were unwell. Most participants reported visiting the local health clinic/post or a nurse for most healthcare needs, and going to the hospital in the city only in case of severe illness or emergency. Distribution of individuals reporting asset-holding, by region (in percentages) 75 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Work Readiness Skills Participants indicated a strong preference for self-employment over salaried employment, especially with the government as the salaries were rather low. Overall, they had a high opinion of their own readiness to work, rating themselves highly in terms of their interpersonal skills and self-confidence.61 Apart from the ability to write CVs or to access finance, land, etc., participants rated themselves as “good” or “neutral” on most items on a five-point scale ranging from very poor to very good. Male participants rated themselves higher (i.e., as being very good) compared to female participants. FGDs also indicated that participants felt they were ready for work and that the training would add the needed polish. Participants also expressed the hope that the training would help with full-time employment or business. Access to Employment/Entrepreneurial Resources Thirty-five percent of all survey respondents reported having tried to find employment in the past six months. Many employed or self-employed participants also reported looking for new jobs. While the most common avenues for looking for new opportunities were participation in job fairs, followed by asking relatives and friends for help, applying directly to employers and registering with public employment office, there are distinct gender differences in the preferred avenue for job search. Female participants reported being more likely to use avenues such as job fairs (51%), seeking the help of friends (29%), and registration with the public employment office (24%) (in that order), while male participants preferred seeking the assistance of friends (54%), followed by job fairs (47%), and direct applications (25%). Only 8% of male respondents reported responding to advertisements on the Internet or radio, an option not used by the females at all. Health Care Health care is a discretionary expenditure and one that is easily overlooked, except in an emergency. The hypothesis underlying the Youth Potential Activity is that with increased income, the participants in the Activity will spend more on their health and health-related expenditures, including family planning. Thirty percent of all survey respondents reported being unwell and taking time off from work due to illness in the past six months. Most reported being unwell due to illnesses such as malaria, typhoid, allergic reactions, and a few reported serious illnesses such as breast and heart problems and kidney infections. Those who sought treatment reported going to a government hospital or a government health station/clinic. Forty-one percent of the respondents reported visiting a health care provider, between 1 and 3 times, over the past six months for themselves or someone in their family. Distribution of individuals reporting preferred provider for health care, by region (in percentages) 61 Note that for some of the baseline study participants (April cohort), WRN! training had started on April 1, 2017 while for others it started mid-Aril 2017. Data collection for all participants started in the second week of April 2017. Effectively, therefore, responses to questions related to work-readiness skills are spread over the duration of WRN! training. 76 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Functional Literacy and Numeracy Skills While not a focus of the Youth Potential Activity, an expected benefit of enrolling in the Activity is improved levels of functional literacy and numeracy. Most participants, male and female, reported being able to read and write fluently, and most had more than just a functional level of literacy (the only exception was Chifra in Afar region). Over two thirds of the participants have completed grade 9 or higher (about 8% have completed TVET or college programs). Survey respondents were given simple tests of literacy and numeracy to assess their ability to read, write, and do basic arithmetic. Over two thirds of the participants demonstrated an ability to read and write with some fluency. Further, 97% of all participants could recognize numbers, and 92% were able do simple addition. Conclusions The YCS is following a subset of youth over a period of 12 months to enable better understanding of: (a) participant-level outcomes; and (b) the extent to which the Youth Potential Activity has improved the employment status and/or income level of a subset of youth participating in the project. The baseline study has collected qualitative and quantitative data on ten indicators for 582 participants in the month of enrollment in the program. This data serves as the benchmark against which the impact of the Activity will be assessed. Over the course of data collection, we noted that the Youth Potential Activity has gotten off to a slow start and has reached only a third of the target population at midpoint in the program. The team also experienced a need for continued communication and management of expectations for both participants and partners. Of the two major government stakeholders,62 the Youth Potential Activity has a strong relationship with the Ministry for Women and Children Affairs, which has oversight of the program; however, there is a need for strengthening the ties with the Ministry of Youth and Sports, which oversees government programming for youth. The focus group discussions revealed that participants had high expectations from the program. In addition to the training, they are expecting to be placed in an internship/WBL opportunity that would lead to a permanent job or be set up with enough financial and other resources for self-employment. This expectation must be managed to avoid a feeling of unfulfilled promises. In addition, the training is likely to generate a lot of demand for the few economic opportunities that are available in the woredas, which has the potential of causing a lot of frustration among those who do not get their desired levels of employment and economic gains. Further, there is an expectation that successful participation in the Youth Potential Activity will result in higher incomes, which in turn would lead to gains in health outcomes and improved levels of functional literacy and numeracy. However, there is nothing in the training curriculum that directly addresses these topics. The training program does not have any training on health care or utilization of health services, nor does it have any activities that provide literacy or numeracy training. The difference between the theoretical construct and practical outcomes needs to be strategically managed. Challenges The team faced some challenges related to data collection. These include: (i) non-availability of data on enrollment of participants in the March/April cohort from SCI; (ii) difficulty in obtaining information on individual income and expenditures, since most participants live with their families; (iii) unavailability of simple easy-to-administer tests of literacy and numeracy relative to the more formal tests such as Early Grade Reading Assessment (EGRA) and Early Grade Math Assessment (EGMA); and (iv) despite all efforts to coordinate with the SCI team, there were instances when the survey team was unable to find the facilitators and or the participants for conducting the surveys. Solutions for these challenges were 62 At the time of award, the major government stakeholder was the Ministry (Bureau at the regional level) of Women, Children, and Youth Affairs. Since then, this has been reorganized into two entities: Ministry of Women and Children Affairs and the Ministry of Youth and Sports. 77 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline found using alternate measures, for example, data from administrative reports was used to replace enrolment data, proxy data on ownership of items was used to determine income, and finally simple tests of literacy and numeracy were developed by the survey team, but the challenges remain. Similarly, the SCI team has promised all help for the next round of data collection, but again the challenge remains. A separate challenge was to do with the context of the participants, we were unable to find out if there were training programs other than the Youth Potential Activity that were available to the participants in the selected woredas. This is important insofar as it could compromise the impact of the interventions and causality of changes in the indicators after one year of enrollment. Recommendations This is a baseline study designed to provide a point of comparison for change resulting from the effects of participation in the Youth Potential Activity. However, during course of data collection a few points of note/recommendations for the implementing team are as follows. (i)The evaluation team noted that there is a need for strengthening the relationship of the Youth Potential team with the Ministry for Youth and Sports and to make the concerned officials more aware of the Youth Potential activities and initiatives. This will help in greater awareness of Youth Potential and provide Youth Potential participants with greater access to programs and resources managed by the Ministry. (ii) Participants have great expectations from the training program. Based on the FGDs, it appears that these expectations may be beyond the scope of Youth Potential program. There is a need for Youth Potential to discuss explicitly and upfront what the Activity can and cannot do. This will help set more realistic expectations on the part of the participants and reduce possible disappointment. (iii) As stated above, the curriculum for Youth Potential does not address the issues of health or literacy and numeracy among the participants. There is a need for a discussion on the role of the Activity and its impact on the issues of health, literacy and numeracy on the participants between the implementers and USAID/Ethiopia. (iv) There is a need for further standardization and transparency in the selection process. While the selection criteria are very clear, the process itself could be improved upon. (v)There is a time lag between the start of the training and the report of the number of participants. In addition, there is a time lag between the participants dropping out of the training program and reporting by the field staff making it a problem for follow-up. In theory, the Youth Potential staff use electronic means for recording data, in practice this is not the case. Perhaps there is a need for better training on the use of electronic recording devices the Youth Potential field staff. (vi) finally, there is a concern regarding the increasing caseload of the facilitators. In order to ensure quality training and follow-up by the facilitators, SCI and its partners should make it a priority to consider this and maintain a manageable caseload. 78 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline ANNEX IV: LIST OF RESEARCH QUESTIONS AND INDICATORS Research Questions – Detailed List Please note that the list below includes questions from the statement of work broken down into their subparts as well as additional questions posed by the evaluation team63. The questions added to the original list provided by the USAID/Ethiopia team are marked with an Asterisk (*). 1) *Description of the Activity (also responds to Question 6) a) *Who are the participants in the Activity? b) *What are the experiences/ training programs offered to the participants? c) *Are there any barriers to participation that are holding back either participants to these programs? (such as language, literacy-numeracy skills, SES, cultural barriers, safety concerns) d) *What proportion of the participants complete the training program? e) *While in the training program, how do the participants perform? f) *How is their performance monitored? g) *How are the participants tracked? h) *What proportion of the participants report using skills learned during the Youth Potential Activity? Questions from Statement of work: 2) To what extent has the income and employment status of study participants changed between enrollment in the Building the Potential of Youth Activity and twelve months following enrollment? Does increased income lead to improved health outcomes among study participants? a) Over the twelve months since joining the Activity, is there a change in the employment status of the participants? b) Type of job held? c) Is there a change in the income level of the participants? d) Is there increased spending on nutrition and preventive health care as a proxy for improved health outcomes (e.g., spending on fruits and vegetables, use of contraceptives, vaccinations for children, doctor visits for self or family) 3) To what extent has participation in the Building the Potential of Youth Activity improved the work readiness skills of study participants? a) Type of skills learned b) Use of skills learned 4) To what extent has participation in the Building the Potential of Youth Activity improved knowledge of and access to employment/entrepreneurial resources for study participants? a) Skills developed 63 The additional questions posed by the evaluation team have been included to make the report more comprehensive and provide a context for the findings of the evaluation. They are not expected to extend the timeline or add to the cost of data collection. 79 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline b) Resources accessed and used 5) To what extent has participation in the Building the Potential of Youth Activity improved functional literacy and numeracy skills of study participants? a) Level of education, any additional activities that provide literacy/numeracy training completed b) Self-reported proxy measures of literacy and numeracy skills, e.g., ability to read street signs, manuals, directions, simple math – addition, subtraction, etc. c) Performance on simple tests of functional literacy/numeracy as a proxy measure 6) How do the outcomes addressed in the questions above vary by participant characteristics (e.g., sex, age, ethnic/linguistic group, SES, marital status, parenting status) and by service delivery partner? a) Data from Q1 used with data on Q2-Q5 will provide data to be used to answer this question. Crosswalk of Study Questions to Indicators Research questions and sub questions Indicators64 Possible sources of data65 1) *Description of the Activity All questions in this section answered for (i) participants in Activity overall; and (ii) for participants in cohort study a) *Who are the participants in the Activity? - # of participants in Activity - # of participants in the cohort study - By gender, SES, Education, age group, marital status, ethnicity, Woreda and region - Review of Quarterly and Annual Activity reports; Youth Potential administrative records - For the cohort in the study, survey and focus groups - Interviews with facilitators and Woreda/Kebele representatives b)*What are the experiences/ training activities offered to the participants? - List and type of activities offered - Curriculum for the Activity - #/percent of participants using coaching and mentoring - #/percent of participants using follow￾up activities c) *Are there any barriers to participation in Youth potential Activity or education in general? (such as language, literacy-numeracy skills, SES, cultural barriers, safety concerns) - Barriers to participation reported by implementers - #/ percent of participants reporting problems/ barriers to participation by type of concerns (i.e. social, educational, cultural, political, economic) d)*What proportion of the participants complete the training Activity? - #/ percent of participants who complete activities by type of Activity, Woreda and region - # percent of participants who do not complete program by type of program, Woreda and region - •# of participants using mentoring /coaching support services by Woreda and region e)*While in the training program, how do the participants perform academically? f) *How is their performance monitored? g) *How are the participants tracked? 2) To what extent has the income and employment status of study participants changed 64 All data disaggregated by program, gender, Woreda, region, SES etc. See Q6. 65 Unless stated otherwise all data will be collected at Baseline, midline and end line. Only exception to this are focus groups with families of participants, these will be conducted only at midline and end line points. 80 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline between enrollment in the Building the Potential of Youth Activity and twelve months following enrollment? Does increased income lead to improved health outcomes among study participants? a) Over the twelve months since joining the Activity, is there a change in the employment status of the participants? • Number (and percentage) of youth reporting new employment, new self￾employment, group employment or better employment as a result of participation in program; • Number (and percentage) of youth reporting internships, job-shadowing, or other paid and unpaid practical experience • Number of days of employment •Survey of Activity Beneficiaries • Focus groups with participants • Interviews with facilitators and Woreda/Kebele representatives • Focus groups with families b)Type of employment c)Is there a change in the income level of the participants • Number (and percentage) of youth beneficiaries reporting increased income; • Number (and percentage) of youth beneficiaries reporting increased assets; • Number (and percentage) of youth beneficiaries reporting increased financial contribution to larger family; d)Is there increased spending on nutrition and preventive health care as a proxy for improved health outcomes • Number (and percentage) of youth beneficiaries with increased spending on food – fruits and vegetables, health care for self or family, contraception, vaccinations for children 3) To what extent has participation in the Building the Potential of Youth Activity improved the work readiness skills of study participants? a) Type of skills learned b) Use of skills learned • Number of youth beneficiaries reporting improved skills; • Number of youth beneficiaries reporting use of skills; • List of skills learned • Number reporting improved literacy and numeracy skills as a result of participation in the Activity •Survey of Activity Beneficiaries • Focus groups with participants • Interviews with facilitators and Woreda/Kebele representatives 4) To what extent has participation in the Building the Potential of Youth Activity improved knowledge of and access to employment/entrepreneurial resources for study participants? a) Type of resources accessed b) Skills developed c) Resources accessed and used • Number of youth beneficiaries reporting access to improved knowledge of and access to employment resources; • Number of youth beneficiaries reporting access to improved knowledge of and access to resources for entrepreneurial work; •Survey of Activity Beneficiaries • Focus groups with participants • Interviews with facilitators and Woreda/Kebele representatives 5) To what extent has participation in the Building the Potential of Youth Activity improved functional literacy and numeracy skills of study participants? 81 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline a) Level of education b) Self-reported proxy measures of literacy and numeracy skills, e.g., ability to read street signs, manuals, directions, simple math – addition, subtraction, etc. ● Number of persons with improved literacy skills as a result of POTENTIAL Activity; ● Number of persons with improved numeracy skills as a result of POTENTIAL Activity •Survey of Activity Beneficiaries • Focus groups with participants • Interviews with facilitators and Woreda/Kebele representatives 6) How do the outcomes addressed in the questions above vary by participant characteristics (e.g., sex, age, ethnic/linguistic group, SES, marital status, parenting status) and by service delivery partner? a) Data from Q1 on YCS sample will provide data to be used to answer this question. ● Gender (Male and Female); ● Age groups (15-19, 20-24, or 25-29); ● Ethnic/linguistic group (“Oromo”, “Amhara”, “Tigray”, or “Somali”); ● SES (Low level, middle level & high level) and ● Marital Status (single, Married or divorced) ...etc. •Survey of Activity Beneficiaries • Focus groups with participants • interviews with facilitators and Woreda/Kebele representatives 82 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline ANNEX V: LIST OF PEOPLE INTERVIEWED AND INFORMATIONAL MEETINGS A total of 48 KIIs and informational meetings were held with stakeholders and implementers in the four regions of Amhara, Afar, Tigray and Oromia. The list of people consulted by region, and within that by Woreda, is below. In addition, 26 focus groups were conducted with current participants of the Activity and 9 focus groups with parents and guardians of the participants. A table showing the focus groups conducted, their location and the number of participants is also included. Focus groups conducted by category and region (in detail) Region Woreda Kebele #of Beneficiary FGD # of FGD Participants # of Parent FGDs # of Parents Amhara Jawi Fandik_01 2 30 1 5 South Achefer Ahuri 2 23 0 0 South Achefer Abchikli 2 9 0 0 South Achefer Lalibela 2 18 1 4 Afar Assayita Kebele_01 2 8 1 3 Chifera Mesigid 4 32 1 5 Tigray Eendamhoni T/Hayimont 2 18 1 7 Embasti 2 17 1 6 Oromia Wolimera Gudo 2 19 1 8 Asgori 2 19 1 7 Yabelo Yubido 4 46 1 5 Total 26 239 9 50 83 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline KIIs conducted by category and region (in detail) Category Afar Amhara Oromia Tigray Total Potential Staff 5 8 5 4 22 TVET/TTI 2 3 3 2 10 Microfinance 0 1 3 2 6 Employers 2 1 0 2 5 Gov’t (micro-enterprise; agriculture) 1 1 2 Total 10 13 11 11 45 84 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline ANNEX VI: SOURCES OF DATA Both primary and secondary data were collected to inform the study and provide baseline data on the current status of program participants. Primary data collected from: ● Surveys with newly enrolled participants of the Youth Potential training program; ● KIIs with (list of people consulted is in Annex 4): (i) Ministry and Bureau officials at the regional, zonal and Woreda levels; (ii) Representatives of the selection committee from the Kebele, implementing partners, facilitators for Youth Potential Activity, and other program staff; and (iii) Other stakeholders such as TTI staff, representatives of the private sector/employers for the participants, and others identified as the study progresses. ● Focus groups conducted with a sample of eligible participants of the Youth Potential training program Secondary data collected from: i. Documents reviewed: a. Youth Potential documents and administrative records: i. Program Summary ii. Activity original statement of work and any amendments iii. Monitoring and evaluation plan/PMP iv. Workplan for Year 3 v. Quarterly and Annual Reports Year 1 and Year 2 vi. Quarterly Report Year 3 Q1 vii. MOUs between SCI and three organizations viii. Spreadsheet of Woredas and active Kebeles with number of participants trained by Woreda and Kebele ix. Calendar of trainings for March and planned for April and May 2017 b. Program training materials: i. BYOB Training manual and facilitators guide ii. YiA resource manual and facilitators guide; and materials for module - myself iii. WRN Ethiopia trainer’s manuals and participant handbooks – modules M0 to M4 iv. Mentor’s guide c. Background articles and other relevant literature i. DHS Ethiopia 2011 - FR255 ii. Ethiopia_2014_EGRA_report_RTI_ (00000002) iii. LSMS Ethiopia ESS3 Community Questionnaire English iv. Other reports such as Program description on USAID website, The World bank reports on current situation in Ethiopia, etc. cited in the body of the report 85 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline ANNEX VII: DATA COLLECTION INSTRUMENTS Instruments included are: ● The survey instruments ● Guides for focus group with Youth Potential participants ● Guides for interviews and informational meeting with: o o implementing partners for the Youth Potential Activity, including facilitators and Woreda officers key stakeholders including officials from the Ministry and Bureaus involved with the program at the regional and Woreda levels 86 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Youth Cohort Survey Questionnaire Midline survey (Please use the scripts for consent of participants). Survey Information to be completed by Enumerator: Name of enumerator ____________________________ Name of Supervisor: Interview date: __________________________ Starting time: _____________________ Location: Region: ____________________ Woreda: _________________ Kebele: ___________________ General Identification 1. ID Number_______________________________________ (fill in by SART) 2. Name of Participant (with grandfather’s name): __________________________________ 3. Participant’s Current address: Personal Tel. (mobile): ______________________, Residence: ___________________ E-mail (if available) or Facebook (if available): __________________________________ Region: ____________________ Woreda: _________________ Kebele: ___________________ 4. Contact information of close relative/neighbor/friends (in addition to contact address): Name: __________________________ Tel: ______________________________ Address Region: ____________________ Woreda: _________________ Kebele: _______________ 5. Age of Participant: ________ 6. Sex: ☐ Male ☐ Female 7. Ethnicity and Language a. Ethnicity: ☐ Afar ☐ Amhara ☐ Guragie ☐ Nuwer ☐ Oromo ☐ Sidamo ☐ Somali ☐ Tigray ☐ Welaita ☐ Other __________________ b. Language Amarigna ☐, Oromigna ☐, Tigrigna ☐ Afarigna ☐ Other ☐ 8. Disability: (check all that apply) ☐ ☐ ☐ ☐ ☐ ☐ No disability Deaf or hard of hearing Blind (one or both eyes) Physical handicap Cognitive disability (mental disability) Other 9. Current marital status of participant (select one): ☐Never married ☐Married ☐Living together ☐Separated/Divorce ☐Widowed 10. Living status: _________Living with parents _____________ Living independently 11. Household Size: _____________ (people living in the same house permanently) 12. Number of Participant’s Children: ________ Total number (zero if none) , ______ Girls ______ Boys 87 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Education status and Functional Literacy/Numeracy 13. Functional literacy (use a letter card, choose one response) ☐ ☐ ☐ ☐ Can’t read & write Can read & write with difficulty Can read & write with some fluency Can read & write very easily 14. Functional numeracy (use a number card) Recognize numbers ☐ ☐ ☐ ☐ ☐ Yes ☐ ☐ No Add simple numbers Yes No Subtract simple numbers Yes ☐ ☐ ☐ No Multiply simple numbers Yes No Divide simple numbers Yes No 15. Education Status (select one response) ☐ ☐ ☐ ☐ ☐ No school Only traditional/non- formal school Grade level or number of years in school ___________ (enter a number from 1- 12; includes grade completed or dropped-out) Completed TVET (drop-outs captured under grade level) Completed College/ University 16. Are you still in school? ☐ Yes ☐ No (skip to 18) 17. If yes, what are you studying? ____________________ (skip to 19) 18. If no, what was the main reason for stopping your education? (select one) ▪ Finished course/ graduated ▪ Failed examinations ▪ Did not qualify for the entrance exam for TVET, College/University ▪ Did not enjoy schooling ▪ Wanted to start working ▪ To get married ▪ Parents did not want you to continue schooling ▪ Economic reasons (could not afford/too poor/needed to earn money to support family) ▪ Other _____________________________________ 19. Education Status of mother (select one response) ☐ ☐ ☐ No school Only traditional/non- formal school Grade level or number of yeas in school ___________ (enter a number from 1- 12; includes grade completed or dropped-out) ☐ ☐ Completed TVET (drop-outs captured under grade level) Completed College/ University 20. Education Status of father (select one response) ☐ ☐ ☐ ☐ No school Only traditional/non- formal school Grade level or number of years in school ___________ (enter a number from 1- 12; includes grade completed or dropped-out) Completed TVET (drop-outs captured under grade level) 88 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline ☐ Completed College/ University Employment Status 21. What is your current employment (income earning) status? Primary job In school or training (skip to 27) Paid/wage/paid-in kind employed Self-employed Self-employed but looking for a job Unemployed (dependent on support from others) (Skip to Q 25) Unemployed but not seeking work for other reasons (example disabilities) Other _____________ 22. If self-employed, what kind of self-employment are you engaged in: ☐ ☐ ☐ Self-owned (skip to Q23) Family owned (skip to Q23) Group owned, a. If group owned, how many people are in the group? __________ (enter number) b. When did you join the group? _________________ ☐ a year or more, ☐ between six months to a year ago, ☐ less than six months ago, 23. Which of the following sectors do you work in (for wage/salary employment or self-employment)? Note to interviewers - For female participants ask – Aside from your own housework, if you work for payment in-cash or kind which of the following sectors do you work in (for wage/salary employment or self-employment)? This is my main job (select one) This is my secondary job (select one) Agricultural + Livestock activities ___________ Trading agricultural products (buying or selling agricultural products) Trading consumer products/items – small shop/kiosk Carpentry, carving, or woodwork, Mechanics, metal work Construction work (brick/stone laying, building, etc.) Daily laborer Electrical work Computer technology and administration Tailoring (making clothes or textiles) Transport service Hairdressing/Barber Providing services (such as coffee/tea, small restaurant, shoeshine, repair and maintenance, etc.) Hotel industry Education Work for family Government NGO Others (specify) 89 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline If response to q23=agriculture, as main job or secondary job ask 23a, else go to 23b 23a. Please tell us: if you are engaged in the following agricultural activities Engaged in Activity six months ago? 0=No 1=Yes Engaged in Activity now? 0=No 1=Yes scope of Activity compared to six months ago (1= expanded 2= same, 3=decreased, 4=n/a) I am growing a single crop in a year I am growing two or more crops in a year I am fattening animals I am rearing sheep I am doing poultry farming I am doing dairy production 23b Is the main job you mentioned in question 23 above self-employment or wage employment? 1= Self-employment individual/group 2=self-employment family__ 3= wage employment _________________ 23c Is the secondary job you mentioned in question 23 above self-employment or wage employment? 1= Self-employment individual/group 2=self-employment family__ 3= wage employment _________________ 24. To what extent are you satisfied with your current main job? ◻ ◻ ◻ ◻ ◻ Mostly satisfied Satisfied Neutral Unsatisfied Mostly unsatisfied 25. Have you tried to find employment/new employment in the past six months? ☐ ☐ Yes No (skip to Q30) 26. If looking for a job ask the questions below 27. How long have you been available for work and actively looking for a job? ◻ ◻ ◻ ◻ ◻ ◻ Less than a week 1-4 weeks 1-2 months 3-6 months 6 months-1 year More than 1 year 28. Which of the following sectors would you wish to work in (for wage/salary employment or self￾employment)? Note to interviewers - For female participants ask – Aside from your own housework, if you work for payment in-cash or kind which of the following sectors do you wish to work in (for wage/salary employment or self-employment)? main job (select one) secondary job (select one) 90 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Agricultural + Livestock activities Trading agricultural products (buying or selling agricultural products) Trading consumer products/items – small shop/kiosk Carpentry, carving, or woodwork, Mechanics, metal work Construction work (brick/stone laying, building, etc.) Daily laborer Electrical work Computer technology and administration Tailoring (making clothes or textiles) Transport service Hairdressing/Barber Providing services (such as coffee/tea, small restaurant, shoeshine, repair and maintenance, etc.) Hotel industry Education Work for family Government NGO Others (specify) 28a. Would you want your main job to be self-employment or wage employment? 1= Self-employment individual/group 2=self-employment family__ 3= wage employment _________________ 28b. Would you want your secondary job to be self-employment or wage employment? 1= Self-employment individual/group 2=self-employment family__ 3= wage employment _________________ If unemployed ask Q29, else go to Q30 29. If unemployed, in your opinion what are the main three reasons that you are not employed (wage/self)? (please put in the first three) Reason one Reason two Reason Three No education Unsuitable vocational education No suitable training opportunities Mismatch between education requirements and that required by the position available No work experience Not enough jobs available Considered too young Discriminatory prejudices (for example, disability, religion, race, appearance, etc.) Low wages in available jobs Poor working conditions in available jobs Don't have the right personal or family connections Lack of access to finance 91 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Lack of land for agriculture Lack of workspace to start a business No response Other Specify: 30. Have you ever refused a job that was offered to you? ☐ ☐ Yes No (skip to 32) 31. Why did you refuse? (select the main reason) ◻ ◻ ◻ ◻ ◻ ◻ ◻ ◻ ◻ ◻ Wages offered were too low Work was not interesting Location was not convenient Work would not match my level of qualifications Work would require too few hours Work would require too many hours Waiting for a better job offer The contract length was too short Saw no possibilities for advancement Other 32. Have you tried to find employment/new employment in the past six months? ☐ ☐ Yes No (skip to Q32b) 32a. How have you tried to find employment/new employment in the past six months? Probe and Check all that apply Through education/training institution By Attending job fairs Registration at a public employment office Registration at a private employment office Direct application for employers, participation in a competition Checking at worksites, farms, factory gates, markets or other assembly places Placing and answering newspaper advertisements Answering advertisements on the internet, etc. Seeking assistance of friends, relatives, colleagues, unions, etc. Other 92 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline 32 b. If you were to create self-employment, what kinds of information would you need? I am not looking to create self-employment Seek assistance of friends, relatives, colleagues, unions, etc. Look for land, building, machinery, equipment to establish own enterprise Arrange for financial resources Apply for permits, licenses Learn new skills Other__________________________________________ Participation in Youth Potential Program 33. When did you enroll/start in the Youth Potential program? ________________ (EC. mm/year) 34. What Youth Potential program activities have you attended? (check all that apply) ☐ ☐ ☐ ☐ ☐ ☐ ☐ ☐ ☐ WRN training (Please know this program is for people who can read) WRPlus training (Please know this program is for people who cannot read) BYOB training PYD/YiA training Technical training at TVET or Farmers Training Center Work Based Learning (apprenticeship, experience, exposure…) Received coaching and advisory services Linkage to source of finance, government/community support/inputs/advise, mentors, etc. Other, specify ______________________________ 35. Are you currently enrolled in any other non-formal training program (like the Youth Potential program)? ☐ ☐ Yes No (skip to 35a) If yes, what training program are you attending? _____________________________ 35a. In the past six months were you enrolled in any formal training program paid for by the Youth Potential Activity)? ☐ ☐ Yes No (skip to 36) 35b. If yes, what training program are you attending? _____________________________ Check all that apply Agriculture (farming, horticulture, irrigation) Animal fattening, animal husbandry Poultry farming Marketing Bakery Carpentry, carving, or woodwork, Mechanics, metal work Construction work (brick/stone laying, building, roads, etc.) Hotel and tourism Electrical work 93 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline ICT, Computer technology and administration Tailoring (making clothes or textiles) Transport services, driving, Car/vehicle repair shop, tire repair shop Hairdressing/Barber Providing services (such as coffee/tea, small restaurant, shoeshine, repair and maintenance, etc.) Other specify 36. Have you previously been enrolled in any non-formal training program? ☐Yes ☐No (skip to 37) If yes, what training program did you attend? _____________________________ 37. Please rate yourself on the following skills (Work Readiness skills) Skills Very Good Good Neutral Poor Very Poor Interpersonal communication Work habits and conduct Job search application skill, writing CV Knowledge of resources of where to look for jobs Business management/improvement skills Skills to start-up a business Knowledge of resources for obtaining loans, seed money, etc. Access resources (finance, land, inputs) Access markets Knowledge of how to obtain license for business Saving habits Setting objectives/goals and preparing personal development plan Improved social bond and team work Build self-confidence Other, specify ___________________________________ 37a. Please tell us if you strongly agree, agree, neither agree or disagree, disagree, strongly disagree with the following statements As compared with six months ago Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree I feel more confident of my abilities I know how to develop a business plan I know how to calculate profit/loss I spend money according to a plan My business has expanded I have better marketing skills I am able to interact better with customers My business has more customers I work more days of the week I have better time management skills I am more patient 94 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline As compared with six months ago Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree I get into fewer fights and arguments I am more persistent and don’t give up as easily I feel more capable to solve problems when they arise I am more respectful of all types of jobs I am more willing to try different kinds of jobs I am more motivated to work 37a. in the past six months, have you applied for a loan 1= Yes 0= No (skip to 38) 37b. Did you receive the loan? 1= Yes (go to 37c) 0= No (skip to 37e) 37c. Was it a group loan or individual loan 1=group loan 2=individual loan 37d. What was the amount? _________________Birr___ (for any of the answers go to 38) 37e. Reason for not obtaining a loan Read the list to the respondent Check all that apply No collateral No agreement within group Did not meet minimum funds requirement I was applying for an individual loan but only group loans were available Forms were too difficult to complete Income and assets 38. What assets do you have at this time (before joining this training}? Do you or your family own: Owned by you (check all that apply) Owned by family (check all that apply) a. House b. Radio/Tape recorder/ MP3 player c. Television d. Mobile Telephones e. Non-mobile telephone f. Own bicycle g. Animal-drawn cart h. Motor cycle/scooter i. Car/truck j. Land for farming, grazing, etc. k. Animals l. Other 1_________________________________ m. Other 2_________________________________ 95 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline n. Other 3_________________________________ 39. How much land do you own? a. _______________________hectare (enter value by converting from local measure to hectare, enter zero if the respondent has no land) 40. What type and how many animals do you have? Animal Number owned (please write number, zero for none) a. Milk cows, oxen or bulls? b. Goat c. Sheep d. Camel e. Horses, donkeys, or mules? f. Poultry, chicken, hens, g. Bee hives h. Fish i. Other 1_______________ Write number and description j. Other 2_____________ 40a. Please tell us about your income level. Compared with six months ago, has your income level: ☐ ☐ ☐ increased decreased stayed the same 41. In the past six months, how have you used your income? Please estimate how much of your total income did you spend on each of these categories? Check all the categories that you spend money on If yes, ask how much was spent in the past six months a. Rent/lease b. Taxation c. Transportation d. Buy food and related goods e. Pay for electricity, water, f. Clothing g. Buy household goods h. Education / school fees i. Health care related expenditures e.g. health insurance, provider fees, medicine, etc. j. Contraceptives and family planning k. Support to other family members l. Invest in business or other assets (including purchase of goats, sheep other animals) m. Savings n. Other 1 (specify) o. Other 2 (specify) p. Other 3 (specify) q. Other 4 (specify) ______________________ 42. Do you currently have savings? YES/NO (if no skip to Q45, also please confirm they are consistent with Q41) a. If yes, how much do you have? ____________ 96 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline 43. How often do you put aside money to save? (only one answer allowed) ◻ ◻ ◻ ◻ ◻ Never A few times a year/seasonal Every month Every week Every day 44. Where do you keep your savings? (check all that apply) ◻ ◻ ◻ ◻ ◻ At home in a safe place In a bank/microfinance institution With someone (friend/relative/other) I trust With an individual who will pay me an interest on my savings Other__________________________ 45. If you do not save why do you not save? (check all that apply) ◻ ◻ ◻ ◻ ◻ ◻ Have no money to save Spend on clothes, shoes, etc. Pay off loans Use money for investment (including animals) Not willing to save Other _______________ Health Related Questions 46. Have you had any illness in the past six months? YES/NO (if no skip to Q47) b. What was your illness? ________________________________ c. Did you take time off work due to illness? ---------------(number of days) d. How often have you visited a health care provider in the past six months? -----------------(number of times) (Note to interviewer – if uncomfortable answering skip item and move to next) 47. How often/how many times have you visited a health care provider in the past six months due to illness in the family? -----------------(number of times) (No skip to 48) 48. Where did you seek advice or treatment the last time you were sick? (multiple response allowed) ◻ ◻ ◻ ◻ ◻ ◻ ◻ ◻ ◻ ◻ ◻ ◻ Govt. Hospital . . ... Govt. Health Center Govt. Health Station/Clinic Govt. Health Post/ Health Extension Worker NGO Health Facility Private Hospital Private Clinic Drug Vendor Shop Traditional Healer Other Specify __________________________ 49. If you have not been sick in the past six months but had to seek healthcare, where would you go? (multiple response allowed) ◻ ◻ ◻ ◻ ◻ ◻ ◻ Govt. Hospital Govt. Health Center Govt. Health Station/Clinic Govt. Health Post/ Health Extension Worker NGO Facility Health Facility Private Hospital 97 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline ◻ ◻ ◻ ◻ ◻ Private Clinic Drug Vendor Shop Traditional Healer Other Specify __________________________ 50. Did you obtain any of the following from your health care provider in the past six months? (Note to interviewer – if uncomfortable answering skip item and move to next) ◻ ◻ ◻ ◻ Medicines for illness Male contraceptives Female contraceptives Advice on nutrition for yourself and your family 51. To what extent do you agree with the following statements Compared to six months ago Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree My health has improved I am less stressed I am less anxious I get fewer headaches I am less angry I eat better food I spend more money on food I take better care of my health I am happier Thank you for your time today and we may be contacting you for any future interviews. Name of interviewer: __________________________ Date: _______________Ending time: __________ 98 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Guide for FGDs - Youth General Introduction and Informed Consent Welcome everyone! My name is _________________________________ from Social Impact, an evaluation company from America. We are conducting a study to assess the effect of the Youth Potential program from selected Woredas in four regions. For this, we are following a cohort of over 400 participants for 12 months. We started the first round in April-May, and we are following up with you now six months later. This group discussion is part of cohort study of the USAID/Youth Potential program, a program to improve youth employment opportunities in Ethiopia. This study will not determine if the program will continue, nor is it to evaluate your performance. It is only to learn about your expectations /experience with the program, use of the training received, the resources available and any challenges. We are very grateful for your time; this discussion should take about an hour and a half. The information you provide to us here is confidential and will not be shared with anyone outside. Please know that we will not share your name with anyone outside the research team, or quote you in our report without your permission, so we also hope that you will provide honest and accurate answers. For the note taker: Location: Region: _____________ Woreda: ______________Kebele: ______________ Focus Group date: __________________________ Starting time: _____________________ Number of males: Number of females: Pass out a sheet of paper to gather information on: Participant name, age, gender, Kebele, level of education, participant signature Focus group guide: 1. Before we begin, kindly; introduce yourself…. 2. Tell us about yourself (5 minutes of ice breaker – show of hands and count) a. How many are married? b. How many live with their parents? Independently? c. How many of you have had technical/vocational training in Potential program? d. How many have had internships, volunteer work, job shadowing opportunities? e. By show of hands, how many of you are working in: ● Wage employment? ● How many are self-employed? In which field? (see reference list below) ● For how many is this the family business? ● How many are in school? ● How many are unemployed? f. How many of you think agriculture is a good opportunity for youth? g. How many of you prefer to be self-employed? Why? h. How many prefer to have wage employment? Why? 99 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline ● Reference list of fields of work ● Agricultural + livestock activities ● Trading agricultural products (buying or selling agricultural products) ● Trading consumer products/items – small shop/kiosk ● Carpentry, carving, or woodwork ● Mechanics, metal work ● Construction work (brick/stone laying, building, etc.) ● Daily laborer ● Electrical work ● Computer technology and administration ● Tailoring (making clothes or textiles) ● Transport service ● Hairdressing/barber ● Providing services (such as coffee/tea, small restaurant, shop) ● Education ● Government ● NGO ● Others (specify) 3. Changes in employment status over the past six months a. How many of you have more than one type of work? If yes in what areas? (Mixed livelihoods – for example, helping family farm and making & selling soap) b. How did the Potential program help you? - Get a job? (Write a CV, find job postings, interview skills) - Start their business? (Give them resources to start their businesses (loan, materials, etc.)?) c. Do you think you could have secured your job or started your business without participating in Potential? d. For those of you who already had a business before joining the program, how has your participation in the program affected your business (increase income, expand services/products) e. For those of you who already had wage based employment before joining the program, how has your participation in the program affected your status (better employment, more income) 4. For those of you who are not employed what have been your biggest challenges in finding employment/starting a business? 5. Relevance of experience with the Youth Potential program a. What did you learn from the Potential training? (For example, gained new hard skills; gained new soft skills; access to finance; access to materials; improved literacy and numeracy; improved perception of self/more confidence) i. What did you like the most? ii. What did you like the least? iii. Please describe your apprenticeship or work-based experience, if you had any. iv. Please describe your mentoring/coaching support, if you had any. b. What new skills are you using as the result of the program? c. Thinking back to five months ago, do you have better access to employment/ entrepreneurship information? d. Beyond job-related support, how do you think your life has or has not changed since participating in Potential? e. What did you learn from the Potential training? (For example, gained new hard skills; gained new soft skills; access to finance; access to materials; improved literacy and numeracy; 6. Income-related questions – Has your income level changed since you participated in the WRN! training? Are you spending money differently now? a. How much of your income do you spend on your family? If you had a steady income, how would you spend the money? 100 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline b. How about food – fruits and vegetables? c. Health care services, like visits to the doctor, ● vaccinations for kids, ● use of contraceptives, etc. d. Do you think your health has improved compared to five months ago? Why? 7. Changes in functional literacy and numeracy a. Has your reading and writing skill improved in the past 4 to 5 months? If yes, was that due to your participation in Youth Potential and if so how? b. Has your addition and multiplication skill improved in the past 4 to 5 months? If yes, was that due to your participation in the Youth Potential and if so how? 8. What additional support do you think you need from Potential to help you find employment, either in wage employment or self-employment? 9. Is there anything else you think we should know about youth and employment in this region? THANK YOU 101 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Interview Guide - Partners General Introduction and Informed Consent Hello! My name is _________________________________ from Social Impact, an evaluation company from America. We are conducting a study to assess the effect of the Youth Potential program, a program to improve youth employment opportunities in Ethiopia. For this, we are following a cohort of over 400 youth participants from selected Woredas in four regions for 12 months. We started the first round in April-May, and we are following up with the youth now six months later. We would like to hear your opinion of the program and its beneficiaries. We are very grateful for your time; this discussion should take about forty minutes. The information you provide to us is confidential and will not be shared with anyone outside. Please know that we will not share your name with anyone outside the research team, or quote you in our report without your permission, so we also hope that you will provide honest and accurate answers. For the note taker: Participant name: ________________________ Company Name: ________________________ Location: Region: _____________ Woreda: ______________Kebele: ______________ Interview date: __________________________ Starting time: _____________________ 1. We would like to learn more about your organization. a. What sector is your company based in? b. In a couple of words, what product/service does your organization provide? c. When was the organization established d. How many people do you employ? (employ less than 10 employees? In between 10 and 50? More than 50?) 2. In what way are you partnering with POTENTIAL Activity? a. What is your motivation for collaboration? b. What kinds of support does your organization provide to the project? c. Are there any differences in the types of support provided by your organization for young men versus young women? ● Are you providing apprenticeships or other work based learning opportunities to youth from Potential program? Have you hired any of them after their apprenticeship? 3. If you already have youth from Potential program at your organization (either as apprenticeships, WBL, or employment), please describe how the experience has been for your organization so far. -Probe: In what ways are youth from Potential different than other youth at your organization? 4. Would you hire future Potential training graduates? Why or why not? 5. Is there any other feedback you think it is important to share with us about the Potential program? THANK YOU 102 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Interview Guide - Employers General Introduction and Informed Consent Hello! My name is _________________________________ from Social Impact, an evaluation company from America. We are conducting a study to assess the effect of the Youth Potential program, a program to improve youth employment opportunities in Ethiopia. For this, we are following a cohort of over 400 youth participants from selected Woredas in four regions for 12 months. We started the first round in April-May, and we are following up with the youth now six months later. We would like to hear your opinion of the program and its beneficiaries. We are very grateful for your time; this discussion should take about forty minutes. The information you provide to us is confidential and will not be shared with anyone outside. Please know that we will not share your name with anyone outside the research team, or quote you in our report without your permission, so we also hope that you will provide honest and accurate answers. For the note taker: Participant name: ________________________ Company Name: ________________________ Location: Region: _____________ Woreda: ______________Kebele: ______________ Interview date: __________________________ Starting time: _____________________ 1. We would like to learn more about your company. a. What sector is your company based in? b. In a couple of words, what product/service does your company provide? c. When did you establish this company? d. How many people do you employ? (employ less than 10 employees? In between 10 and 50? More than 50?) 2. Do you know an Activity or a project called Youth potential? If yes, how was it known to you? a. Are you collaborating with them? in what way? b. Are you providing apprenticeships or other work based learning opportunities to youth from Potential program? 3. What is your motivation for providing apprenticeship and/or work based learning opportunities to youth? a. Have you hired any of them after their apprenticeship? 4. What is your hiring perspective for the next year? -Probe: Is your sector in general growing? Are there likely to be new jobs available for young people? 5. How do you typically fill vacancies/recruit for entry-level jobs? - Probes – how do they advertise jobs? How common is word of mouth? Etc.) - Probe: Which entry-level jobs are the most difficult to fill? Why? 6. Please describe the types of jobs available for young people in entry-level positions at your enterprise. -Probe: Are there any differences in the types of jobs available for young men versus young women? 7. Which skills are most important for entry-level positions (probe on hard and soft skills)? 8. If you already have youth from Potential program at your company (either as apprenticeships, WBL, or employment), please describe how the experience has been for your company so far. -Probe: In what ways are youth from Potential different than other youth at your company? 9. Would you hire future Potential training graduates? Why or why not? 10. Is there any other feedback you think it is important to share with us about the Potential program? THANK YOU 103 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Guide for FGDs – Parents/Guardians General Introduction and Informed Consent Welcome everyone! My name is _________________________________ from Social Impact, an evaluation company from America. We are conducting a study to assess the effect of the Youth Potential program, a program to improve youth employment opportunities in Ethiopia. For this, we are following a cohort of over 400 youth participants from selected Woredas in four regions for 12 months. We started the first round in April-May, and we are following up with the youth now six months later. This group discussion will not determine if the program will continue, nor is it to evaluate your children’s performance. It is only to learn about your children’s expectations /experience with the program, the resources available and any challenges. We want to get the first￾hand perspectives of parents/guardians of Youth Potential Activity participants. We are very grateful for your time; this discussion should take about an hour. The information you provide to us here is confidential and will not be shared with anyone outside. Please know that we will not share your name with anyone outside the research team, or quote you in our report without your permission, so we also hope that you will provide honest and accurate answers. For the note taker: Location: Region: _____________ Woreda: ______________Kebele: ______________ Focus Group date: __________________________ Starting time: _____________________ Number of males: Number of females: Pass out a sheet of paper to gather information on: Participant name, gender, Kebele, participant signature 1. Before we begin, kindly; introduce yourself…. 2. Tell us about yourself (5 minutes of ice breaker – show of hands and count) a. How many of you are parents of participants in POTENTIAL program vs guardians? b. How many children do you have? c. How many of them live at home? d. How many of you own a house? e. How many of you own land for agriculture? If so, do your children help you with farming? f. How many of you have your own business? If so, do your children help you with your family business? g. How many of you have a child (children) who are excited to be participating in Potential program? 3. What do you know about the Potential program? a. How did you hear about the Youth Potential program? b. Did you encourage your child to join Potential? Why or why not? c. What is the reputation of Youth Potential program in your community? 4. What type of skills is Potential helping your child gain, if any? - Probe with examples if needed: -Agriculture -Self-employment (open shops, find a business idea, customer service, how to calculate profit, marketing, etc.) -Accessing financing -Soft skills (confidence, job applications, interview skills, communication, being proactive, etc.) -Technical/vocational skills (hairdressing, tailoring, bakery, welding, mechanics, construction, driving, etc.) 5. Have you noticed any changes in your child since he or she joined Potential? (For example, has more confidence, saves more money, is more responsible, has new hard skills, etc.) 6. What kinds of work would you like to see for your children? (Probe on agriculture, off-farm self-employment 104 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline opportunities) a. What are the main barriers for youth to start their own business in your community? b. What are the main barriers for youth to obtain wage employment? c. Do you think there are enough jobs for youth in your community? 7. How is Potential impacting your child in terms of employment opportunities? (Probe on both starting new businesses, and gaining wage employment.) a. How do you think Potential has, or has not, helped your child overcome these barriers to start their own business? b. How do you think Potential has or has not helped your child overcome these barriers to get employed? 8. If your household had more income, what type of health-related expenses would you spend more money on? (For example, preventive care, vaccinations, reproductive health, etc.) - Probe – has your child’s employment made any difference to how much you spend or she/he spends? 9. Is there any other feedback you think it is important to share with us about the Potential program? THANK YOU 105 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Guide for KII – YP Facilitators and Woreda Officers General Introduction and Informed Consent Welcome! My name is _________________________________ from Social Impact, an evaluation company from America. We are conducting a study to assess the effect of the Youth Potential program from selected Woredas in four regions. For this we are following a cohort of over 400 participants for 12 months. We started the first round in April-May, and we are following up now six months later. This KII is part of cohort study of the USAID/Youth Potential project. This interview is to learn about your thoughts on the Youth Potential project. We are very grateful for your time; this interview should take about half an hour. The information you provide to us here is confidential and will not be shared with anyone outside. Please know that all information will be combined together for report, we will not quote you in our report without your permission, so we hope that you will provide honest and accurate answers. Location: Region: _____________ Woreda: __________Kebele: ______________ Office_____________ Position of the participant_____________________ (Ask Q1 and Q2 only for new facilitators – don’t need to repeat if we interviewed facilitator at the baseline.) 1. What is your role on the project? How long have you been involved with it? 2. Can you describe the project activities? (Note: for new facilitators ask questions below in general and for the rest ask as below) Thinking of the cohort from March/April 2017: 3. How are youth doing in Potential training program over the past 6 months? a. What proportion of youth are completing the soft skills training? (PYD, WRN, etc.) The business training (BYOB)? b. What are the main reasons youth drop-out of the training? What have you found to be most helpful in overcoming the drop-out challenges? c. What are lessons learned in working with youth to prepare their personal development plans? What about the business idea plans? d. What is the experience with illiterate youth? Is there any support for literacy/numeracy? e. What proportion of youth receive hard skills training? f. What are the most common topics for hard skills training? What additional hard skills topics are youth requesting training? (e.g. driving, construction, baking, etc.?) g. How many youth have opportunities for work based learning? Which are most common – internships, job shadowing, volunteering, other? h. What are challenges in youth experiences with work based learning? What about challenges with employers? 4. Support for youth in self-employment/starting their own businesses a. How are youth identifying viable business ideas in their communities? b. How are youth identifying agriculture opportunities in their communities? (e.g. vegetable production, fattening, chicken production, etc.) c. How many youth receive coaching/mentoring? On average, how many sessions do they receive? What has been working well with coaching/mentoring, what has been challenging? d. How are youth getting support to get start-up finance for their business idea? How are they accessing the materials and equipment they need? What about land for agriculture opportunities? e. Are youth showing interest in starting group business with their peers? 5. Support for youth in finding wage employment a. What is the experience so far of youth finding jobs/wage employment after training? In what sectors and types of occupations are they finding jobs? b. What sort of job placement support is Potential providing? What are the main challenges in being able to offer this support? c. What is the experience in working with employers for the work based learning, and job 106 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline placement? 6. What are the challenges in tracking and following up on trained beneficiaries? What have you found to work well in tracking the youth? a. How many youth in your cohort migrated to a new town/region to find more employment opportunities? 7. Are youth sharing any of their experiences and challenges related to health problems and health outcomes? 8. What is your experience with Woreda government partners and other stakeholders (TVET, TTIs, credit association/microfinance, etc.)? 9. Is there anything else you would like to share with us about how youth are doing in income and employment status, or general challenges/lessons learned? THANK YOU 107 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Guide for KII – Microfinance Providers General Introduction and Informed Consent Welcome! My name is _________________________________ from Social Impact, an evaluation company from America. We are conducting a study to assess the effect of the Youth Potential program from selected Woredas in four regions. For this we will follow a cohort of over 400 participants for 12 months. This KII is part of cohort study of the USAID/Youth Potential project. This interview is to learn the firsthand perspective of microfinance providers. We are very grateful for your time; this interview should take about half an hour. The information you provide to us here is confidential and will not be shared with anyone outside. Please know that all information will be combined together for report, we will not quote you in our report without your permission, so we hope that you will provide honest and accurate answers. For the note taker: Name of Participant:__________________________ Name of Institution: ____________________ Location: Region: _____________ Woreda: ______________Kebele: ______________ Interview date: __________________________ Starting time: _____________________ 1. What is your role at the microfinance institution? How long have you been working here? 2. How did you start partnering with the Potential program? 3. What share of your overall clients are young people? 4. What types of financial products and services do you offer to young people – e.g. loans, savings accounts, etc.? Are these the same financial products available to Potential youth, or do they have access to different products? (Probe – do they have any group products for youth available?) 5. What types of businesses do Potential youth who apply for your loans have? What is the quality of the business plans they share with you? 6. What is the average size of loans you give to Potential youth, and what is the range of the loan size? 7. What do youth typically present as collateral when they apply for loans? 8. In what ways, if any, are youth from Potential different than other youth who apply for loans at your microfinance institution? (Probe – Is there any difference in repayment?) THANK YOU 108 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline ANNEX VIII: DATA TABLES AND GRAPHS Additional Data tables and Graphs, Regression Analysis and Expenditures by Category Demographic characteristics of midterm participants, by region (%) Level of satisfaction with current job, by sex and age group (%) 109 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Average monthly expenditure for select participants, by region and sex (Birr, USD) In Birr Regions Month Afar Amhara Oromiya Tigray Regional Total June 4300* 581 1092 132 1026 July 584 288 586 1153 697 August 1022 422 586 1437 929 September 927 299 646 1102 801 In USD Regions Month Afar Amhara Oromiya Tigray Regional Total June 188* 25 48 6 188 July 25 13 26 50 25 August 45 18 26 63 45 September 40 13 28 48 40 Note: Region average is calculated based on relative shares of each region in total number of respondents interviewed. *Omitted the outlier for Afar where Q20q_Howmuch (i.e., asset expenditure was 45000 (Q1_ID==169754312). Determinants of Employment Using Regression Analysis Determinants of employment using regression analysis (odds ratios; dependent variable: employed=1) Table 1: Please tell us about your income level. Compared with six months ago, has your income changed? Freq. Percent Cum. Increased 325 60.98 60.98 Decreased 71 13.32 74.3 Stayed the same 137 25.7 100 Total 533 100 Table 2: Cross tab between sector of employment and those reporting income increase/decrease (frequency) No Yes Total In school or training 9 2 11 Paid/wage/paid-in kind 13 34 47 Self-employed 47 162 209 Self-employed but looking for job 48 89 137 Unemployed (dependent 84 37 121 110 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Unemployed but not seeking Job 5 1 6 Other 2 0 2 Total 208 325 533 Table 3: Likelihood of saying that income increased ml_Q40a1 Odds Ratio Std. Err. z- value |P>z| [95% Conf. Interval] gender 1.38 0.33 1.35 0.18 0.86 2.20 age_gr1 (15-18 yrs) 0.95 0.30 -0.16 0.88 0.51 1.77 age_gr3 (25-30 yrs) 0.73 0.20 -1.15 0.25 0.42 1.25 HH_size1 (1-2) 1.08 0.33 0.27 0.79 0.60 1.96 HH_size3 (5-10) 1.35 0.33 1.22 0.22 0.83 2.18 HH_size4 (10-15) 0.66 0.62 -0.44 0.66 0.11 4.16 marital_status2 (Married) 0.95 0.25 -0.18 0.86 0.57 1.59 marital_status3 (Other) 0.92 0.41 -0.19 0.85 0.38 2.22 Ref: (Paid/wage/paid-in kind employed) ml_Q213 (Self-employed) 1.44 0.77 0.68 0.49 0.50 4.13 ml_Q214(Self-employed but looking for job) 0.92 0.51 -0.15 0.88 0.31 2.71 ml_Q37bY (Received loan) 2.19 1.05 1.64 0.10 0.86 5.60 Sector of employment (Ref: Agriculture) mlQ231b2 (Trading) 4.23 2.42 2.52 0.01 1.38 12.99 mlQ231b3 (Manufacturing and constr.) 7.01 4.30 3.18 0.00 2.11 23.31 mlQ231b4 (Daily Lab) 5.58 3.91 2.45 0.01 1.41 22.05 mlQ231b5 (Service) 5.06 2.50 3.28 0.00 1.92 13.32 mlQ231b6 (Professional) 6.79 4.11 3.17 0.00 2.08 22.22 Region: (Ref: ) Tigray Location_Region2 (Afar) 0.95 0.42 -0.12 0.90 0.40 2.24 Location_Region3 (Amhara) 0.36 0.14 -2.68 0.01 0.17 0.76 Location_Region4 (Oromia) 0.50 0.19 -1.78 0.08 0.24 1.07 _cons 0.61 0.25 -1.21 0.23 0.28 1.36 Logistic regression Log likelihood -301.41 Number of observations 533 LR chi2(19) 110.17 Prob > chi2 0.000 Pseudo R2 0.1545 111 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Table 4: Fairlie decomposition employed Coef. Std. Err. z P>z [95% Conf. Interval] % Share= (coeff/Total explained) gender 0.010 0.003 3.080 0.002 0.004 0.017 0.18 lincome 0.045 0.006 8.180 0.000 0.035 0.056 0.79 age_gr 0.001 0.002 0.740 0.457 -0.002 0.005 0.02 HHsize 0.000 0.001 0.350 0.725 -0.001 0.002 0.00 marstat 0.005 0.003 1.410 0.157 -0.002 0.011 0.08 Q37bY 0.006 0.002 2.640 0.008 0.002 0.011 0.11 region -0.011 0.008 -1.400 0.161 -0.026 0.004 -0.19 Panel II 1.00 Legend Total change in employment from baseline to midline Total change in employment from baseline to midline explained by the factors other than treatment Group variables: age_gr: age_gr1 (15-18) age_gr2 (19-24) age_gr3 (25+) HHsize: HH_size1 (1-2) HH_size2 (3-5) HH_size3 (6-10) HH_size4 (11+) marstat: marital_status1 marital_status2 marital_status3 (never married, married, other) region: Location_Region1 (Tigray), Location_Region2 (Afar), Location_Region3 (Amhara), Location_Region4 (Oromia)) Number of obs 1,066 N of obs G=0 (Baseline) 533 N of obs G=1 533 Pr(Y!=0G=0) 0.74 Pr(Y!=0G=1) 0.47 Difference 0.26 Total explained 0.06 112 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline Expenditure by category Table 5: Distribution of the median monthly household expenditures by region and sex (in USD) Expenditure Category Midline Expenditures in USD* Distribution by region (median expenditures) Distribution by sex (median) Total Mean expenditure Total median expenditure Afar Amhara Oromiya Tigray Male Female n 533 533 95 172 171 95 304 229 Rent/lease 70 48 22 44 79 52 52 39 Taxation 32 13 13 50 6 19 7 20 Transportation 17 11 13 10 10 11 13 11 Buy food and related goods 69 44 35 72 22 66 44 52 Pay for electricity, water, 13 9 13 5 9 5 9 10 Clothing 37 26 44 22 26 30 35 22 Buy household goods 32 17 35 17 13 22 17 17 Education / school fees 43 22 13 31 26 21 48 13 Health care related expenditures 17 11 13 9 8 13 11 13 Contraceptives and family planning 2 1 0 2 1 0 2 1 Support to other family members 47 31 46 26 26 44 44 24 Invest in business or other assets 288 175 87 153 262 214 192 175 Savings 94 43 124 26 26 87 52 31 Other expenditures 12 9 66 9 0 0 11 9 Total expenditure 228 *Exchange rate 22.9 Birr = 1 USD (using rate from April to allow for comparability with baseline) 113 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline ANNEX IX: STUDY TEAM DISCLOSURES OF CONFLICT OF INTEREST 114 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline 115 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline 116 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline 117 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline 118 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline 119 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline ANNEX X: STUDY TEAM PROFILES Dr. Deepika Chawla, Team Leader, brings over 15 years of experience in project management and evaluation (including impact assessments) of education, workforce development and health programs in the US as well as in several countries around the world. In the course of her career she has worked in and supported projects in Africa (Ethiopia, Liberia, Malawi, Morocco, Nigeria, and Sudan), Eastern Europe and South Asia (India, Afghanistan, Bangladesh). She has successfully managed several multi￾million-dollar USAID funded programs. Dr. Chawla has designed and implemented several quantitative and qualitative evaluations of education and health programs, as well as training and capacity building programs in the United States and overseas. She has also designed and supported the monitoring and evaluation of several USAID funded projects overseas to study the impact and effectiveness of these programs. These evaluations have included the conduct of paper-pencil and web-based surveys, site visits, focus groups and interviews, as well as reviews of administrative data. Proficient in qualitative and quantitative data analysis, Dr. Chawla has led research studies on child welfare programs, trafficking in persons, labor force, etc., in Eastern Europe. In Asia, she has conducted an analysis of factors affecting participation of girls in education in rural areas of India and identified the principal determinants that act as barriers to continued participation and retention in the education cycle. In another study in India, she analyzed the factors affecting academic achievement for urban boys and girls and the barriers to girls’ achievement in school. In Bangladesh, she analyzed the data from the Demographic Health Survey to identify the demand and utilization of contraception among urban and rural women. She is currently analyzing the factors affecting reading outcomes for children in South Asia. Dr. Chawla has developed National Education Accounts, a rapid assessment tool for mapping expenditure flows in the education sector according to the sources-to-financing agents-to-uses framework, which is increasingly being used by policy makers and donors interested in improving education outcomes at the state and national level. She has worked with in-country teams to build capacity for the conduct of future NEAs. In addition to successfully managing multimillion-dollar programs, she has been actively involved in new business development and knowledge management for the Education for Development division of Creative Associates and prior to that at Abt Associates. Dr. Chawla holds an Ed.D. in education administration, training and policy studies, and a Ph.D. in psychology with a focus on psychometrics and education assessment. Mr. Ermias Assefa Eshetu, Local Expert, brings over 10 years of monitoring and evaluation and development experience to the team. Mr. Eshetu’s has experience in qualitative and quantitative research, data collection, analysis, and management, conducting community discussions, recruiting research participants, preparing planning, budgeting, and M&E manuals, and in preparing questionnaires using statistical packages. Mr. Eshetu also has extensive experience developing and utilizing socio￾economic and demographic surveys, such as Behavioral Surveillance Survey (BSS); Demographic Health survey (DHS); Agricultural samples; Food security; small, medium, and large-scale industries and construction; educational, health, nutrition, and others. As a member of the study team, Mr. Eshetu was responsible for: participating in the development of data collection instruments; training data collectors and supervisors; following the performance of a data collection firm; ensuring that study outputs are of high quality and useful; identifying a sampling frame for primary qualitative and/or quantitative data collection during all phases of fieldwork; conducting data 120 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline collection, including interviews and FGDs with stakeholders and households; compiling project monitoring and government data by liaising with implementing, funding, and government stakeholders, and; participating in data analysis, presentation of findings, and report writing. Mr. Eshetu holds an M.A. in Measurement & Evaluation from Addis Ababa University, an M.Sc. in Economics (Specialized Agricultural Economics) from Haromaya University, and a B.Sc. in Information Systems from Addis Ababa University. Ms. Estera Barbarasa, International Expert, brings over 10 years of international workforce development, youth employment, labor markets, and monitoring and evaluation experience to the team. She is experienced in conducting mixed methods research, designing data collection instruments, facilitating focus groups, conducting KIIs, writing technical reports, and conducting youth employment and labor market evaluations. She is experienced in program design, project implementation, and evaluation with IDB, USAID, and World Bank. Ms. Barbarasa served as lead researcher of a thematic evaluation of the Inter-American Development Bank (IDB) Multilateral Investment Fund (MIF's) youth employment and entrepreneurship projects in Latin America and the Caribbean, evaluating 20 projects in eight countries to assess youths’ labor market outcomes. She also provided technical assistance in overseeing the design and completion of a national labor market assessment in El Salvador identifying jobs and skills trends, and she co-authored a report for the World Bank on “New and Promising Approaches in Youth Employment Programs,” which assessed youth employment design across 19 projects in 15 countries (including sub-Saharan Africa); highlighted innovations in skills training, youth self-employment in rural contexts, and youth and agricultural value chains; and analyzed M&E methods to track programs’ impact on job and income outcome metrics for youth. Ms. Barbarasa has worked in El Salvador, Ethiopia, India, Liberia, Malawi, and Morocco. She holds an M.P.P. in economic development from Harvard Kennedy School, and a B.A. in political science from University of Chicago. Mr. Worku Ambelu, Evaluation Methods Specialist for the USAID/Ethiopia Performance Monitoring and Evaluation Services Activity implemented by Social Impact, brings 20 years of development work expertise to the team. Mr. Ambelu possesses a wealth of knowledge in project monitoring and evaluation, planning, statistics, assessments and research activities gained from different organizations. Presently, as Evaluation Methods Specialist, he is responsible for coordinating all the evaluation activities and providing oversight on monitoring and learning activities. Mr. Ambelu coordinated and provided tool development, data analysis and report review support to the Graduation with Resilience to Achieve Sustainable Development (GRAD) final evaluation, Community Managed Disaster Risk Reduction (CMDRR) final evaluation, Ethiopia Gender Analysis and Youth Potential Baseline Survey. Before joining Social Impact, Mr. Ambelu was an M&E Expert for the World Bank/VNG International Consulting firm on a program called Promoting Basic Services - Ethiopia Social Accountability Program 2 (ESAP2), the objective of which was to capacitate citizens and citizen groups to hold accountable service providers and government officials for the basic services they provide in Agriculture, Education, Health, Water and Sanitation, and Rural Road sectors. Mr. Ambelu monitored if implementing partners had equipped citizens, service providers and government officials with the necessary capacity such as assessing the aforementioned basic services against the service standards, and Mr. Ambelu assessed if proper tool utilization in identifying service issues were being utilized (this includes the application of a 121 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline community score card, citizen report card, participatory planning and budgeting, and public expenditure and tracking survey tools). FGDs were intensively used as ongoing evaluation tools to identify the implementation status of projects by different implementing partners and to identify opportunities and challenges. Prior to that, Mr. Ambelu was working with World Learning Inc. to lead overall M&E issues for the School Community Partnership Serving OVC affected by HIV/AIDS in Primary Schools project. Mr. Ambelu spearheaded a wide variety of tasks from conducting annual and baseline surveys, developing database systems, producing survey/research reports and training staff on M&E, data management, indicator development. Mr. Ambelu assumed the Monitoring and Evaluation Specialist and Database Manger positions for the USAID-Community Government Partnership program that was implemented by World Learning and other partners to enhance quality and equity in primary education. He worked as the Monitoring and Evaluation Officer for the health PEPFAR project called CASCADE (Communities and Schools for Children Affected by HIV/AIDS). Before re-joining World Learning in 2012, Mr. Ambelu was at the Population Council as a Research Coordinator working on Poverty, Gender, Youth, HIV/AIDS and Reproductive Health programs. His role in the organization included facilitating baseline, end line and impact evaluation surveys/research protocols, coordinating data collection and data entry, preparing data entry templates, developing sampling methodology, sampling respondents, producing data tables, interpreting results, producing reports, and disseminating findings. The youth survey, coordinated by Mr. Ambelu, focused on a range of youth issues such as Education, Livelihoods, Access and Utilization of Services, Participation and Parent￾Child Relationships, Female Genital Mutilation and Male Circumcision, Alcohol and Chat/Khat, etc. This included both qualitative and quantitative data analysis and report writing. He also contributed to the production of service statistics and routine monitoring reports. Most importantly, Mr. Ambelu’s research findings informed policy makers and project implementers to take different actions. Mr. Ambelu has a strong statistical background receiving his M.Sc. and B.Sc. degrees in the field of Statistics from the Addis Ababa University and is also well trained in different statistical and qualitative software. Ms. Hika Alemu, Monitoring, Evaluation, Accountability, and Learning Specialist for the USAID/Ethiopia Performance Monitoring and Evaluation Services Activity, brings over 12 years of experience in development as a Performance MEL Adviser, Senior Program Officer, M&E Specialist, M&E Officer, Project Coordinator and Facilitator and Trainer for three USAID funded international NGOs; and two USAID MEL contractors with strong experience in education as well. Ms. Alemu has worked in Ethiopia with international development firms and agencies such as Social Impact Inc., Population Council Inc, Management Systems International(MSI), EngenderHealth Inc., and Food for the Hungry(FH). As a Performance, Monitoring Advisor for Social Impact, she has been supporting the USAID/Ethiopia Mission and its Implementing Partners (IPs) in terms of strengthening its Monitoring, Evaluation and Learning System and was responsible for the management of the USAID/Ethiopia Mission and its IPs MLE system and capacity development. During her work at Population Council Inc., Ms. Alemu revised and designed the M&E system, developed the project’s log frame and produced project proposals to donors. As a manager, she exhibited excellency in clearly articulating expectations and holding staff accountable for results, and for providing support and input where needed. During her work at MSI, Ms. Alemu was extensively and actively involved in developing, refining, and documenting the USAID/Ethiopia PMP/M&E Plan and project proposal to ensure that USAID and government requirements were met, and that PMP/M&E indicators informed decision￾making. As M&E Officer for EngenderHealth Inc., Ms. Alemu oversaw the overall MEL of four local 122 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline partners at 31 project sites. She facilitated different trainings for partners and for EngenderHealth Inc. staff on basic monitoring tools and their utilization. Ms. Alemu has completed training in MEAL, basic research methodologies, and HIV/AIDS prevention, and possesses good knowledge of various software packages for both data entry and processing, such as MS Access, MS Excel, SPSS, STATA, EventSmart and Epinfo. Ms. Alemu holds an M.A. in Development Economics from Unity University, a B.A. in Economics from Alpha University College, and a B.Sc. in Biology from Addis Ababa University. Dr. Basab Dasgupta, Senior Technical Expert, possesses more than 18 years of research experience and 13 years of experience designing, implementing and managing evaluations, assessments and surveys within an international development context. Currently as one of the SI’s Technical Directors in Impact Evaluation team, Dr. Dasgupta is participating in several performance and impact evaluations as Senior Technical Expert and leading projects as Principal Investigator in Social Impact. For example, Basab, as Project Director, is leading Early Grade Reading Program in Haiti (USAID funded), and Girls Education Challenges in Ghana (DFID funded). He is also leading the impact evaluation component of the Young African Leadership Program in 14 East African countries. As Senior Technical Expert, Basab is also working in several projects such as, Evaluation of ‘Honduras Threshold Program (MCC funded)’; and Impact Evaluation of ‘Lesotho Metlong Dam and Water Supply (MCC funded)’. Prior to joining SI, Dr. Dasgupta worked as a Senior Consultant/Economist for the World Bank for more than 11 years. As an economic researcher and evaluation specialist in the World Bank Water global practice, he conducted (i) quantitative and qualitative analysis to assess the impact of institutional bottlenecks in the water sector on access to improved water, improved sanitation, poverty and human development outcomes, (ii) water resource management and functionality of water schemes and points, (iii) informal water markets in Nigeria, Palestine and Tanzania, and (v) prepared policy notes by stock taking of impact and performance evaluation in the water and sanitation sector. In the Economic Policy Unit, South Asia region of the World Bank, he designed the strategy and methodology for evaluation of the Impact of the Telangana Rural Inclusive Growth Project in India. As a core team member, Dr. Dasgupta prepared scoping paper for M&E, and designed the evaluation methodology to evaluate the impact of Nutrition-Water-Sanitation linkages under the Rural Inclusive Livelihood Project for Andhra Pradesh and Telengana states in India. In his roles at the World Bank Sustainable Development Network, Dr. Dasgupta worked on Economic Sector Work on infrastructure development in LAC region, evaluated a housing subsidy program in South Africa and worked on inclusive growth and housing investment in Sub Saharan Africa, Turkey and Tunisia. Dr. Dasgupta contributed to several poverty assessment reports, analyzed wage and employment trends in India, global price shocks on rural poor in Bangladesh, shortcomings in the functioning of labor markets and governments in Europe and Central Asia, poverty in the European Union, and the financial crisis across five countries such as Turkey, Montenegro, Bulgaria, Armenia and Romania. He also spearheaded evaluations of the social assistance program on work disincentives in Armenia, support for poor and disadvantaged area program in Indonesia, and mediation and a community legal empowerment program in Indonesia. He led an analysis of the quality of education and health services in Indonesia and developed an analytical model for a cost-benefit analysis to evaluate alternative infrastructural amenities in Brazil. Dr. Dasgupta has coauthored several articles in peer reviewed journals, World Bank Working papers series and contributed to Book Chapters. 123 Ethiopia Performance Monitoring and Evaluation Service Final Evaluation Report, Youth Cohort Study Midline During his stay in India, Dr. Dasgupta worked as Consultant for IFPRI for 2 years and Research Associate in Indian Institute of Management, Ahmedabad for more than 3 years. Dr. Dasgupta received his Ph.D. in Economics from the University of Connecticut and M.Phil. in Economics from Jawaharlal Nehru University. He is fluent in English, Bengali and Hindi. U.S. Agency for International Development Entoto Street PO Box 1014 Addis Ababa, Ethiopia