YOUTH EMPLOYMENT PROJECT MIDTERM EVALUATION REPORT (UNDP) DECEMBER 9, 2016 This publication was produced for review by the United States Agency for International Development. It was prepared by Jeffrey Tines, Noman ul Haq Siddiqui, Nadeem Akhtar, Muhammad Sadiq, Tehmina Tanveer, and Syed Zohaib Iqbal Zaidi, Management Systems International, A Tetra Tech Company. YOUTH EMPLOYMENT PROJECT MIDTERM EVALUATION REPORT (UNDP) Contracted under Order No. AID-391-C-15-00004 Performance Management Support Contract DISCLAIMER This report is made possible by the support of the American people through the United States Agency for International Development (USAID). The contents are the sole responsibility of Management Systems International and do not necessarily reflect the views of USAID or the United States Government. Youth Employment Project: Midterm Evaluation Report (UNDP) iii CONTENTS Acronyms............................................................................................................................v Project Summary .............................................................................................................. 1 Executive Summary.......................................................................................................... 3 Evaluation Purpose and Questions.................................................................................................. 3 Project Background............................................................................................................................. 3 Key Findings and Conclusions.......................................................................................................... 3 Recommendations............................................................................................................................... 4 Evaluation Purpose and Questions.................................................................................. 5 Evaluation Questions.......................................................................................................................... 5 Project Background .......................................................................................................... 5 Theory of Change................................................................................................................................ 6 Evaluation Methods and Limitations............................................................................... 6 Data Collection.................................................................................................................................... 6 Evaluation Limitations and Mitigation Measures........................................................................... 8 Findings .............................................................................................................................. 9 Q1: How effective are the project’s targeting, recruiting, and selection strategies? ............................................................................................................................... 9 Q2: How is the program affecting individual trainees?.............................................................12 Q3: How, if at all, has the perception of insecurity changed in targeted towns?...............20 Q4: How are stakeholders involved in the project? .................................................................25 Q5: Is the project scalable and can local resources be leveraged to transition it out of donor dependence? ...............................................................................................27 Conclusions ...................................................................................................................... 29 Recommendations .......................................................................................................... 30 Annexes............................................................................................................................ 32 Annex 1: Evaluation Scope of Work.............................................................................................32 Annex 2: Assignment Work Plan...................................................................................................38 Annex 3: Sample by UNDP Training Partner .............................................................................69 Annex 4: Data Collection Instruments.........................................................................................71 Annex 5: Sampling ...........................................................................................................................161 Annex 6: Marginalization Index ....................................................................................................162 Annex 7: Effect of Distance on Recruiting.................................................................................167 Annex 8: Conflict of Interest Disclosures.................................................................................168 Youth Employment Project: Midterm Evaluation Report (UNDP) iv LISTS OF TABLES AND FIGURES List of Tables Table 1: Project Summary................................................................................................................................................ 1 Table 2: Surveys Conducted, Planned Versus Actual................................................................................................ 7 Table 3: Profile of Trainees Relative to Karachi.......................................................................................................11 Table 4: Cost-Effectiveness of Recruiting Strategies...............................................................................................12 Table 5: Transition of Alumni Between States of Employment............................................................................14 Table 6: Alumni Who Would Pay for Similar Training in Future.........................................................................27 Table 7: Profile of Survey Respondents......................................................................................................................70 Table 8: Surveys Required, Planned, and Completed ...........................................................................................161 Table 9: Dimensions, Indicators, Definitions, and Weights for the CMI..........................................................163 Table 10: CMI Calculation for all 18 Towns...........................................................................................................165 Table 11: Regression Analysis of Effects of Distance on Recruiting..................................................................167 List of Figures Figure 1: Youth Employment Project Training Partners.......................................................................................... 2 Figure 2: Distribution of Trainees by Town................................................................................................................ 9 Figure 3: Employment Status Before and After Training........................................................................................14 Figure 4: Average Weekly Working Hours Before and After Training..............................................................16 Figure 5: Median Personal Monthly Income Before and After Training .............................................................17 Figure 6: Job Satisfaction of Employed Alumni .........................................................................................................17 Figure 7: How Training Contributes to Reduced Crime and Violence..............................................................20 Figure 8: Survey Respondents’ Contribution to Household Income, Household Affairs, and Neighborhood Wellbeing ............................................................................................................................21 Figure 9: Perceived Level of Crime and Violence....................................................................................................21 Figure 10: Impact of Crime and Violence on Households.....................................................................................22 Figure 11: Perceived Impact of Religious Groups....................................................................................................22 Figure 12: Perceived Impact of Political Parties........................................................................................................23 Figure 13: Perceptions of Income-Generating Opportunities ..............................................................................23 Figure 14: Perceptions of Poverty Level ....................................................................................................................24 Figure 15: Residents’ Satisfaction with Government Services..............................................................................24 Figure 16: Marginalization Index, Location of Training Venues, and Percentage of Trainees by Town....166 Youth Employment Project: Midterm Evaluation Report (UNDP) v ACRONYMS CBO Community-Based Organization CMI Composite Marginalization Index DO Development Objective MITI Memon Industrial and Technical Institute MoU Memorandum of Understanding MSI Management Systems International NGO Non-governmental Organization P&D Planning and Development Department PHMA IKTK Pakistan Hosiery Manufacturers and Exporters Association Institute of Knitwear Technology Karachi SMS Short Message Service SPO Strengthening Participatory Organization STEVTA Sindh Technical Education and Vocational Training Authority STU Satellite Training Unit TVET Technical Vocational Education and Training UNDP United Nations Development Program USAID United States Agency for International Development VTC Vocational Training Center Youth Employment Project: Midterm Evaluation Report (UNDP) 1 PROJECT SUMMARY Table 1 summarizes basic information about the Karachi Youth Employment Project. TABLE 1: PROJECT SUMMARY Title/Field Project/Activity Information Contract/agreement numbers AID-391-IO-15-00004 Contracting/agreement officer’s representative (COR/AOR) Start date January 7, 2015 Completion date January 6, 2018 Location Karachi Implementing partner(s) UNDP USAID/Pakistan Mission Strategic Framework objectives addressed DO3: Increased stability in focus areas DO2: Improved economic status of focus populations Budget Youth Employment Project: Midterm Evaluation Report (UNDP) 2 FIGURE 1: YOUTH EMPLOYMENT PROJECT TRAINING PARTNERS Youth Employment Project: Midterm Evaluation Report (UNDP) 3 EXECUTIVE SUMMARY Evaluation Purpose and Questions The midterm evaluation of the UNDP-implemented Youth Employment Project aimed to assess whether the project is on track to meet training and employment targets, examine the efficacy of recruitment strategies, gauge stakeholder involvement, and assess the project’s potential for scalability and sustainability. USAID/Pakistan expects to use the evaluation results to guide future stabilization programming, including the design of the second part of Let Girls Learn, a USAID-funded program that seeks opportunities to foster public-private partnerships to advance girls’ education. The evaluation will also help identify potential course corrections before the project ends in January 2018. The evaluation addresses five questions: 1. How effective are the program’s targeting, recruiting, and selection strategies? 2. How is the program affecting individual trainees? 3. How, if at all, has the perception of insecurity changed in targeted neighborhoods? 4. What is the stakeholder involvement in the projects? 5. Is the project scalable and can local resources be leveraged to transition it out of donor dependence? Project Background The project focuses on improving the quality of skilled labor available in Karachi, with a focus on enhancing employment prospects for youth (aged 18-29 years) in marginalized neighborhoods identified as priorities by USAID. The project’s theory of change hypothesizes that training marginalized/at-risk youth from Karachi’s violence-prone areas will improve their employment prospects, reduce violent extremism, and contribute to establishing a foundation for long-term economic growth and prosperity. Trained individuals are expected to become agents of change in their society and contribute to reducing aggregate levels of violence and extremism in the city. The project focuses on vocational training to build the capacities of youth to engage in the job market in strategically selected industries that are important in the project areas. It works through 10 public and private technical vocational education and training (TVET) institutes and satellite training units (STUs) it has established within eight large-scale textile manufacturers. Key Findings and Conclusions UNDP effectively attracted and selected most of its beneficiaries from marginalized neighborhoods and towns that USAID identified as priorities. UNDP’s practice of strategically collaborating with training institutes and satellite training units in the targeted neighborhoods contributed to the positive recruiting outcomes since the proximity of training facilities to trainees’ homes is an important determinant of their decisions about seeking and completing training, particularly for women. UNDP’s direct recruiting strategy relied largely on print media (newspaper ads, pamphlets, and brochures), internal announcements in its facilities, and direct contact with communities through social mobilization and community visits. However, most trainees heard about the opportunity from family and Youth Employment Project: Midterm Evaluation Report (UNDP) 4 friends. Without knowing the extent to which each direct recruiting method contributed to word-of￾mouth, it is not possible to assess the cost effectiveness of the different methods. However, since word￾of-mouth incurred no direct costs, it would substantially boost the cost-effectiveness of the direct methods that contributed most to word-of-mouth advertising. Training has created new employment by increasing the employment rate, decreasing the unemployment rate, and increasing labor force participation rates among trainees. It also helped trainees obtain better employment—many women shifted from self-employment to more stable wage employment, employed women are working more hours, employed men and women are earning higher incomes, and employed men and women are more satisfied with their jobs. However, the fact that 50 percent of alumni reported being unemployed or out of the labor force suggests either a mismatch between training topics and market demands or that the training is not equipping trainees with the hard and soft skills demanded by potential employers. Evidence from employers suggests a variety of issues. The evaluation findings suggest that better employment opportunities contribute to reducing crime and violence. The primary mechanism through which employment produces these positive outcomes is through productive employment taking the place of “negative” activities. However, the evidence for this is largely hearsay and not based on reported behavioral change. Communities, community organizations, and the private sector collaborated with UNDP on various aspects of design and implementation. Collaboration included working with community-based organizations (CBOs) and non-governmental organizations (NGOs) to leverage their connections with communities to inform youth of the training opportunities; working with community leaders to identify potential trainees, particularly girls; and soliciting feedback from private sector employers on the skills they required and the performance of trainees placed with employers. Private sector training institutes also supported the project by providing stipends to trainees and covering in-kind costs such as space, machinery, training materials, utilities, and transportation to trainees working in their facilities. Furthermore, the government allows UNDP to use Sindh Technical Education and Vocational Training Authority (STEVTA) training institutes—albeit with compensation. Recommendations  If USAID/Pakistan or UNDP wish to extend the project to additional towns, UNDP should establish relationships with training institutes and satellite centers in, or near, the targeted areas. Proximity of the training venue to the homes of potential trainees will attract more applicants and is especially important for women. Expanding training capacity, perhaps by working in additional neighborhoods and with more training partners, will be critical to meeting training targets.  UNDP needs to carefully examine the trades or occupations in which it provides training and work with the private sector to identify the skills required in these occupations and trades. It then needs to ensure that its training, in hard and soft skills, is focused on meeting market demands. More active collaboration with the private sector to identify employment opportunities and the required skills, and ensure that the training is effectively teaching these skills, may improve post-training employment outcomes and encourage the private sector to more actively support the training program.  UNDP needs to ensure that its implementing partners adapt their curricular content to the needs of the private sector. They should also engage the private sector in validating the curricula and ensuring that trainees acquire the soft and technical skills required by private sector employers. Given the importance employees, especially women, place on proximity to the workplace, UNDP should adopt a strategy for matching training at a particular venue to the sectors and skills required in the specific locality. Youth Employment Project: Midterm Evaluation Report (UNDP) 5  UNDP’s implementing partners need to build their capacities in placement, vocational counseling, and employment services. Without these key services, placement and employment rates are likely to remain low. EVALUATION PURPOSE AND QUESTIONS The midterm evaluation of the Youth Employment Project implemented by UNDP aimed to assess whether the project is on track to meet training and employment targets, examine the efficacy of recruitment strategies, gauge stakeholder involvement, and assess the project’s potential for scalability and sustainability. USAID/Pakistan expects to use the evaluation results to guide future stabilization programming, including the design of the second part of Let Girls Learn, a USAID-funded program that seeks opportunities to foster public-private partnerships to advance girls’ education. The evaluation will also help identify potential course corrections before the project ends in January 2018. Evaluation Questions The evaluation focused on answering the following five questions identified in the scope of work (Annex 1) and assignment work plan (Annex 2): 1. How effective are the program’s targeting, recruiting, and selection strategies? 2. How is the program affecting individual trainees? 3. How, if at all, has the perception of insecurity changed in targeted neighborhoods? 4. What is the stakeholder involvement in the projects? 5. Is the project scalable and can local resources be leveraged to transition it out of donor dependence? PROJECT BACKGROUND Launched in 2015, the Youth Employment Project continues UNDP’s efforts to improve the quality of skilled labor available in Karachi. The project builds on UNDP’s experience implementing similar projects in Karachi, specifically the Gender Promotion through Skills Development in Garment Factories project (2006–2012) and the Promoting Employment and Productivity in Garment Industry project that started in 2012. In contrast to these earlier UNDP projects, which were implemented in cities across the country, the current partnership between UNDP and USAID focuses on youth (18–29 years of age) in Karachi. Planning started in January 2015, and the project began implementation in earnest in the second half of 2015. UNDP plans to train 13,160 youth from selected areas of Karachi within a 3-year window and has set a target of placing 75 percent of alumni in jobs. While the project initially focused on three priority areas—namely Lyari, Korangi, and Sultanabad—it recently expanded coverage to other towns in Karachi.1 1 UNDP’s first annual report states: “Recently USAID has allowed the project to expand its geographical boundaries and coverage to other poor, marginalized and vulnerable neighborhoods of Karachi such as Landhi, Malir, Shah Faisal, Orangi, SITE areas and Baldia. As a result, the project managed to increase its outreach and share the program benefits to other potential and deserving youth living in these parts of Karachi. Youth Employment Project: Midterm Evaluation Report (UNDP) 6 UNDP is implementing the project in close collaboration with 10 technical vocational education and training (TVET) institutes, 6 public and 4 private. In addition, UNDP has built on relationships with garment manufacturing factories established through previous projects and set up satellite training units with eight large-scale textile manufacturers. (For a list of all UNDP training partners, see Annex 3.) These satellite units offer trainings related to textile manufacturing—either domestic or industrial—with course duration ranging between one and two months. Recently, UNDP launched an internship program that provided students with three months of training split between formal training at a partner institute and on-the-job (internship) training in a related industry setting. To increase outreach and improve targeting, UNDP partnered with Strengthening Participatory Organization (SPO), an NGO that specializes in community mobilization. The resource centers SPO established in project areas work with social mobilizers and CBOs to encourage youth to apply for training, track trainees, and assist with job placement. Theory of Change The project’s theory of change hypothesizes that training marginalized/at-risk youth from Karachi’s violence-prone areas will improve their employment prospects, reduce violent extremism, and contribute to establishing a foundation for long-term economic growth and prosperity. Trained and employed individuals are expected to become agents of change in their society and contribute to reduced aggregate levels of violence and extremism in the city. EVALUATION METHODS AND LIMITATIONS The evaluation applies a quasi-experimental design aimed to measure project outcomes by incorporating both a pre-post comparison of alumni and a comparison of the pre-training status of current trainees to the post-training status of the alumni. The pre-training status of current trainees serves as a counterfactual (i.e., represents what the current status of alumni would have been if they had not participated in training) to compare with the post-training status of alumni as an alternative method of measuring impact. The ability to draw two comparisons for a set of outcome variables allowed the evaluation team to triangulate changes in outcome variables and enhance the reliability of its findings. Data Collection The evaluation used mixed methods to answer the evaluation questions. Primary data collection methods included surveys, semi-structured interviews, and group interviews. (See Annex 4 for the data collection instruments.) The evaluation also conducted a structured survey of a random sample of alumni and current trainees and administered group interviews to a subset of survey respondents to obtain more detailed information to answer how and why questions. Finally, the evaluation used key informant interviews with a purposive sample of respondents from government, industry, UNDP project staff, and UNDP training partners to learn their perspectives on the project. The evaluation also analyzed data from a variety of secondary sources. UNDP provided demographic information on trainees (both current and alumni) which served as the sampling frame for the survey and group interviews. To assess the prevalence of crime in Karachi’s towns, the evaluation team compiled and mapped crime reports for Karachi from the Express Tribune newspaper. To provide context and comparisons where applicable, the team also analyzed datasets from the Pakistan Bureau of Initially, the project faced difficulty in convincing garment companies and training institutes to recruit youth trainees from just three areas‐ Lyari, Korangi and Sultanabad.” (p. 7) Youth Employment Project: Midterm Evaluation Report (UNDP) 7 Statistics. A literature review of the available discourse on the subject provided background information for triangulation. Survey UNDP provided demographic data on trainees, including their town of residence, which served as the sampling frame for the survey. The evaluation team selected a simple random sample of alumni from all 18 UNDP training partners in Karachi and conducted a census survey of current trainees. The evaluation team calculated sample sizes that allowed them to detect a 10 percentage point change in the proportion of trainees employed as a result of the training, with an alpha of 0.05. Table 2 indicates the numbers of planned versus completed surveys. Annex 5 describes the sampling approach in more detail. TABLE 2: SURVEYS CONDUCTED, PLANNED VERSUS ACTUAL Respondent Type Number of Surveys Planned Number of Surveys Conducted Current trainees 201 217 Alumni 500 186 Total 701 403 Because of the difficulties of obtaining the necessary government permissions for fieldwork, the team conducted the surveys at training institutes and satellite training centers. The team used the mobile phone numbers provided by UNDP to contact the trainees and invite them to participate in the survey at the training institute where they received training. Surveyors attempted to contact trainees at least three times a day for two consecutive days; if they got no response after two days, they marked the case as a non-response and replaced it. Twenty-four percent of sampled individuals had their mobile phones switched off, and another 19 percent did not answer repeated phone calls, even after they had been sent a short SMS introducing the survey purpose. Ten percent were unavailable or unable to travel to training institutes for the survey. Only 5 percent of sampled individuals refused to take part in the survey. Overall, 32 percent of sampled individuals were available for the survey. Group Interviews The evaluation team conducted group interviews with a subset of survey respondents to acquire more detailed qualitative information to substantiate findings from the survey. The team conducted eight group interviews stratified by sex, training status, and employment status, with up to four participants in each group interview. The team purposively selected participants from the available pool of survey respondents by prioritizing respondents from areas that USAID categorized as marginalized, namely Lyari, Korangi, and Sultanabad. Key Informant Interviews The team conducted 32 key informant interviews with a variety of stakeholders including government representatives (3), UNDP project staff (3), SPO social mobilization staff (4), representatives of UNDP training partners (10), employers of alumni (5), experts on Karachi city (5), and representatives of industry trade associations (2). Youth Employment Project: Midterm Evaluation Report (UNDP) 8 Secondary Review In addition to collecting primary data using the methods outlined above, the evaluation team studied UNDP’s reports in detail. The team also reviewed relevant academic journal articles and policy documents, and analyzed relevant government statistics on labor force and employment to draw comparisons. Evaluation Limitations and Mitigation Measures The evaluation team made extensive efforts to ensure the data collected were valid and reliable. This section presents the limitations arising as a result of the evaluation design and field procedures. It also explains the mitigation measures the team took to minimize the impact of these limitations on the reliability and validity of the data collected. 1. Because of the difficulty of obtaining the necessary government permissions to conduct fieldwork in Karachi, the evaluation team was not able to visit trainees at their homes. The team considered a telephone survey; however, they deemed that option infeasible in view of the length of the questionnaire. Instead, they conducted the survey at the parent training institute of sampled trainees. To the extent that respondents’ willingness and ability to travel to another location for the interviews is related to anticipated outcomes (e.g., unemployed individuals may be more likely to come to the interview), conducting the interviews outside the home may have introduced bias. To help mitigate this source of potential bias, MSI offered respondents Rs. 500 to compensate them for their time and any costs associated with travel. 2. To reduce non-response rates, trainees were given Rs. 500 for taking part in the survey. Given the low response rate, this amount was perhaps lower than optimal. While it could be argued that monetary incentives bias responses, according to Toepoel, 2 they do not bias responses, and may instead contribute to more detailed information. 3. The potential for low response rates became apparent during field testing of the survey instrument. To mitigate this potential issue, the evaluation team provided replacements in addition to the original 701 sampled trainees, stratified by sex and town of residence. Despite this mitigation measure, the response rate was low. The team completed 403 surveys instead of the planned 701. The low response rate raised the potential for bias since trainees who came for the survey may be systematically different from those who did not. 4. Because of limited time and access, the evaluation team interviewed only 10 industry representatives. The evidence on employers’ perspectives is thus quite thin. While the responses illustrate issues and perspectives, they are not necessarily generalizable to the universe of employers. 5. The survey asked alumni a set of recall questions about their pre-training status. In some cases, the recall period was longer than one year, so the responses have the potential for some degree of unreliability. To mitigate this, the team limited recall questions to topics that trainees could reasonably be expected to remember. For example, the survey included questions on employment status and personal monthly incomes, but not on feelings of insecurity before the training. Another measure to mitigate the effect of recall bias was to train enumerators to encourage respondents to think carefully about the question and use as much time as they needed before answering recall 2 Toepoel, Vera. 2012. “Effects of Incentives in Surveys.” Handbook of Survey Methodology for the Social Sciences: 209-223. http://link.springer.com/chapter/10.1007%2F978-1-4614-3876-2_13. Youth Employment Project: Midterm Evaluation Report (UNDP) 9 questions. Additionally, comparing the pre-training status of current trainees to the pre-training status of alumni validated the reliability of responses to recall questions. FINDINGS Q1: How effective are the project’s targeting, recruiting, and selection strategies? Q1(a): Is UNDP selecting beneficiaries from marginalized/at-risk neighborhoods? The analysis addresses this question by examining whether beneficiaries (alumni and current trainees selected for USAID-supported training programs) are more likely to come from marginalized/at-risk towns than from other towns.3 The analysis uses two measures to identify marginalized/at-risk towns, (1) the three towns USAID identified as initial priorities in the cooperative agreement4 and (2) a marginalization index developed by the evaluation team (see Annex 6). A positive correlation between either marginalization measure and the percentage of alumni and current trainees from a town would suggest that UNDP is selecting beneficiaries disproportionally from marginalized/at-risk towns.5 UNDP’s records indicate that it has trained 3,542 individuals under the USAID-funded project. Sixty-two percent came from one of the three towns USAID identified as initial priorities, and 61 percent came from the towns the marginalization index ranked as the five most marginalized/at risk (Figure 2). FIGURE 2: DISTRIBUTION OF TRAINEES BY TOWN The evaluation team’s interviews asked UNDP and implementing partner staff how they selected trainees. While the question did not ask specifically about geographic selection criteria, 10 of 12 3 The analysis focuses on towns instead of neighborhoods because neighborhoods are not well defined, making it difficult to consistently identify the neighborhood in which a trainee lived. 4 Korangi, Lyari, and Sultanabad (a neighborhood of Kimari Town). 5 A more appropriate analysis would weight the percentages of trainees from a particular town by the town’s population (especially the population within the targeted age range). However, the most recent census data date from 1998, there are no projected population estimates at the town level, and the populations of the towns have probably changed substantially since 1998. Youth Employment Project: Midterm Evaluation Report (UNDP) 10 respondents alluded to geography and 6 specifically stated that they selected trainees from certain towns. Distance between trainees’ homes and the training facility strongly influenced the choice of training facility, especially for women. A regression analysis of the effect of distance on the percentage of trainees recruited from various towns found a statistically significant negative correlation between distance from the training venue and the proportion of applicants and trainees from the town.6 Furthermore, the effect of distance was stronger for women than for men (see Annex 7 for details of the regression analysis). Conclusions: The data show that UNDP attracted trainees largely from marginalized towns as identified by USAID and the evaluation’s marginalization index. The travel distance between the training facility and a trainee’s home was an important factor affecting recruiting; the fact that UNDP had numerous facilities within the priority towns contributed to its ability to attract individuals from these locations. Q1(b): Are the recruiting strategies attracting marginalized/at-risk individuals and is UNDP selecting marginalized/at-risk individuals from among the pool of recruits? This question differs from question 1(a) because it focuses on individual characteristics instead of characteristics of a town. To answer the question, the evaluation team first had to define “marginalized/at-risk” at the individual level. Because there is no standard definition, the analysis uses data from the surveys of alumni and current trainees to develop a profile along seven dimensions related to marginalization, all measured pre-training: household income, poverty score, likelihood of poverty, literacy, educational attainment, employment rate, and percentage employed. It then matches the characteristics of respondents to those of the population of Karachi to determine whether respondents were more or less “marginalized” than the general population. Because UNDP did not provide data on applicants, it is not possible to answer the first part of the question, i.e., whether recruiting strategies attracted marginalized/at-risk individuals. However, because UNDP did not use geographic criteria to select trainees from among the pool of applicants (see findings for question 1(a)), it is reasonable to assume that the “marginalization” status of applicants is similar to that of alumni and current trainees. To answer the second part of the question, i.e., whether UNDP selected marginalized/at-risk individuals from among the pool of recruits, the analysis compared the characteristics of alumni and current trainees at the time they started training to the values of these characteristics for the target population (i.e., 18–29 years of age) in Karachi (Table 3). The data in Table 3 suggest that, prior to training, UNDP trainees are more “marginalized” than their age-group peers in urban Karachi. They are less likely to be employed, less likely to be in the labor force, more likely to be impoverished, and have substantially lower median household incomes. 6 The analysis used a crude measure of distance between the geographic centers of the towns, i.e., the driving distance from the geographic center of the town in which a trainee lived to the center of the town in which he or she was trained. Youth Employment Project: Midterm Evaluation Report (UNDP) 11 TABLE 3: PROFILE OF TRAINEES RELATIVE TO KARACHI Characteristics Alumni (Pre￾training) and Current Trainees Karachi (18-29 Years of Age) Aged 18-29 years (%) 86% 21% Literate (%) 97% 90% In labor force (%) 36% 43% Employment rate (%) 74% 87% Educational attainment Secondary or lower (%) 50% 64% Higher secondary or above (%) 50% 36% Poverty score (mean) 74.9 73.5 Maximum likelihood of being under the poverty line of $1.25/day (average %) 37% 27% Maximum likelihood of being under the national poverty line (average %) 38% 30% Median monthly household income (Rs) 20,000 32,000 Household income of Rs 20,000 or below (%) 50% 52% Q1(c): What is the relative cost-effectiveness of the different recruiting strategies? To answer this question, the evaluation team first used data from the surveys of alumni and current trainees to determine which recruiting strategies attracted them to the training opportunity. It then collected data from UNDP on the costs of each recruiting method, estimated the total number of successful applicants (i.e., accepted into the training program) attracted by each method, and calculated the cost per successful applicant by dividing the cost of the method by the number of successful applicants it attracted. The survey of trainees (alumni and current trainees) asked respondents to indicate all the ways they had heard about the training opportunity. Family and friends emerged as the most frequently mentioned sources, with internal announcements (i.e., announcements made within the training locations) and newspaper ads as distant second and third most frequently mentioned. The data suggest that UNDP employed a fairly narrow recruiting strategy that relied largely on word-of-mouth and print media, with word-of-mouth and internal announcements—both with no direct costs—being the most effective. Table 4 ranks the recruiting methods from most to least cost-effective. These estimates suggest that, of the direct methods, print media (i.e., newspaper ads and brochures) and industrial and community visits were the most cost-effective. The remaining methods entailed substantially higher average cost per trainee.7 7 The data did not permit gaining a deeper understanding of the contribution of each of the direct methods to word-of-mouth. The ranking of methods by cost-effectiveness implicitly assumes that they contributed equally to word-of-mouth. Because the question asked which methods alumni and current trainees had heard of, and not which were most influential in their decision to apply, the analysis also assumes that the methods were equally influential in attracting successful applicants. Youth Employment Project: Midterm Evaluation Report (UNDP) 12 TABLE 4: COST-EFFECTIVENESS OF RECRUITING STRATEGIES Recruiting Method Effectiveness of Method (% of Respondents) 8 Total Cost of Method (Rs) Number of Successful Applicants Cost per Successful Applicant (Rs) Family and friends 62% N/A N/A N/A Internal announcements 14% N/A N/A N/A Newspaper ads 9% 500,000 362 1,381 Pamphlets, brochures, flyers 6% 1,000,000 149 6,723 Industrial and community visits 2% 500,000 71 7,085 Social mobilization 1% 1,500,000 38 39,896 Radio show 0% 500,000 0 -- All methods 4,000,000 619 6,463 / 1,193a a. UNDP spent Rs 6,463 per individual recruited directly by a method. Factoring in the contribution of the direct methods to word-of-mouth (family and friends) and the costless, but effective, internal announcements, implies that UNDP spent about Rs 4,000,000 to recruit 3,356 trainees at an average cost of Rs 1,193 per trainee. Q2: How is the program affecting individual trainees? Q2(a): To what extent are trainees obtaining new or better jobs relative to similar individuals who do not receive training? The evaluation team assessed various employment characteristics to determine whether trainees are getting “new or better jobs” through the project, relying largely on survey data. The factors include labor force participation rate, employment rate, employment type, contract length, income, working hours, and job satisfaction. The team also analyzed the degree to which jobs are relevant to trainings received, and whether implementing partners have provided adequate support in job placement. Taken together, analysis of these findings shows how the program affected the employment of beneficiaries. The analysis also explored the relationship between indicators of better jobs and factors such as training type, gender, prior education, and location to gain further insights into how and for whom training was effective. Sixty percent of surveyed alumni felt that the training they had received through the project had a positive impact on their employment. Women were more likely than men to believe their employment had improved (65 percent of women and 48 percent of men). Seventy-four percent of employed alumni—and 45 percent of unemployed alumni—believed that the training had a positive impact. A current trainee explained how he expected the training to improve his job prospects: “I feel that this certification will aid future job prospects. This certification is great not just because of the UNDP name, but also because it’s free. Especially when compared to other certifications that are costly and still don’t affect our job prospects, I find this to be a great opportunity.” – Male current trainee, Gulshan Town When asked to identify all the ways in which the project had positively affected employment from a defined list of effects, alumni most frequently mentioned that it had improved their skill set (34 percent 8 Percentage of survey respondents who mentioned each method. Youth Employment Project: Midterm Evaluation Report (UNDP) 13 of 115 responses), increased their confidence in finding a job (26 percent), and increased their income (22 percent). Employment Status9 The data suggest that UNDP training resulted in higher labor force participation rates. Participating in the labor force means that an individual either is working or has sought work in the past seven days. Among current trainees, approximately 53 percent of respondents were in the labor force prior to starting their training (Figure 3). Sixty-four percent of alumni reported participating in the labor force prior to starting training, a value that increased to 67 percent post-training. The pre-post comparison of alumni suggests that labor force participation increased by three percentage points, while the comparison between the post-training situation of alumni and the pre-training situation of current trainees suggests that labor force participation increased by 14 percentage points. A t-test of the labor force participation rates among alumni does not show a statistically significant difference between before and after training.10 The data also suggest that UNDP training increased the percentage of trainees who were employed— i.e., the percentage of respondents who reported having worked in the past seven days. Half of alumni reported being employed in the seven days prior to the survey, compared to 40 percent of alumni and 26 percent of current trainees who were employed prior to entering the program. The 10 percentage point difference between pre- and post-training employment for alumni is significant at 10 percent. 11 The 24 percentage point difference in employment rates between alumni (post-training) and current trainees provides further evidence that training increased incomes (interpreting the current trainees as the counterfactual). The proportion of alumni who are unemployed fell by 7 percentage points after they received training (Figure 3), and this difference is statistically significant. Similarly, the post-training unemployment rate among alumni is 9 percentage points lower than the 26 percent among current trainees (Figure 3) which further supports a conclusion that training reduced unemployment. Male and female alumni experienced similar changes in employment pre- and post-training: men’s employment increased from 55 percent to 65 percent, and women’s increased from 34 percent to 44 percent. Women have a harder time finding and keeping employment because of societal pressures and insecurity in Karachi. When asked to describe barriers to employment, 3 of 16 female group interview participants explained that their families do not approve of their working far from home, and 4 mentioned the judgement of neighbors even with family permission. “[My] family doesn’t mind me working but outsiders do, people who see us leave the house, those who live in the neighborhood. Things are better now, but factory work is difficult.”– Female current trainee, Lyari Town 9 For this research question, the evaluation team calculated rates of respondents who are employed, unemployed, and not in the labor force based on the number of all respondents who were asked the question. Individuals are designated as employed if they have a job or business or claim to have done work for pay, profit, or family gain for at least one hour in the seven-day period in question. If they are not employed, respondents are considered unemployed if they have been seeking work in the same seven-day period. All others are designated as outside the labor force, and they are not excluded from the calculation. Therefore, the three categories always equal 100 percent. 10 t=0.749, p=0.228 11 t=-1.32, p=.094 Youth Employment Project: Midterm Evaluation Report (UNDP) 14 FIGURE 3: EMPLOYMENT STATUS BEFORE AND AFTER TRAINING Older trainees seemed to benefit from training more than younger trainees. The percentage of alumni between 18 and 29 years old who were employed increased from 40 percent to 47 percent between pre- and post-training, while the percentage of those above age 29 who were employed increased from 45 percent to 67 percent. In fact, a correlation between age and post-training employment is positive and significant.12 A transition matrix (Table 5) shows that most alumni who were employed prior to training were employed after training, and that most who were not in the labor force prior to training remained out of the labor force after training. However, 34 percent of previously unemployed alumni gained employment after training, and 36 percent who were not previously in the labor force joined the labor force after training. TABLE 5: TRANSITION OF ALUMNI BETWEEN STATES OF EMPLOYMENT Post-Training Employment State Employed (%) Unemployed (%) Not in Labor Force (%) Pre￾Training Status (%) Pre-Training Employment State Employed (%) 85% 7% 8% 40% Unemployed (%) 34% 40% 27% 24% Not in Labor Force (%) 22% 14% 64% 36% Post-Training Status (%) 50% 17% 33% This transition matrix shows the (row) percentage of respondents who transitioned from the pre-training employment state to the post-training employment state. The effect of training on employment varied substantially by course. Employment among alumni of the Industrial Machine Operator training course increased by 57 percentage points—from 33 percent 12 t=0.131, p=0.074 Youth Employment Project: Midterm Evaluation Report (UNDP) 15 before training to 90 percent after. Employment for alumni of both Fashion Designing and Merchandising trainings increased by 11 and 16 percentage points, respectively. Employment rates for Industrial Stitching and Dress Making did not increase. Only 22 percent of alumni who had taken Industrial Stitching were employed both before and after training. In general, training related to stitching began with low levels of employment, and training had little effect on employment. 13 The percentage of alumni employed increased in all towns, with Landhi showing the largest increase (53 percent before training compared to 83 percent after) and Lyari the smallest (28 percent before training compared to 31 percent after). Jamshed had the lowest employment both before and after training (5 percent and 20 percent, respectively). This may be preliminary evidence that beneficiaries from Lyari and Jamshed Town are confronting a particularly difficult employment environment. Employment Type The survey data revealed no major shift in employment type for men before and after training. Women, however, shifted away from self-employment (41 percent of 41 employed women alumni before training compared to 31 percent after training) toward paid employment (54 percent before training to 67 percent after). The comparisons between employment types for alumni (post-training) and current trainees (the counterfactual) corroborate the results of the pre-post comparison for alumni. Men from either group are about as likely to be paid employees (95 percent of 40 alumni compared to 90 percent of 21 current trainees) and self-employed (5 percent for both groups), while alumni women are more likely than current trainees to be paid employees (67 percent of 57 alumni compared to 36 percent of 36 current trainees) and less likely to be self-employed (31 percent of 57 alumni compared to 58 percent of 36 current trainees). The shift in employment type for women represents better employment since paid employment provides a more stable income than self-employment. Training did not have a significant effect on the percentage of trainees with an employment contract. However, having a contract was positively and significantly correlated with job satisfaction—94 percent of 23 employed alumni with a contract were satisfied with their jobs, compared to 70 percent of 55 employed alumni without a contract, a statically significant difference. 14 This indicates that contracts do improve the attractiveness of job opportunities for young people in Karachi. This is especially true for written contracts. All of the 17 alumni with written contracts were satisfied with their jobs, compared to 5 of the 6 alumni with verbal contracts. Working Hours A comparison of employed alumni’s pre- and post-training weekly hours of work revealed an average decrease of five hours worked per week for men and an increase of six hours per week for women (Figure 4). The comparison between post-training alumni and current trainees also suggests that training reduced average hours worked for men (by two hours per week) and increased work hours for women (by five hours per week). The difference between the working hours of all employed alumni before and after training is statistically insignificant.15 The differences for men and women are also insignificant.16 13 This paragraph reports results for courses with a sufficient number of observations for analysis. 14 t=-1.3, p=0.0987 15 t=0.093, p=0.93 16 Men t=0.46, p=0.55; women t=-1.17, p=0.24 Youth Employment Project: Midterm Evaluation Report (UNDP) 16 FIGURE 4: AVERAGE WEEKLY WORKING HOURS BEFORE AND AFTER TRAINING Income The analysis considers only personal income from employment and excludes individuals with no income. Median incomes therefore exclude respondents who are not employed or not participating in the labor force. A comparison of pre- and post-training average monthly income shows a substantial increase for both men and women, with a larger percentage increase for women—97 percent compared to 17 percent for men (Figure 5). This is an understandable result given that UNDP focused much more on those trades that benefit women. The comparison between post-training income of alumni and pre-training income of current trainees (the counterfactual) also suggests that women are earning substantially more than they would have had they not participated in the training. For men, however, the comparison suggests that their earnings are lower than they would have been otherwise. Because the counterfactual is not a rigorous one, the comparison of alumni’s pre- and post-training income is a more reliable estimate of the effects of the training. The median monthly income of employed alumni before and after training increased by more than 100 percent for participants in the Industrial Stitching ($19 to $57), Industrial Machine Operator ($57 to $133), and Merchandising ($76 to $159) training courses. Median income for alumni who had completed the Dress Making training dropped from $38 to $29. Job Satisfaction Satisfaction with a job is an indicator of overall job quality that may incorporate dimensions of salary, work environment, and consistency with interests or lifestyle. A greater percentage of UNDP alumni reported being satisfied with their post-training employment (77 percent of 97 respondents) than with their jobs before training (62 percent of 73 respondents); however, this difference is not statistically significant.17 Women reported a greater change in job satisfaction than men (Figure 6). The somewhat lower percentage of current trainees who reported being satisfied with their pre-training jobs (58 17 t=-0.958, p=0.173 Youth Employment Project: Midterm Evaluation Report (UNDP) 17 percent of 57 respondents) suggests that training has increased job satisfaction relative to what it would have been without training. FIGURE 5: MEDIAN PERSONAL MONTHLY INCOME BEFORE AND AFTER TRAINING FIGURE 6: JOB SATISFACTION OF EMPLOYED ALUMNI When asked to explain why they were satisfied or dissatisfied with their jobs, respondents most frequently cited proximity of the workplace to their homes (23 percent of 79 respondents) and better salary (18 percent of 79 respondents). Outside of these two main reasons, men were more likely than women to mention opportunities for career growth and job security, while women placed more importance on a lack of discrimination and appropriate working hours. This may indicate that men are more concerned with long-term considerations when assessing employment while women look towards the conditions of their work environment. Among the few alumni who were dissatisfied with their jobs, the most common reason cited was a low salary (85 percent of 8 respondents). Youth Employment Project: Midterm Evaluation Report (UNDP) 18 Proximity of home to the workplace also emerged as the most frequently cited reason for pre-training job satisfaction (34 percent of 83 respondents) which underscores the importance of providing trainees with skills relevant to their specific neighborhoods. Three of eight female current trainees also mentioned this consideration during group interviews. Being able to obtain employment near their homes is particularly important for women. “[A good job] should be near home. People don’t consider it good if a girl steps out of the house or goes to far-away places. People indulge in malicious gossip about girls then. If our workplace is close to home, it will be more acceptable socially.” – Female current trainee, Lyari Town When asked to define the characteristics of a good job, 42 percent of 31 group interview participants cited a good work environment, 39 percent mentioned good working hours, and 32 percent noted providing transportation to and from work. Job Placement and Retention Forty-eight percent of 186 alumni survey respondents reported that they received some type support from their training institute after the course was completed. The most common type of support that alumni received was help in searching for jobs (17 percent of 186 respondents) and being placed in a job directly (10 percent of respondents). Only 3 percent were placed in an apprenticeship, and 2 percent received career counseling. During group interviews, just 6 of the 15 alumni participants noted some form of support from the institutes. This included direct job offers and follow-ups. According to an unemployed male alumnus: “The institute contacts us periodically to ask us our employment status, and that’s about it.” Alumni survey respondents reported various levels of support across training institutes. All 22 respondents who trained through Gul Ahmed reported receiving some kind of support after completing training. Other large training partners provided support to less than half of their alumni. These included Memon Industrial and Technical Institute (MITI) (30 percent of 26 respondents), Pakistan Hosiery Manufacturers and Exporters Association Institute of Knitwear Technology Karachi (PHMA IKTK) (43 percent of 37 respondents), and Vocational Training Center (VTC) (G) Kalari Lyari (29 percent of 21 respondents). In interviews, training partners emphasized their formal links with the private sector as a strategy to place graduate trainees into jobs (9 of 18 partners interviewed). Job retention rates were relatively high among employed alumni. Eighty-three percent of 111 alumni who have had a job since they completed training are still in the first job they obtained. Interviews with industry leaders indicated that program graduates’ attitudes and soft skills often impact their job retention rates. Training partners stressed this as well. Seventy-five percent of 12 training partners interviewed noted that trainee behavior separates beneficiaries who obtain a job and those who are unable to utilize the training properly. “The graduates of this project are not properly oriented and prepared to work with the company. They do not have the attitude and mindset that future workers should have. They are not interested in working eight hours a day and they do not work with males and females in the same environment. Females who are coming out of the UNDP project are only interested in working at home.” – Industry partner, Gul Ahmed Industry Relevance of Jobs to Training Thirty-six percent of 97 employed alumni were working in occupations that matched their training course; this rate was equal for both men and women. Both the Quality Checker (95 percent of 18 Youth Employment Project: Midterm Evaluation Report (UNDP) 19 employed respondents) and Industrial Machine Operator (71 percent of 22 employed respondents) trainings had a high rate of job matching. The rate of matched jobs declines sharply for other trainings. Only 2 of 7 Industrial Stitching graduates and 2 of 12 Fashion Designing graduates were in occupations matched to their training. The lowest rates of job matching were from the Dress Making (6 percent of 6 respondents) and Merchandising (0 percent of 7 respondents) trainings. Industry leaders did not express satisfaction with the performance of Youth Employment Project graduates. Only one out of six interviewed felt that the graduates had better on-the-job performance compared to other new recruits. Three mentioned problems with the skill sets of graduates, while three also mentioned problems with the attitudes of Youth Employment Project alumni. As one noted: “The new recruits that are hired through word-of-mouth are more motivated than the ones that come from the VTCs. Their technical capacity is limited, but they are more interested in working than VTC graduates.” – Industry partner Project beneficiaries were aware of the importance of both hard and soft skills in obtaining employment. Forty-two percent of the 31 group interview participants said that interpersonal/communication skills or proper behavior were skills that employers in their industry look for when hiring. Fifty-two percent discussed having certified skills or a proper education as a factor in being hired. However, the variety of skills provided to program beneficiaries does not address what group interview participants felt was the greatest challenge for young people to obtain desirable jobs—corruption in hiring. Thirty-nine percent of the 31 participants said that nepotism and bribes were the greatest challenge. This kind of corruption in the broader Karachi job market is likely a major impediment for trainees who are being recruited from the most marginalized areas of the city. As one alumnus expressed: “Jobs are not given out on merit. Those responsible for hiring are often highly incompetent and only care about personal references. Ultimately, more often than not, the deserving ones are usually left out.” – Male alumnus, North Nazimabad Town Q2(b): What effect is the program having on keeping youth away from violent groups or activities? This evaluation question focuses on testing the link between skill development, employment, and reduction in crime and violent extremism. Four of five experts interviewed believed it was somewhat likely that vocational training and employment would lead to a reduction in crime and violent extremism. Two experts believed the scale of the project was too small to have any impact, while one cautioned that alumni who do not find jobs may become frustrated. UNDP project staff, SPO staff, and representatives of training institutes, industry, and trade associations also agreed with the hypothesis. Ninety-two percent of 25 respondents believed vocational training would keep youth away from crime and violent extremism. Only one respondent somewhat disagreed with the statement that there was a link between vocational training and participation in criminal or extremist activities. The surveys of alumni and current trainees also asked respondents if they thought vocational training— for youth such as themselves—would deter youth from participating in violent extremism or crime, and 87 percent of 403 respondents said that it would. The survey asked respondents to explain why, and the most frequent responses were by changing attitudes, increasing employment, and leading youth to use their time more productively. At the aggregate level, for all towns, 43 percent of respondents indicated that training and employment would reduce crime since youth would have less time available for Youth Employment Project: Midterm Evaluation Report (UNDP) 20 “negative activities”; 38 percent believed that training and employment would lead to less crime; and 16 percent believed training would lead to a change in attitudes and hence a reduction in crime and violent extremism. Figure 7 shows the percentage of respondents who gave each reason disaggregated by town. It is interesting to note that in Korangi and Lyari, a larger proportion of respondents thought that the reduction in crime and violence occurs because of a better use of time, while in other towns—especially Landhi, Malir, and Shah Faisal—a greater proportion of respondents believed that the reduction in crime and violence is due to engaging in better employment. FIGURE 7: HOW TRAINING CONTRIBUTES TO REDUCED CRIME AND VIOLENCE The survey asked respondents whether they contributed to household income, general household affairs, and the wellbeing of their neighborhoods in any way, in order to assess whether these positive outcomes could be associated with the project. Figure 8 presents results disaggregated by sex. Comparisons of pre- to post-training responses found little change for men or women on these three indicators, with the exception that women were more likely to contribute to household income after training than before, a difference that is significant with a p-value of 0.01. Comparisons of post-training responses to current trainees (the counterfactual) suggest larger effects for both sexes on contributions to household income and, for men, on contributions to neighborhood activities. Q3: How, if at all, has the perception of insecurity changed in targeted towns? In the absence of a baseline for this question, change in perception of insecurity cannot be measured over time. Instead, the evaluation discusses the current state of perceptions about insecurity in the targeted towns, and this could serve as a baseline for future evaluations. This analysis could also inform decisions on selecting towns in Karachi for similar future activities. This section presents data from the surveys of alumni and current trainees on various dimensions of security, such as the perceived level of crime and violence, its effect on households, the effects of religious and political groups, and perceptions of employment opportunities and poverty. Youth Employment Project: Midterm Evaluation Report (UNDP) 21 FIGURE 8: SURVEY RESPONDENTS’ CONTRIBUTION TO HOUSEHOLD INCOME, HOUSEHOLD AFFAIRS, AND NEIGHBORHOOD WELLBEING The analysis for question 2(b) found that an overwhelming majority of respondents believed that training reduced the likelihood of young people’s engaging in criminal or violent activities. All five experts interviewed for the evaluation believed crime in Lyari, Korangi, and Sultanabad had decreased over time, and four attributed the change to the ongoing Rangers operation in the city. Experts were unable to predict whether the lull in violence would be permanent. Respondents reported higher levels of crime and violence in Lyari, Malir, Shah Faisal, and Korangi than in other towns (Figure 9) but believed that the impact of crime and violence on their households was greater in Lyari, Korangi, Landhi, and Jamshed (Figure 10). FIGURE 9: PERCEIVED LEVEL OF CRIME AND VIOLENCE Youth Employment Project: Midterm Evaluation Report (UNDP) 22 FIGURE 10: IMPACT OF CRIME AND VIOLENCE ON HOUSEHOLDS The survey also asked about the impact of religious groups and political parties on the respondents or their households. Respondents perceived the effect of religious groups as largely positive in Korangi, Lyari, Landhi, and Shah Faisal (Figure 11). Political parties had little effect on respondents in all towns except Landhi (Figure 12). FIGURE 11: PERCEIVED IMPACT OF RELIGIOUS GROUPS Youth Employment Project: Midterm Evaluation Report (UNDP) 23 FIGURE 12: PERCEIVED IMPACT OF POLITICAL PARTIES Survey respondents from Jamshed were more likely than those from other towns to believe they had few, if any, employment opportunities, but respondents were not optimistic about the employment prospects in any of the towns (Figure 13). Respondents from Lyari and Landhi were more likely than those from other towns to believe poverty levels were high (Figure 14). FIGURE 13: PERCEPTIONS OF INCOME-GENERATING OPPORTUNITIES Youth Employment Project: Midterm Evaluation Report (UNDP) 24 FIGURE 14: PERCEPTIONS OF POVERTY LEVEL Respondents were also asked to rate the effectiveness of government services in their neighborhoods. Government services referred to municipal services such as water, sanitation, education, and health. Respondents from Lyari were the most dissatisfied with government services, while those from Jamshed were most satisfied (Figure 15). FIGURE 15: RESIDENTS’ SATISFACTION WITH GOVERNMENT SERVICES Conclusions: The findings suggest that two of the targeted towns (Korangi and Lyari) have some of the highest overall levels of perceived insecurity among the towns in Karachi. Perceived levels of crime and violence in these towns, and their impacts on households, were among the highest in the city. Respondents from these towns were more likely than others be negatively affected by the actions of political parties and more likely to believe that religious groups had a positive influence. While the differences between towns were not large, residents of the targeted towns perceived fewer Youth Employment Project: Midterm Evaluation Report (UNDP) 25 employment opportunities and higher levels of poverty than respondents from most other towns in Karachi. Q4: How are stakeholders involved in the project? Various stakeholder groups have been involved in planning and/or implementing the UNDP’s Youth Employment Project, and several stakeholder groups have agreed to share the cost of some of the activities. The findings highlighted below are based on 25 in-depth interviews with implementing partner staff (12), social mobilizers (4), and representatives of government18 (3) and industry (6), as well as quantitative data collected from current trainees and alumni. Q4(a): What was the community’s involvement in planning and implementing this project? UNDP and its training partners implemented a robust system for mobilizing youth from the targeted communities. Since UNDP did not engage directly with the communities, it engaged a local organization, SPO, to lead community mobilization activities. SPO engaged local youth networks such as peace clubs and local youth community organizations to help mobilize potential trainees. All of the social mobilizers interviewed mentioned that the collaboration with the community was strong. For example: “Yes, the community played a strong role, as it helped the implementing partner to identify the required youth and provided the space for conducting awareness sessions and setting up an information desk during the mobilization.” – SPO representative Further, SPO conducted meetings with community elders and social workers to identify and recruit youth for the trainings. Some of the training partners also mentioned that they meet with community elders, elected members, or local CBOs to raise awareness and induct trainees. One of the training partners mentioned: “We have links with counselors. We meet them, especially women, so that they can introduce the project to the people of their areas.” – Training partner representative Q4(b): What was the private sector’s involvement in planning and implementing this project? UNDP involved the private sector through various means and for different reasons. Essentially the private sector is involved in training UNDP trainees and providing them job opportunities. For training purposes, UNDP signed a memorandum of understanding (MoU) with eight businesses, such as Gul Ahmed, to use their satellite training units as training facilities. A UNDP representative explained: “We have signed MoUs with different industries which spell out roles and responsibilities [of each party]. The scope is to train new workers and also to upgrade the skills of existing workers. These MoUs are for the industries that have the satellite training units where UNDP supports in-house training by training the trainers, providing manuals, reviewing curricula, and also holding workshops with industry and academics. Last year it organized three such workshops.” – UNDP representative 18 One government representative was from Sindh Technical Education and Vocational Training Authority (STEVTA), one was from the Planning and Development Department (P&D), and one was from Benazir Bhutto Shaheed Youth Development Program (BBSYDP). Youth Employment Project: Midterm Evaluation Report (UNDP) 26 In its work with the satellite training centers, UNDP approaches the associated business and provides a short-term trainer to conduct a three-hour training; the business does not request the assistance from UNDP. UNDP counts each training participant as a trainee, even though the participants are employed at the time of the training, which helps UNDP meet its training targets. Another key role for the private sector is providing employment to graduates of UNDP-supported training programs. UNDP uses at least three strategies to place graduates. First, a job placement officer collects data on graduates’ skills and knowledge and shares these profiles with potential employers. Second, employers communicate directly with UNDP’s training partners for semi-skilled workers in the occupational areas in which the partners provided training. UNDP’s partners also refer their graduates to private sector employers so that they can apply for, and potentially be placed in, jobs. Third, training partners may directly hire graduates from their satellite training units. Private sector employers the team interviewed also suggested improvements to UNDP’s approach to training. These included improving the training curricula because they believed that the content of the existing training courses does not adequately equip youth with the skill sets they require (four of four respondents). Additionally, representatives of two employers believed it was important for training centers to introduce “soft skills” training into their programs so that their graduates develop better work ethics and behavior (two of four respondents). Q4(c): What was the involvement of other key stakeholders in planning and implementing this project? Government institutes, CBOs, and NGOs have been involved in planning and implementing the project to some extent. SPO and local youth community organizations have helped UNDP identify and recruit trainees. Further, UNDP recently engaged the Small and Medium Enterprise Development Authority (SMEDA) to provide entrepreneurship training to graduates of technical training courses who are interested in self-employment. The government’s Planning and Development Department (P&D) supports UNDP by serving as a member of the Youth Employment Project steering committee. On a regular basis, UNDP updates P&D on progress and issues relating to implementation, and P&D helps resolve some of these issues. The government also allows UNDP to sign MoUs with STEVTA and its affiliated vocational training institutions to provide technical training. In return for the government support, UNDP provides the government training institutes with funds for raw materials and also pays a fee per trainee. A representative of STEVTA explained: “We have been working with UNDP since August/September of last year. We have 52 institutes in Karachi; however, UNDP wanted to focus on underprivileged areas. UNDP is providing partner government training institutes with money for raw materials, and a per trainee fee. STEVTA provides the infrastructure and organizational staff … training institutions get 6,000 rupees per trainee under different budget headings such as mobilization, transport, raw material, utility, etc.” – STEVTA representative Q4(d): What, if any, cost-sharing has the project benefitted from? Stakeholders have provided minimal cost sharing. Private sector enterprises have not contributed to any of the costs associated with the training in the vocational training centers. However, some private sector enterprises have contributed to operational and material costs when UNDP’s implementing partners provided on-site training through the satellite training units. Youth Employment Project: Midterm Evaluation Report (UNDP) 27 UNDP alumni have not expressed much interest in paying for training (Table 6). The alumni survey asked respondents if they would be willing to pay for similar training in the future. Overall, less than half of 186 respondents said they would be willing to pay for training. Those who were employed were more likely to be willing to pay compared to those who were unemployed, and those matched to jobs in the areas in which they were trained were more likely than those not matched to be willing to pay for future training. TABLE 6: ALUMNI WHO WOULD PAY FOR SIMILAR TRAINING IN FUTURE Alumni Respondents Count Yes (%) No (%) Not Sure (%) Participating in labor force? Yes 115 36% 54% 10% No 40 35% 55% 10% Total 155 36% 54% 10% Employed? Yes 83 42% 47% 11% No 72 29% 62% 9% Total 155 36% 54% 10% Working in field in which trained? Yes 33 49% 45% 6% No 50 39% 48% 13% Total 83 42% 47% 11% In conclusion, neither alumni nor the private sector are very interested in assuming any financial responsibility related to the vocational training that is currently being provided by UNDP’s implementing partners. The reason the private sector has little interest in contributing financially to the program is because, in their experience, the training does not adequately prepare potential employees for the job market. Q5: Is the project scalable and can local resources be leveraged to transition it out of donor dependence? Q5(a). To what extent are project activities scalable? In individual interviews, the evaluation team asked 26 respondents including experts (5), representatives of UNDP (2), UNDP training partners (10), industry (4), associations (2), and government (3) whether they believed the project should be scaled up to train more youth, and if yes, what support should be provided. All respondents believe that there is potential to scale up the project across the three target communities and shared some suggestions for scaling up. These included:  All 26 respondents stated that the project should be extended geographically. Future efforts should focus on providing vocational training in towns other than Korangi, Lyari, Jamshed, and Malir. Making training opportunities more convenient to more households will likely increase enrollment and future employment.  Nine of 16 representatives of UNDP and industry emphatically stated that the quality of training needs to improve. They suggested that the private sector should be involved more in developing content to ensure its relevance to the needs of the industry. Another way to enhance the quality of the courses is to increase their duration. Presently, industry representatives believe Youth Employment Project: Midterm Evaluation Report (UNDP) 28 that the short-term courses offered by UNDP do not adequately equip youth with the technical and soft skills required by the industry. One of the industry representatives made his point: “For achieving targets, [the] project needs to understand and study the relationship [gap] between supply and demand. For example, the laundry sector has a lot of potential which has yet to be addressed. Dry and wet operations are in demand. But the training institution has no experience in this. There is a need to put capital expense in introducing new courses.” – Industry representative  Two of five experts suggested that in addition to expanding geographical coverage, the government should be engaged more in such projects. They were convinced that the current practice of engaging with the government only through their participation on the steering committee and use of government training centers is insufficient. They suggested that government should provide more support in recruitment, selection, revising curricula, and providing job placement and follow-up services.  UNDP representatives and training partners also stated that scale-up plans should consider expanding the age range of the trainees. Currently, the range is from 18 to 29 years, but several training partner representatives stated that for greater impact the age range for trainees should be increased to 35 years. They suggested that this was an important factor for vocational training and for creating better employment opportunities for women. Explaining the reasons, one of the training partner representatives said: “…girls with the age of 18 or 19 years are not very keen and willing to [be] involved in the industry; parents usually don’t allow their girls to get involved in the industry due to the young age, so we motivated them through counselling ... Age limit is a big challenge; if it is increased, we can complete the target quickly.” – Implementing partner Q5(b). Could the trainings continue without USAID funding? If yes, how? If no, why not? During individual interviews with 2 UNDP representatives, 10 training partners, 5 experts, and 3 government representatives, evaluators asked about the likelihood that the project would be sustainable without external funding. All respondents believed that sustaining a project of this nature without donor funding would be challenging. They believed that, without donor funding, the budget for training would be much smaller but that training may continue in some of the same skill areas, albeit with fewer trainees. They also believed that most efforts for job placements and employment would end. The evaluation team’s observation of UNDP’s training partners found that none of them were following international placement/employment services best practices. Without this capacity, the partners will not be able to continue entrepreneurship training for graduates who are interested in setting up home or community-based enterprises. Also, the partners will not be able to provide job placement services, liaise with the private sector, or provide follow-up services to graduates who are looking for employment opportunities. Likewise, the partners will have to reduce the number of training courses offered because they will not be able to hire as many trainers. The partners will not necessarily stop offering trainings but may only be able to offer courses that they can finance themselves, or that can be supported by industry or the government. Another challenge that the UNDP training partners face is that the youth who receive their training do not have the financial capacity to pay for the courses (see findings for question 4). Consequently, if a Youth Employment Project: Midterm Evaluation Report (UNDP) 29 third party does not cover the costs of each training course, then many of the existing courses will have to be discontinued. The three stakeholder groups made the following suggestions for improving prospects for sustainability.  Eight of the 12 UNDP representatives and independent experts proposed to engage the private sector so that they could financially support the training. However, the degree to which the private sector would be interested in financing these trainings is unclear.  Three of the five experts proposed engaging philanthropic organizations such as foundations to financially support the project. These respondents thought that some foundations may be interested in supporting a project of this nature because of their commitment to social responsibility. The government representatives did not express any interest in providing additional support to the project other than allowing the project to use STEVTA centers.  All of the government representatives believed the project would not continue without donor funding. In conclusion, the sustainability of the project is questionable if it continues to be implemented as it has been during the first eighteen months. If one of the key stakeholder groups, private sector industry, has serious reservations about the quality of the training which is currently being provided to youth, then it is not likely to invest in the project unless its reservations are addressed. CONCLUSIONS Given that it did not use location as a specific criterion, UNDP effectively attracted and selected a majority of its beneficiaries from marginalized towns as defined by USAID priorities and the evaluation team’s marginalization index. The fact that many of UNDP’s training facilities are located in priority and marginalized towns was a key factor in its success because the proximity of training facilities to trainees’ homes was an important determinant of individuals’ decisions about seeking and completing training, particularly for women. UNDP’s direct recruiting strategy relied largely on print media (newspaper ads, pamphlets, and brochures), internal announcements in its facilities, and direct contact with communities through social mobilization and community visits. Its direct strategy fueled a very effective secondary word-of-mouth effect that reached far more people than any of the direct recruiting methods. Without knowing the extent to which each direct recruiting method contributed to word-of-mouth, it is not possible to assess the cost effectiveness of the different methods. Training has increased the employment rate, decreased the unemployment rate, and increased labor force participation relative to pre-training conditions and relative to outcomes if the trainees had not received training. It was also effective in helping trainees obtain better employment—many women shifted from self-employment to more stable wage employment, employed women were working more hours, employed men and women were earning higher incomes, and employed men and women were more satisfied with their jobs. However, the fact that 50 percent of alumni reported being unemployed or out of the labor force suggests either a mismatch between training topics and market demands or that the training is not equipping trainees with the hard and soft skills demanded by potential employers. Evidence from employers suggests that the latter is an issue. Youth Employment Project: Midterm Evaluation Report (UNDP) 30 The evaluation finds some evidence to support the development hypothesis that better employment opportunities contribute to reducing crime and violence. The primary mechanism through which employment produces these positive outcomes is by productive employment taking the place of “negative” activities. However, the evidence for this is largely hearsay and not based on reported behavioral change. Except for garment manufacturers’ involvement in the design of the satellite training units, the evaluation team found little evidence that stakeholders were substantially involved in the project design. Furthermore, stakeholder engagement in implementation was largely passive—i.e., agreeing to work with UNDP but not particularly proactively. UNDP’s community mobilization activities appear to have successfully engaged communities in helping to identify potential trainees. Its private sector-based satellite training centers engaged a few private sector businesses directly in implementation, but otherwise the private sector’s role seems to have been limited to employing graduates—although private sector representatives did not believe graduates were well prepared, and some preferred to hire “off the street.” Government engagement included allowing UNDP to use STEVTA training institutes— albeit with compensation. Financial cost-sharing has been minimal, however, and private sector stakeholders may not see much value in supporting training that they believe does not adequately prepare graduates for the jobs they have to offer. Most trainees (a small majority) are not willing to pay for the training, although alumni who are employed, and particularly those employed in the field in which they were trained, are more likely than others to be willing to pay for training. In this context, it is difficult to see how the project is sustainable without donor support. Greater private sector engagement in project design and implementation may have improved the likelihood that graduates would have acquired the skill sets that the private sector is looking for. The fact that many potential employers believed the graduates had not acquired the necessary skills may explain the lower income for male alumni relative to the counterfactual. The finding that only 36 percent of employed alumni are working in occupations for which they were trained implies that the implementing partners are not focusing sufficiently on the overall target of employment. Furthermore, alumni who are working in the occupational area for which they were trained are more satisfied with their jobs than those working in other occupations. If UNDP’s implementing partners can more effectively match youth to training courses for occupational areas where there is a high probability of their obtaining employment, it is likely that job satisfaction will improve. RECOMMENDATIONS  If USAID/Pakistan or UNDP wish to extend the project to additional towns, UNDP should establish relationships with training institutes and satellite centers in, or near, the targeted areas. Proximity of the training venue to the homes of potential trainees will attract more applicants and is especially important for women. Expanding training capacity, perhaps by working in additional neighborhoods and with more training partners, will be critical to meeting training targets.  UNDP needs to carefully examine the trades or occupations in which it provides training and work with the private sector to identify the skills required in these occupations and trades. It then needs to ensure that its training, in hard and soft skills, is focused on meeting market demands. More active collaboration with the private sector to identify employment opportunities and the required skills, and ensure that the training is effectively teaching these skills, may improve post-training employment outcomes and encourage the private sector to more actively support the training program. Youth Employment Project: Midterm Evaluation Report (UNDP) 31  UNDP needs to ensure that its implementing partners adapt their curricular content to the needs of the private sector. It should also engage the private sector in validating the curricula and ensuring that trainees acquire the soft and technical skills required by private sector employers. Given the importance employees, especially women, place on proximity to the workplace, UNDP should adopt a strategy for matching training at a particular venue to the sectors and skills required in the specific locality.  UNDP’s implementing partners need to build their capacities in placement, vocational counseling, and employment services. Without these key services, placement and employment rates are likely to remain low. To build their capacity, the implementing partners should: o Counsel, orient, and match youth to vocational training courses that are consistent with occupations for which the private sector is recruiting semi-skilled and skilled workers. o Ensure that the curricular content of all vocational training courses complies with the demands of the private sector. o Strengthen the job matching, job placement, and follow-up services so that: (a) a greater number of alumni transition into sustainable employment; and (b) the job retention rate is improved.  UNDP and its implementing partners must strengthen their engagement with the following key stakeholder groups: (a) private sector enterprises in the economic sectors for which they train youth; (b) government entities that are responsible for initiatives that target youth, skills development, job creation, and employment; and (c) entrepreneurship and microfinance service delivery providers. Youth Employment Project: Midterm Evaluation Report (UNDP) 32 ANNEXES Annex 1: Evaluation Scope of Work Youth Employment Project: Midterm Evaluation Report (UNDP) 33 Youth Employment Project: Midterm Evaluation Report (UNDP) 34 Youth Employment Project: Midterm Evaluation Report (UNDP) 35 Youth Employment Project: Midterm Evaluation Report (UNDP) 36 Youth Employment Project: Midterm Evaluation Report (UNDP) 37 Youth Employment Project: Midterm Evaluation Report (UNDP) 38 Annex 2: Assignment Work Plan Youth Employment Project: Midterm Evaluation Report (UNDP) 39 Youth Employment Project: Midterm Evaluation Report (UNDP) 40 Youth Employment Project: Midterm Evaluation Report (UNDP) 41 Youth Employment Project: Midterm Evaluation Report (UNDP) 42 Youth Employment Project: Midterm Evaluation Report (UNDP) 43 Youth Employment Project: Midterm Evaluation Report (UNDP) 44 Youth Employment Project: Midterm Evaluation Report (UNDP) 45 Youth Employment Project: Midterm Evaluation Report (UNDP) 46 Youth Employment Project: Midterm Evaluation Report (UNDP) 47 Youth Employment Project: Midterm Evaluation Report (UNDP) 48 Youth Employment Project: Midterm Evaluation Report (UNDP) 49 Youth Employment Project: Midterm Evaluation Report (UNDP) 50 Youth Employment Project: Midterm Evaluation Report (UNDP) 51 Youth Employment Project: Midterm Evaluation Report (UNDP) 52 Youth Employment Project: Midterm Evaluation Report (UNDP) 53 Youth Employment Project: Midterm Evaluation Report (UNDP) 54 Youth Employment Project: Midterm Evaluation Report (UNDP) 55 Youth Employment Project: Midterm Evaluation Report (UNDP) 56 Youth Employment Project: Midterm Evaluation Report (UNDP) 57 Youth Employment Project: Midterm Evaluation Report (UNDP) 58 Youth Employment Project: Midterm Evaluation Report (UNDP) 59 Youth Employment Project: Midterm Evaluation Report (UNDP) 60 Youth Employment Project: Midterm Evaluation Report (UNDP) 61 Youth Employment Project: Midterm Evaluation Report (UNDP) 62 Youth Employment Project: Midterm Evaluation Report (UNDP) 63 Youth Employment Project: Midterm Evaluation Report (UNDP) 64 Youth Employment Project: Midterm Evaluation Report (UNDP) 65 Youth Employment Project: Midterm Evaluation Report (UNDP) 66 Youth Employment Project: Midterm Evaluation Report (UNDP) 67 Youth Employment Project: Midterm Evaluation Report (UNDP) 68 Youth Employment Project: Midterm Evaluation Report (UNDP) 69 Annex 3: Sample by UNDP Training Partner Sr # Name of Partner Type of Partner Number of Survey Respondents 1 Eastern Garments Pvt Ltd Factory – Satellite Training Unit (STU) - * 2 Kings Apparel Pvt Ltd Factory - STU -** 3 Soorty Enterprises Factory - STU 6 4 Gul Ahmed Apparel Div Factory - STU 22 5 Gul Ahmed Home Textile Factory - STU -* 6 Rajby Industries Factory - STU -*** 7 Artistic Milliner Factory - STU 1 8 Al Karam Textiles Factory - STU -*** 9 Government Vocational Training Centre [Girls] (GVTC), Korangi No. 2.5 Government TVET institute 29 10 Government Polytech Institute [Women] (GPI), Korangi No.3.5 Government TVET institute 27 11 Government Vocational Training Centre [Girls] (GVTC), Korangi No. 5 Government TVET institute 28 12 Government Vocational Institute [Women] (GVI), Nayabad Khadda, Lyari Government TVET institute 25 13 Government Vocational Training Centre [Women] (GVTC), Kalri, Lyari Government TVET institute 52 14 Pak-Korean Garment Technology Institute (PKGTI), Nazimabad Campus Private TVET institute 8**** 15 Memon Industrial & Technical Institute (MITI), Korangi Karachi Private TVET institute 65 16 Pakistan Hosiery Manufacturers & Exporters Association Institute (PHMA) Private TVET institute 43 17 Pakistan Readymade Garments Technical Training Institute (PRGTTI) Private TVET institute 77 18 Habib University Foundation - Institute of Advancing Careers & Talents Private TVET institute 20 19 Government Polytech Institute [Women] (GPI), Landhi Government TVET institute -***** Total 403 * Three alumni in sample frame – none randomly sampled ** Factory unavailable to conduct survey *** Information not provided in sample frame **** PKGTI is no longer operational – respondents were invited to the nearest TVET institute ***** GPI-Landhi has recently partnered with UNDP – no alumni available to sample at the time of the survey Youth Employment Project: Midterm Evaluation Report (UNDP) 70 TABLE 7: PROFILE OF SURVEY RESPONDENTS Town (n) Age (average men) Age (average women) Education (% matric & below – men) & below Education (% matric – women) Received Religious Education (% men) Received Religious Education (% women) Never Married (% men) Never Married (% women Household Size (mean) D Number of ependents Mother Tongue - Urdu Mother Tongue - Sindhi Mother Tongue - Balochi Mother Tongue – Punjabi & Siraiki Poverty Score Median Household Income Years Years % % % % % % # # % % % % Scale Rs Baldia (3) 20.5 41.0 50 100 0 0 100 0 5.75 3.5 50.0 - - 50.0 77.27 31,500 Bin Qasim (7) 27.0 28.5 66.7 25.0 33.3 0 33.3 0 10.5 4.0 45.8 31.3 - 22.9 63.40 27,300 Gadap (1) - 35.0 - 0 - 0 - 0 9 7.0 100 - - - 82.95 35,000 Gulberg (2) 23.5 - - - 0 - 100 - 5.0 2 50.0 - - - 86.21 40,000 Gulshan (8) 25.6 29.2 37.5 60.0 0 0 72.7 60.0 6.9 1.8 56.3 37.5 - - 85.74 30,000 Jamshed (32) 23.5 27.0 93.6 86.3 3.9 10.5 96.2 51.4 5.4 0.4 61.9 - - 25.8 81.99 25,000 Kimari (12) 24.0 19.5 0 96.8 25.0 3.0 75.0 88.1 5.3 0.4 21.6 - - 12.0 68.15 28,000 Korangi (103) 24.5 23.0 42.7 80.4 21.5 14.9 77.9 78.2 7.3 1.3 73.1 5.4 2.6 13.5 81.01 25,000 Landhi (50) 26.3 27.1 33.3 62.4 20.0 12.1 61.8 59.0 7.0 1.6 58.9 8.3 5.2 14.1 73.28 21,800 Liyaqatabad (5) 21.9 25.1 82.4 - 0 - 100 - 8.6 3.1 74.1 - - 25.9 85.15 29,000 Lyari (89) 25.1 22.7 75.3 62.6 11.0 26.5 100 93.7 7.7 0.6 12.3 19.1 24.0 11.8 75.88 20,000 Malir (37) 27.3 23.4 33.4 98.3 12.2 28.0 99.3 72.5 6.5 2.0 87.5 3.4 0.3 8.8 81.55 29,500 New Karachi (6) 20.7 25.0 100 - 0 0 92.9 50.0 8.3 3.9 62.5 37.5 - - 85.99 30,000 North Nazimabad (4) 26.0 21.0 - 100 33.3 0 100 100 6.0 3.0 60.0 40.0 82.32 36,000 Orangi (11) 25.4 22.0 95.4 - 29.0 0 74.1 100 7.3 3.2 62.3 - - 12.6 79.24 26,000 S.I.T.E (1) 26.0 - 0 - 0 - 100 - 4.0 0 - - - - 75.75 30,000 Saddar (5) 24.5 24.0 100 100 0 0 70.0 100 6.3 1.6 54.6 - - 9.1 83.35 32,000 Shah Faisal (26) 23.8 25.3 33.3 66.7 0 2.6 95.8 100 6.9 0.9 91.1 - - 5.6 83.70 32,000 Youth Employment Project: Midterm Evaluation Report (UNDP) 71 Annex 4: Data Collection Instruments Questionnaire for Alumni Youth Employment Project: Midterm Evaluation Report (UNDP) 72 Youth Employment Project: Midterm Evaluation Report (UNDP) 73 Youth Employment Project: Midterm Evaluation Report (UNDP) 74 Youth Employment Project: Midterm Evaluation Report (UNDP) 75 Youth Employment Project: Midterm Evaluation Report (UNDP) 76 Youth Employment Project: Midterm Evaluation Report (UNDP) 77 Youth Employment Project: Midterm Evaluation Report (UNDP) 78 Youth Employment Project: Midterm Evaluation Report (UNDP) 79 Youth Employment Project: Midterm Evaluation Report (UNDP) 80 Youth Employment Project: Midterm Evaluation Report (UNDP) 81 Youth Employment Project: Midterm Evaluation Report (UNDP) 82 Youth Employment Project: Midterm Evaluation Report (UNDP) 83 Youth Employment Project: Midterm Evaluation Report (UNDP) 84 Youth Employment Project: Midterm Evaluation Report (UNDP) 85 Youth Employment Project: Midterm Evaluation Report (UNDP) 86 Youth Employment Project: Midterm Evaluation Report (UNDP) 87 Youth Employment Project: Midterm Evaluation Report (UNDP) 88 Youth Employment Project: Midterm Evaluation Report (UNDP) 89 Youth Employment Project: Midterm Evaluation Report (UNDP) 90 Youth Employment Project: Midterm Evaluation Report (UNDP) 91 Youth Employment Project: Midterm Evaluation Report (UNDP) 92 Youth Employment Project: Midterm Evaluation Report (UNDP) 93 Youth Employment Project: Midterm Evaluation Report (UNDP) 94 Youth Employment Project: Midterm Evaluation Report (UNDP) 95 Youth Employment Project: Midterm Evaluation Report (UNDP) 96 Youth Employment Project: Midterm Evaluation Report (UNDP) 97 Youth Employment Project: Midterm Evaluation Report (UNDP) 98 Youth Employment Project: Midterm Evaluation Report (UNDP) 99 Questionnaire for Current Trainees Youth Employment Project: Midterm Evaluation Report (UNDP) 100 Youth Employment Project: Midterm Evaluation Report (UNDP) 101 Youth Employment Project: Midterm Evaluation Report (UNDP) 102 Youth Employment Project: Midterm Evaluation Report (UNDP) 103 Youth Employment Project: Midterm Evaluation Report (UNDP) 104 Youth Employment Project: Midterm Evaluation Report (UNDP) 105 Youth Employment Project: Midterm Evaluation Report (UNDP) 106 Youth Employment Project: Midterm Evaluation Report (UNDP) 107 Youth Employment Project: Midterm Evaluation Report (UNDP) 108 Youth Employment Project: Midterm Evaluation Report (UNDP) 109 Youth Employment Project: Midterm Evaluation Report (UNDP) 110 Youth Employment Project: Midterm Evaluation Report (UNDP) 111 Youth Employment Project: Midterm Evaluation Report (UNDP) 112 Youth Employment Project: Midterm Evaluation Report (UNDP) 113 Youth Employment Project: Midterm Evaluation Report (UNDP) 114 Youth Employment Project: Midterm Evaluation Report (UNDP) 115 Youth Employment Project: Midterm Evaluation Report (UNDP) 116 Youth Employment Project: Midterm Evaluation Report (UNDP) 117 Youth Employment Project: Midterm Evaluation Report (UNDP) 118 Youth Employment Project: Midterm Evaluation Report (UNDP) 119 Youth Employment Project: Midterm Evaluation Report (UNDP) 120 Group Interview Alumni Basic information sheet (Note: This sheet will be distributed to each participant before the interview and collected after the interview. Provide assistance to the participant if any of the information is missing) Name Course attended Age Current Residential Address District Town Name of Union Council Institution name Year graduated Contact number Email Participant number (1-4) *This table will be copied four times. Once for each participant Group Interview Alumni Instructions for interviewer: Properly introduce the participants-script provided below. Please make sure that you solicit permission for recording the interview from each participant. If a respondent declines to record the interview, then take detailed notes and use the backside of pages or additional pages if required to write your notes (use notebook if required). All the questions require answers unless stated otherwise in skip instructions. Please assign an alphanumeric code such as P1, P2, etc. to each participant. Each participant is expected to respond to each question. If a participant confirms the response from another participant, then input the code “Do” and “the participant number of the person whose response is confirmed by another participant”. For example, if participant number 3 (P3) confirms the response from participant number 1 (P1), then code participant number 3´s response by coding it as follows: Do P1. Youth Employment Project: Midterm Evaluation Report (UNDP) 121 Introduction: Assalam-o-Alaikum (Greetings). Hope you are having a good day and thank you for the meeting. My name is ____________. I am a member of the team that is conducting field work for the midterm evaluation of youth workforce development projects funded by USAID and implemented by UNDP/Amantech in different parts of Karachi. The evaluation aims to determine whether the project under which you received training is on track to meet its training and employment targets, and whether we could bring about any improvements at this stage. We would like to hear your views about the project and your experience with it. The information that you share with us will only be used for this evaluation and will remain confidential. I. Does Participant 1 agree to participate in this interview and give permission to record? Yes / No II. Does Participant 2 agree to participate in this interview and give permission to record? Yes / No III. Does Participant 3 agree to participate in this interview and give permission to record? Yes / No IV. Does Participant 4 agree to participate in this interview and give permission to record? Yes / No Note: If all participants say yes, continue the interview. If no, try to motivate all of the participants by answering their questions and explaining the importance of recording the interview. Background information Q. 1. How was your experience with the training? P1 P2 P3 P4 Q. 2. Why did you choose your specific training course? P1 P2 P3 P4 Q. 3. How, if at all, has the training course that you received affected your life? What changes have occurred as a result of completing that training course that you were not doing previously? P1 P2 P3 P4 Youth Employment Project: Midterm Evaluation Report (UNDP) 122 Q. 4. How did you learn about the training course that you completed? (Circle all that apply for each participant) Recruitment Strategy Participant 1 Participant 2 Participant 3 Participant 4 Radio show 1 1 1 1 Billboard 2 2 2 2 Internal announcements 3 3 3 3 SMS 4 4 4 4 Cable airing production 5 5 5 5 Social media 6 6 6 6 Newspaper ads 7 7 7 7 Promotion at sports events 8 8 8 8 Street theaters 9 9 9 9 Door to door activity 10 10 10 10 Open house 11 11 11 11 Refer a student campaign 12 12 12 12 Family, Friends, or Social Mobilizers 13 13 13 13 TV 14 14 14 14 Other (Specify): 15 15 15 15 Generally, what is the most influential method that would encourage youth like yourself to apply for such trainings? (Circle up to three responses for each participant) Recruitment Strategy Participant 1 Participant 2 Participant 3 Participant 4 Radio show 1 1 1 1 Billboard 2 2 2 2 Internal announcements 3 3 3 3 SMS 4 4 4 4 Cable airing production 5 5 5 5 Social media 6 6 6 6 Newspaper ads 7 7 7 7 Promotion at sports events 8 8 8 8 Youth Employment Project: Midterm Evaluation Report (UNDP) 123 Recruitment Strategy Participant 1 Participant 2 Participant 3 Participant 4 Street theaters 9 9 9 9 Door to door activity 10 10 10 10 Open house 11 11 11 11 Refer a student campaign 12 12 12 12 Family, Friends, or Social Mobilizers 13 13 13 13 TV 14 14 14 14 Other (Specify): 15 15 15 15 Q. 4.1. Why do you think that this recruitment method is more effective than others? P1 P2 P3 P4 Q. 5. How were you selected for this training? What are your views about the selection process? P1 P2 P3 P4 Q. 6. In your opinion, what are the characteristics of a good job? P1 P2 P3 P4 Q. 7. For youth like you, what are the factors that determine how much you earn? P1 P2 P3 P4 Q. 8. If you had not completed your training course, then what do you think you would be doing now? Why? P1 P2 P3 P4 Youth Employment Project: Midterm Evaluation Report (UNDP) 124 Q. 9. In your opinion, what are the skills that employers in your industry are looking for when hiring? P1 P2 P3 P4 Q. 9.1. What are the greatest challenges for young people like you to get the jobs that you want? P1 P2 P3 P4 *** UNEMPLOYED Ask the following questions only if the group is currently unemployed, otherwise skip to Q14. Q. 10. Why are you currently unemployed? Were you ever employed since you completed the training? P1 P2 P3 P4 Q. 10.1. If you were previously employed in a job related to your training, why did you leave that job? P1 P2 P3 P4 Q. 10.2. How is being unemployed affecting you and your household? P1 P2 P3 P4 Youth Employment Project: Midterm Evaluation Report (UNDP) 125 EMPLOYED Ask the following questions only if the group is currently employed, otherwise skip to Q14. Q. 11. Which of the following best describes your current employment status? Daily wage worker, salaried employee, self-employed, or an unpaid family worker? # Paid employment (daily wage) Paid employment (salaried) Self￾employed Unpaid family work Participant 1 4 3 2 1 Participant 2 4 3 2 1 Participant 3 4 3 2 1 Participant 4 4 3 2 1 Q. 11.1. Please describe your current job? P1 P2 P3 P4 Q. 11.2. How did you end up in this occupation? This question is meant to probe how participants end up in different occupations, rather than how they obtained their current job P1 P2 P3 P4 Q. 12. How does your current job compare to any job that you were doing before the training? What changes came about due to the training? P1 P2 P3 P4 Q. 13. What kind of support, if any, are you receiving or have ever received from the [implementing partner responsible for placement]? P1 P2 P3 P4 Youth Employment Project: Midterm Evaluation Report (UNDP) 126 Q. 14. Based on your experience, what can we improve in the recruiting, training, and placement process? P1 P2 P3 P4 Q. 15. In terms of employment, what would better encourage, attract, and retain youth in jobs? P1 P2 P3 P4 Q. 16. To what extent do you agree with the following statement: a project that focuses on vocational training and employment is the correct approach to keep marginalized and at-risk Karachi youth away from negative activities (especially crime and violent extremism)? Do you strongly agree, somewhat agree, somewhat disagree, or strongly agree, or neither agree nor disagree with the statement? # Strongly Somewhat agree Neither Somewhat disagree Strongly disagree Don’t know Participant 1 5 4 3 2 1 99 Participant 2 5 4 3 2 1 99 Participant 3 5 4 3 2 1 99 Participant 4 5 4 3 2 1 99 Q. 1.1. Why do you agree or disagree? Please explain in detail. P1 P2 P3 P4 Q. 17. Anything else that you would like to mention? P1 P2 P3 P4 Thank you for your time and patience. Youth Employment Project: Midterm Evaluation Report (UNDP) 127 Group Interview Current Trainees Basic information sheet (Note: This sheet will be distributed to each participant before the interview and collected after the interview. Provide assistance to the participant if any of the information is missing) Name Course attended Age Current Residential Address District Town Name of Union Council Institution name Year graduated Contact number Email Participant number (1-4) *This table will be copied four times. Once for each participant Group Interview Current Trainees Instructions for interviewer: Properly introduce the participants-script provided below. Please make sure that you solicit permission for recording the interview from each participant. If a respondent declines to record the interview, then take detailed notes and use the backside of pages or additional pages if required to write your notes (use notebook if required). All the questions require answers unless stated otherwise in skip instructions. Please assign an alphanumeric code such as P1, P2, etc. to each participant. Each participant is expected to respond to each question. If a participant confirms the response from another participant, then input the code “Do” and “the participant number of the person whose response is confirmed by another participant”. For example, if participant number 3 (P3) confirms the response from participant number 1 (P1), then code participant number 3´s response by coding it as follows: Do P1. Youth Employment Project: Midterm Evaluation Report (UNDP) 128 Introduction: Assalam-o-Alaikum (Greetings). Hope you are having a good day and thank you for the meeting. My name is ____________. I am a member of the team that is conducting field work for the midterm evaluation of a couple of youth workforce development projects funded by USAID and implemented by UNDP and Amantech in different parts of Karachi. This evaluation aims to determine whether we could bring about any improvements in training or job placements for youth like you. We would like to hear your views about the project. The information that you share with us will only be used for this evaluation and will remain confidential. I. Does Participant 1 agree to participate in this interview and give permission to record? Yes / No II. Does Participant 2 agree to participate in this interview and give permission to record? Yes / No III. Does Participant 3 agree to participate in this interview and give permission to record? Yes / No IV. Does Participant 4 agree to participate in this interview and give permission to record? Yes / No Note: If all participants say yes, continue the interview. If no, try to motivate all of the participants by answering their questions and explaining the importance of recording the interview. Background information Q. 1. How is your experience with the training so far? P1 P2 P3 P4 Q. 2. Why did you choose your specific training course? P1 P2 P3 P4 Q. 3. How, if at all, do you believe this training course will affect your life in the future? P1 P2 P3 P4 Youth Employment Project: Midterm Evaluation Report (UNDP) 129 Q. 4. How did you learn about the training course? (Circle all that apply for each participant) Recruitment Strategy Participant 1 Participant 2 Participant 3 Participant 4 Radio show 1 1 1 1 Billboard 2 2 2 2 Internal announcements 3 3 3 3 SMS 4 4 4 4 Cable airing production 5 5 5 5 Social media 6 6 6 6 Newspaper ads 7 7 7 7 Promotion at sports events 8 8 8 8 Street theaters 9 9 9 9 Door to door activity 10 10 10 10 Open house 11 11 11 11 Refer a student campaign 12 12 12 12 Family, Friends, or Social Mobilizers 13 13 13 13 TV 14 14 14 14 Other (Specify): 15 15 15 15 Q. 5. Generally, what is the most influential method that would encourage youth like yourself to apply for such trainings? (Circle up to three responses for each participant) Recruitment Strategy Participant 1 Participant 2 Participant 3 Participant 4 Radio show 1 1 1 1 Billboard 2 2 2 2 Internal announcements 3 3 3 3 SMS 4 4 4 4 Cable airing production 5 5 5 5 Social media 6 6 6 6 Newspaper ads 7 7 7 7 Promotion at sports events 8 8 8 8 Youth Employment Project: Midterm Evaluation Report (UNDP) 130 Recruitment Strategy Participant 1 Participant 2 Participant 3 Participant 4 Street theaters 9 9 9 9 Door to door activity 10 10 10 10 Open house 11 11 11 11 Refer a student campaign 12 12 12 12 Family, Friends, or Social Mobilizers 13 13 13 13 TV 14 14 14 14 Other (Specify): 15 15 15 15 Q. 5.1. Why do you think that this method is more effective than others? P1 P2 P3 P4 Q. 6. How were you selected for this training? What are your views about the selection process? P1 P2 P3 P4 Q. 7. In your opinion, what are the characteristics of a good job? P1 P2 P3 P4 Q. 8. For youth like you, what are the factors that determine how much you earn? P1 P2 P3 P4 Q. 9. In your opinion, what are the skills that employers in your industry are looking for when hiring? P1 P2 P3 P4 Youth Employment Project: Midterm Evaluation Report (UNDP) 131 Q. 9.1. What are the greatest challenges for young people like you to get the jobs that you want? P1 P2 P3 P4 *** UNEMPLOYED Ask the following questions only if the group was unemployed before starting the training, otherwise skip to Q12. Q. 10. Why were you unemployed before the training? Were you ever employed? P1 P2 P3 P4 Q. 10.1. If you were previously ever employed in a job, why did you leave that job? P1 P2 P3 P4 Q. 10.2. How did being unemployed affect you and your household? P1 P2 P3 P4 EMPLOYED Ask the following questions only if the group was employed before starting the training, otherwise skip to Q12. Q. 11. Which of the following best describes your employment status before training? Daily wage worker, salaried employee, self-employed, or an unpaid family worker? # Paid employment (daily wage) Paid employment (salaried) Self￾employed Unpaid family work Participant 1 4 3 2 1 Participant 2 4 3 2 1 Participant 3 4 3 2 1 Participant 4 4 3 2 1 Youth Employment Project: Midterm Evaluation Report (UNDP) 132 Q. 11.1. Could you please describe your current job? P1 P2 P3 P4 Q. 11.2. How did you end up in that occupation? This question is meant to probe how participants ended up in their occupations, rather than how they obtained their job. P1 P2 P3 P4 Q. 12. What kind of support, if any, will you receive from your training institute in getting a job after you complete your training? P1 P2 P3 P4 Q. 13. Based on your experience, what could be improved in the recruiting, training, and placement process? P1 P2 P3 P4 Q. 14. In terms of employment, what would better encourage, attract, and retain youth in jobs? P1 P2 P3 P4 Q. 15. To what extent do you agree with the following statement: a project that focuses on vocational training and employment is the correct approach to keep marginalized and at-risk Karachi youth away from negative activities (especially crime and violent extremism)? Do you strongly agree, somewhat agree, somewhat disagree, or strongly agree, or neither agree nor disagree with the statement? # Strongly Somewhat agree Neither Somewhat disagree Strongly disagree Don’t know Participant 1 5 4 3 2 1 99 Participant 2 5 4 3 2 1 99 Participant 3 5 4 3 2 1 99 Participant 4 5 4 3 2 1 99 Youth Employment Project: Midterm Evaluation Report (UNDP) 133 Q. 15.1. Why do you agree or disagree? Please explain in detail. P1 P2 P3 P4 Q. 16. Anything else that you would like to mention? P1 P2 P3 P4 Thank you for your time and patience. Youth Employment Project: Midterm Evaluation Report (UNDP) 134 Key Informant’s Interview Experts Instructions for Interviewer: Properly introduce yourself to the respondent – script provided below. The interviewer should have sufficient knowledge about the project before conducting this interview. Please make sure that you solicit permission for recording the interview. If the respondent declines, take detailed notes and use the back side of pages or additional pages if required to write your notes. All questions require answers unless stated otherwise in skip instructions. Please provide the respondent with a brief description of the projects prior to starting. Introduction: Assalam-o-Alaikum (Greetings). Hope you’re having a good day. Thank you for the meeting. My name is ____________. I am a member of the team that is conducting field work for the midterm evaluation of youth workforce development projects funded by USAID and implemented by UNDP and Amantech in different parts of Karachi. The evaluation aims to determine whether the project is on track to meet its training and employment targets, and whether we could bring about any improvements at this stage. We would like to hear your views about the project and general political economy conditions in the city. The information that you share with us will only be used for this evaluation and will remain confidential. You will not be quoted on any information you provide, without your prior written permission. A. Do I have your consent to proceed with this interview? Yes/No ______ If no, stop interview, but complete the respondent information table below. B. Do I have your permission to record this interview for our notes? Yes/No _______ Explain to the respondent that it will be very difficult for you to conduct the interview and take detailed notes simultaneously, if you don’t record the interview. C. Respondent information 1. Full name: 2. Designation: 3. Organization: 4. Department: 5. Address: 6. Contact:  At least one valid contact number (mobile or landline) is mandatory a. Phone: b. E-mail: Briefly introduce the two projects, their purpose, main activities, training areas, and job placement efforts. Youth Employment Project: Midterm Evaluation Report (UNDP) 135 Q 1. How would you characterize marginalized or at-risk youth in Karachi? In your opinion, what are the principle reasons why youth become marginalized or at-risk? Q 2. Based upon the reasons that you have identified, which towns and neighborhoods would you consider to have the highest rates of marginalized or at-risk youth? Why? Q 2.1. Could you share reasons why youth could be more marginalized in Lyari, Korangi, and Sultanabad compared to other areas in Karachi? Q 2.2. Has the security, crime, economic, and political situation in these towns changed lately? Has the situation deteriorated or improved? Q 3. In your opinion, what are the best strategies to identify the most at-risk and marginalized youth in Karachi? Q 4. In your opinion, what type of role do marginalized or at risk youth usually play in their local communities in Karachi? Q 5. The Youth Workforce Development Projects support marginalized/at-risk youth by providing them with vocational training and supporting them in finding a job. To what extent do you agree with the following statement: a project that focuses on vocational training and employment is a correct approach to help marginalized and at-risk youth in Karachi to become more positive actors in their local communities? Do you strongly agree, somewhat agree, somewhat disagree, strongly disagree, or neither agree nor disagree with the statement? Youth Employment Project: Midterm Evaluation Report (UNDP) 136 Strongly agree Somewhat agree Neither agree nor disagree Somewhat disagree Strongly disagree Don’t know 5 4 3 2 1 99 Q 5.1. Please share reasons for your agreement or disagreement? Are you aware of any evidence supporting your views? Q 5.2. To what extent do you agree with the following statement: a project that focuses on vocational training and employment contributes to keep marginalized and at-risk Karachi youth away from negative activities (particularly crime and violent extremism)? Do you strongly agree, somewhat agree, somewhat disagree, or strongly agree, or neither agree nor disagree with the statement? Strongly agree Somewhat agree Neither agree nor disagree Somewhat disagree Strongly disagree Don’t know 5 4 3 2 1 99 Q 5.3. Please share reasons for your agreement or disagreement? Are you aware of any evidence supporting your views? Q 6. Considering the social, political, and economic situation in Karachi, in your opinion, how best could we engage the youth to ensure that they don’t get involved in negative activities, especially crime and violent extremism? Q 7. What other type of support, if any, do you believe that marginalized/at-risk youth should receive in order to facilitate their potential to become positive actors in their local communities? Youth Employment Project: Midterm Evaluation Report (UNDP) 137 Q 8. How could we ensure that marginalized or at-risk youth of Karachi remains gainfully employed? Q 9. What role, if any, do you believe that local community members, government, and the private sector should play in the support of marginalized/at-risk youth in Karachi? Q 10. What recommendations do you have on how best to ¨scale up¨ these projects in Karachi or other parts of the country so that other youth benefit from such initiatives? Which locations in Karachi would most benefit from such training and employment projects? Q 11. Under what conditions could the trainings continue without USAID funding? Who else could fund them? Q 12. In your opinion, how sustainable will the impact of employment training for marginalized/at-risk youth be in Karachi? What components can help in ensuring sustainability? Q 13. What sectors are most in need of skilled workers? Q 14. What technical skills are these sectors most in need of? Youth Employment Project: Midterm Evaluation Report (UNDP) 138 Q 15. What are the best strategies to identify youth to train in these skills that they can find employment? Q 16. Anything else that you would like to mention? Anything important that we have not yet discussed? Thank you for your time and patience. Youth Employment Project: Midterm Evaluation Report (UNDP) 139 Key Informant Interview Government Officials Covers STEVTA, and any other government department that is already aware of and working with the projects, such as the Sindh P&DD. Instructions for Interviewer: Properly introduce yourself to the respondent – script provided below. The interviewer should have sufficient knowledge about the project before conducting this interview. Please make sure that you solicit permission for recording the interview. If the respondent declines, take detailed notes and use the back side of pages or additional pages if required to write your notes. All questions require answers unless stated otherwise in skip instructions. Please provide the respondent with a brief description of the projects prior to starting. Introduction: Assalam-o-Alaikum (Greetings) Hope you’re having a good day. Thank you for the meeting. My name is ____________. I am a member of the team that is conducting field work for the midterm evaluation of youth workforce development projects funded by USAID and implemented by UNDP and Amantech in different parts of Karachi. The evaluation aims to determine whether the projects are on track to meet training and employment targets, and whether we could bring about any improvements at this midterm stage. We would like to hear your views about the project(s) and its activities. The information that you share with us will only be used for this evaluation and will remain confidential. You will not be quoted on any information that you provide, without your prior written permission. A. Do I have your consent to proceed with this interview? Yes/No ______ If no, stop interview, but complete the respondent information table below. B. Do I have your permission to record this interview for our notes? Yes/No _______ Explain to the respondent that it will be very difficult for you to conduct the interview and take detailed notes simultaneously, if you don’t record the interview. C. Respondent information 1. Full name: 2. Designation: 3. Organization: 4. Department: 5. Address: 6. Contact:  At least one valid contact number (mobile or landline) is mandatory a. Phone: b. E-mail: Briefly introduce the two projects, their purpose, main activities, training areas, and job placement efforts. Youth Employment Project: Midterm Evaluation Report (UNDP) 140 Q. 1. How do you see the current economic and political conditions in Karachi? How are these conditions affection the youth? Q. 2. Do you believe that certain localities in Karachi are more affected in terms of negative impact on youth? Which areas are more affected and how? Q. 3. What steps is the government taking in Karachi to productively engage and train youth, create employment opportunities, and save youth from getting into crime and violence? Are any major policies or strategies being implemented or planned for the near future? Q. 4. What sectors are most in need of skilled workers in Karachi? Q. 5. What technical skills are these sectors most in need of? Q. 6. What are the best strategies to identify youth to train in these skills that they can find employment? Q. 7. What are the key challenges and opportunities on the demand side of the labor force? Youth Employment Project: Midterm Evaluation Report (UNDP) 141 Q. 8. What are the key challenges and opportunities on the supply side of the labor force? Q. 9. What is the nature of your department / organization’s involvement in these projects that UNDP / Amantech is implementing? Q. 10. What is the best way to engage the government in such projects? Q. 11. What is the best way to engage the private sector in such projects? Q. 12. What is the best way to engage the community in such projects? Q. 13. How likely would it be for the training to continue without USAID funding? Under what conditions could the trainings continue without this funding? Who else could fund them? Could the government share costs in anyway? Q. 14. In your opinion, what is the best approach for scaling such projects? Could the government scale such projects on its own, if yes how? If no, why not? What conditions would need to be in place for such projects to scale? Youth Employment Project: Midterm Evaluation Report (UNDP) 142 Q. 15. What sort of impact do you think such projects have on the trainees and their households? Do such projects contribute to changes at the neighborhood level? How? Q. 16. What are the most important lessons that you have learned from such projects? Lessons that we could apply to these projects. Please share your thoughts on each of the following aspects: Q. 16.1. Identification of trainees, recruitment, and selection Q. 16.2. Training Q. 16.3. Placement in jobs, and follow-up Q. 17. To what extent do you agree with the following statement: a project that focuses on vocational training and employment is the correct approach to keep marginalized and at-risk Karachi youth away from negative activities (especially crime and violence)? Do you strongly agree, somewhat agree, somewhat disagree, or strongly agree, or neither agree nor disagree with the statement? Strongly agree Somewhat agree Neither agree nor disagree Somewhat disagree Strongly disagree Don’t know 5 4 3 2 1 99 Q. 17.1. Why do you agree or disagree? Q. 18. Anything else that you would like to mention? Anything important that we have not discussed? Thank you for your time and patience Youth Employment Project: Midterm Evaluation Report (UNDP) 143 Key Informant Interview Industry Includes private sector owners or managers, representatives of trade associations. Some of the private sector owners or managers should also be from amongst the training partners. Instructions for Interviewer: Properly introduce yourself to the respondent – script provided below. The interviewer should have sufficient knowledge about the project before conducting this interview. Please make sure that you solicit permission for recording the interview. If the respondent declines, take detailed notes and use the back side of pages or additional pages if required to write your notes. All questions require answers unless stated otherwise in skip instructions. Please provide the respondent with a brief description of the projects prior to starting. Introduction: Assalam-o-Alaikum (Greetings). Hope you’re having a good day. Thank you for the meeting. My name is ____________. I am a member of the team that is conducting field work for the midterm evaluation of youth workforce development projects funded by USAID and implemented by UNDP and Amantech in different parts of Karachi. The evaluation aims to determine whether the project is on track to meet its training and employment targets, and whether we could bring about any improvements at this stage. We would like to hear your views about the project and general requirements and conditions of employment in your industry. The information that you share with us will only be used for this evaluation and will remain confidential. You will not be quoted on any information you provide, without your prior written permission. A. Do I have your consent to proceed with this interview? Yes/No ______ If no, stop interview, but complete the respondent information table below. B. Do I have your permission to record this interview for our notes? Yes/No _______ Explain to the respondent that it will be very difficult for you to conduct the interview and take detailed notes simultaneously, if you don’t record the interview. Briefly introduce the two projects, their purpose, main activities, training areas, and job placement efforts. C. Respondent information 1. Full name: 2. Designation: 3. Organization: 4. Department: 5. Address: 6. Contact:  At least one valid contact number (mobile or landline) is mandatory a. Phone: b. E-mail: Youth Employment Project: Midterm Evaluation Report (UNDP) 144 Q. 1. Tell us about your business. What are your main products or services? Q. 2. What sorts of technical skills does your business require the most in producing or delivering your products or services? Q. 3. What kind of skilled workforce does your sector require the most? Q. 4. What types of contracts, if any, do you use when hiring these new employees? Why do you use them? Q. 5. How do firms like yours in this sector, recruit the type of skilled workers that are needed the most? (Circle all that apply) i. Job expo ii. Newspaper advertisement iii. Web advertisement iv. HR recruitment firms v. Head hunting vi. Through relevant universities or training institutes vii. All of the above viii. Other (specify)________________________ Q. 5.1. Which methods are most effective in finding and recruiting skilled workers, and why? Q. 6. What kinds of institutions or organizations do you usually contact to recruit new employees with the skill Youth Employment Project: Midterm Evaluation Report (UNDP) 145 sets that you are looking for? Why? Q. 7. Would you rely more on internal trainings for your workforce in these skills, or on the trainings and courses offered by public and private sector training institutes? Why? If the firm is working with the Youth Workforce Development Projects that are being implemented by AMANTECH/UNDP or any of its training institutes continue interview. Otherwise, skip to Q18. Q. 8. Could you please describe what type of collaboration, if any, your firm has had with the AMANTECH or UNDP program in the following areas? If you have not had any collaboration please say so. Q. 8.1. Recruitment Q. 8.2. Training Q. 8.3. Job Matching Q. 8.4. Job Placement Q. 8.5. Others (Specify ________________) Q. 9. How is your firm’s collaboration with AmanTech/UNDP working so far? What are the main results? Youth Employment Project: Midterm Evaluation Report (UNDP) 146 Q. 10. How could the collaboration between your firm and AmanTech/UNDP be improved? What is missing? What is needed to improve the entire supply chain from recruiting to working? Q. 11. Are there any other training institutes besides Amantech and UNDP that you regularly work with? Yes / No: ___________ Q. 12. Why do you work with them? How do they compare? Q. 13. What recommendations do you have on how best to scale up the youth workforce development projects in other parts of Karachi, or elsewhere in Pakistan? What will be the key factors or requirements? Q. 14. What sort of problems, if any, have you had with the retention of YWFD graduates you have recruited? Q. 15. How would you evaluate the YWFD project’s graduates you have hired in the following areas? Are you highly satisfied, somewhat satisfied, unsatisfied, highly unsatisfied, or do you have no opinion? HS SS NO SU HU Technical capacity 1 2 3 4 5 Practical skills 1 2 3 4 5 Problem solving skills Repair/ maintenance/ 1 2 3 4 5 Behavior 1 2 3 4 5 Communication skills/ report writing 1 2 3 4 5 Safety consciousness 1 2 3 4 5 Quality assurance 1 2 3 4 5 HS=Highly satisfied, S=Somewhat satisfied, NO=No opinion, SU=Somewhat unsatisfied, HU=Highly unsatisfied Youth Employment Project: Midterm Evaluation Report (UNDP) 147 Q. 15.1. Do you know of any examples of really good or really bad vocational training graduates from these projects that have worked for you? Q. 16. How does the on-the-job performance of graduates from these partner training centers compare to other new recruits whom you have hired? In what ways are they similar or different to other new recruits that you have recently hired? ____________________________________________________________________________________ ____________________________________________________________________________________ Q. 17. In your opinion, how will the private sector, local communities and youth benefit from the type of projects that AMANTECH/UNDP is implementing which focuses on preparing marginalized/at-risk youth for gainful employment? Q. 18. What sectors are most in need of skilled workers? Q. 19. What technical skills are these sectors most in need of? Q. 20. What are the best strategies to identify youth to train in these skills so that they may get good jobs? Q. 21. To what extent do you agree with the following statement: a project that focuses on vocational training and employment is the correct approach to keep marginalized and at-risk Karachi youth away from negative activities (especially crime and violent extremism)? Do you strongly agree, somewhat agree, somewhat disagree, or strongly agree, or neither agree nor disagree with the statement? Strongly agree Somewhat agree Neither agree nor disagree Somewhat disagree Strongly disagree Don’t know 5 4 3 2 1 99 Youth Employment Project: Midterm Evaluation Report (UNDP) 148 Q. 21.1. What are your reasons for agreement or disagreement? Q. 22. Anything else that you would like to mention? Anything important that we may have missed? Thank you for your time and patience. Youth Employment Project: Midterm Evaluation Report (UNDP) 149 Key Informant Interview Implementing Partners Implementing partners include UNDP, AMANTECH, and other training partners. Instructions for Interviewer: Properly introduce yourself to the respondent – script provided below. Please make sure that you solicit permission for recording the interview. If the respondent declines, take detailed notes and use the back side of pages or additional pages if required to write your notes. All questions require answers unless stated otherwise in skip instructions. Introduction: Assalam-o-Alaikum (Greetings). Hope you’re having a good day. Thank you for the meeting. My name is ____________. I am a member of the team that is conducting field work for the midterm evaluation of youth workforce development projects funded by USAID and implemented by your organization in different parts of Karachi. The evaluation aims to determine whether the project you are implementing is on track to meet its training and employment targets, and whether we could bring about any improvements at this midterm stage. We would like to hear your views about the project(s) and its activities. The information that you share with us will only be used for this evaluation and will remain confidential. A. Do I have your consent to proceed with this interview? Yes/No ______ If no, stop interview, but complete the respondent information table below. B. Do I have your permission to record this interview for our notes? Yes/No _______ Explain to the respondent that it will be very difficult for you to conduct the interview and take detailed notes simultaneously, if you don’t record the interview. C. Respondent information 1. Full name: 2. Designation: 3. Organization: 4. Department: 5. Address: 6. Contact:  At least one valid contact number (mobile or landline) is mandatory a. Phone: b. E-mail: Youth Employment Project: Midterm Evaluation Report (UNDP) 150 Q. 1. Please provide us with a brief overview of the youth workforce development project that your organization is implementing. (Project background, your role, objectives, aims, targets) Q. 2. What is the profile and location of your target group for this program? How do you define your target group? Q. 3. What challenges, if any, have you faced in achieving the targets so far? Q. 3.1. What strategies, if any, have you developed and applied to achieve the targets? Q. 4. In your opinion, how successful have you been in recruiting the target group for these projects? Would you say that you have been successful, partially successful, or unsuccessful? Successful Partially successful Unsuccessful Don’t know 3 2 1 99 Q. 4.1. Why do you think the recruiting strategy was successful or unsuccessful? Youth Employment Project: Midterm Evaluation Report (UNDP) 151 Q. 5. Please describe the overall strategy that your organization has used to inform potential candidates about the training, and encourage them to apply (recruitment strategy). (This question is about the overall recruitment strategy. Specific recruitment activities or methods are covered in the following questions.) Q. 5.1. Why did you choose this strategy? Q. 5.2. In what ways, if any, has your recruitment strategy evolved over time? If so, why has it changed? Q. 5.3. What recruitment activities or methods have you implemented under this strategy? Please state in order of priority from most used to least. Q. 5.4. Have you matched recruitment activities to different communities? If yes, how? If not, why not? Please give specific examples by community. Q. 5.5. Based upon your lessons learned, which recruitment activities have been the most cost-effective at recruiting marginalized or at-risk youth? Why do you think they have been effective? Q. 5.6. What kinds of activities have not been cost-effective at recruiting marginalized or at￾risk youth? Why do you think they are not effective? Youth Employment Project: Midterm Evaluation Report (UNDP) 152 Q. 6. After recruiting, selecting and matching youth to a vocational training program, about what percentage of the participants have dropped out of the program? (Don’t require a precise number) ___________% Q. 6.1. In your opinion, what have been the principle reasons why participants drop out of the program? Q. 6.2. What mitigation efforts, if any, have you introduced to reduce trainee drop outs? To what extent have these mitigation efforts been effective? Q. 7. How do you place your graduate trainees into jobs? What is the process? Q. 8. In what ways, if any, do you follow up on your graduates, whether employed or unemployed? For how long do you follow up with alumni? Q. 9. With respect to this project, what is the nature of collaboration, if any, that exists between your institution and the private sector? Q. 10. With respect to this project, what is the nature of collaboration, if any, that exists between your institution and the communities with which you are working? What are the pros and cons of collaborating with communities on such projects? Youth Employment Project: Midterm Evaluation Report (UNDP) 153 Q. 10.1. In what ways, if any, have you engaged community members such as parents, social workers, community members, during the program review, planning and implementation? Q. 10.2. How can the involvement of community members be improved in the future? Q. 11. With respect to this project, what is the nature of collaboration, if any, with relevant government departments or officials? What are the pros and cons of collaborating with the government on such projects? Q. 12. Are there any other stakeholder groups that you have engaged in the implementation of this project? If yes, which stakeholders and what has been the nature of the collaboration? What are the pros and cons of collaborating with these other stakeholders? Q. 13. What kind of cost-sharing arrangements, if any, have you made with the private sector thus far in the project? Please share some examples. Q. 14. Imagine that USAID funding for the project stopped. How likely would it be for the training to continue without USAID funding? Would it be very likely, somewhat likely, neither likely nor unlikely, somewhat unlikely, or highly unlikely? Very likely Somewhat likely Neither likely nor unlikely Somewhat unlikely Highly unlikely Don’t know 5 4 3 2 1 99 Youth Employment Project: Midterm Evaluation Report (UNDP) 154 Q. 14.1. Please share some reasons to elaborate on your answer above. Q. 15. Under what conditions could the trainings continue without this funding? Who else could fund them? Q. 16. What recommendations do you have on how best to scale-up the project in Karachi or other parts of the country? Which recruiting and training strategies should be utilized in a larger-scale program? What is your strongest component? Q. 17. What do you consider to be the two or three most important lessons that you have learned from this project so far? Q. 18. Based on these lessons learned, what recommendations would you propose to further improve the outcomes of this project? Q. 19. What sectors are most in need of skilled workers? Q. 20. What technical skills are these sectors most in need of? Youth Employment Project: Midterm Evaluation Report (UNDP) 155 Q. 21. What are the best strategies to identify youth to train in these skills that they can find employment? Q. 22. Are there any characteristics that separate a candidate who will likely complete training and get a job vs those who will not utilize the training properly? Q. 23. To what extent do you agree with the following statement: a project that focuses on vocational training and employment is the correct approach to keep marginalized and at-risk Karachi youth away from negative activities? Do you strongly agree, somewhat agree, somewhat disagree, or strongly agree, or neither agree nor disagree with the statement? Strongly agree Somewhat agree Neither agree nor disagree Somewhat disagree Strongly disagree Don’t know 5 4 3 2 1 99 Q. 23.1. Why do you think that? Q. 24. Anything else that you would like to mention? Anything important that we have not discussed? Thank you again for your time and patience Youth Employment Project: Midterm Evaluation Report (UNDP) 156 Key Informant Interview Social Mobilizers Instructions for Interviewer: Properly introduce yourself to the respondent – script provided below. Please make sure that you solicit permission for recording the interview. If the respondent declines, take detailed notes and use the back side of pages or additional pages if required to write your notes. All questions require answers unless stated otherwise in skip instructions. Introduction: Assalam-o-Alaikum (Greetings). Hope you’re having a good day. Thank you for the meeting. My name is ____________. I am a member of the team that is conducting field work for the midterm evaluation of youth workforce development projects funded by USAID and implemented by UNDP and Amantech in different parts of Karachi. The evaluation aims to determine whether the projects are on track, and whether we could bring about any improvements. We would like to hear your views about the project that you are involved in. The information that you share with us will only be used for this evaluation and will remain confidential. You will not be quoted on any information. A. Do I have your consent to proceed with this interview? Yes/No ______ If no, stop interview, but complete the respondent information table below. B. Do I have your permission to record this interview for our notes? Yes/No _______ Explain to the respondent that it will be very difficult for you to conduct the interview and take detailed notes simultaneously, if you don’t record the interview. C. Respondent information 1. Full name: 2. Designation: 3. Organization: 4. Department: 5. Address: 6. Contact:  At least one valid contact number (mobile or landline) is mandatory a. Phone: b. E-mail: Youth Employment Project: Midterm Evaluation Report (UNDP) 157 Q 1. Please share a description of the local community where you have conducted social mobilization activities for this project, emphasizing on the following aspects: Type of description Description 1.1 Socio-economic 1.2 Security and political 1.3 Ethnic and cultural 1.4 Other (Specify): ____________________ Q 2. Could you please describe the planning, if any, that is usually involved in selecting geographic areas and for your mobilization activities for this project? Q 3. Please provide a description of the activities that you implement at the community level for this project? Additionally, which population are each of these activities targeting? Description of activities Targeted population Q 4. To what extent have each of the following groups played a role in your mobilization efforts i.e. identifying and encouraging youth to apply for the trainings? Have they had a strong role, a weak role, or no role at all in mobilization efforts? Stakeholder Strong role Weak role No role Don’t know Community 3 2 1 99 Current trainees 3 2 1 99 Graduates 3 2 1 99 Q 4.1. Has their participation been helpful or not? Ask if at least one group played a strong or weak role in Q4 Youth Employment Project: Midterm Evaluation Report (UNDP) 158 Q 4.2. How could these groups further support your mobilization activities? Ask if at least one group played a strong or weak role in Q4 Q 5. How do you identify the communities and the marginalized or at-risk youth that you need to mobilize for receiving training under this project? Q 6. What criteria do you apply to select the neighborhoods for selecting youth? Q 7. What criteria do you apply for selecting youth in these neighborhoods? Q 8. What are the principle challenges you have encountered in your mobilization activities at the community and at the individual levels? 8.1 At the community level 8.2 At the individual level Q 9. Since the initiation of this project, what kind of changes, if any, have you noticed in the neighborhoods where you are working? Why have these changes occurred? Q 10. In what ways, if any, do the project beneficiaries (trainees) contribute to the situation in their households? Youth Employment Project: Midterm Evaluation Report (UNDP) 159 Q 11. In what ways, if any, do the project beneficiaries (trainees) contribute to the situation in their neighborhoods? Q 12. Have you observed any significant changes in the behaviors or life style of the youth who have received trainings, as compared to those that do not? Yes No (Skip to Q13) Don’t know 2 1 99 Q 12.1. Could you please describe the changes you have observed? Q 13. In your opinion, how successful have the recruitment activities been in engaging marginalized or at-risk youth? Would you say they have been successful, somewhat successful, somewhat unsuccessful, or unsuccessful? Successful Somewhat successful Somewhat unsuccessful Unsuccessful Don’t know 4 3 2 1 99 Q 13.1. Why do you think they were successful or unsuccessful? Q 14. What suggestions or recommendations do you have for improving the project’s mobilization activities? Q 15. In your opinion, how effective is this youth workforce development project been at engaging participants in employment activities? Has it been very effective, somewhat effective, somewhat ineffective, very ineffective, or neither effective nor ineffective? Very effective Somewhat effective Neither effective nor ineffective Somewhat ineffective Very ineffective Don’t know 5 4 3 2 1 99 Youth Employment Project: Midterm Evaluation Report (UNDP) 160 Q 15.1. Why do you think it was effective or ineffective? Q 16. In your opinion, how effective has this project been at contributing to the economic wellbeing of participant’s household and neighborhood? Has it been very effective, somewhat effective, somewhat ineffective, very ineffective, or neither effective nor ineffective? Very effective Somewhat effective Neither effective nor ineffective Somewhat ineffective Very ineffective Don’t know 5 4 3 2 1 99 Q 16.1. Why do you think it was effective or ineffective in this regard? Q 17. To what extent are participants promoting the project to their family, friends or community members? Are they promoting it strongly, promoting it weakly, or not promoting it at all? Strongly promoting Weakly promoting Not promoting Don’t know 3 2 1 99 Q 17.1. If they are, how are they promoting the project? If they are not, why not? Q 18. Anything else that you would like to mention? Anything important that we have not yet discussed? Thank you for your time and patience. Youth Employment Project: Midterm Evaluation Report (UNDP) 161 Annex 5: Sampling The evaluation assessed the impact of training by comparing the pre- and post-training status of alumni as well as the pre-training status of current trainees to the post-training status of alumni, where the pre￾training status of current trainees represented the counterfactual, i.e., what the current status of alumni would have been had they not participated in training. Evaluators designed the survey sample to be able to detect a 10 percentage point increase in the proportion of trainees employed because of training. To test this hypothesis with 95 percent confidence and 90 percent power, the sample sizes required were 119 and 183 for current trainees and alumni respectively. However, a high non-response was expected considering that, in the absence of government permissions for fieldwork, trainees will be contacted through mobile phone numbers provided by UNDP and will be invited to participate in interviews at the training institutes in which where they were trained. The team therefore decided to conduct 500 surveys of alumni (from a sampling frame of 3,210) and 201 of current trainees (representing a complete census)—although the actual number of surveys completed for alumni is much lower than planned. Table 8 shows the numbers of surveys required, planned, and completed: TABLE 8: SURVEYS REQUIRED, PLANNED, AND COMPLETED Respondent Type Total Number of Trainees Number of Surveys Required Planned Completed Current 201 119 201 217 Alumni 3,210 183 500 186 Total 3,411 302 701 403 UNDP provided data on trainees’ sex, town of residence, training course attended, and contact number. The list served as the sampling frame from which the evaluation team drew the sample of alumni and current trainees. The team selected simple random samples stratified by sex and town of residence. The sample was allocated across strata to ensure the appropriate presentation of each stratum in the sample. The analysis used design weights to remove the possible bias introduced due to non￾proportional allocation of the sample across strata. In addition to the pre-selected sample of 500 alumni, the team also selected a random replacement sample from each stratum. Youth Employment Project: Midterm Evaluation Report (UNDP) 162 Annex 6: Marginalization Index To identify marginalized/at-risk towns, the evaluation team developed a composite marginalization index (CMI) using town level information on 19 indicators which are organized into three dimensions: actual crimes that occurred over the past year, trainees’ perceptions of insecurity in their neighborhoods, and other socio-economic indicators. The team gathered crime data for two randomly selected dates in each month from July 2014 to June 2015 from the daily Express Tribune newspaper. The team obtained perceptions of insecurity and other socio-economic indicators from the surveys of alumni and current trainees. Crime data on 18 towns of interest included data on murders, shootings, bomb attacks, target killings, deaths, robbery, and other crimes. For perceptions of insecurity, the surveys of trainees (current trainees and alumni from UNDP and Amantech) asked a set of questions on the levels of crime and violence, political parties’ and religious groups’ activities in their neighborhoods, and the effects of crime and violence on their households. Additionally, the survey asked respondents about their perception of poverty in their neighborhoods, the availability of income-generating opportunities, and their satisfaction with basic government services. For other socio-economic indicators, the evaluators derived measures of educational attainment, availability of economic opportunities in neighborhoods, poverty score, and average household income from the surveys. Details and definitions of crimes and other indicators used in the construction of composite marginalization index are given in Table 9. The composite marginalization index follows a nested weighting structure that applies equal weights across dimensions and equal weights for each indicator within a dimension. The weights assigned to each indicator are listed in Table 9. Considering variation in the nature and unit of measurement, for each indicator the team calculated a relative number using the formula given below, and these relative numbers (instead of actual numbers) were used to calculate the value of the composite marginalization index for each town. 𝑅𝑖𝑗 = 𝑋𝑖𝑗 − 𝑋𝑖(𝑚𝑖𝑛) 𝑋𝑖(𝑚𝑎𝑥) − 𝑋𝑖(𝑚𝑖𝑛) Where 𝑅𝑖𝑗 = Relative value of indictor i for town j 𝑋𝑖𝑗 = Actual value of indictor i for town j 𝑋𝑖(𝑚𝑖𝑛) = Minimum value of indictor i for all towns 𝑋𝑖(𝑚𝑎𝑥) = Maximum value of indictor i for all towns The CMI for each town is then the weighted sum of the relative numbers calculated from the actual numbers given in Table 10 and based on the definitions in Table 9. Youth Employment Project: Midterm Evaluation Report (UNDP) 163 TABLE 9: DIMENSIONS, INDICATORS, DEFINITIONS, AND WEIGHTS FOR THE CMI Dimension Indictor Definition Weight Crime Murder Number of deaths due to shooting by criminals or as a result of personal enmity 1/21 Shooting Number of people injured in shootings/crossfire 1/21 Bomb Attacks Number of deaths due to explosives, e.g., grenades, vehicle bombs, etc. 1/21 Targeted Attack Number of deaths or injuries due to targeted attacks on individuals 1/21 Deaths Number of deaths that do not fit any of the categories above. Examples include suicides, body found in parts of the city, drug overdose, criminals killed by rangers, etc. 1/21 Robbery Number of robberies 1/21 Other Number of any other crime reported that does not fit the categories above but is listed in the crime and misdemeanor section of the Express Tribune, e.g., injured during a brawl, weapons/drugs confiscated, etc. 1/21 Perception of Insecurity Effects of political parties’ activity Percentage of respondents who think political parties’ activities in their neighborhood negatively affect his/her household 1/21 Effects of religious groups’ activity Percentage of respondents who think religious groups’ activities in their neighborhood negatively affect his/her household 1/21 Level of crime and violence Percentage of respondents who reported “a lot of crime and violence” in their neighborhood 1/21 Effects of crime and violence Percentage of respondents who reported crime and valance “largely” affected their neighborhood 1/21 Level of poverty Percentage of respondents who think level of poverty in their neighborhood is “high or very high” 1/21 Lack of income￾generating opportunities Percentage of respondents who think in their neighborhood there are “no opportunities” to generate income 1/21 Satisfaction with government services Percentage of respondents who were “mostly or very dissatisfied” with government services in their neighborhood 1/21 Youth Employment Project: Midterm Evaluation Report (UNDP) 164 Dimension Indictor Definition Weight Other Socio￾Economic Indictors Educational attainments Percentage of respondents with “middle or low grades” before starting training 1/15 Economically inactive population Percentage of respondents who were “not in labor force” before starting training 1/15 Unemployment rate Percentage of respondents who were “unemployed” before starting training 1/15 Poverty score Each town’s average household’s poverty score gap from 100 (i.e., 100 – town’s average household’s poverty score) 1/15 Average income Each town’s average household’s monthly income gap from 40,000 (i.e., 40,000 – town’s average household monthly income) 1/15 Youth Employment Project: Midterm Evaluation Report (UNDP) 165 TABLE 10: CMI CALCULATION FOR ALL 18 TOWNS Towns Crimes Perception of Insecurity Other Socio-Economic Indicators CMI Murder Shooting Bomb Attacks Targeted Attack Deaths Robbery Other Crimes Effects of Political Parties’ Activity Effects of Religious Groups’ Activity Level of Crime and Violence Effects of Crime and Violence Level of Poverty Lack of Income-Generating Opportunities Satisfaction with Government Services Below or Equal to Middle Economically Not Active Unemployed Poverty Score Median Household Income No No No No No No No % % % % % % % % % % Scale Rs Baldia 2 3 0 0 4 0 0 24.0 13.5 3.9 42.9 23.3 39.7 79.9 7.6 39.0 12.7 77.27 31,500 0.4098 Bin Qasim 0 4 0 0 0 0 0 19.6 6.1 9.8 0.0 42.8 36.4 56.9 8.3 23.9 26.9 63.40 27,300 0.3947 Gadap 3 6 2 0 4 0 1 0.0 0.0 0.0 17.3 25.7 46.7 26.7 0.0 25.7 0.0 82.95 35,000 0.2368 Gulberg 3 5 0 2 0 0 1 22.5 3.8 14.1 5.8 20.1 13.2 63.7 3.0 34.8 11.8 86.21 40,000 0.3017 Gulshan 1 1 1 2 1 3 2 10.6 2.8 0.0 3.3 19.9 42.0 38.6 3.1 47.8 15.7 85.74 30,000 0.3324 Jamshed 1 2 0 1 4 1 6 17.7 8.4 3.5 8.8 19.0 39.1 51.9 13.5 46.5 23.1 81.99 25,000 0.4294 Kimari 0 0 0 0 2 0 0 4.0 0.0 0.0 3.8 43.1 29.3 58.4 23.9 59.6 19.8 68.15 28,000 0.3741 Korangi 2 12 0 3 1 1 0 16.8 6.3 5.6 8.5 26.3 43.6 48.5 8.3 43.5 27.3 81.01 25,000 0.4550 Landhi 6 2 1 0 1 2 9 15.8 5.2 8.4 16.1 28.6 44.5 49.2 12.6 33.7 18.5 73.28 21,800 0.5186 Liyaqatabad 4 2 0 0 4 1 0 42.3 0.0 11.0 9.1 29.5 41.7 51.2 2.7 35.8 27.7 85.15 29,000 0.3957 Lyari 3 11 1 0 15 0 3 8.0 1.4 12.8 32.8 40.6 47.4 66.7 10.6 35.4 34.3 75.88 20,000 0.5760 Malir 1 2 0 1 1 1 2 10.1 4.6 5.1 9.9 26.5 35.8 52.9 0.5 42.5 11.7 81.55 29,500 0.3141 New Karachi 2 6 0 0 1 2 1 28.6 5.5 10.9 6.9 20.7 44.6 60.6 4.1 36.9 12.7 85.99 30,000 0.3764 North Nazimabad 3 3 0 1 0 2 3 10.6 3.2 15.4 20.6 17.3 39.9 32.4 5.6 52.0 18.9 82.32 36,000 0.3825 Orangi 3 3 4 5 4 1 1 10.1 4.7 7.5 12.9 26.9 34.0 60.3 3.4 40.9 18.1 79.24 26,000 0.4740 S.I.T.E. 1 0 1 1 3 1 0 8.3 0.0 4.3 8.0 36.3 34.9 57.1 0.0 36.3 25.4 75.75 30,000 0.3348 Saddar 3 4 0 0 4 2 0 18.9 9.7 5.5 9.1 20.8 49.7 32.6 4.5 40.5 22.9 83.35 32,000 0.3778 Shah Faisal 1 1 0 0 1 0 0 24.6 11.7 10.6 19.9 12.3 25.4 54.8 1.9 45.1 30.5 83.70 32,000 0.3491 Youth Employment Project: Midterm Evaluation Report (UNDP) 166 FIGURE 16: MARGINALIZATION INDEX, LOCATION OF TRAINING VENUES, AND PERCENTAGE OF TRAINEES BY TOWN Youth Employment Project: Midterm Evaluation Report (UNDP) 167 Annex 7: Effect of Distance on Recruiting TABLE 11: REGRESSION ANALYSIS OF EFFECTS OF DISTANCE ON RECRUITING Variable Estimated Coefficient Standard Error t-statistic P>t All Applicants (N=51, Adjusted R-squared=0.06) Distance -0.002060 0.000984 -2.1 0.041 Constant 0.056123 0.019800 2.83 0.007 Men (N=28, Adjusted R-squared=0.02) Distance -0.001100 0.000760 -1.44 0.157 Constant 0.043771 0.016009 2.73 0.009 Women (N=43, Adjusted R-squared=0.11) Distance -0.004990 0.002366 -2.11 0.045 Constant 0.119512 0.045094 2.65 0.014 The dependent variable is the road distance between the geographic center of the town in which an applicant lives and the center of the town in which they received training. Records are the percentage of trainees from each training location/home location pair. Youth Employment Project: Midterm Evaluation Report (UNDP) 168 Annex 8: Conflict of Interest Disclosures The conflict of interest disclosures have been removed to protect the confidentiality of team members. They are available from MSI/PERFORM on request. U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523