end FINAL REPORT PERFORMANCE EVALUATION OF THE SENEGAL YOUTHMAP JEUNES AGRICULTEURS PROJECT February 2016 This publication was produced at the request of the United States Agency for International Development. It was prepared by Eric Allemano, Alioune Dieng and the Project Team of the Management Support and Technical Analy￾sis Services Project, implemented by the Pragma Corporation. i YOUTHMAP SENEGAL: JEUNES AGRI￾CULTEURS PILOT PROJECT1 PERFORMANCE EVALUATION DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government.                                                              1 YouthMap Senegal also included two other projects: YouthMap Casamance and Innove4Africa. ii Table of Contents ACRONYMS ............................................................................................................................................................... iii EXECUTIVE SUMMARY ........................................................................................................................................... 1 INTRODUCTION AND BACKGROUND ......................................................................................................... 9 Purpose of the Evaluation ......................................................................................................................................... 10 Overview of the Jeunes Agriculteurs (JA) Project .............................................................................................. 11 The Synapse Center Training Program .................................................................................................................. 12 Coordinating Partners ............................................................................................................................................... 13 Activity, or Implementing Partners ......................................................................................................................... 14 Cohort Classification.................................................................................................................................................. 15 METHODOLOGY ................................................................................................................................................... 16 Primary Data Collection ............................................................................................................................................ 16 Secondary Data Use ................................................................................................................................................... 17 Cost-effectiveness and cost-benefit calculations ................................................................................................. 17 LIMITATIONS TO THE STUDY .......................................................................................................................... 17 FINDINGS BY KEY QUESTION .......................................................................................................................... 19 KEY CONCLUSIONS ............................................................................................................................................. 36 RECOMMENDATIONS .......................................................................................................................................... 37 KEY LESSONS LEARNED ...................................................................................................................................... 41 OVERALL RECOMMENDATIONS FOR FUTURE PROGRAMMING ....................................................... 41 ANNEXES Annex 1 – Evaluation Scope of Work Annex 2 – Inception Report Annex 3 – PMP Annex 4 – Monitoring Table Annex 5 – Cost per job information Annex 6 – Cost-Benefit Analysis Annex 7 – Youth FGD Results Annex 8 – List of KII Institutions iii ACRONYMS AFR/SD USAID Africa Bureau Office of Sustainable Development ANPEJ National Agency for the Promotion of Youth Employment ARD Regional Development Agency ASESCAW Amicale socio-éducative, sportive et culturelle des agriculteurs du Walo CFA French Community of Africa CIH Centre d’Initiation à l’Horticulture CNAFP Centre National de Formation d’Appui à la Formation Professionnelle CNFPT Centre National de Formation Professionnel et Technique CONFEDES Confédération des Ministères de la Jeunesse et des Sports des Pays Ayant le Français en Par￾tage CREEJ Centre des Ressources pour l'Emploi des Jeunes CRREJ Centre Régional de Ressources pour l’Emploi des Jeunes ENSA Ecole Nationale Supérieure d’Agronomie EPT Ecole Polytechnique de Thiès FAO Food and Agriculture Organization FGD Focus Group Discussion FONGIP Fonds de Garantie des Investissements Prioritaires ICT Information and Communication Technologies ILO International Labor Organization IR Intermediate Result IYF International Youth Fund JA Jeunes Agriculteurs KII Key Informant Interview M&E Monitoring and Evaluation MSTAS Management Support and Technical Analysis Services Program NGO Non-governmental Organization PMP Performance Monitoring Plan RENCJES National Network of Young Entrepreneurs’ Clubs of Senegal SSA Sub-Saharan Africa TOT Training of Trainers UGB Université Gaston Berger USAID United States Agency for International Development 1 EXECUTIVE SUMMARY EVALUATION BACKGROUND AND PURPOSE The Jeunes Agriculteurs project was part of YouthMap, a multi-country initiative managed by the International Youth Foundation (IYF) and supported by USAID that seeks to inform how donors, international and local NGOs and gov￾ernments understand and address the challenges facing Africa’s youth. Senegal is one of seven Sub-Saharan African countries where IYF and various international and local partners have conducted on-the-ground assessments and supported innovative programs and evidence-based approaches to positive youth development. In 2012, IYF, under the YouthMap/Senegal Innovation Fund, invited Synapse Center, a Senegalese NGO located in Dakar, to submit a concept paper to implement an employability and entrepreneurship program in Senegal. The sub-grant was approved with a total budget of $500,440 as per the signed 1246SEN-GA grant agreement and updated in subsequent con￾tracts. The project purpose, as outlined in the 1246SEN-GA, was to “provide entrepreneurship, ICT, agriculture, and life skills training for 400 disadvantaged youth in Saint-Louis, Dakar and Kolda”. Because of funding constraints, the region of Kolda was replaced by Thiès, and the number of beneficiaries was dropped from 400 to 320. Purpose of Evaluation The contract to conduct the final evaluation of the Jeunes Agriculteurs (JA) project was carried out by The Pragma Corporation under the Management Support and Technical Assistance Services (MSTAS) Program. Dr. Eric Alle￾mano, Dr. Alioune Dieng and Dr. Babacar Diouf made up the field team that carried out the assessment during the month of August, 2015. A total of 177 beneficiary youth and 38 project partner representatives (including USAID Dakar) were interviewed in three regional zones (Dakar, Thiès, St. Louis) through a combination of Key Informant Interviews (KIIs) and Focus Group Discussion (FGD) interviews. This evaluation is intended to assess the multifac￾eted impacts of the YouthMap Jeunes Agriculteurs (JA) Project, implemented by Synapse Center, on youth beneficiar￾ies, to analyze the effectiveness of the project’s implementation and describe the strengths/challenges related to JA performance. The report is designed to generate pragmatic practices/lessons learned that should be considered for scale-up of future programs of this nature. These could be utilized by a broad range of stakeholders; including USAID, the Government of Senegal, IYF, Synapse Center, and other youth-focused NGOs and donor organizations inter￾ested in effectively integrating youth into agriculture programs. METHODOLOGY The performance evaluation of the JA project used a mixed-methods approach to assess effectiveness, impact, sus￾tainability and scalability. The evaluation team studied qualitative data and some quantitative data from IYF and Syn￾apse Center and used its pre- and post-test data on beneficiary youth to develop a profile of project results in terms of employment creation or finding by beneficiary youth. Primary Data Collection With the assistance of Synapse Center, all JA youth were contacted by telephone and asked to participate in the eval￾uation. The177 youth who came to the three research locations for interviews represented approximately 61% of the total original JA population (288) who had completed the training. During the first three weeks of August, 2015, the Evaluation Team interviewed 177 of the 288 youth who had completed the JA program in 2013 and 2014. Through a purposive sampling process, 60 youth participants were selected for KIIs, among the 177 who had filled out the questionnaires. In this regard, at each research location the KIIs were conducted with youth of both sexes grouped from each of the three cohorts of the project (crop production, food processing and produce marketing), and selected in a manner which effectively reflected the range of demographic, educational, and employment back￾ground and experience characteristics found among the respondents. Fifteen FGDs were conducted with youth dur￾ing the field research. As with the KIIs, the researchers tried to achieve a balance of participant profiles within each focus group: particularly sex and employment status. The average size of a focus group was about 10 youth. Secondary Data Use The principal sources of secondary data on the JA project were those provided by IYF and Synapse Center. The JA pre- and post-test results were particularly useful information sources. These data were utilized principally to gener￾ate information about employment outcomes for the entire group of 288 program graduates. 2 LIMITATIONS TO THE STUDY There are four categories of limitations to the study: 1. The timing of the field research 2. Sampling of youth to be interviewed 3. Documentation gaps 4. Complexity in determining project management overhead costs FINDINGS BY KEY QUESTION Evaluation Question 1: How effective were the approaches taken for each of the four program compo￾nents (entrepreneurship, agriculture2, ICT, and life skills) in achieving the activity’s goals? Findings and Conclusions Life skills was the first module of training, followed by entrepreneurship and technical skills (related to the agriculture sector value chain). In 2013, Synapse Center developed a pre-training test of youth attitudes and skills and a post-test at the end of the last training cycle in 2014. All 288 recruited youth took the tests. Before starting the program, par￾ticipants rated themselves fairly highly (either excellent or very good) across many of the categories, with the exception of conflict management. Program participants scored themselves highly on soft skills (i.e. accountability for results, time management, and creativity), relative to their skills in areas directly applicable to launching and operating a business. This shows the clear need for future programs to address those types of hard skills necessary to operate successfully in a professional environment. Although the questions in the post-test were not all the same, the results of the post-training test show that program participants felt more confident in basic aspects of entrepreneurship. The Agricultural Training component of the program is referred to as “Technical Training” by Synapse Center. It was provided by consultants in the fields of crop production, food processing and produce marketing and was evaluated by the youth trainees. During the FGDs and KIIs, many youth characterized the technical training as inadequate (de￾spite assertions by Synapse Center that the trainers were highly qualified), particularly among youth in the crop pro￾duction cohort in the Dakar Region. Overall, 54% of all respondents rated the technical training as average, passable, or poor. The issues raised by the youth included the lack of preparation for classes, not coming on time, postponing classes and using theoretical rather than practical methods. The qualifications of the trainers were not raised as an issue. The Youth-Map Monitoring Framework for Senegal (Annex 3) provides data for a program focused on self￾employment. Mentoring and coaching were provided during the training process but only sporadically after the pro￾ject, despite original plans. At the end of the project, 175 of the 288 youth who completed the training stated that they had created some form of self-employment. Thirty-eight found salaried jobs (on their own – many appearing to be short-term or seasonal in nature), 13 returned to studying and 62 remained unemployed. Overall, the JA project produced mixed results in meeting its stated objectives. Targets met or surpassed: Number of assessments; Attendees at dissemination events; Systems strengthened; Applicants to the program (target of 400; 1,609 applications received); Mentoring and coaching service targets were met; the numbers of learning expeditions and retreats were met; the expected numbers of business plans and busi￾ness development services were exceeded. Targets not met • Resources leveraged. The target was to raise $125,000 in both cash and in-kind contributions. However, the Monitoring Table reports only $4,520 of in-kind contributions3. • The number of youth trained was expected to be 320. Because of attrition during training (a loss of 10 percent of the trainees), only 288 youth finished the program4.                                                              2 Synapse Center used the term “Technical Training” for this module, since it included training on food processing and marketing. 3 Synapse reports that there was a total of $39,255 in pro bono, not including the value of land offered to the youth in the crop production cohorts. 4 While the number of youth who completed all of the project activities is 288, 320 participants were enrolled. The 32 participants who dropped out, mainly because of the strikes in Universities, completed about 75% of the project activities according to Synapse Center. 3 Ambiguously Defined Targets Several targets were ambiguous. For example, “Number of businesses improved”. In general the project focused on creating new employment more than on improving existing employment5. Overall Cost-Effectiveness The program was implemented for a total cost of just over $1.0 million, according to figures provided to the Evalua￾tion Team by IYF. Before the start of the JA project in 2013, a total of 28 youth reported some form of existing em￾ployment: 20 stated that they were self-employed while eight said that they had salaried employment. Employment in this context is defined as full time self-employed or salaried. According to a 2014 survey carried out by program sub￾implementer Synapse, 213 out of a total of 288 program participants who completed the JA training program either improved existing enterprises or created new enterprises. Some found salaried employment by the time the program had ended. This represents about 74% of the program participants who completed the JA training program.6 In addi￾tion, the average cost per employment position created under the program was approximately $4,701 per job. This cost was compared against a network of 11 recent (implemented over the last decade) USAID-supported employ￾ment creation and workforce development programs obtained through targeted desk research. A summary table on these studies is included in Annex 5. The Evaluation Team found that for these programs the average cost per job created ranged from a low of $1,100 to a high of approximately $12,000; with the vast majority of the programs falling in the range of $1,200-$5,000. Within this overall context, the per capita employment generation cost for the JA program fell at the high end of that band. set of comparator programs; most of which focused in large part on job training and employment matchning activities. Intuitively, within the context of a country with a per capita income in 2014-15 of approximately $1,100, this would appear to represent a very high cost per job leveraged in terms of op￾portunity cost. Evaluation Question 2: What was the activity’s overall impact on youth beneficiaries? What benefits did the targeted youth receive from participating across each of the training components? Findings: Within the participant group, 175 had created their own enterprises, 38 were in salaried employment, 13 had gone back to their studies and 62 were unemployed. In other words, 74% or nearly three-quarters of the participants who completed the training were employed in some form or another, surpassing the project’s target. It appears that many of the cases of post-training self-employment are extensions of the previous self-employment reported on the ques￾tionnaires and KIIs with the participants. During visits to the sites of participant enterprises (3 in the Dakar Region, 3 in Thiès and 1 in St. Louis7 ), the Evaluation Team learned that the participants sometimes used either prize money provided by the JA project or funds raised through small family loans to “spin off” an enterprise from a family enterprise. According to the KIIs and FGDs with the youth, as well as with the project partners, the lack of access to finance was cited as a critical constraint experienced in starting a business. The break-down of the coordinating com￾mittees appears to be a major cause of the lack of institutional follow-up on financing. Overall the lack of a coherent strategy for addressing the access to finance constraint facing participants was a critical constraint on the robust and sustainable impact on Business Plan implementation and income growth through the program. Net Impact on Income and Cost/Benefit Outcomes The Evaluation Team surveyed a total of 174 respondents.8 Of these, 94 respondents were identified who reported income at the beginning and/or end of the program (those that reported zero income at both the pre and post￾program stages were in general regarded as exhibiting a high likelihood of reporting error) An analysis of variance for                                                              5 While the intent of the JA project was employment creation, in some cases the project enabled youth to improve or upgrade existing self￾employment activities. The evaluators noted this in the case of a coffee processing and marketing enterprise in Thiaroye and a restaurant in Dakar that they visited. In any event, there was no target set for improving existing enterprises. 6 A determination was made by the project team that the pre and post-participation employment data gathered by Synapse was the most rigor￾ous and complete source of available information on employment outcomes for JA participants. 7 Heavy rain made it impossible to visit more than one youth employment initiative in the St. Louis region, as the enterprises were mainly rice or vegetable production in areas distant from paved roads. 8All 288 graduates of the JA training program were invited to participate in the interviews. However, 174 participants actually appeared at the regional interview settings and participated in KII’s. Of these 94 provided information on before and after-program earnings which the Evalua￾tion Team determined to reflect a reasonable level of accuracy (as explained further in the text). The employment information was generated from the Synapse data base, which covered all 288 program participants. Unfortunately, this data base did not contain systematic earnings infor￾mation for the participants, and so could not be utilized for purposes of cost-benefit and income impact analyses. 4 self-employment and wage earnings reported before and after the program was undertaken for the 94 referenced program participants. Overall, average annual earnings for the group increased by about $626 for the participant group members. For the 32 female participants included in the analysis, the average difference in annual wages was $639; while for the 59 male participants included it was approximately $561. These numbers seem to indicate some “push” impact from the program on average participant incomes. There was a significantly significant increase in mean earnings at a 90% confidence level. At the same time it is worth noting that for all the samples the standard deviation exceeds the pre- and post-program average income differential. This could in turn in part reflect the relatively small sample size (particularly for the sex-specific sub-groups), as well as the heterogeneous nature of the program partici￾pants. The Evaluation Team also carried out a cost/benefit analysis for the sample of 94 participants referenced above, utiliz￾ing the following assumptions: • Costs were based on the total $1.0 million program cost figure, scaled down in proportion to the number of beneficiaries included in the sample, and applied over the 2013-2014 period • Income benefits for year 1 were the incremental pre and post-program income stream for each participant, which were then adjusted upward by 5% annual during the remaining 29 years for which program related benefits were counted for a total of 30 years of projected benefits • A discount rate of 10% was applied to the stream of costs and benefits. 9 The results indicated that the NPV for the entire sample at a 10% discount rate was approximately $288,000 and the internal rate of return was about 7%. This indicates that the project as designed/implemented did not generate a fi￾nancial rate of return that would normally be considered acceptable for a donor employment/training program (a 10- 12% rate of return is normally considered the minimum acceptable rate for a donor-supported project). Program￾matic costs appear to be extremely high in relation to the aggregate income generation benefits, particularly given the size of the beneficiary population. This in turn generates critical questions regarding key program design and imple￾mentation parameters of the program. First, the program overall seems to have targeted a small number of beneficiaries in relation to the volume of funding being provided (roughly about 320 total beneficiaries originally envisaged for a budget of just over $1 million million). That is about $3,130 per beneficiary as originally planned – presumably a somewhat high cost per capita for an employment generation/training initiative. This is particularly true within the context of a country with per capita income about one-third of that level. In addition, once we take into account the inevitable “wastage” rate in terms of participants who either dropped out or were unable to establish businesses or find salaried employment, it was virtually inevitable that the costs per job were going to become high by international comparative standards. Evaluation Question 3: To what extent was the capacity of various activity partners strengthened through JA and what were the outcomes of these efforts? Findings: For the most part, the JA project focused on training trainers among the staff in the activity partners. The staff who benefitted from the ToT training went on to conduct the training on life skills and entrepreneurship for the JA pro￾ject youth. The technical trainers were independent consultants rather than staff members of the activity partners. Only five of the 12 activity (implementing) partners have formally agreed to integrate the IYF modules on life skills and entrepreneurship in their respective curricula. The overall lack of coordination and follow-up to the JA training appears to be largely due to the collapse of the regional coordinating committees set up in each region. As a result, Synapse was not able to respond rapidly to problems related to how the numerous implementation partners were (or were not) carrying out their various tasks. The program did provide very useful training of trainers activities cov￾ering the life skills and entrepreneurial skills modules, and initial preparation of sustainability plans by stakeholders. At the same time, it remains unclear that the project was able to materially and sustainably strengthen the institutional capacity of a network of partners to effectively provide these services, given the extra costs of orchestrating activities such as retreats, field trips and ICT training.                                                              9 The actual calculations carried out for the cost-benefit analysis are included in Annex 6. 5 Evaluation Question 4: What are identified trends in youth employment data with respect to sex, sectors of interest, employee retention, and employment quality (e.g., part-time/full-time, salary, hours, location, and promotions)? Similar questions will be asked regarding youth in self-employment. Findings: When responding to the pre-test designed and administered by Synapse Center in 2013, 185 individuals, or 58% of the youth who completed the training said that they already had some experience in the field of agribusiness. In most cases, these initiatives had been on family farms or in informal family enterprises. When asked what field they would like to develop as entrepreneurs, the most sought-after fields were crop production (67% of responses), marketing (52% of responses) and food processing (34% of responses) as shown in the table below. Synapse Center designed the training program with these youth goals in mind. The respondents were fairly evenly divided among the three project regions. Male respondents were more numerous than their female counterparts, in proportions similar to the sex breakdown of the original 288 youth trainees. Male and female respondents had similar profiles in terms of post￾project salaried work or self-employment. Both sexes were three to four times likelier to be self-employed than sala￾ried. In the Thiès Region, self-employment was seven to nine times more common than salaried employment. Youth in all three locations had similar combined education levels in terms of secondary and higher education with 88% for Thiès, 93% for Dakar, and 87% for Saint Louis10. In relation to salaried employment retention/employment quality trends, there is very little that can be effectively inferred since only eight participants at the start of the program and only 38 participants after completion of the program reported any type of salaried employment (and in many instances these appeared to be part-time positions). In this regard, because the focus of the program on self￾employment opportunities in the agricultural sector obviates the relevance of this analysis, as well as eliminating the possibility of generating a sample size through which reasonably rigorous results could be generated. Since in the end most of the non-self employment positions both before and after program particpation were part-time in nature and covered a very small portion of the particpant group, it was determined that detailed tracking of outcomes for this limited and un-targeted sub-group would not yield relevant or statistically valid/interpretable results. Evaluation Question 5: What is the likelihood of sustainability of program activities? Findings: Overall, as discussed earlier relation to comparative experience with other programs of this nature costs per job generated were relatively high, while the cost/benefit analysis indicated a low net financial return to project activities. Although Synapse Center prepared a sustainability plan for the project, the overall sustainability of the JA program activities is relatively weak owing to a lack of a permanent institutional base for the initiative. The program was organized in a somewhat ad-hoc manner, with components hosted by or contracted to a variety of partners. Moreo￾ver, the fact that just five implementing partners have agreed to integrate the JA training in their own activities im￾plies that many of the program activities are unlikely to continue. The agriculture/technical training activities are the most vulnerable since they were organized by external trainers rather than staff of the host institutions. These train￾ers were paid by the JA project budget and there is no documented indication that the implementing partners will be able to support their costs. The curricula for the cultivation, food processing and marketing training are the property of the trainers and were not handed over to the host implementing partners of the JA training unless the trainers were staff members of those institutions, other than those cases where the curricula were already in use. On the other hand, the fact that staff of the implementing partners were trained on the life skills and entrepreneur￾ship module increases the likelihood that these aspects of the training will continue. Question 6: Considering the financial and institutional context, which activity components can be scaled up and which stakeholders are best placed to support and are actively interested in scaling-up? Findings: An expanded or future JA project needs a firmer institutional framework. The project suffered from a plethora of at times inadequately coordinated partners. On the one hand, the coordinating partners mobilized to support the training process in each region, and to provide follow-up support in obtaining financing for youth employment initiatives, withdrew their support because they were expected to act voluntarily, without cost reimbursement from                                                              10 One hundred and twenty four youth had some secondary education while only thirty seven had completed secondary education. Ninety-five participants had some level of higher education but only three had completed a degree program in higher education. 6 Synapse Center. On the other hand, the implementing partners (with some exceptions11) did not integrate the train￾ing process in their activities. Evaluation Question 7: How did partnership formation support the building of local ownership and en￾courage innovation? Findings: Partnership formation was a relatively weak aspect of the JA project. Over 20 organizations were involved in coordi￾nating and implementing the management of the project. The coordinating partners formed committees in each re￾gion in order to guide and support the training process. However, none of the committees lasted beyond the first year of the project. The KIIs with the partner representatives revealed that these organizations had asked for reimbursement of travel and meeting costs (even at the local level), but these costs had not been budgeted and Synapse was unable to cover them. It was either not made clear to these institutions that their services were to be provided pro bono, or they were insufficiently sensitized about the philosophy and objectives of the JA project. Since Synapse Center was unable to find additional funds, the coordinating committees ceased to meet. Local ownership seems scattered and uncertain, given the need to fund many of the JA activities ranging from retreats to field trips, mentoring and inputs to training such as plots of land, kitchens and cooking appliances (for the food processing co￾horts). As mentioned earlier, only five of the implementing partners have formally agreed to integrate the life skills and entrepreneurship modules into their existing curricula. It is uncertain if other partners will follow suit. KEY LESSONS LEARNED 1. Choosing the most suitable partners for a “JA2” project is critically important. The Senegalese universities are strike-prone and the extended student strikes at the Ecole Polytechnique of Thiès and the Gaston Berger Univer￾sity of St. Louis effectively shut down the JA training that was hosted on their campuses. In addition, training programs held at formal educational institutions may operate on the academic year schedule, rather than by the calendar year. This problem occurred at CNAFP in Thiès. 2. Chambers of Commerce may be more suitable training partners, as their mission is business promotion, and they operate throughout the calendar year. Committed agricultural training institutions like the HIC of St. Louis and ASESCAW of Ross Bethio (St. Louis Region) are flexible and have a suitable hands-on informal approach to train￾ing. 3. Sufficient resources are needed in order to attract and keep competent and committed trainers. The technical training of the JA project was provided by outside consultants who were not paid in accordance with market rates in Senegal12. As a result, youth reported that some technical trainers were poorly prepared or relied only on classroom instruction instead of taking the trainees to nearby experimental plots to learn about cultivation techniques. The youth told the evaluators that the technical trainers often came late or suddenly postponed the training to the next day. 4. Strong incentive mechanisms should be put in place to ensure that the engagement of training partners positively impacts program outcomes. At a minimum, the reviews/responses from participants regarding the quality of the training activities should be reviewed and acted on if necessary. 5. Adequate time is needed to build skills, particularly in the area of financing, where youth participants did not have prior experience. The interviewed youth found that financial management was the weak point of an otherwise good quality module on entrepreneurship. A more integrated “institutional outreach” approach to link project participants up with viable sources of finance may be required as well to significantly improve the impact of this type of initiative in future. 6. At a more strategic level, it is worth considering whether or not the JA training/outreach strategy and associated set of implementation tools represents a cost-effective and viable intervention approach to driving agricultural entrepreneurship and income growth. Typically, the range of technical and market development and related institutional strengthening interventions required to drive successful agricultural/agribusiness outcomes under a donor support strategy and program go significantly beyond those available under a workforce development                                                              11 These appear to be implementing partners who receive funding from Synapse Center’s Challenge program. 12 The training consultants were paid 8,000 CFA Francs an hour. Dr. Alioune Dieng and Dr. Babacar Diouf the highly experienced Senegalese consultants who participated in the evaluation, stated that 23K to 25K CFA Francs are normal rates for senior consultants in Senegal. 7 focused initiative. The results of both the cost-effectiveness and cost/benefit analysis reported above certainly seem to suggest that this is the case for the JA Program. 7. It is extremely important that significant changes/adaptations in program focus be carefully reviewed with and approved by USAID from the twin vantage points of their prospective impact on cost-effectiveness and their relevance for the attainment of core programmatic outcomes in advance of their adoption by the program implementer and sub-implementers. 8. Future employment generation-focused initiatives of this nature should have a much stronger focus on concrete outcome-related performance indicators and related targets including specific cost-efficiency and income genera￾tion-related indicators. OVERALL CONCLUSIONS • The JA Program exhibited a solid overall employment creation or job placement rates for program participants who completed the program, with 74% of respondents reporting some form of employment at the time of the survey. At the start of the program only 28 youth reported some form of employment (20 were self-employed and eight were salaried workers. • Self-reported average annual earnings for those participants who both responded to the survey and reported in￾come either pre- or post-program increased by about $626 (150%). This is likely a result of youth moving from unemployment to employment, rather than increases in income associated with movement from one type of em￾ployment to another. • According to survey results, participants increased their social and personal empowerment with a robust im￾provement in conflict management and professional ethics and modest gains in personal skills as a result of the program. Improvements were lowest for the applied technical skills related to the agricultural sector value chain. • According to cost-benefit calculations looking solely at reported earnings, the JA Program generated an internal rate of return on project financial costs (the excess of quantifiable benefits of the project over USAID’s financial costs) of approximately 7%, which is below commonly accepted minimum rate of return standards for donor programs (the World Bank minimum standard is 10-12%). This implies that the overhead costs for the program, which comprise about half of total program costs, were relatively high in relation to the direct income benefits generated for program participants. • The Program developed and provided rigorous and highly effective training curricula in the life-skills and entre￾preneurship areas. The materials and training provided in critical technical areas – particularly related to agricul￾tural and agribusiness training - were significantly less effective, according to feedback from the youth interviewed during the field research. • The feedback loop between implementation and JA Program management appears to have been too tenuous to facilitate training program adjustments which could have strengthened key technical components of the training program where problems arose. • Lack of progress in facilitating financial access for program participants under the JA program limited its overall impact on participant incomes. • The complex network of institutional partnerships through which the JA Program operated made it difficult to promote accountability and shared expectations among partners, and to maintain uniform performance standards across JA training programs. • Inadequate trainer compensation policies appear to have significantly hampered the quality of the technical train￾ing activities. • The sustainability of JA activities is open to considerable doubt. In large part this is caused by the lack of clarity among partner institutions regarding the key elements of the sustainability strategy behind the JA Program, and the role of each partner in executing that strategy. OVERALL RECOMMENDATIONS FOR FUTURE PROGRAMMING • The JA project can best be seen as a pilot experiment in enabling youth to create employment in the agricultural value chain. Its successor could prepare youth for employment creation with more training on financial manage￾ment, and achieve a higher level of success by providing more guidance and follow-up on financing for the youth enterprises. It would also require better targeted technical and agronomic training, • Synapse Center had between five and eight of its staff working on the JA project. Among these staff, the Project Manager and Project Coordinator dedicated 100 percent of their time to supervising project activities. Any 8 future JA project will need a dedicated project manager in the home office to manage relations with the implementing partners and at least two or three support staff to oversee the training and identify problems that need attention. These staffing requirements may be difficult to fund for a local institution interested in managing a JA-style project like the one implemented by Synapse Center. • A future project of this nature should incorproate at its starting point an anlysis of prospective labor market demand trends at the sub-sectoral level; in order to assess where employment growth opportunities ar elikely to be greatest by sub-industry and region; in order to better focus/orient training programs in a manner which will maximize prospective impact on employment and income growth. • Public-private partnerships are valuable tools for training, internships and post-training follow-up and support for youth entrepreneurs. These partnerships need to be negotiated and set up in advance. The roles and responsibili￾ties of each partner must also be made clear, especially if they are expected to provide pro bono services such as mentoring and supervision. • The project was overly reliant on poorly-paid consultants for the technical training. Future JA training needs an institutional base to manage it while developing and keeping talented trainers. • Future initiatives like the JA project should focus on youth with at least secondary-level education. If training is to be provided for out-of-school youth with little education, a separate program or program component is needed. • Use JA graduates as resources in future iterations of the project. The successful graduates can provide strategic advice to trainees on how best to set up and manage their enterprises. The youth graduates can also share their experiences in funding, marketing and customer relations, among other areas of concern. • Post-training follow-up is critical. Mentoring was provided during the training but with the exception of one part￾ner in St. Louis, not after the training13. Follow-up support finding financing for youth projects is also essential. There are at least two sources of funding for productive projects, and one is designed specifically for youth. However, youth trainees need support and guidance to help them with the application process. • Serious consideration should be given as to whether a workforce development/training styled initiative is an ef￾fective centerpiece intervention mechanism for driving agriculture/agribusiness development at the farm or value chain level. If a determination is made to move forward with a renewed program of this nature, it needs to be effectively integrated with broader agricultural intervention programs focused on improving agricultural value chain competitiveness. Otherwise the prospective impact of a follow-on JA Program to build agricultural entre￾preneurship capacity is likely to be severely muted. • An effective M&E system is vital to the effectiveness of future programming for youth development. Ideally, the system should include: 1. Clear terms of reference or memoranda of understanding for each project partner 2. A results framework for each major project activity, supporting the results framework for the project. 3. A management system which systematically collects and effectively collates and utilizes collected data to de￾tect problems in project implementation, and to formulate targeted corrective measures. 4. Targets to track cost-effectiveness and cost-benefit of program activities                                                              13 Synapse Center notes that 5 ENSA students received land from their mentors. In Ross Bethio, 10 trainees gained access to land with the help of their mentors. In Sangalkam (Dakar Region) 10 other trainees had similar support from mentors. Finally, in Thiès, Synapse Center states that the “community of mentors” funded15 of the youth entrepreneurs’ projects. 9 INTRODUCTION AND BACKGROUND Senegal is an emerging market country with ambitious growth aspirations. However over the last decade Senegal has been outperformed by Sub-Saharan Africa, which grew at an average rate of 6% whereas growth in Senegal averaged only 3.3% since 2006. Senegal also performed poorly as compared with non-resource-rich Sub-Saharan countries. Moreover, while output per capita grew slowly, the volatility of growth was greater than that of other West African Economic and Monetary Union countries. Within Senegal, poverty remains high at 46.7% according to the 2011 Pov￾erty household survey, and the number of poor has risen during the 2006-2011 period. Given an estimated annual population growth of 2.5%, gross domestic product (GDP) growth remains well below the rate necessary for signifi￾cant poverty reduction14. With the support of Feed the Future, the US Government’s global hunger and food security initiative, the Govern￾ment of Senegal’s growing investment in agriculture (by over 10 percent per year) has opened the door for progress. In 2009, the country finalized its Comprehensive Africa Agriculture Development Program Investment Plan,15 setting a long-term vision for agricultural development as the primary driver of economic growth. The government also signed a New Alliance for Food Security and Nutrition Cooperation Framework in November 2013, paving the way for increased investment in the agriculture sector.16 Youth un- and underemployment is a serious development challenge in Senegal. Many youth do not possess the skills for employment and young entrepreneurs often face challenges of limited resources and business networks. These initiatives also have weak donor support; typically their resources come only from government funds. Senegalese youth have traditionally lacked adequate institutional support to overcome critical employability shortcomings stem￾ming from lack of experience and inadequate networking relationships within the business community17. These fac￾tors are compounded by enabling environment rigidities which limit private investment and job creation. The Jeunes Agriculteurs was a pilot project created to ‘test’ sustainable and effective responses to these needs, focused on the agriculture sector. The Jeunes Agriculteurs project was part of YouthMap, a multi-country initiative managed by the International Youth Foundation (IYF) and supported by USAID that seeks to inform how donors, international and local NGOs and gov￾ernments understand and address the challenges facing Africa’s youth. Senegal is one of seven Sub-Saharan African countries where IYF and various international and local partners have conducted on-the-ground assessments and supported innovative programs and evidence-based approaches to positive youth development. From the perspective of IYF, YouthMap was not a traditional development project, but a pilot program designed to test the viability of in￾novative employability approaches. In YouthMap the assessment component was followed by pilot Innovation Fund activities, decided upon and sculpted at the discretion of local USAID Mission personnel at the time. The objectives of these projects were to test promising practices and inform the mission’s youth activities. The YouthMap context is relevant to an assessment of this Innovation Fund project. • The YouthMap Cooperative Agreement (CA) included broad language with respect to Innovation Funds: They were limited “one-time” $500K budgets. • An example of CA language: “The YM Africa Innovation Fund will provide seed funding to implement pro￾grammatic options and then learn from and scale successful new efforts.” • YouthMap was USA/Washington-driven but local projects were implemented at the discretion of the country missions. • Mission involvement was uneven due to staff turnover. • Likewise, the YouthMap approach/budget appears to have been impacted by high turnover in USAID/W AORs starting in 2013, coupled with a simultaneous institutional USAID shift away from core Education Goal 2 Innovation Fund activities (WFD) to Goal 1 (EGRA) activities.                                                              14 http://www.worldbank.org/en/country/senegal/overview 15 http://www.nepad-caadp.net/text-version/library-country-status-updates.php 16 http://www.feedthefuture.gov/country/senegal 17 http://www.cres-sn.org/sites/default/files/article_diene_brookings.pdf 10 In 2011, IYF conducted an assessment in five regions in Senegal in partnership with a team of international and local researchers that included Senegalese youth. Building on YouthMap assessment results as well as IYF experience in youth employability and entrepreneurship in Africa and aligned with USAID/Senegal priorities in its Feed the Future initiatives, IYF decided to pilot an innovative youth employability and entrepreneurship program in the agriculture sector, targeted at rural, unemployed and vulnerable youth populations in Saint-Louis. A year later, in 2012, IYF, un￾der the YouthMap/Senegal Innovation Fund, invited Synapse Center, a Senegalese NGO located in Dakar18, to submit a concept paper to implement an employability and entrepreneurship program in Senegal. As outlined in the docu￾ment presenting the guidelines for submitting the concept paper, the funding for the sub-grant was approved with a total budget of $500,440 as per the signed 1246SEN-GA grant agreement and updated in subsequent contracts19. The project purpose, as outlined in the 1246SEN-GA, was to “provide entrepreneurship, ICT, agriculture, and life skills training for 400 disadvantaged youth in Saint-Louis, Dakar and Casamance”. Because of funding constraints, the region of Kolda was replaced by the region of Thiès, and the number of beneficiaries was dropped from 400 to 320. Purpose of the Evaluation The contract to conduct the final evaluation of the Jeunes Agriculteurs (JA) project was carried out by The Pragma Corporation under the Management Support and Technical Assistance Services (MSTAS) Program. Dr. Eric Alle￾mano, Dr. Alioune Dieng and Dr. Babacar Diouf made up the field team that carried out the assessment during the month of August, 2015. A total of 177 beneficiary youth and 38 project partner representatives (including USAID Dakar) were interviewed in three regional zones (Dakar, Thiès, St. Louis) through a combination of Key Informant Interviews (KIIs) and Focus Group Discussion (FGD) interviews. The results of this empirical work, supplemented by the utilization of an employment database for all program participants (288) provided by Synapse Center, provided the empirical base for the analysis. These data were supplemented by desk analysis of a range of reference and re￾porting documents. This evaluation is intended to assess the multifaceted impacts of the YouthMap Jeunes Agriculteurs (JA) Project, implemented by Synapse Center, on youth beneficiaries, to analyze the effectiveness of the project’s implementation and describe the strengths/challenges related to JA performance. The report is designed to generate pragmatic prac￾tices/lessons learned that should be considered for scale-up of future programs of this nature. These could be utilized by a broad range of stakeholders; including USAID, the Government of Senegal, IYF, Synapse Center, and other youth-focused NGOs and donor organizations interested in effectively integrating youth into agriculture programs. This performance evaluation is intended primarily to assess the effectiveness and impact on beneficiaries of the Sene￾gal Jeunes Agriculteurs project. The evaluation also seeks to determine the sustainability and scalability of the project in order to inform similar youth training and job creation projects that may be implemented in the future. The evalu￾ation was designed to provide a comprehensive analysis of all elements of the project, including youth recruitment and training as well as employment creation and placement. It responds to the evaluation questions in depth, ques￾tion by question. It provides an assessment of the degree to which JA project objectives and outcomes have been achieved, with a particular focus on how the program has contributed to employability prospects for youth benefi￾ciaries. The target audience of this evaluation includes key stakeholders ranging from USAID and the Government of Senegal to IYF, private companies, and youth-serving NGOs/donors. The evaluation of the JA project used a mixed-methods approach to assess effectiveness, impact, sustainability and scalability. The evaluation team received qualitative data from IYF and Synapse Center, the local implementing partner in Senegal, and made use of Synapse’s performance monitoring and quantitative data obtained from local partners                                                              18 “Founded in 2002, Synapse is a living community, bringing together learning and entrepreneurial leaders and emerging entrepreneurs. Our vision and positive values are based on principles that guide the development of systems and processes that serve as the backbone to the free￾dom to innovate and undertake .Our originality lies in our structured articulation of innovation, excellence and continuous learning and our anchor to the local context.” http://www.synapsecenter.org/mission . Synapse Center focuses on developing youth employability, social entre￾preneurship and opportunities for co-working. 19 This is the official YouthMap JA grants line-item amount. $500K was the grant amount to Synapse, with an $8K difference consisting of re￾lated banking fees, wire transfers, and other transaction costs 11 involved in providing training and facilitating job placement and creation among youth. In addition, data were collected from questionnaires filled out by 175 beneficiary youth as well as focus groiup discussions and key informant interviews with selected youth . The research was structured to answer seven key questions and six sub-questions: • Evaluation Questions on Program Effectiveness and Impact • Evaluation Question 1: How effective were the approaches taken for each of the four program components in achieving the activity’s goals?   • Sub-question 1a: How useful have internal assessments by USAID, IYF and the Synapse Center been for designing and/or improving training/employment activities?   • Sub-question 1b: How were youth qualifications and corresponding employer characteristics matched?   • Evaluation Question 2: What was the activity’s overall impact on youth beneficiaries? What benefits did the targeted youth receive from participating across each of the training components?   • Sub-question 2a: Employment Creation and Net Income Impact • Evaluation Question 3: To what extent was the capacity of various activity partners strengthened through JA and what were the outcomes of these efforts?   • Evaluation Question 4: What are identified trends in youth employment data with respect to sex, sectors of interest, employee retention, and employment quality (e.g., part-time/full-time, salary, hours, location, and promotions)? Similar questions will be asked regarding youth in self-employment • Evaluation Questions on Program Sustainability and Scalability • Evaluation Question 5: What is the likelihood of sustainability of program activities? • Sub-question 5a: Activity Costs • Sub-question 5b: What are the factors facilitating or inhibiting the sustainability of activity components beyond the JA (e.g. regional councils, partnership formation, etc.)?   • Question 6: Considering the financial and institutional context, which activity components can be scaled up and which stakeholders are best placed to support and are actively interested in scaling-up?   • Evaluation Question 7: How did partnership formation support the building of local ownership and encourage innovation? Overview of the Jeunes Agriculteurs (JA) Project Before the JA project began, IYF assisted Senegalese youth to find work from 2007 to 2009 with funding from the Microsoft Corporation. The focus of these efforts was to place youth in salaried jobs in existing enterprises. Accord￾ing to Synapse Center, that initiative achieved a placement rate of 68%. In another project with IYF, RTI, the Minis￾tries of Education and Youth of Senegal, Synapse Center was contracted to teach life skills to 4,430 youth via 166 trained facilitators. In 2012, Synapse Center saw an opportunity for youth employment in the agribusiness value chain due to available unused land in Senegal and a growing need for food self-sufficiency which coincided with a Government policy to en￾courage youth to seek employment in agriculture. After reviewing a mapping exercise of youth-serving organizations in Senegal, Synapse Center proposed what was to become the Jeunes Agriculteurs project to IYF. At the end of 2012 it received a grant of $508,939 from IYF to create and manage the project, implemented at a total cost of $1,833,000 including IYF’s overhead costs in supervising the JA project. A PMP for the JA project was provided to the team by Synapse Center, but a Monitoring Table (Annex 4) was used for project management purposes instead. This change appears to have been approved by USAID/Senegal. At the same time the project’s Results Framework does not provide quantified performance indicators that would guide the monitoring of project performance beyond the activity and output level. The YouthMap Semi-Annual Report (Octo￾ber 2014 to March 2015) and the YouthMap Monitoring Table for Senegal for the same period were used to help fill in data gaps. Further data on project performance and results, particularly for March 2014 through June 2015 were gathered from Synapse Center and the partner implementing organizations in Senegal. Although the JA Project’s Results Framework did not provide target numbers of youth to be trained or employment to be found or created, Synapse Center initially intended in 21 months to support 400 young Senegalese aged 16 to 12 29 and to create 120 businesses and/or social ventures in agricultural value chains. In addition to the training, Synapse Center and its partners organized retreats, site visits and counseling sessions for the participants, which received positive ratings from the youth. One of the project activities was to assist participating youth to obtain financing for their employment projects and there were cash prizes given by Synapse Center to youth with outstanding business plans. Some of the partner organizations are specialized in assisting youth in obtaining financing but there appears to have been a lack of adequate follow-up on this component of the project, according to KIIs with financing institutions and youth responses to the questionnaire. The project was to have operated in the Kolda, Dakar, Thiès and St. Louis regions. However, Synapse Center eventually decided to scale back the effort for cost reasons and finally recruited 320 youth for the project with a core focus on creating individual enterprises. The recruitment was carried out in the Dakar, Thiès and St. Louis regions. According to the Synapse Center managers interviewed, the JA project underwent several changes in scope and tim￾ing. The program officially started December 10, 2012, and was to end on August 31, 2014. However, it was subse￾quently extended by one month and was then officially closed on September 30, 2014. Synapse Center continued to run the project informally until January 2015 in order to provide follow-up support to the youth cohorts trained dur￾ing 2014. The Synapse Center Training Program Between January 2013 and March 2013, the Synapse project team worked on identifying institutional partners in the three target regions, selecting and training trainers, developing the training documentation and curricula. The youth were recruited by the project’s implementing partners. Of the 320 youth who were enrolled in the JA training, 288 completed their training. The recruitment for the program was done in three phases: eligibility determination, prelim￾inary identification and final selection of candidates. Eligibility for Participating in the JA Program The eligibility criteria for the youth participants were: • Between 16 and 29 years of age; • Residing in one of the three regions covered by the project: Dakar, Thiès or Saint Louis; • Having a project idea or experience in the field of agribusiness; • Wanting to develop a project in the production, processing or marketing of agricultural products; and • Being motivated and ready to participate in all program activities. The applicants’ level of education was not considered relevant. As a result, some youth were admitted to the JA pro￾gram with only Koranic or some primary education. Most applicants had some level of secondary education, and some were at the Master’s level. Preliminary Identification The preliminary identification of youth beneficiaries was done through the following means: • The involvement of local partners and stakeholders (resource persons, youth-serving organizations, and agribusi￾ness associations) that organized information sessions for potential youth applicants • Schools and Universities • Press releases in the three target regions and through this video: https://vimeo.com/69151565 • The website of Synapse Center (http://www.Synapse Centercenter.org/ plus outreach by its partners and in￾formed resource persons • Social networks • The broadcast of a video spot in recruitment sessions and on the web (https://vimeo.com/61379433) • Word of mouth and the "snowball" effect through the involvement of former beneficiaries of Synapse Center programs in the targeted areas As a result of this strategy, Synapse Center received a total of 1,609 applications. Final Selection The selection process was done through the following steps: 13 • Submission of application forms completed by the youth candidates • Evaluation of the applications by Synapse Center • Interviews to confirm the accuracy of the information provided Selected candidates took a pre-test to assess their attitudes and self-described level of skills related to the training program. After the selection process had been completed, 319 youth enrolled in the program. Of these, 288 com￾pleted the program successfully, including 164 men and 124 women. • Age: 143 (50%) youth were aged 21 to 25 while 98 (34%) were between the ages of 26 and 29. Only 47 (16%) persons were aged 16 to 20. • Marital status: By and large, the youth were single. Only 33 (12%) of the 288 who completed the program were married. Education: The youth chosen for the JA program exhibited fairly heterogeneous educational characteristics. The larg￾est number (124 persons, or 43%) had some secondary education. The profile of the 288 youth based on their edu￾cational levels is as follows: • Less than primary = 1% or 3 persons • Koranic or Primary = 9% or 26 persons • Secondary = 43% or 124 persons • Complete Secondary = 13% or 37 persons • University or tertiary = 33% or 95 persons • Complete university or tertiary = 1% or 3 persons Some of the project partners who manage the training found that mixing educational levels was not always produc￾tive. Some trainers had to explain certain concepts in Wolof to a subsection of students without French language abilities. According to Synapse Center, mixing educational levels was organized so that the educated youth could help those who were illiterate to develop their business plans. The training was carried out in two cycles. The youth were grouped into 16 thematic cohorts, each of which was trained for seven to eight months. The first cycle of eight cohorts trained a total of 134 youth (79 men and 55 women) during 2013. The second comprised a larger number of beneficiaries: a total of 154 youth (85 men and 69 women) in eight cohorts, trained during 2014.20 The first eight cohorts started their training between late March and mid-April 2013, finishing between October and November that year.21 The eight 2014 cohorts all started their train￾ing activities between February and April 2014 and completed them between September and December 2014. The documentation provided to the Evaluation Team indicates that a pre-test of youth who successfully applied to join the training program was conducted at the start of the JA project. A post-test was administered at the end of the project. Coordinating Partners Synapse Center mobilized 20 organizations spread over the three project regions to serve as coordinators of the recruitment and the training processes, listed below by region. These partners were responsible for project coordination, monitoring and internal evaluation. Some supervised the consultants hired by Synapse Center to pro￾vide the technical training and some organized field trips for the JA youth trainees. The professional background and skill sets of the consultants corresponded with the three types of technical training organized by the project. Some consultants, agronomists, were specialists in crop production. Others, who provided instruction for the food processing cohorts, were specialists in restaurant management and/or food preservation. The last group of consultants was specialized in marketing. According to Synapse Center, the technical consultants were experts in their respective fields. However, with some exceptions, the youth beneficiaries generally rated the                                                              20 Thirty-one of the 319 youth who began the JA training process did not complete the program because of student strikes at the Gaston Ber￾ger University in St. Louis and the Ecole Polytechnique in Thiès. A total of 288 youth completed the training (all cohorts combined.) 21 The staggered starts of the cohorts were linked to the availability of trainers and, in the case of the crop production cohorts, seasonal weather considerations. 14 technical training poorly. Some organizations (*) also served as implementing partners, hosting the youth training pro￾cess by providing training space, arable land or equipment. Dakar Region 1. Fonds de Garantie des Investissements Prioritaires (FONGIP) 2. Maison du Consommer Sénégalais 3. Mamelle Jaboot 4. The CEDEPS of Rufisque 5. City Hall of Guédiawaye* 6. Réseau national des clubs de jeunes entrepreneurs du Sénégal (RENCJES) * 7. Exode Urbain* 8. Fédération des Producteurs de Sangalkam* Thiès 1. UTC du Sine 2. Commune de Lam-Lam 3. Service Régional du Commerce 4. Chamber of Commerce* 5. Ecole Polytechnique de Thiès (EPT)* 6. Ecole Nationale de Sciences Agronomes (ENSA)* 7. Centre National de Formation d’Appui à la Formation Professionnelle (CNAFP)* Saint Louis 1. Regional Development Agency (ARD) 2. Chamber of Commerce* 3. Centre Régional des Ressources pour l'Emploi des Jeunes (CREEJ)* 4. Centre d’Initiation à l’Horticulture (CIH) * 5. Departmental Rural Development Service of Saint Louis) 6. Amicale socio-éducative, sportive et culturelle des agriculteurs du Walo (ASESCAW)* Activity, or Implementing Partners The activity, or implementing, partners were organizations responsible for managing the training process. The training was hosted by 12 implementing partners. Some had expertise in horticulture or rice cultivation, food processing and produce marketing. Others were youth-serving organizations and others were business associations or administrative bodies. These partners were the following: Dakar Region 1. City Hall of Guédiawaye 2. Exode Urbain 3. Fédération des Producteurs de Sangalkam 4. Réseau national des clubs de jeunes entrepreneurs du Sénégal (RENCJES) Thiès 1. Chamber of Commerce 2. Ecole Polytechnique de Thiès (EPT) 3. Ecole Nationale de Sciences Agronomes (ENSA) 4. Centre National d’Appui à la Formation Professionnelle et Technique (CNAFPT) Saint Louis 1. Centre d’Initiation à l’Horticulture (CIH) 2. Centre Régional des Ressources pour l'Emploi des Jeunes (CRREJ) 3. Université Gaston Berger (UGB) 4. Amicale socio-éducative, sportive et culturelle des agriculteurs du Walo (ASESCAW) 15 Cohort Classification Depending on their background and aspirations, the beneficiary youth were trained in one of three distinct thematic cohorts. These were: a) crop production, b) food processing and 3) marketing. Two to three cohorts were trained in each of the three target regions in each year of the project. • The production cohorts received technical training primarily in horticulture or rice cultivation, depending on the region. • The food processing cohorts learned skills, including making and preserving food products such as fruit juice and syrups, macaroni, couscous or semolina products. Some of the youth in these cohorts learned catering and restaurant management skills. • The marketing cohorts focused on selling raw or processed fruits, vegetables or grains. Each cohort began its learning program with two common modules: life skills training and entrepreneurship. These modules were adapted from training materials developed by IYF for use in other countries. The evaluators found lit￾tle reference to Senegal or agriculture in the case studies. The third module, technical training, covered mainly agri￾culture (crop production) or food processing or marketing, depending on the cohort. These three modules com￾prised 80 hours of training in 20 four-hour sessions scheduled over three to four months (depending on the cohort or location). The balance of the training included retreats, counselling sessions on themes such as financing and site visits to locations relevant to the central theme of each cohort (production, food processing or marketing). See be￾low for more details. Original plans for the JA project called for a fourth module, on ICT skills. However, the final module was not imple￾mented due to a lack of computers and Internet access at many of the training sites. In addition, the limited education of some of the participants made it difficult for them to master ICT skills, according to Synapse Center. The two common modules were complemented by additional learning activities that were also organized by Synapse Center. These were as follows: Introductory learning and planning activities designed to prepare the participants to benefit fully from the training on life skills, entrepreneurship and technical skills. These activities were scheduled and implemented as fol￾lows: • Learning expeditions (month 1). These consisted of trips designed to help the participants learn about the agribusiness system by observing it (and other relevant systems) firsthand. All the cohorts had a learning ex￾pedition as a prelude to their training; each learning expedition lasted a day or longer, depending on the na￾ture or geographic location of the training. • Retreats (month 2). The retreats (one per cohort) each lasted three days and provided a forum to enable the participants to clarify what they were being called on to do, individually and collectively. Among the out￾comes of the retreats was the formulation of individual action plans. On-going sensitization and learning activities were designed to reinforce skill acquisition and translate them into concrete results such as business plans. Months 2 to 6: • One thematic session per month with business leaders, finance specialists and experts on societal issues. These sessions enabled the youth to interact with, learn from and network with practitioners and leaders in their re￾spective fields of agribusiness. • One debriefing session per month was held to conduct peer reviews of progress, overcome obstacles and ex￾plore alternative ways of developing individual projects. • Ten coaching and mentoring sessions were organized in between training sessions. Each lasted four hours and was led by business professionals. Months 5 and 6: • Two innovation labs, designed to help youth to increase their capacity to analyze local markets and design and implement appropriate sustainable business responses, were held over a period of 30 to 45 days for each cohort 16 in Saint Louis and Thiès.22 The labs required suitable land for learning and perfecting cultivation methods, which was provided by the project partners in the St. Louis and Thiès regions. Months 5, 6 and 7: • Two financing platforms per cohort were held, each lasting four hours. These sessions exposed the youth to bankers, investors, philanthropists and heads of micro-credit unions. The sessions were intended to enable the JA youth to gain a better knowledge of the requirements and procedures involved in obtaining financing. The training was carried out by consultants provided by the JA Project’s partner organizations in the case of life skills and entrepreneurship or hired by Synapse Center for the technical training. The life skills and entrepreneurship train￾ers had a training of trainers program, held by Synapse Center, to learn how to use the training materials provided by IYF. These materials were originally written in English and translated into French. The technical trainers, who were considered subject-matter specialists, developed their own training materials and methods, depending on the theme of the youth cohorts they were contracted to train. In filling out questionnaires and during KIIs and FGDs some of the youth told the evaluators that they had not been given copies of all of the training materials to keep. The FGDs with the youth also revealed that their having to cover their own transportation costs to go to the training locations was an obstacle to optimal participation in the training for many youth. METHODOLOGY The performance evaluation of the JA project used a mixed-methods approach to assess effectiveness, impact, sus￾tainability and scalability. The evaluation team studied qualitative data and some quantitative data from IYF and Synapse Center and used its pre- and post-test data on beneficiary youth to develop a profile of project results in terms of employment creation or finding by beneficiary youth. Primary Data Collection With the assistance of Synapse Center, all JA youth were contacted by telephone and asked to participate in the eval￾uation. The177 youth who came to the three research locations for interviews represented approximately 61% of the total original JA population (288) who had completed the training. The participants were given refreshments and a meal at each research location. In addition, their travel expenses were reimbursed. During the first three weeks of August, 2015, the Evaluation Team interviewed 177 of the 288 youth who had com￾pleted the JA program in 2013 and 2014. The first round of youth interviews was held on the premises of Synapse Center and two institutional partners in the Dakar region that had hosted the training. The second round was held during the second week of research at the Chamber of Commerce of Thiès, one of the project partners. The third and final round of interviews was held the following week at project partner locations in the St. Louis region. At each research location, youth interviews and FGDs were held on Mondays, Tuesdays and Wednesdays. Thursdays were dedicated to KIIs with project partner representatives, and the research team visited selected enterprises created by JA youth on Fridays. Through a purposive sampling process, 60 youth participants were selected for KIIs, among the 177 who had filled out the questionnaires. In this regard, at each research location the KIIs were conducted with youth of both sexes grouped from each of the three cohorts of the project (crop production, food processing and produce marketing), and selected in a manner which effectively reflected the range of demographic, educational, and employment back￾ground and experience characteristics found among the respondents. Fifteen FGDs were conducted during the field research. As shown in the table below, six FGDs were held in the Dakar Region, Five in Thiès and four at different locations in the St. Louis Region. As with the KIIs, the researchers tried to achieve a balance of participant profiles within each focus group: particularly sex and employment status. The average size of a focus group was about 10 youth. The questionnaires were designed in English before the start of the field work and translated into French for use in the field. The youth questionnaires were used to gather data on the following themes:                                                              22 The labs were organized for the cohorts involved in crop production, which did not concern the cohorts in the Dakar region. 17 1. Demographic data, including age, sex, marital status, employment status, residence location and contact infor￾mation 2. Educational background up to the highest level of education attained 3. Pre-project employment (if any) and annual salary received 4. Post-project employment (if any) and annual salary received or net profits made 5. Assessment of the modules comprising the training program, including life skills, entrepreneurship and tech￾nical skills 6. Evaluation of the trainers 7. Strengths and weaknesses of the training program Interviews were generally conducted in French. However, some interviews were conducted in Wolof with youth who had little knowledge of French. Table 1: Data Collection among JA Youth Zone Individual Youth Inter￾views (KIIs) Individual Youth Ques￾tionnaires completed without KIIs Number of Focus Groups Participants in FGDs Dakar 25 incl. 6 women 56 6 48 incl. 19 women Thiès 20 incl.6 women 58 5 45 incl. 25 women Saint Louis 15 incl.3 women 63 4 46 incl. 14 women Total 60 177 15 139 incl. 58 women Source: Pragma Evaluation Team A non-experimental evaluation design was used for the evaluation because it was not possible to interview youth applicants who were not accepted to the JA project. Moreover, the post-training survey of all 288 participating youth from both training cycles lacked a control group. The data gathered from the questionnaires were keyed into an Ex￾cel database each week during the field work. SPSS was used to generate tables with cross-tabulations and percent￾ages. Secondary Data Use The principal sources of secondary data on the JA project were those provided by IYF and Synapse Center. The JA pre- and post-test results were particularly useful information sources. These data were utilized principally to gener￾ate information about employment outcomes under the program for the entire group of 288 program graduates. Cost-effectiveness and cost-benefit calculations The cost-effectiveness of the project’s employment generation interventions was assessed by calculating the average cost per job created across all program participants and comparing that against a benchmark of average per job cost for other similar training/job generation programs (described in more detail subsequently). The cost/benefit calcula￾tions were developed for a sample of the participants for whom reliable pre and post-program income data is availa￾ble (as discussed subsequently, this approach was adopted in order to avoid distortions likely to result from under￾reporting of income by some applicants, who either did not state an income level or reported zero both pre and post participation. Overall, this cautious approach to data selection will tend to lend if anything an upward bias to the results. The differential between pre and post-program earnings was projected out over a 30 year period and compared with a two-year stream of project costs (for 2013 and 2014), at a 10% discount rate, to provide a discounted net present value of the financial benefits to the participants. An internal rate of return on project costs was calculated in this manner. LIMITATIONS TO THE STUDY There are four categories of limitations to the study: 1. The timing of the field research 2. Sampling of youth to be interviewed 3. Documentation gaps 4. Complexity in determining causal relationships related to employment and income growth The research was conducted during the month of August, 2015, which meant that the youth in the cohorts that had 18 finished their training in November and December, 2014 had less than a year of income to report in cases where in￾terviewed youth reported creating their own enterprises. In an optimal situation, the youth would have reported at least a year of earnings. In situations where JA youth have focused on planting seedlings for mangoes or other tree crops, earnings related to their participation in the project will not be available for another two to three years. As a result there may not have been a sufficiently lengthy period of time between project completion and assessment work to permit a fully accurate gauging of the long-term impact of the program on the employment and earnings of program participants. Overall this might be expected to bias downward somewhat the results of the analysis. In ad￾dition the timing of the assessment, occurring so quickly after completion of the program, creates ambiguities in rela￾tion to assessing the sustainability of program outcomes; as it requires that inferences be drawn regarding motiva￾tional disposition of program partners, based on empirical observations and interview processes. It should be noted as well that the nature of the JA project was to prepare youth to create their own employment in the agriculture value chain of crop production, food processing or produce marketing. All of the 288 graduates of the JA project training were invited to participate in the interview process. However, because of the rural focus of the project, some youth in the crop production cohorts were in remote locations (especially in the rice-growing areas of the St. Louis Region) and had constraints in travelling to the research locations. Youth who were in salaried jobs (a minority) may have had difficulty in taking a working-day off to participate in the research. Family obligations could have represented a constraint for some female graduates of the training program. Finally, torrential rains in the St. Louis Region made it difficult for some JA youth to travel from remote locations to the Ross Bethio interviewing lo￾cation. The research team did not have access to certain important documents related to project impact. These were nota￾bly the results of the youth evaluation of the technical trainers, conducted by Synapse Center in its post-project eval￾uation. Furthermore, Synapse Center did not provide the research team with copies of the MoUs that it said had been signed with the implementing partners that agreed to continue the JA training program after the end of the USAID Innovation Fund financing under IYF aegis. This has created additional ambiguity in terms of assessing which partners are actually ready to continue the training process, particularly since follow-on funding has not been pro￾vided for this process except for five of the 12 implementing partners. Other exceptions are ENSA, which has spe￾cialized staff, land and equipment near Thiès, and the Centre d’Initiation à la Horticulture in St. Louis, both of which are in the process of integrating the JA training modules in their curricula. As for determining whether the training was the sole or the key factor in job creation there is some ambiguity in the documentation about youth baseline characteristics. The Synapse Center pre-test states that 58 percent of the re￾cruited youth said they had “prior experience in agribusiness” and some already had some form of employment in agriculture or food processing. At the same time it is clear that most of the interviewed participants did indicate that they felt they had benefited overall from their training experience. In this regard the analysis implicitly assumes that the training provided has had a reasonably direct relationship with subsequent employment outcomes for program participants. It was difficult to obtain precise and credible data on income. One factor linked to this is cultural, in that Senegalese are reluctant to report their earnings and financial resources to strangers (the research team in this case.) It was apparent that some of the pre-project income reported was seasonal or part-time earnings. A further limita￾tion to the reporting of income is that some of the youth cohorts had completed their training less than a year be￾fore the research was carried out in August, 2015. As a result, some employed youth had less than 12 months of in￾come to report. Site visits and KIIs revealed that some of the youth were either starting fruit-tree nurseries or had planted mango seeds to start orchards. It will be several years before these trees can generate income. Moreover, for youths with multiple sources of (part-time) income, accurate quantification is particularly difficult. In terms of cost-benefit analysis related results, it should be noted that this type of analysis necessarily omits non-pecuniary ben￾efits to the individual trainees; as well as indirect benefits deriving from institutional capacity building among partner entities. The evident benefits of the project were operational skills and emotional/attitudinal development related to business management gained by the youth. The research team found that there was little concrete evidence of capacity-build￾ing among the implementing partner organizations beyond the strengthening of their staff who had been trained by the project to use the life skills and entrepreneurship modules in future training initiatives. Four months after the field research, Synapse Center informed Pragma that five of the implementing partners are replicating the JA training with 19 funding from Synapse Center. More precise information about the new cycle of training has not been provided and the research team cannot verify how many youth are enrolled and whether any changes have been made in the meth￾odology of the first JA project. FINDINGS BY KEY QUESTION In total there are 13 evaluation questions. Seven of these are key questions, four of which concern the effectiveness and impact of the project on the participating youth while three relate to the project’s sustainability and scalability. The remaining six questions are sub-questions of four of the key questions. Evaluation Questions on Program Effectiveness and Impact The implementation of the JA project was initially supposed to follow a work plan leading to the intermediate results presented in the Results Framework (Annex 4). However, as noted earlier, in practice a monitoring table was used in place of the Results Framework. Evaluation Question 1: How effective were the approaches taken for each of the four program compo￾nents (entrepreneurship, agriculture, ICT, and life skills) in achieving the activity’s goals? Findings Life skills was the first module of training, followed by entrepreneurship and finally technical skills (related to the agri￾culture sector value chain). ICT skills were not covered in the training program, reportedly because of budgetary constraints and a lack of computers and Internet connections at the training sites. Without it, however, youth are faced with many difficulties ranging from a lack of information about markets for their products and services to con￾straints to bookkeeping and financial management. The ICT component of the training was dropped early during the implementation of the program. The eight hours dedicated to ICT training was not enough for the majority of partici￾pants (either illiterate or with a low level of education) to master computer skills and/or office applications. On the other hand, participants from the Universities (ENSA, UGB and EPT) already mastered what they were supposed to learn in the area of ICT applications. In addition, the services proposed by different providers including were over budget .It was then decided, in accordance with IYF, to add the eight hours initially to be dedicated to ICT to the en￾trepreneurship component, where more support was needed. 23 Pre-test to Post-Test Changes among JA Participants. In 2013, Synapse Center developed a pre-training test of youth attitudes/skills and a post-test at the end of training cycle in 2014. All 288 recruited youth took the tests. Personal (Life) Skills Development. The pre-test was designed to establish a baseline of youth attitudes and self￾described skills related to enterprise creation and management. The first baseline test analyzed self-confidence, communication and related skills which were further developed in the life skills module of the training program. Be￾fore starting the program, participants rated themselves fairly highly (either excellent or very good) across many of the categories, with the exception of conflict management. The Synapse data do not distinguish between male and female responses. Table 2: JA Youth Personal Skills Profile Pre-Test Self Confi￾dence Relationship building skills Teamwork skills Conflict man￾agement skills Personal presen￾tation skills Professional eth￾ics Excellent 33% 29% 25% 10% 22% 27% Very good 30% 25% 40% 24% 35% 30% Good 33% 35% 29% 54% 36% 35% Average 3% 9% 5% 10% 6% 7% Weak 1% 2% 1% 2% 1% 1% Total 100% 100% 100% 100% 100% 100% Source : Synapse Center Pre-Test                                                              23 IYF communicated directly with USAID/Senegal and USAID/W (YouthMap was centrally- funded and managed out of Washington) on all program decisions, which are also reflected in semiannual reports. 20 Conflict management and professional ethics saw the largest gains from program participants after the program. The proportion of youth rating their skills in these areas as either excellent or very good increased from 34% to 52% and from 57% to 74%, respectively. Table 3: JA Youth Personal Skills Profile Post-Test Self-Confi￾dence Relationship building skills Teamwork skills Conflict man￾agement skills Personal presentation skills Professional ethics Excellent 34% 22% 32% 14% 21% 29% Very good 41% 42% 40% 38% 49% 45% Good 22% 29% 24% 40% 27% 23% Average 3% 6% 4% 6% 3% 3% Weak 0% 1% 0% 2% 0% 0% Total 100% 100% 100% 100% 100% 100% Source : Synapse Center Post-Test Entrepreneurship Training. The second key dimension of the training was entrepreneurial skills. The self-assess￾ment pre-test in 2013 yielded the following results: 0% 10% 20% 30% 40% 50% 60% Self Confidence Relationship building skills Teamwork skills Conflict management skills Personal presentation skills Professional ethics Chart 1 JA Youth Personal Skills Profile Pre‐Test Excellent Very good Good Average Weak 0% 10% 20% 30% 40% 50% 60% Self‐Confidence Relationship building skills Teamwork skills Conflict management skills Personal presentation skills Professional ethics Chart 2 JA Youth Personal Skills Profile Post Test Excellent Very good Good Average Weak 21 Table 4: JA Youth Entrepreneurship Skills Pre-Test Ac￾counta￾bility for re￾sults Mastery of the steps in creating a business Ability to prepare a business plan Mastery of the legal procedures in setting up a business Ability to find fi￾nancing Time man￾agement Creativity Excellent 29% 2% 0% 0% 1% 14% 11% Very good 31% 6% 9% 4% 3% 25% 27% Good 26% 15% 20% 14% 15% 37% 41% Average 10% 34% 32% 25% 31% 16% 19% Weak 4% 43% 39% 57% 50% 8% 2% Total 100% 100% 100% 100% 100% 100% 100% Source : Synapse Center Pre-Test Program participants scored themselves highly on soft skills (i.e. accountability for results, time management, and cre￾ativity), relative to their skills in areas directly applicable to launching and operating a business. This shows the clear need for future programs to address these types of skills. Although the questions in the post-test were not all the same, the results suggest that program participants felt more confident in basic aspects of entrepreneurship. In the post test the youth were asked, “Have you mastered the following skills?” The responses were not broken down by sex. 0% 10% 20% 30% 40% 50% 60% Accountability for results Mastery of the steps in creating a business Ability to prepare a business plan Mastery of the legal procedures in setting up a business Ability to find financing Time management Creativity Chart 3 JA Youth Entrepreneurship Skills Pre‐Test Excellent Very good Good Average Weak Table 5: JA Youth Entrepreneurship Skills – Post-Test Answers Mastery of the steps in creating a business Ability to pre￾pare a business plan Mastery of the legal pro￾cedures in setting up a business Ability to find financing Yes 66% 72% 55% 42% No 7% 7% 15% 14% Partial 25% 17% 26% 39% I am not sure 2% 4% 3% 5% Not applicable 0% 0% 1% 0% Total 100% 100% 100% 100% Source : Synapse Center Post-Test 22 Agricultural Training. This component of the program is referred to as “Technical Training” by Synapse Center. It was provided by consultants in the fields of crop production, food processing and produce marketing and was evalu￾ated by the youth trainees. During the FGDs and KIIs, youth characterized the technical training as inadequate, par￾ticularly among youth in the crop production cohorts in the Dakar Region. Criticism included the reliance of some trainers on academic lecturing rather than practical activities in the field. During a FGD during the evaluation, some of the youth who were students at ENSA in Thiès said that they knew more about agronomy than the trainer24. Student strikes at the Gaston Berger University in St. Louis and the Ecole Polytechnique in Thiès, which hosted some production cohorts, effectively closed down these institutions. As a result, these institutions were unable to deliver the technical training to the concerned JA youth. The following table presents the findings on the Agricultural/Tech￾nical skills training as expressed by the youth interviewed during the final evaluation of the JA project. Overall, 54% of the respondents rated the technical trainings as average, passable, or poor. Table 6: Satisfaction with JA Agricultural/Technical Skills Training Dakar Thies Saint Louis Total N % N % N % N % Male Re￾spondents Excellent 2 3.8% 1 2.4% 1 3.8% 4 3.4% Very good 7 13.5% 2 4.9% 5 19.2% 14 11.8% Good 4 7.7% 2 4.9% 0 0% 6 5.0% Average 11 21.2% 12 29.3% 7 26.9% 30 25.2% Passable 0 0% 2 4.9% 3 11.5% 5 4.2% Poor 5 9.6% 2 4.9% 0 0% 7 5.9% Total 29 55,8% 21 51.2% 16 61.5% 66 55.5% Female Re￾spondents Excellent 2 3.8% 9 22.0% 0 0% 11 9.2% Very good 7 13.5% 3 7.3% 2 7.7% 12 10.1% Good 5 9.6% 1 2.4% 2 7.7% 8 6.7% Average 6 11.5% 6 14.6% 6 23.1% 18 15.1% Passable 3 5.8% 1 2.4% 0 0% 4 3.4% Poor 0 0% 0 0% 0 0% 0 0% Total 23 44,2% 20 48.8% 10 38.5% 53 44.5% Source: Pragma Evaluation (Post-project)                                                              24 Synapse has indicated its disagreement with this statement, although it was recorded by an author of this study. 0% 10% 20% 30% 40% 50% 60% 70% 80% Mastery of the steps in creating a business Ability to prepare a business plan Mastery of the legal procedures in setting up a business Ability to find financing Chart 4 JA Youth Entrepreneurship Skills – Post‐Test Yes No Partial I am not sure Not applicable 23 However, female respondents were more favorable than their male counterparts in evaluating the technical training, particularly in the Thiès Region. The food processing cohorts were essentially female and they had training in host organizations specialized in food processing training such as the Centre National de Formation Professionnelle et Technique in Thiès, whose director gave special attention to the training. Twenty-two percent of the female re￾spondents in Thiès rated the technical training as “excellent” where are less than four percent of the respondents (male and female) rated the technical training as “excellent” in the Dakar and St. Louis Regions. None of the female respondents scored the technical training as “poor” while nearly six percent of the male respondents did so. The PMP Indicators A PMP was prepared for the JA project but was replaced by the Youth-Map Monitoring Framework for Senegal (An￾nex 3). The Monitoring Framework provides data for a program entirely focused on self-employment, which was reportedly approved by USAID, according to IYF (the use of the Monitoring Framework was referred to in the Youth Map Semi-Annual Report ending March 31, 2013). As a result, there was no focus on job placement per se. Mentoring and coaching were provided during the training process but with a few exceptions, not afterward, as was originally planned. The Monitoring Framework called for the provision of “business services.” Some guidance was provided on financial management and “funding platforms” were held to guide the youth to sources of business loans. Although many youth cited a lack of funding as a major constraint to developing their enterprises, Synapse informed the evaluators that it is working with the Fonds de Garantie des Investissements Prioritaires (FONGIP) to support the financing of the young entrepreneurs’ projects. However, Synapse has not provided the evaluators with figures on the number of youth beneficiaries or the amounts of funding raised. At the end of the project, 175 of the 288 youth who completed the training stated that they had created some form of self-employment. Thirty-eight found salaried jobs (on their own – many appearing to be short-term or seasonal in nature), 13 returned to studying and 62 remained unemployed. These data are summarized below. Overall, the JA project produced mixed results in meeting its stated objectives as laid out in the Monitoring Table (below). In some areas, project achievements were over the target figures in Table 7 (below) while in other areas the achievements were below the targets, as summarized below. Targets met or surpassed: • Number of assessments • Attendees at dissemination events • Systems strengthened • Applicants to the program (target of 400; 1,609 applications received) • Mentoring and coaching service targets were met 0% 10% 20% 30% 40% 50% 60% %%%% Dakar Thiès Saint Louis Total Chart 5 Satisfaction with JA Agricultural/Technical Skills Training Excellent Very good Good Average Passable Poor 24 • The numbers of learning expeditions and retreats were met • The expected numbers of business plans and business development services were exceeded. Targets not met: • Resources leveraged. The target was to raise $125,000 in both cash and in-kind contributions. However, the Monitoring Table reports only $4,520 of in-kind contributions25. • The number of youth trained was expected to be 320. Because of attrition during training (a loss of 10 percent of the trainees), only 288 youth finished the program26. Ambiguously Defined Targets Several targets were ambiguous. For example, “Number of businesses improved”. While some of the JA participants were self-employed (often in some form of family farming), the project focused on creating employment more than on improving existing employment. The target, “Number of youth gaining employment as a result of program participation” appears to imply finding sala￾ried work. While some youth found or returned to salaried work, job hunting was not a project objective. Similar ambiguity surrounds the target of “Number of youth transitioning to further education as a result of program partici￾pation,” which is subject to potential misinterpretation for the JA project. In this regard, according to Synapse Cen￾ter, the 13 JA trainees who returned to studies at the end of the training did so because they wanted to complete their studies. Table 7: Comparison of Project Targets and Achievements 2011-12 Completed FY2013 Annual Target Actual Total FY2014 Annual Target Actual Total Total Number of assessments completed 1 0 - - 1 Number of people who attended the dissemination event and other program events 100 0 200 - 200 Number of partnerships created in sup￾port of innovation fund program N/A 10 12 12 Resources leveraged for YouthMap Senegal (USD value) N/A 62,500 62,500 4,520 4,520 Cash 31,250 - 31,250 - - In-kind (buildings, equipment & ser￾vices) 31,250 31,250 4,520 4,520 Number of systems27 strengthened N/A 1 - 1 - 1 Number of youth applying for program N/A 200 782 200 827 1,609 Male - - - Female - - - Number of youth participating in train￾ing programs N/A 160 164 160 124 288 Male 80 79 80 55 134 Female 80 85 80 69 154 Number of youth completing the pro￾gram N/A 128 - 128 135 288 Male - 82 164                                                              25 Synapse asserts that there was a total of $39,255 in pro bono, not including the value of land offered to the youth in the crop production cohorts. 26 While the number of youth who completed all of the project activities is 288, 320 participants were enrolled. The 32 participants who dropped out, mainly because of the strikes in Universities, completed about 75% of the project activities according to Synapse Center. 27 The Monitoring Table does not specify what “systems” are concerned. 25 Female - 53 124 Number of youth receiving mentoring services N/A 120 - 120 123 288 (only during the training) Male 72 80 164 Female 48 43 124 # of youth receiving coaching services N/A 128 160 128 123 288 (during training) Male 76 89 80 164 Female 52 71 43 124 # of learning expeditions organized N/A 8 8 8 8 8 Number of retreats organized N/A 8 8 8 8 8 Number of business plans created N/A 96 33 96 198 288 # of youth receiving business development services N/A 96 180 96 255 288 (only during the training) Male 48 99 152 164 Female 48 81 103 124 Number of businesses created N/A 30 10 30 56 175 56 Number of businesses improved N/A 30 31 30 116 N/A (no post-pro￾ject follow-up) Number of youth gaining employment following program participation N/A 100 - 100 - 38 personal initia￾tives, as finding sal￾aried work was not a project ob￾jective Male 50 - 50 25 Female 50 - 50 13 Number of youth setting up businesses following program participation N/A 30 10 30 56 175 Male 5 41 112 Female 5 15 63 Number of youth transitioning to fur￾ther education as a result of program participation N/A 96 96 38 youth returned to studies, having not found or cre￾ated employment Male 48 48 Female 48 48 Findings: It is difficult to assess objectively the JA project’s impact on the youth beneficiaries because there was no evidence provided to the evaluation team of any testing of gains in knowledge and skills by the trainers at the end of the training cycles.28 The youth gave their own self-evaluations of what they had gained through the training process and its com￾ponents, as well as evaluation comments about the three main components of the training program. One weakness of the JA program is that the performance assessment is heavily focused on self-perceptions of skills gained. Future programs should have a more rigorous M&E framework to measure results more objectively. The training on life skills and entrepreneurship appears to have had the greatest impact on the participants in terms of their evaluations of the quality of the three main modules of the JA training program. According to the question￾naire, KII and focus group results, the technical training provided less substance than many participants expected. The participants were asked to evaluate the quality and relevance of three key dimensions of the technical dimensions of the JA training program: 1. The training modules or thematic components of the overall training program                                                              28 Synapse Center affirms that gains in knowledge and competence were assessed but the evaluation team did not receive these data. 26 2. The trainers, by module 3. The trainers, by project partner institution Youth Reactions to the Training Modules As shown in Table 8, the life skills module, designed by IYF, received the highest ratings. In all three regions, this module was rated “Very good” by 50% of the respondents. Entrepreneurship had the next highest ratings with 49% of the respondents rating it as “Good”. The technical skills module (which was either crop production, food pro￾cessing or crop cultivation depending on the youth cohort) had the worst ratings. Overall, 41% of the respondents found this module to be “Passable.” In the St. Louis region, this module was rated “Passable” by 51% of the respond￾ents. Table 8: Youth Evaluation of the Quality of the JA Training Program Modules Modules Dakar Thiès Saint Louis Total N % N % N % N % Life Skills Very good 26 50% 22 50% 26 49% 74 50% Good 18 34% 13 30% 20 38% 51 34% Fairly good 5 10% 7 16% 5 9% 17 11% Passable 3 6% 1 2% 2 4% 6 4% Poor 1 2% 1 1% Total 52 100% 44 100% 53 100% 149 100% Entrepreneurship Very good 10 19% 11 27% 9 18% 30 21% Good 32 61% 15 37% 24 47% 71 49% Fairly good 6 12% 10 24% 13 25% 29 20% Passable 4 8% 4 10% 5 10% 13 9% Poor 1 2% 1 1% Total 52 100% 41 100% 51 100,0% 144 100% Technical Training Very good 4 8% 10 24% 1 4% 15 12% Good 14 27% 6 14% 7 26% 27 22% Fairly good 9 17% 3 7% 2 7% 14 12% Passable 17 32% 18 43% 14 52% 49 41% Mediocre 3 6% 3 7% 3 11% 9 7% Poor 5 10% 2 5% 7 6% Total 52 100% 42 100% 27 100% 121 100% Source : Pragma Evaluation Team August, 2015 In terms of sex differences, Table 8.1 below shows that male respondents had a slightly more favorable opinion of the Life Skills module than females. Twenty-six percent of the young men rated the Life Skills module as “Very good” compared with 23.3 percent of the young women. Among male respondents, 18.5 percent felt that the module was “Good” compared with 15.8 percent of the female respondents. Male respondents were also likelier than females to rate the module as “Passable” or “Poor”, although the percentages in these categories were below five percent for both sexes (no young women rated the module as “Poor”). Table 8.1 : Youth Evaluation of the Quality of the JA Training Program Modules Life Skills Total Dakar Thies Saint Louis N % N % N % N % Male Respond￾ents Very good 15 28.8% 8 18.6% 15 29.4% 38 26.0% Good 8 15.4% 8 18.6% 11 21.6% 27 18.5% Fairly good 3 5.8% 5 11.6% 2 3.9% 10 6.8% Passable 3 5.8% 1 2.0% 4 2.7% Poor 1 2.3% 1 .7% Total 29 55.8% 22 51.2% 29 56.9% 80 54.8% Female Re￾spondents Very good 11 21.2% 13 30.2% 10 19.6% 34 23.3% Good 10 19.2% 5 11.6% 8 15.7% 23 15.8% Fairly good 2 3.8% 2 4.7% 3 5.9% 7 4.8% Passable 1 2.3% 1 2.0% 2 1.4% Total 23 44.2% 21 48.8% 22 43.1% 66 45.2% Source : Pragma Evaluation Team August, 2015 Table 8.2 shows that the youth reactions to the Entrepreneurship module were somewhat less favorable than those 27 regarding the Life Skills module but were generally positive for both sexes. Female respondents rated the module more favorably than males, however. Twelve percent of the young women rated it as “Very good” whereas 8.5 per￾cent of young men gave it this rating. In the “Good” category, males were more favorable than females: 28.4 percent compared with 21.3 percent for females. On the other hand, negative scores were more numerous among the young men than the young women. Nearly six and a half percent of the young men gave the module a rating of “Passable” or “Poor’ compared with 3.5 percent among young women, none of whom rated it as “Poor”. Table 8.2 : Youth Evaluation of the Quality of the JA Training Program Modules Entrepreneurship Total Dakar Thies Saint Louis N % N % N % N % Male Re￾spondents Very good 4 7.7% 1 2.5% 7 14.3% 12 8.5% Good 19 36.5% 6 15.0% 15 30.6% 40 28.4% Fairly good 2 3.8% 9 22.5% 5 10.2% 16 11.3% Passable 4 7.7% 3 7.5% 1 2.0% 8 5.7% Poor 1 2.5% 1 .7% Total 29 55,8% 20 50.0% 28 57.1% 77 54.6% Female Re￾spondents Very good 6 11.5% 10 25.0% 1 2.0% 17 12.1% Good 13 25.0% 8 20.0% 9 18.4% 30 21.3% Fairly good 4 7.7% 1 2.5% 7 14.3% 12 8.5% Passable 1 2.5% 4 8.2% 5 3.5% Total 23 44,2% 20 50.0% 21 42.9% 64 45.4% Source : Pragma Evaluation Team August, 2015 As for the Technical Training module (Table 8.3), female respondents were somewhat more positive than males in their evaluation although both sexes rated the module much less favorably than the Life Skills and Entrepreneurship modules. Nine percent of the young women scored the Technical Skills module as “Very good” while only 3.4 per￾cent of males did so. The “Good” scores were very similar, however (11.8 percent for young men and 10.1 percent for young women). A large gap was evident in the “Passable” category, with 25.2 percent of male respondents using this rating as opposed to 15.1 percent of their female counterparts. The young men were more severe than the young women in the “Mediocre” and “Poor” categories. The combined scores for young men were 10.1 percent compared to 3.4 percent for young women, none of whom used the “Poor” category. Youth Evaluations of the Trainers The Life skills trainers were rated as “Very good” by 45% of the respondents while 53% found the entrepreneurship trainers to be “Good.” Only 3% of the technical trainers were rated as “Very good”. The highest score for this group of trainers was “Passable”, a rating given by 31% of the respondents. These findings echo the comments of the youth during the focus group discussions. The Entrepreneurship module produced gains in areas such as developing a busi￾ness plan and a marketing strategy. Financial management, however, was insufficient, according to the KIIs and FGDs. Table 8.3 : Youth Evaluation of the Quality of the JA Training Program Modules Technical Training Total Dakar Thies Saint Louis N % N % N % N % Male Re￾spondents Very good 2 3.8% 1 2.4% 1 3.8% 4 3.4% Good 7 13.5% 2 4.9% 5 19.2% 14 11.8% Fairly good 4 7.7% 2 4.9% 6 5.0% Passable 11 21.2% 12 29.3% 7 26.9% 30 25.2% Mediocre 2 4.9% 3 11.5% 5 4.2% Poor 5 9.6% 2 4.9% 7 5.9% Total 29 55.8% 21 51.2% 16 61.5% 66 55.5% Female Re￾spondents Very good 2 3.8% 9 22.0% 11 9.2% Good 7 13.5% 3 7.3% 2 7.7% 12 10.1% Fairly good 5 9.6% 1 2.4% 2 7.7% 8 6.7% Passable 6 11.5% 6 14.6% 6 23.1% 18 15.1% Mediocre 3 5.8% 1 2.4% 4 3.4% Total 23 44.2% 20 48.8% 10 38.5% 53 44.5% Source : Pragma Evaluation Team August, 2015 28 When asked about the technical training module, the youth in the KIIs and FGDs said that many of the trainers were mediocre and not always well prepared. The youth in the production cohorts in the Dakar region were particularly disappointed with the lack of field activity in this module. Table 9: Youth Evaluation of the Trainers in the JA Project Trainers/Modules Total Dakar Thiès Saint Louis N % N % N % N % Life Skills Very good 25 45% 24 52% 22 39% 71 45% Good 21 38% 15 33% 25 43% 61 38% Fairly good 8 15% 6 13% 8 14% 22 14% Passable 1 2% 1 2% 2 4% 4 3% Total 55 100% 46 100% 57 100% 158 100% Entrepreneurship Very good 6 11% 5 10% 12 21% 23 14% Good 33 61% 24 51% 28 49% 85 53% Fairly good 12 22% 14 29% 15 26% 41 25% Passable 3 6% 5 10% 2 4% 10 6% Total 54 100% 48 100% 57 100% 159 100% Technical Training Very good 1 2% 2 5% 1 3% 4 3% Good 12 22% 10 23% 16 52% 38 29% Fairly good 15 28% 8 18% 5 16% 28 22% Passable 15 28% 18 41% 7 23% 40 31% Mediocre 6 11% 5 11% 2 6% 13 10% Poor 5 9% 1 2% 6 5% Total 54 100% 44 100% 31 100% 129 100% Source : Pragma Evaluation Team, August, 2015 Finally, when asked to rate the project’s technical partners, the Thiès Chamber of Commerce received the highest rating (46% of the youth who had training there rated it as “Very good”). The next best score in the “Very good” category was Exode Urbain, in the Dakar Region where 22% of the respondents in that region rated it at that level. “Good” scores were given to RENCJES, the Thiès Chamber of Commerce and ASESCAW by 35%, 31% and 21% of the concerned respondents. Otherwise, “Very good” and “Good” scores were given by fewer than 20% of respond￾ents overall. Table 10: Youth Evaluation of the Project Partners Partners Evaluation of the Trainers Provided by the Project Partners Total Very Good Good Mediocre Poor N % N % N % N % N % Dakar Partners RENCJES 6 13% 16 35% 1 2% 23 50% Exode urbain 10 22% 11 24% 1 2% 22 48% ITA 1 2% 1 2% Total 16 35% 28 61% 1 2% 1 2% 46 100% Thiès Partners University of Thiès 1 3% 1 3% Chamber of Commerce 18 46% 12 31% 2 5% 3 8% 35 90% ENSA 1 3% 2 5% 3 8% Total 18 46% 13 33% 5 13% 3 8% 39 100% Saint Louis Partners CIH 6 14% 8 19% 3 7% 1 2% 18 43% CRREJ 2 5% 1 2% 1 2% 4 10% Université Gaston Ber￾ger 1 2% 1 2% 2 5% ASESCAW 4 10% 9 21% 4 10% 17 40% AfricaRice 1 2% 1 2% Total 12 29% 19 45% 8 19% 3 7% 42 100% Source : Pragma Evaluation Team August, 2015 During the KIIs and FGDs, many participants expressed disappointment that the project did not give them a certifi￾cate or diploma at the end of training. Senegal is a very diploma-conscious country and the lack of certification of the training by a recognized organization, such as the Ministry of Youth, is a handicap to youth efforts to find funding or salaried employment. 29 Overall Cost-Effectiveness: The JA program was implemented for a total cost of just over $1.0 million, according to figures provided to the Eval￾uation Team by IYF. The format of costs provided to the Evaluation Team by IYF is as follows29:   Table 11: IYF YouthMap Project Costs Description Amount ($) 1. Personnel 247,454 2. Grants 508,939 3. Travel 71,141 4. Indirect 173,782 Total Expenses 1,001,316 Regarding local costs, several KIIs with implementing partners suggested that certain elements of the project, such as two-day youth retreats held at hotels or host institutions during the Life Skills module, cost more than they were worth. Although the youth found the retreats to be very valuable experiences, it is uncertain whether the costs of the retreats can be covered by the organizations that have agreed to continue the JA training unless they are able to raise extra funds. On the other hand, certain activities such as the operating costs of the regional coordinating com￾mittees were not budgeted. Overall, as discussed earlier, the costs of the project seem high in relation to comparative experience with other programs of this nature in relation to costs per job generated, while the cost/benefit analysis indicated a very low net financial return to project activities. According to a 2014 survey carried out by program sub-implementer Synapse, 213 out of a total of 288 program par￾ticipants who completed the JA training program found employment by the time the program had ended. This represents about 74% of the program participants who completed the JA training program. By way of comparison, total youth employment for Senegal as reported by the World Bank/ILO averaged approximately 15% during the 2010-14 period. It is of course difficult to directly compare these figures due to differences in employment outcomes by region, educational level, and other socio-demographic factors; as well as the fact that the JA Program is designed to target needy rural job seekers. At the same time relevant past donor labor market research for Senegal indicates that unemployment rates in urban centers have at times been more than triple that found in rural areas. This is largely due to the high attachment of rural youth to seasonal employment in the agricultural sector, as well as the “pull-effect” of relatively high public sector wages on the urban youth labor market. The study thus finds that youth unemployment rates in urban areas are more than triple those in rural areas.30 Overall the comparative data on youth unemployment rates in Senegal and employment placement rates under JA should be interpreted with care. Nonetheless the data provides useful comparative information about placement rates in comparison with those found in the national youth labor market. In addition the average cost per employment position created under the program was approximately $4,701 per job. This cost was compared against a network of 11 recent (implemented over the last decade) USAID-supported em￾ployment creation and workforce development programs obtained through targeted desk research. A summary table on these studies is included in Annex 5). The Evaluation Team found that for these programs the average cost per job created ranged from a low of $1,100 to a high of approximately $12,000; with the vast majority of the programs falling in the range of $1,200—5,000. Most of these programs provided more intensive training support than the JA program, with in general a greater focus on technical training, and many also included enterprise-level technical sup￾port as well). Within this overall context, the per capita employment generation cost for the JA program seems relatively high in relation to the majority of comparator programs; most of which incorproate a signifcant job training and employment outreach focus. Intuitively, within the context of a country with a per capita income in 2014-15 of approximately $1,100, this would appear to represent a relatively high cost per job leveraged in terms of opportunity cost.                                                              29 The following table and the description of the list of costs correspond to Sub-question 5a: What are the Activity Costs? 30 “Understanding Youth Employment Outcomes in Senegal,” World Bank, ILO, UNESCO, 2007. 30 Sub-question 1a: How useful have internal assessments by USAID, IYF and the Synapse Center been for designing and/or improving training/employment activities? Findings: USAID and IYF had little direct involvement with the design of the JA project. Under contract with IYF, Synapse Cen￾ter designed the project, based on the profile of the youth and their aspirations. The Senegalese Ministry of Agricul￾ture was not consulted during the design phase although some of its training institutions like ENSA hosted the train￾ing of some of the cultivation cohorts of youth trainees. One lesson learned is that the Ministry’s technical training services should have been involved in designing and/or implementing the technical training on cultivation and harvest￾ing, which proved to be the weakest part of the JA project. Two private food-processing companies in Dakar (Maison du Consommer Sénégalais and Mamelles Jaboot were involved in the project as hosts for site visits and retreats. The surveyed youth found these experiences to be very valuable but it is unclear whether these companies were involved in the project design. Available JA project documentation does not indicate how or whether it was based on similar initiatives in Senegal or other countries. There was not a well-developed project monitoring and evaluation system. Synapse sent brief (7 or 8-page) quarterly reports to IYF. They are mostly lists of activities accomplished or pending. The research team did not find evidence that formal internal assessment led to improving the training content or management. Problems such as the collapse of the regional coordination committees were not documented or remedied. Difficulties in train￾ing youth with very mixed levels of education became apparent in the first year of operations but neither IYF nor Synapse Center intervened to modify the recruiting procedures used by the implementing partners. The Youth-Map Senegal Monitoring Table (Annex 4) was used as the effective project monitoring framework rather than the PMP and the Results Framework.31 In the case of the latter document, types of outputs were specified but not numbers or percentages. There is no clear indication of how the Monitoring Table has been used to measure results or correct problems that were detected during the course of the program, such as the quality and effectiveness of the technical skills module or access to financing for business plans created by the youth32. Although these gaps were evident in 2013, it does not appear that measures were taken to correct them in 2014. Sub-question 1b: How were youth qualifications and corresponding employer characteristics matched? Findings: The focus of the JA project was on developing entrepreneurship rather than salaried employment. Therefore, there were no activities directly related to placing youth with employers. The documents provided to the Evaluation Team do not describe how or why this decision was made. However, the Director of Synapse Center informed the team that in his view there are limited opportunities for salaried employment in the Senegalese agribusiness sector. In addition, in his view, jobs are likely to be poorly paid and offer little opportunity for advancement. Overall, Synapse Center felt that its area of strength was in self-employment and that the resources provided from the grant from IYF would not cover a program with two focus areas. As a result, the determination was made to focus exclusively on entrepreneurship. Evaluation Question 2: What was the activity’s overall impact on youth beneficiaries? What benefits did the targeted youth receive from participating across each of the training components? Sub-question 2a: Employment Creation and Net Income Impact Findings and Conclusions: According to data collected by Synapse Center at the end of the training in 2014, of the 288 participants who had completed the training program, 213 were self-employed or had found employment (74%). As noted above in our analysis of cost-effectiveness, this compares to about a 15% rate of unemployment among Senegalese Youth overall during the 2010-14 period. Overall this would appear to indicate adequate but not very impressive employment crea￾tion performance by the project in relation to economy-wide job placement performance for youthful job seekers (taking into account as referenced earlier that a direct comparison with national level youth employment figures                                                              31 USAID Senegal was aware of this change. The Monitoring Table is also referenced in the April 2013 Youth-Map Semi-Annual Report. 32 Synapse Center had a competition for well-designed business plans and gave cash prizes for the best ones. 31 should be interpreted cautiously). Within the participant group, 175 had created their own enterprises, 38 were in salaried employment, 13 had gone back to their studies and 62 were unemployed. In other words, 74% or nearly three-quarters of the participants who completed the training were employed in some form or another. It appears that many of the cases of post-training self-employment are extensions of the previous self-employment reported on the questionnaires and KIIs with the participants. During visits to the sites of participant enterprises (3 in the Dakar Region, 3 in Thiès and 1 in St. Louis33), the Evaluation Team learned that the participants sometimes used either prize money provided by the JA project or funds raised through small family loans to “spin off” an enterprise from a family enterprise. Table 12 : Employment creation by region Region Self-employed Salaried Return to studies Unemployed Dakar 45 11 0 15 Thiès 69 9 11 20 Saint-Louis 61 18 2 27 Total 175 38 13 62 According to the KIIs and FGDs with the youth, as well as with the project partners, the lack of access to finance was cited as a critical constraint experienced in starting a business. In meetings with the Centre Régional de Ressources pour l’Emploi des Jeunes (CRREJ), a youth-serving government body in St. Louis, the Evaluation Team learned that this body is formally authorized to help youth to formulate and budget business plans. Such plans will be automatically accepted by the Crédit Mutuel Sénégalais for loans at 7% with no collateral. The CRREJ was an implementing partner of the JA project, but there was no follow-up on obtaining financing for the youth enterprise plans. Overall the lack of a coherent strategy for addressing the access to finance constraint facing participants was a critical constraint on the robust and sustainable impact on Business Plan implementation and income growth through the program. Net Impact on Income and Cost/Benefit Outcomes The Evaluation Team surveyed a total of 174 respondents.34 Of these, 94 respondents were identified who reported income at the beginning and/or end of the program. Those that reported zero income at both the pre and post￾program stages were in general regarded as exhibiting either unemployment at both stages or a high likelihood of reporting error, reflecting the sensitivity attached to earnings/income reporting for many respondents; and so were excluded from the income analysis work undertaken by the Evaluation Team. If anything this would tend as refer￾enced earlier to impart an upward bias to the results, as it would exclude from the analysis any participants who ac￾tually experienced either no income gain or a net income loss after participating in the program (e.g. if in fact they became unemployed after completing the training sessions). An analysis of variance for self-employment and wage earnings reported before and after the program was undertaken for the 94 referenced program participants. The re￾sults are reported in Table 12. They are provided for the group overall and are also broken down by sex, for 32 fe￾male respondents and 59 male respondents. For the remaining three program participants, the Evaluation Team was not able to identify whether they were male or female; these three program participants are excluded from the “to￾tal, male and female” analysis. Table 13: Self-Employment and Wage Earnings Pre- and Post-Program Group Sample Size U.S. Dollar Average of An￾nual Baseline Earnings Average of An￾nual Endline Earnings Average of Difference in Earnings Std Dev of Dif￾ference in Earnings Female 32 400 1,072 639 1,071 Male 59 459 1,020 561 647                                                              33 Heavy rain made it impossible to visit more than one youth employment initiative in the St. Louis region, as the enterprises were mainly rice or vegetable production in areas distant from paved roads. 34All 288 graduates of the JA training program were invited to participate in the interviews. However, 174 participants actually appeared at the regional interview settings and participated in KII’s. Of these 94 provided information on before and after-program earnings which the Evalua￾tion Team determined to reflect a reasonable level of accuracy (as explained further in the text). The employment information was generated from the Synapse data base, which covered all 288 program participants. Unfortunately this data base did not contain systematic earnings infor￾mation for the participants, and so could not be utilized for purposes of cost-benefit and income impact analyses. 32 Total, male and female 91 438 1,038 588 816 All respondents with non-zero pre- and post-program income 94 416 1,042 626 817 Overall, self-reported average annual earnings for those participants who both responded to the survey and reported income either pre- or post-program increased by about $626 (150%). This is likely a result of youth moving from un￾employment to employment, rather than increases in income associated with movement from one type of employ￾ment to another. For the 32 female participants included in the analysis, the average difference in annual wages was $639; while for the 59 male participants included it was approximately $561. These numbers seem to indicate some “push” impact from the program on average participant incomes. There was a significantly significant increase in mean earnings at a 90% confidence level. At the same time it is worth noting that for all the samples the standard deviation exceeds the pre- and post-program average income differential. This could in turn in part reflect the relatively small sample size (particularly for the sex--specific sub-groups), as well as the heterogeneous nature of the program partici￾pants. The Evaluation Team also carried out a cost/benefit analysis for the sample of 94 participants referenced above, utiliz￾ing the following assumptions: • Costs were based on the total program cost figure of $1,001,316, scaled down in proportion to the number of beneficiaries included in the sample, and applied over the 2013-2014 period; • Income benefits for year 1 were the incremental pre and post-program income stream for each participant, which were then adjusted upward by 5% annual during the remaining 29 years for which program related benefits were counted for a total of 30 years of projected beenfits, which is reasonably standard for a lifetime incremental income projection exercise of this nature). • A discount rate of 10% was applied to the stream of costs and benefits (absolute minimum discount rate to be applied according to World Bank standards).35 The results indicated that the NPV for the entire sample at a 10% discount rate was approximately $288,160 and the internal rate of return was about 7%. This indicates that the project as designed/implemented did not generate a fi￾nancial rate of return that would normally be considered acceptable for a donor employment/training program (as noted earlier, a 10-12% rate of return is normally considered the minimum acceptable rate for a donor-supported project).36 In addition the cost-benefit exercise was carried out for the larger sample of 174 beneficiaries, which as noted earlier included a large number which did not report income before or after the program (which the team feels is largely traceable to reluctance to accurately report income sources/levels). The cost figures which were in￾cluded were correspondingly scaled down to reflect the fact that only one-third of the program beneficiaries were included in this calculation. The results for this larger group yielded an IRR of 0.4%, effectively underscoring that the conservative data inclusion approach applied by the evaluation team in no way understated program impact – if any￾thing it had the opposite effect. At the same time, it is worth noting that by definition cost/benefit analysis does not capture non-income benefits. Programmatic costs appear to be somewhat high in relation to the aggregate income generation benefits, particularly given the size of the beneficiary population. This in turn generates critical questions regarding key program design and implementation parameters of the program. First, the program overall seems to have targeted a small number of beneficiaries in relation to the volume of funding being provided (roughly about 320 total beneficiaries originally envisaged for a budget of just over $1 million). That is approx. $3,130 per beneficiary as originally planned – a comparatively high cost per capita for an employment generation/training initiative. This is particularly true within the context of a country with per capita income about one-sixth of that level. In addition, once we take into account the inevitable “wastage” rate in terms of participants who either dropped out or were unable to establish businesses or (in those few cases where the trainees were interested in pursuing wage employment) find salaried employment, it was virtually inevitable that the costs per job were going to become very high by international comparative standards. In this regard, the program’s focus on life skills productivity and incomes and/or developing agribusiness units. At the                                                              35 The actual calculations carried out for the cost-benefit analysis are included in Annex 6. 36 The World Bank, Handbook on Economic Analysis of Investment Operations, 1998, Technical Appendix, paragraph 20 states “The Bank tradition￾ally has not calculated a discount rate, but has used 10-12% as a notional figure for evaluating Bank-financed projects.” 33 same time, as noted earlier, the technical training provided may not have been sufficiently focused across the range of key technical, organizational, agri-input supply, basic financial planning/management, and market linkage/outreach interventions typically required to improve farm and agribusiness income and employment generation performance on a sustainable basis. Moreover, such interventions are typically implemented within a broader context of targeted technical assistance, development finance institutional linkages and agribusiness value chain institutional strengthening support interven￾tions. These are categories of programmatic support that virtually by definition the JA Program could never have been expected to effectively develop and implement. In some sense, there are elements of the program that seem more akin to an urban/semi-urban workforce development/job placement program. These have been applied instead to an agriculture/agribusiness development context that may have required a much more integrated and targeted set of intervention tools/mechanisms to have yielded a tangible and sustainable impact. These conclusions appear to flow clearly from the cost/benefit analysis and per capita cost exercises, and are also amply reinforced by the inductive findings from the FGD and KII interview data. Evaluation Question 3: To what extent was the capacity of various activity partners strengthened through JA and what were the outcomes of these efforts? Findings: For the most part, the JA project focused on training trainers among the staff in the activity partners. The staff who benefitted from the ToT training went on to conduct the training on life skills and entrepreneurship for the JA pro￾ject youth. The technical trainers were mostly independent consultants rather than staff members of the activity part￾ners. During the field research in Senegal the evaluators learned that37 five of the 12 activity (implementing) partners had formally agreed to integrate the IYF modules on life skills and entrepreneurship in their respective curricula. These partners are: • RENCJES (National Network of Young Entrepreneurs’ Clubs of Senegal) • CNAFP (Centre National d’Appui à la Formation Professionnelle) • The Chamber of Commerce and Centre National d’Appui à la Formation Professionnelle et Technique (CNFPT) of Thiès • The Centre d’Initiation à l’Horticulture (CIH) in St. Louis • ASESCAW at Ross Bethio (St. Louis Region) In addition, the Centre d’Initiation à la Horticulture (CIH) benefitted from the donation of an experimental plot of land by Synapse Center and is continuing to offer post-training support to the JA youth in the St. Louis peri-urban area. The overall lack of coordination and follow-up to the JA training appears to be largely due to the collapse of the regional coordinating committees set up in each region. As a result, Synapse was not able to respond rapidly to prob￾lems related to how the numerous implementation partners were (or were not) carrying out their various tasks. Ul￾timately, the project was not able to materially and sustainably strengthen the institutional capacity of a network of partners to effectively provide these services. When interviewed by the Evaluation Team, the members of the coor￾dinating committees said that there was a lack of funding from Synapse Center for their activities (meetings and site visits to youth projects) and the committees, therefore, ceased functioning within the first year of the project.38 Other members said that their services were not properly used by the project. For example, during a key informant interview with a member of the evaluation team, the General Secretary of the Chamber of Commerce of St. Louis said that her office had proposed a “business incubator” model to the JA project to provide post-training follow-up to the youth initiatives but that this suggestion was not followed. According to Synapse Center the rent for the space needed for the activity was too high, and the proposal was not couched as a “business incubator” initiative.                                                              37 In late 2015 Synapse Center informed the evaluators that the five listed partners are JA anchor partners and are still involved in the training of additional youth through Synapse Center’s Challenge program. 38 Synapse Center said that these expenditures had not been budgeted when the JA project was designed and launched. 34 Evaluation Question 4: What are identified trends in youth employment data with respect to sex, sectors of interest, employee retention, and employment quality (e.g., part-time/full-time, salary, hours, location, and promotions)? Similar questions will be asked regarding youth in self-employment. Findings: When responding to the pre-test designed and administered by Synapse Center in 2013, 185 individuals, or 58% of the youth who completed the training said that they already had some experience in the field of agribusiness. In other words, they had already been asked to participate in or develop income-generating activities. In most cases, these initiatives had been on family farms or in informal family enterprises. When asked what field they would like to develop as entrepreneurs, the most sought-after fields were crop production (67% of responses), marketing (52% of responses) and food processing (34% of responses) as shown in the table below. Synapse Center designed the train￾ing program with these youth goals in mind. Table 13: Youth Aspirations In what field would you like to be an entrepreneur? (more than one choice was possible) Percent Total responses Production 75% 215 Storage 8% 23 Transportation 3% 8 Food processing 37% 107 Marketing 57% 165 Other 2% 7 Total of respondents 288 Source: Synapse Center Youth Survey The respondents were fairly evenly divided among the three project regions. Male respondents were more numerous than their female counterparts, in proportions similar to the sex division of the original 288 youth trainees. One ex￾planation of this division was the predominance of youth in rice cultivation in the Saint Louis Region, where rice growing is traditionally a male occupation. In Thiès, some participants were recruited from the Ecole Nationale Supé￾rieure d’Agronomie (ENSA), an institution with a predominately male student population. Male and female respondents had similar profiles in terms of post-project salaried work or self-employment. Both sexes were three to four times likelier to be self-employed than salaried. In the Thiès Region, self-employment was seven to nine times more common than salaried employment. Table 14: Employment Status of JA Respondents by Region and Sex (Post- project) Sex of respondents Total Male Female N % N % N % Total Employed Dakar Self Employed 30 53% 15 27% 45 80% Salaried 7 14% 4 7% 11 20% Total 37 66% 19 34% 56 100% Thiès Self Employed 48 62% 21 26% 69 88% Salaried 7 9% 2 3% 9 12% Total 55 71% 23 29% 78 100% Saint Louis Self Employed 34 43% 27 34% 61 77% Salaried 11 14% 7 9% 18 23% Total 45 57% 34 43% 79 100% All regions 137 76 213 Source: Synapse Center, September 2015 35 Youth in all three locations had similar combined education levels in terms of secondary and higher education with 88% for Thiès, 93% for Dakar, and 87% for Saint Louis. However, Thiès had significantly more participants with higher education (50%) than Dakar (30%) and Saint Louis (14%) had. Table 15: Employment Status of JA Respondents by Region and Educational Attainment (Post￾Project) Educational Attainment (partial or complete) Total Less primary Koranic/Primary Secondary Higher Education N % N % N % N % N % Dakar Self empl. 0 0% 3 5% 27 48% 15 27% 45 80% Salaried 0 0% 1 2% 8 14% 2 4% 11 20% Total 0 0% 4 7% 35 63% 17 30% 56 100% Thiès Self empl. 0 0% 9 12% 27 35% 33 42% 69 88% Salaried 0 0% 0 0% 3 4% 6 8% 9 12% Total 0 0% 9 12% 30 38% 39 50% 78 100% Saint Louis Self empl. 1 1% 7 9% 46 58% 7 9% 61 77% Salaried 0 0% 2 3% 12 15% 4 5% 18 23% Total 1 1% 9 11% 58 73% 11 14% 79 100% All regions 1 22 123 67 213 Source: Synapse Center, September 2015 In relation to employment retention/employment quality trends, there is very little that can be effectively inferred since only 24 participants at the start of the program and only 38 participants after completion of the program reported any type of salaried employment (and in many instances these appeared to be part-time positions). In this regard the shift in program focus from an early stage to one which was focused almost entirely on self-employment opportunities in the agricultural sector has obviated the relevance of this analysis, as well as eliminating the possibility of generating a sample size through which reasonably rigorous results could be generated. Since in the end most of the non-self employment positions both before and after program particpation were part-time in nature and covered a very small portion of the particpant group, it was determined that detailed tracking of outcomes for this limited and un-targeted sub-group would not yield relevant or statistically valid/interpretable results. Evaluation Questions on Program Sustainability and Scalability Evaluation Question 5: What is the likelihood of sustainability of program activities? Sub-question 5b: What are the factors facilitating or inhibiting the sustainability of activity compo￾nents beyond the JA (e.g. regional councils, partnership formation, etc.)? Findings: Although Synapse Center prepared a sustainability plan for the project the overall sustainability of the JA program activities is relatively weak owing to a lack of a permanent institutional base for the initiative. The program was organized in an ad-hoc manner, with components hosted by or contracted to a variety of partners. Moreover, the fact that relatively few implementing partners have agreed to integrate the JA training in their own activities implies that many of the program activities are unlikely to continue with the exception of the five partners participating in Synapse Center’s Challenge project. The agriculture/technical training activities are the most vulnerable since they were organized by external trainers rather than staff of the host institutions. These trainers were paid by the JA pro￾ject budget and there is no documented indication that the implementing partners will be able to support their costs. The curricula for the cultivation, food processing and marketing training are the property of the trainers and were not handed over to the host implementing partners of the JA training. On the other hand, the fact that staff of the implementing partners were trained on the life skills and entrepreneurship module increases the likelihood that these aspects of the training will continue. 36 Question 6: Considering the financial and institutional context, which activity components can be scaled up and which stakeholders are best placed to support and are actively interested in scaling-up? Findings: An expanded or future JA project needs a firmer institutional framework. The project suffered from a plethora of poorly coordinated partners. On the one hand, the coordinating committees mobilized to support the training process in each region and to provide follow-up support in obtaining financing for youth employment initiatives, withdrew their support because they were expected to act voluntarily, without cost reimbursement from Synapse Center. On the other hand, the implementing partners (with a few exceptions) did not integrate the training process in their activities. ENSA in Thies is a notable exception in that it found the JA approach to agricultural self-employ￾ment to be innovative and appropriate and plans to integrate this approach in its formal curricula. Evaluation Question 7: How did partnership formation support the building of local ownership and en￾courage innovation? Findings: Partnership formation was a relatively weak aspect of the JA project. Over 20 organizations were involved in coordi￾nating and implementing the management of the project. The coordinating partners formed committees in each re￾gion in order to guide and support the training process. However, none of these committees lasted beyond the first year of the project. The KIIs with the partner representatives revealed that these organizations had asked for reimbursement of travel and meeting costs (even at the local level) but these costs had not been budgeted and Synapse was unable to cover them. It was either not made clear to these institutions that their services were to be provided pro bono, or they were insufficiently sensitized about the philosophy and objectives of the JA project. Since Synapse Center was unable to find additional funds, the coordinating committees ceased to meet. Eleven of the coordinating partners also served as implementing partners. However, the apparent overlap of func￾tions seems to have created uncertain regarding project implementation responsibilities, rather than helping promote long-run commitment to developing/maintaining follow-on programs. Two of the partners (CIH in St. Louis and the CNAFP in Thiès) informed the evaluators that their operational abilities were strengthened by the JA project. How￾ever, it is difficult to say how or whether other JA project partners will undertake further youth training initiatives without additional funding. Local ownership seems scattered and uncertain, given the need to fund many of the JA activities ranging from retreats to field trips, mentoring and inputs to training such as plots of land, kitchens and cooking appliances (for the food processing cohorts). As mentioned earlier, although five of the implementing part￾ners have continued the JA training with funding from a Synapse Center project, it remains highly uncertain if other partners will follow suit. KEY CONCLUSIONS • The JA Program exhibited a solid overall employment creation or job placement rates for program participants who completed the program, with 74% of respondents reporting some form of employment at the time of the survey. At the start of the program only 28 youth reported some form of employment (20 were self-employed and eight were salaried workers. • Self-reported average annual earnings for those participants who both responded to the survey and reported in￾come either pre- or post-program increased by about $626 (150%). This is likely a result of youth moving from unemployment to employment, rather than increases in income associated with movement from one type of em￾ployment to another. • According to survey results, participants increased their social and personal empowerment with a robust im￾provement in conflict management and professional ethics and modest gains in personal skills as a result of the program. Improvements were lowest for the applied technical skills related to the agricultural sector value chain. • According to cost-benefit calculations looking solely at reported earnings, the JA Program generated an internal rate of return on project financial costs (the excess of quantifiable benefits of the project over USAID’s financial costs) of approximately 7%, which is below commonly accepted minimum rate of return standards for donor programs (the World Bank minimum standard is 10-12%). This implies that the overhead costs for the program, 37 which comprise about half of total program costs, were relatively high in relation to the direct income benefits generated for program participants. • The Program developed and provided rigorous and highly effective training curricula in the life-skills and entre￾preneurship areas. The materials and training provided in critical technical areas – particularly related to agricul￾tural and agribusiness training - were significantly less effective, according to feedback from the youth interviewed during the field research. • The feedback loop between implementation and JA Program management appears to have been too tenuous to facilitate training program adjustments which could have strengthened key technical components of the training program where problems arose. • Lack of progress in facilitating financial access for program participants under the JA program limited its overall impact on participant incomes. • The complex network of institutional partnerships through which the JA Program operated made it difficult to promote accountability and shared expectations among partners, and to maintain uniform performance standards across JA training programs. • Inadequate trainer compensation policies appear to have significantly hampered the quality of the technical train￾ing activities. • The sustainability of JA activities is open to considerable doubt. In large part this is caused by the lack of clarity among partner institutions regarding the key elements of the sustainability strategy behind the JA Program, and the role of each partner in executing that strategy. RECOMMENDATIONS This section presents the recommendations for each of the research questions and sub-questions whose finding and conclusions are presented above. Evaluation Questions on Program Effectiveness and Impact Evaluation Question 1: How effective were the approaches taken for each of the four program compo￾nents in achieving the activity’s goals? • First and foremost, a similar project should include ICT training. Without it, youth are faced with many difficulties ranging from a lack of information about markets for their products and services to constraints to bookkeeping and financial management. • In the area of entrepreneurship, youth responses to questions asked during the FGDs demonstrated that financial management was a weak point in this training module. A renewed training program must revisit the time and re￾sources allocated to training on financial management. • Moreover, and most importantly, a much stronger program of technical training is required in order for beneficiary youth to acquire the skills needed to achieve increased crop production, improved methods of food processing and effective methods of marketing. The questionnaire results, KIIs and the focus group discussions all emphasized the weakness of the technical training and the urgent need for improvement in this area. Specific rec￾ommendations include: o Pay the trainers more attractive fees. Synapse Center paid the trainers 8,000 CFA francs an hour, which is one-third of the going rate for the best national trainers. The low fees were probably a reason some trainers were absent or shifted their training sessions at the last moment. o Provide more supervision to ensure that the trainers use suitable training methods such as projectors to show PowerPoint presentations, interactive discussions with the trainees and more practical skill-building, particularly in experimental plots (production cohorts). o Require the trainers to evaluate the youth’s learning outcomes, citing both gaps and achievements. • Ensure that the youth benefit from mentoring after the training process. Among the implementing partners, only the CIH of St. Louis is currently providing post-training/mentoring support to former youth trainees. (Five other implementing partners continue to provide JA training with funding from a separate Synapse Center project. • Ensure that overhead costs are kept moderate. The overhead costs incurred by IYF in supervising the JA project appear to have been significantly greater than the value of the $508,000 grant resources that Synapse Center used to cover all local costs. • Finally, it is critically important that PMP indicators be outcome-focused to a greater degree. Almost all of the indicators and related targets in the final monitoring plan were input/output-based; thereby limiting the focus on concrete impact-related program results. Also, there were no indicators that related to cost-effectiveness or 38 related to the net income impact of program outcomes – these should be added to future programmatic initia￾tives. Sub-question 1a: How useful have internal assessments by USAID, IYF and the Synapse Center been for designing and/or improving training/employment activities? • Quarterly and semi-annual reports should provide more quantitative data as appropriate to track progress and changes toward reaching objectives, including lessons learned from each training cycle. Sub-question 1b: How were youth qualifications and corresponding employer characteristics matched? There are no recommendations for this research sub-question since the JA project was focused entirely on self-em￾ployment. The youth who reported salary income had essentially found their jobs on their own. Evaluation Question 2: What was the activity’s overall impact on youth beneficiaries? What benefits did the targeted youth receive from participating across each of the training components? • The Evaluation Team recommends that future initiatives on training for youth employment develop a less com￾plex recruitment process and divide the youth by educational levels. Clearer criteria for recruiting youth are needed. • The life skills module is of high quality and should be maintained. The entrepreneurship module needs strength￾ening in the area of financial management. In reviewing the training materials, evaluators noticed that there was little reference to Senegal or agriculture in the case studies. These omissions need to be corrected in future edi￾tions. • If a similar initiative to the JA project is designed, the technical training module will need to be thoroughly re￾viewed for the pertinence and quality of instruction. The host institutions should be responsible for supervising the trainers and providing quality control of the instruction. The youth evaluation of this module should also be used to help redesign the module. • The beneficiary youth should receive personal copies of the training materials. According to findings from the questionnaires and KIIs with youth, there were instances when the youth did not keep copies of the training ma￾terials after the end of their course. For those with limited competence in French, Wolof versions are needed. Senegal has a well-developed system of literacy materials in national languages, including materials on key aspects of agriculture, used by the government extension services for training farmers. • Finally, a completion-of-training certificate or diploma should be given to participants who successfully complete the training. Sub-question 2a: Employment Creation and Net Income Impact • Overall the project was not particularly cost-effective in relation to comparative per capita job creation costs of similar programs internationally. Nor did it generate an acceptable internal rate of return in relation to normal “cut-off” standards. Taken together with the results of the surveys and the FGD and KII discussions, this implies that the targeting and execution of key program training elements – both life skills training and technical training – were inadequate to generate robust employment/income generation results in relation to per capita program costs. This in turn indicates that in the future, the design of similar training programs will need to be seriously re￾shaped as indicated above prior to any attempted roll-out of the program. • It may well be worth rethinking whether this type of program design is appropriately suited for providing tar￾geted technical/training support for agricultural development focused initiatives. There is typically a broad range of technical/training and material support needs encountered/addressed under donor-supported agricultural/agri￾business development programs. These frequently range from enhanced cultivation practices, to improved farm budget estimation/management, to agro-input supply network development, to marketing support, to financial access development/support for agricultural/agribusiness client groups. The results of the evaluation indicate that it is very much an open question whether a workforce development/entrepreneurship initiative, which almost definitionally is not set up to address in depth the range of critical institution-building constraints which typically need to be impacted under an agricultural/agribusiness livelihoods program, can/should be relied upon as a primary service delivery mechanism for a program focused on enhancing agricultural employment and incomes. AFR/SD and the Senegal Mission may want to consider whether a more systematic farm unit and/or value chain level intervention approach may be required to generate the agriculture/agribusiness entrepreneurship impact 39 being sought through this type of program initiative. Evaluation Question 3: To what extent was the capacity of various activity partners strengthened through JA and what were the outcomes of these efforts? • A renewed JA project (or a similar initiative) that includes a network of local implementation partners needs to operate with greater accountability to its beneficiaries and partners. • As a fund provider or contractor of services, a follow-on project would need to prepare clear memoranda of understanding or contracts that specify services and products that each partner is to provide within specific time frames. • The services to be provided by local implementing partners need to be incorporated into the budget. • If services are to be in-kind contributions to a project, this fact would need to be made clear in the contract or memorandum of understanding. • A new or renewed JA project would need to have a well-developed monitoring and evaluation system to ensure that corrective action is taken when problems are detected. For example, when the first cohorts of youth gave negative evaluations to the trainers in the technical skills module, it would have been appropriate to have taken rapid action to address the serious problems highlighted in the youth evaluations. Evaluation Question 4: What are identified trends in youth employment data with respect to sex, sec￾tors of interest, employee retention, and employment quality (e.g., part-time/full-time, salary, hours, location, and promotions)? Similar questions will be asked regarding youth in self-employment Since the JA project was not concerned with salaried employment, the evaluation comments refer to self-employ￾ment issues. A future JA project or similar initiative should do a more detailed pre-project assessment in several ar￾eas: • Assess the demand for agricultural produce in the targeted regions and identify high-potential crops/products. • Use findings from the completed JA project to craft strategies catering to the needs and interests of young women, such as more training and post-training follow-up on food processing and marketing. • Evaluate constraints in food storage, food processing and marketing in different areas of the country. • Negotiate funding options for graduates of the training program and design training modules to prepare youth trainees to apply for funding for bankable projects at the end of or after the training. • The background and educational levels of the participants was too heterogeneous to apply uniform training mate￾rials effectively. Either the selection process needs to focus on a more homogeneously defined set of participants or a future project would need to prepare training and post-training services more effectively tailored to: o Youth with little formal education o Youth with post-secondary general education o Youth with post-secondary education in agronomy and related fields. • The latter approach would, of course, entail additional programmatic costs for what is already a high-cost pro￾gram in relation to core employment and income generation benefits. It would be important to determine appro￾priate approaches to lower the cost of the program. Evaluation Questions on Program Sustainability and Scalability Evaluation Question 5: What is the likelihood of sustainability of program activities? Sub-question 5a: Activity Costs • More rigorous budgeting and cost accounting may be necessary for future initiatives like the JA project. The lack of monitoring tools like an operational results framework made budget management difficult for Synapse Center and IYF and needs to be corrected in future initiatives. • Future JA-like initiatives that rely on a network of service providers for training, mentoring and guidance need to work from a Logical Framework that defines the roles, activities, outputs and outcomes of services provided by project partners. • Reviewing the results of the first JA project should inform the choice of service providers in a future project with similar objectives. The costs of their services should be calculated in a realistic fashion. 40 Sub-question 5b: What are the factors facilitating or inhibiting the sustainability of activity compo￾nents beyond the JA (e.g. regional councils, partnership formation, etc.)? • Five implementing partners have agreed to continue the JA training and are funded by a different Synapse Center project. However, it is unclear if they have the independent financial resources to do so once project funding fin￾ishes. These institutions have staff trained on life skills and entrepreneurial skills but only a few of them, like ENSA, have staff and facilities to replicate technical training in areas such as cultivation methods. • Synapse, which has proven expertise in life skills training, could provide trainers for that component of the pro￾gram should the need arise. Question 6: Considering the financial and institutional context, which activity components can be scaled up and which stakeholders are best placed to support and are actively interested in scaling-up? • The Evaluation Team feels that the Chambers of Commerce could provide the best institutional base for training and post-training follow-up. The Chambers of Commerce are also able to issue certificates or diplomas to gradu￾ates of their programs. • ENSA, the Ecole Nationale de Sciences Agronomes of Thiès, informed the Evaluation Team that it would like to introduce a JA-like entrepreneurship focus to its diploma program. This agricultural university, whose mission has been to train technicians and researchers for governmental institutions, believes it is important to train students for self-employment in agriculture. • The financing institutions focused on youth services should partner proactively with the “business incubator” methodology offered by the Chambers of Commerce to provide a solid platform for creating and funding youth enterprises. At the same time, future initiatives of this kind should partner with progressive commercial finance institutions to develop a greater understanding of the profitability associated with the provision of financial services to viable agriculture/agribusiness entrepreneurs and expanding commercial linkages with this client base. • At the international level, the CONFEDES (Confédération des Ministères de la Jeunesse et des Sports des Pays Ayant le Français en Partage)39 could be well-positioned to provide support for scaling up the achievements of the JA project. ILO, FAO and bilateral donors like USAID are also in a position to support future youth self-em￾ployment initiatives in the agribusiness value chain. Foundations like IYF and interested partners like Microsoft could also become engaged in supporting scaling-up strategies. Evaluation Question 7: How did partnership formation support the building of local ownership and en￾courage innovation? Apart from two implementing partners, there is little concrete evidence of local ownership and a commitment to encourage innovation. As mentioned earlier, ENSA in Thiès and CIH in St. Louis are perhaps the only organizations that are making plans for continuing JA-type training (ENSA) or are currently engaged in mentoring JA graduates (CIH). • Ownership and sustainability of youth employment initiatives in Senegal could best be piloted by key youth￾serving organizations. These include the following: o RENCJES (National Network of Young Entrepreneurs’ Clubs of Senegal), is an organization that could certify the training, provide post-training follow-up, and facilitate access to financing. o ANPEJ (National Agency for the Promotion of Youth Employment) is a branch of the Ministry of Youth and could provide policy support for encouraging youth self-employment in the agriculture sector. It is also linked with the Fonds de Financement de l’Entreprenariat Jeune with access to the Crédit Mutuel Sénégalais fund for youth employment. • The youth-serving organizations would need to develop cooperative links with business networks and related training through the regional chambers of commerce. • Future JA-type initiatives could include funding for prizes for innovative and sustainable partnerships. Similarly, rewards for outstanding partnerships could include study tours, further training and/or introduction to interna￾tional partners and services. Networks of Chambers of Commerce could play a catalytic role in such strategies.                                                              39 CONFEDES (the Confederation of Ministries of Youth and Sports of Countries Sharing the Use of French) is part of the inter￾national francophone movement and develops initiatives and cooperation among the ministries of youth and sports in member states. 41 KEY LESSONS LEARNED 1. Choosing the most suitable partners for a “JA2” project is critically important. The Senegalese universities are strike-prone and the extended student strikes at the Ecole Polytechnique of Thiès and the Gaston Berger Univer￾sity of St. Louis effectively shut down the JA training that was hosted on their campuses. In addition, trainings held at formal educational institutions may operate on the academic year schedule, rather than by the calendar year. This problem occurred at CNAFP in Thiès. 2. Chambers of Commerce may be more suitable training partners, as their mission is business promotion, and they operate throughout the calendar year. Committed agricultural training institutions like the HIC of St. Louis and ASESCAW of Ross Bethio (St. Louis Region) are flexible and have a suitable hands-on informal approach to train￾ing. 3. Sufficient resources are needed in order to attract and keep competent and committed trainers. The technical training of the JA project was provided by outside consultants who were not paid in accordance with market rates in Senegal. As a result, youth reported that technical trainers were poorly prepared or relied only on class￾room instruction instead of taking the trainees to nearby experimental plots to learn about cultivation tech￾niques. The youth told the evaluators that the technical trainers often came late or suddenly postponed the train￾ing to the next day. 4. Strong incentive mechanisms should be put in place to ensure that the engagement of training partners positively impacts program outcomes. At a minimum, the reviews/responses from participants regarding the quality of the training activities should be reviewed and acted on if necessary. 5. Adequate time is needed to build skills, particularly in the area of financing, where youth participants did not have prior experience. The interviewed youth found that financial management was the weak point of an otherwise good quality module on entrepreneurship. A more integrated “institutional outreach” approach to link project participants up with viable sources of finance may be required as well to significantly improve the impact of this type of initiative in future. 6. At a more strategic level, it is worth considering whether or not the JA training/outreach strategy and associated set of implementation tools represents a cost-effective and viable intervention approach to driving agricultural entrepreneurship and income growth. Typically, the range of technical and market development and related institutional strengthening interventions required to drive successful agricultural/agribusiness outcomes under a donor support strategy and program go significantly beyond those available under a workforce development focused initiative. The results of both the cost-effectiveness and cost/benefit analysis reported above certainly seem to suggest that this is the case for the JA Program. 7. Future employment generation-focused initiatives of this nature should have a much stronger focus on concrete outcome-related performance indicators and related targets including specific cost-efficiency and income genera￾tion-related indicators. This should also include targets for youth placement or job creation. Training/improved skills, coaching/professional development, private sector engagement/relevant training are in the end intermediate results to getting youth into employment and raising their incomes. OVERALL RECOMMENDATIONS FOR FUTURE PROGRAMMING • The JA project can best be seen as a pilot experiment in enabling youth to create employment in the agricultural value chain. Its successor could prepare youth for employment creation with more training on financial manage￾ment, and achieve a higher level of success by providing more guidance and follow-up on financing for the youth enterprises. It would also require better targeted technical and agronomic training, • Synapse Center stated that it had between five and eight of its staff working on the JA project. Among these staff, according to Synapse Center, the Project Manager and Project Coordinator dedicated 100 percent of their time to supervising project activities. Any future JA project will need a dedicated project manager in the home office to manage relations with the implementing partners and at least two or three support staff to oversee the training and identify problems that need attention. These staffing requirements may be difficult to fund for a local institution interested in managing a JA-style project like the one implemented by Synapse Center, unless it obtains additional funds. • A future project of this nature should incorporate at its starting point an analysis of prospective labor market demand trends at the sub-sectoral level; in order to assess where employment growth opportunities are likely to 42 be greatest by sub-industry and region; in order to better focus/orient training programs in a manner which will maximize prospective impact on employment and income growth. • Public-private partnerships are valuable tools for training, internships and post-training follow-up and support for youth entrepreneurs. These partnerships need to be negotiated and set up in advance. The roles and responsibili￾ties of each partner must also be made clear, especially if they are expected to provide pro bono services such as mentoring and supervision. • The project was overly reliant on inadequately-paid/motivated consultants for the technical training. Future JA training needs an institutional base to manage it while developing and keeping talented trainers. • Future initiatives like the JA project should focus on youth with at least secondary-level education. If training is to be provided for out-of-school youth with little education, a separate program or program component is needed. • Use JA graduates as resources in future iterations of the project. The successful graduates can provide strategic advice to trainees on how best to set up and manage their enterprises. The youth graduates can also share their experiences in funding, marketing and customer relations, among other areas of concern. However, adequate logistical and financial support will be required for such an initiative. • Post-training follow-up is critical. Mentoring was provided during the training but with the exception of one part￾ner in St. Louis, not after the training40. Follow-up support finding financing for youth projects is also essential. There are at least two sources of funding for productive projects, and one is designed specifically for youth. However, youth trainees need support and guidance to help them with the application process. • Serious consideration should be given as to whether a workforce development/training styled initiative is an ef￾fective centerpiece intervention mechanism for driving agriculture/agribusiness development at the farm or value chain level. If a determination is made to move forward with a renewed program of this nature, it needs to be effectively integrated with broader agricultural intervention programs focused on improving agricultural value chain competitiveness. Otherwise the prospective impact of a follow-on JA Program to build agricultural entre￾preneurship capacity is likely to be severely muted. • An effective M&E system is vital to the effectiveness of future programming for youth development. Ideally, the system should include: ¾ Clear terms of reference or memoranda of understanding for each project partner ¾ A results framework for each major project activity, supporting the results framework for the project. ¾ A management system which systematically collects and effectively collates and utilizes collected data to de￾tect problems in project implementation, and to formulate targeted corrective measures. ¾ Targets to track cost-effectiveness and cost-benefit of program activities Management actions to correct problems detected by M&E procedures. ¾ Clear targets to track cost-effectiveness and cost-benefit of program activities.                                                              40 Synapse Center notes that 5 ENSA students received land from their mentors. In Ross Bethio, 10 trainees gained access to land with the help of their mentors. In Sangalkam (Dakar Region) 10 other trainees had similar support from mentors. Finally, in Thiès, Synapse Center states that the “community of mentors” funded15 of the youth entrepreneurs’ projects. ANNEXES 1 ANNEX 1 YouthMap: Senegal Final Evaluation Scope of Work These terms of reference are for a performance evaluation of the pilot YouthMap activity in Senegal. The activity aims to strengthen the skills of young people so that they are better equipped to enter the job market, either as entrepreneurs or employees. SENEGAL A. Background The YouthMap Senegal - Jeunes Agriculteurs (JA), is a pilot Innovation Fund activity under International Youth Fund’s (IYF) USAID-funded YouthMap initiative. Building on the YouthMap assessment, USAID/Senegal’s 2012-2016 Country Development Cooperation Strategy (an IR calls for better educated youth) and Feed the Future Initiative, the JA activity was designed to address Senegal’s youth unemployment and under-employment concerns. The activity is underpinned by the development hypothesis that if youth are provided with comprehensive employability and entrepreneurship training experience that is rigorous, practical, integrated, and aligned with labor market demand, then their chances of gainful employment or establishment of successful enterprises will significantly increase. The JA pilot activity’s intended outcomes are to prepare participants for successful employment and/or enterprise development across Senegal’s agriculture value chains. The results framework is presented in the figure below. Activity outcomes include:  Improved access to training, knowledge, and skills  Improved prospects of employment and self-employment of youth  Increased sustainability and scale through public-private partnerships 2 Results Framework The intermediate results (IRs) are expected to lead to the achievement of the larger YouthMap goal. Key concepts are defined here to ensure clarity and common understanding. Intermediate Result 1: Improved access to training and knowledge/skills IR1 refers to training and knowledge that Youth Map beneficiaries obtained. Training is focused on entrepreneurship, ICT, and life skills tailored to the agribusiness sector. Through their participation in the training, youth acquire knowledge and skills, which allows them to productively engage in the labor market and/or pursue self-employment opportunities. Intermediate Result 2: Improved prospects of employment/self-employment of youth IR2 refers to new opportunities offered to the beneficiaries in terms of employability and self￾employment. Through the combined support from trainers, mentors and coaches, a supportive environment is created to enable participants to gain access to job opportunities and creation of business ventures in the agricultural sector. Intermediate Result 3: Improved capacity of local organization as a result of participation in the program: IR3 refers to local organization capacity on youth programs designing, implementation, monitoring and evaluation. IYF is responsible for strengthening the capacity of Synapse Center (the local implementing partner) to deliver high quality youth programming. Also, throughout the activity Synapse and IYF strengthen the capacity of partners in the regions to effectively deliver trainings in life skills, entrepreneurship, ICT and youth employability and entrepreneurship program designing, monitoring and evaluation. Goal: Enhanced participation of youth with relevant skills in the agriculture sector to stimulate growth of jobs and innovation in Senegal Strategic Objective: Youth trained and equipped with relevant skills to generate both income and employment opportunities in the agriculture sector Result 1: Improved access to training and knowledge/skills Result 2: Improved prospects of employment/self￾employment of youth Result 3: Increased sustainability and scale through public￾private partnerships 3 The program indicators for JA are attached in Annex A. B. Evaluation Purpose and Use The evaluation is intended to assess the multifaceted impacts on youth beneficiaries, the effectiveness of program implementation, and the strengths/challenges related to JA performance. A summative analysis of all JA elements (e.g., recruitment and selection, training, and business development services as outlined below) will inform the degree to which objectives were met. The evaluation report will capture/share promising practices/lessons that should be considered for scale￾up of future agriculture-based programs of this nature, and be used by key stakeholders ranging from USAID and the Government of Senegal to IYF, Synapse Center, and other youth-serving NGOs/donors interested in integrating youth into agriculture programs. C. Program Implementation and M&E Approach The JA activity is an entrepreneurship and agriculture skills training program that provided 320 young people, ages 16 to 29, with entrepreneurship, agriculture, ICT, and life skills training. Youth participants benefited from a blended training approach which includes experiential learning activities, such as learning expeditions, exchanges, innovation labs, training workshops, and business development services. The activity – managed by IYF and its implementing partner, Synapse Center (Synapse) – focused its implementation activities around Senegal’s geographic hubs of Dakar, Thiès, and Saint-Louis. Key elements of the activity to examine include:  Program design  Participant selection  Life skills training  ICT training  Entrepreneurship training  Business development services  Public-private partnerships in sup-port of youth exposure to agribusiness  USAID’s role  M&E (baseline and exit surveys)  Cost analysis Youth-Map Senegal set up an overall M&E system that covers the following components: • A tracking system was used to track progress on key indicators. Monitoring focuses on the program’s ability to deliver services as planned, tracking output and outcome and indicators (detailed below). IYF developed the template in consultation with YouthMap implementing partner Synapse, and Synapse was responsible for tracking and submitting information to IYF as part of its quarterly reporting. • Baseline, exit, and follow-up surveys of youth participants were developed by Synapse in collaboration with IYF and administered to gather more specific information about youth participants and program performance. Synapse administered a baseline survey at activity outset. Upon completion of training cycle, Synapse administered an exit survey with youth. A follow-up survey was administered six months after youth have completed internships. If sufficient information is available on a group of youth who applied but were not selected, there may be an opportunity to provide a comparison with a group that did participate, which would 4 provide the basis for a better understanding of the impact of this program. This will be determined during the inception phase. • An end-of-program evaluation was conducted to assess the program impact on beneficiaries and capture overall program learnings and good practices for future internship programs. D. Evaluation Questions This evaluation will gather data related to JA activity achievements and their impact on the intended beneficiaries (disaggregated by gender, to the extent possible), incorporating the following questions: Program Effectiveness and Impact  How effective were the approaches taken for each of the four program components in achieving the activity’s goals? o Entrepreneurship o Agriculture o ICT o Life Skills  How useful have internal assessments been for designing and/or improving training/employment activities?  How were intern qualifications and corresponding employer characteristics matched? In the final report, provide a profile of the interns and qualifications and include a description of the intern selection process and criteria. Also propose potential improvements based on analysis, and include section on summary statistics of interns.  What was the activity’s overall impact on youth beneficiaries? What benefits did the targeted youth receive from participating across each of the training components?  How the capacity of various activity partners strengthened through JA and what was the outcomes of these efforts?  What are identified trends in intern employment data1 with respect to gender, sectors of interest, employer retention, and employment quality (e.g., part-time/fulltime, salary, hours, location, and promotions)?  What was the cost-effectiveness of the program? Define how cost-effectiveness is measured and if sampling was used whether that sampling applies to the entire population.  What are the activity costs? Include detailed analysis of cost components—this will impact data requests from the existing project team. If costs are high make recommendations on which cost components could be dropped or reduced.  How do activity costs compare to similar activities (can only be done if data is made available by USAID on similar programs—publically available reports will be checked as well but it is unlikely that sufficient data will be available)?  What was the financial rate of return of the activity? Please explain the methodology used for the calculation. 1 Note that the interns are university graduates and hence unemployment needs to be compared to the unemployment among youth of similar levels of education, say unemployment among university graduates or say youth with tertiary education. 5 Sustainability and Scalability  What are the factors facilitating or inhibiting the sustainability of activity components beyond the JA (e.g. regional councils, etc.)?  Considering the financial and institutional context, which activity components can be scaled, and which stakeholders are best placed to support and are actively interested in scaling-up (e.g., government, private sector, university system, and international donors)?  How did partnership formation support the building of local ownership and encourage innovation? E. Evaluation Design and Methodology 1) Design and Methodology This performance evaluation will primarily be a mixed methods and non-experimental. Depending on data availability for youth not selected into the JA a quasi-experimental design may be used (see below). Data will be collected from a variety of sources including:  Primary data collection o Interviews with program staff from IYF and Synapse, JA trainers, youth training graduates and dropouts (if possible), youth who have started businesses, private sector/agribusiness companies, and others (i.e. regional council members). o Focus groups with the activity participants o E-survey of youth participants o Interviews with other key informants such as: USAID staff, government, civil society, and other donors/projects involved in youth employment  Secondary data o Existing project data such baseline, exit, and follow-up surveys of youth participants will be provided by Synapse in collaboration with IYF o Activity performance data; o Document review In order to address bias, the focus group participants will be randomly selected as is feasible, and should proportionally match the number of participants and gender distribution between the regions. In addition to the data collection methods above, the evaluation team will be provided contact information for youth that applied but were not accepted into the program by USAID, IYF or Synapse. If the data is provided within one week of the project start, the evaluation team will collect data from this group either through interviews or an e-survey to use as a comparison to program participants. Quasi-experimental impact analysis through matching or regression discontinuity techniques will be conducted if possible. 2) Data Analysis As part of the inception report developed before the start of new data collection, the evaluation team will include a data analysis plan that details the quality of the IYF/Synapse data received prior to 6 the start of data collection; what steps need to be taken to address any gaps in that data; how the key informant interviews and focus group data will be analyzed; what procedures will be used to analyze qualitative data from key informant and other stakeholder interviews; and how the evaluation will weigh and integrate qualitative data from these sources with quantitative data from the participant survey. The evaluation team will analyze the qualitative data from the interviews and focus groups for trends. The evaluation team will analyze program service statistics, beneficiary demographic data, attrition and retention information, field trip reports, and other program data to support a triangulated, critical analysis of the evaluation objectives. The evaluation team will also analyze data previously collected through JA baseline and exit surveys administered to youth. 3) Limitations for proposed design and methodology While the mixed methods design aims to arrive at unbiased findings, conclusions, and recommendations by bringing together a variety of sources, there is always the possibility of introducing bias through interviews or focus groups. Additionally, an e-survey will always have the possibility of a respondent bias based on who chooses to complete the questionnaire. The evaluation team will be mindful of these potential issues and should mitigate this bias as much as possible. F. Logistics Within Senegal, the evaluation team will focus their data collection on three geographic areas:  Dakar (40 JA graduates in cohort 1 / 45 in cohort 2)  Thiès (60 JA graduates in cohort 1 / 61 in cohort 2)  Saint-Louis (60 JA graduates in cohort 1 / 62 in cohort 2) I. Team Composition and Schedule The core team will consist of the Evaluation Team Leader (expat) and a Local Expert (Senegal). The evaluation team will be supported remotely by the MSTAS Headquarters team, and will be supported by a logistics coordinator/translator, to arrange transport, meetings, follow up on telephone surveys and translate from the local language into French. The Evaluation Team Leader must have experience conducting mixed-methods evaluations for USAID, including tool development, data collection, and data analysis for interviews, focus groups, and surveys. Additionally, the Evaluation Team Leader must have experience working with databases and datasets in order to analyze the existing program data. Work experience in Senegal or Francophone Africa is preferred. French fluency is required. 7 The following is an illustrative level of effort table covering the evaluation, which will be finalized through the budget agreed upon between USAID and MSTAS. It is expected that this effort would be carried out over a 6-8 week period beginning o/a June 1, 2015. Position Desk Review Travel Methodology, Design, and Tool Development and Surveys/Focus Groups/Inter￾views/Analysis/Initial Drafting/Draft Final Report/Presentation (Field) Final Report Total Evaluation Team Leader 8 2 27 5 42 Local expert, Senegal 6 - 27 1 34 Admin Support/Translato r, Senegal - - 27 - 27 Total LOE 14 2 81 6 103 8 Estimated Schedule of Activities Activity Time Frame Notes Initial analysis of quantitative data provided by IYF and desk review and, Initial briefing with Africa/SD and Senegal mission via conference call June 29 –July 2 (4 days) This review will be conducted to determine the quality and usability of the data, and to inform on the evaluation design. Data gaps and limitations will be identified. The review will be done prior to travelling to Senegal Inception Report July 3-7 (3 days) The inception report will be based on the analysis of the IYF data and desk review, and will serve as the basis of the evaluation design. Submit Inception Report to Africa/SD July 8 Requires USAID input/comments and approval prior to travel Time for USAID Review July 9 – July 15 (5 days) Evaluation Team travel to Senegal July 18 (1 day) The Evaluation Team will depart for Senegal after the inception report is finalized and approved by USAID. Initial briefing with USAID/Senegal personnel July 20 To review evaluation questions, methodologies and schedule. Adapt and or develop data collection tools, and Evaluation Team meets with partners, stakeholders and conducts key informant interviews in Dakar July 21 – July 27 (6 days) Evaluation Team collects data in Thies July 28 – August 2 (6 days including travel) Conduct key informant interviews, focus groups and review additional data that may be unique to this hub. Evaluation Team collects data in Saint-Louis August 2 -8 (6 days including travel) Conduct key informant interviews, focus groups and review additional data that may be unique to this hub. Follow up with key informant interviews (if needed), and the local implementing partner. Clean and analyze data, and write preliminary draft evaluation report August 10-14 (4 days) 9 Draft and Present findings and recommendations to USAID/Senegal (provide draft 2 page summary), and finalize draft evaluation report August 15-19 (4 days) Make presentation to USAID August 20 Travel from Senegal August 21 Submit Draft Evaluation Report August 28 Draft report is submitted for comments and approval. Finalize Evaluation Report 1 day Based on comments and suggestions received from USAID, the report will be revised. Submit Final Evaluation Report September 11 II. Expected Deliverables The evaluation team will be expected to produce the following deliverables: • Inception Report, including final evaluation questions, data collection and data analysis plan, and a work plan to guide the overall evaluation • Data collection tools, including: o Key informant interview guide o Youth focus group guide o Youth survey and employer survey to measure their satisfaction with the quality of graduates and better understand the quality of jobs • Debrief for Mission staff and program stakeholders on evaluation findings • Draft evaluation report, including all questions listed above, for review by IYF and USAID • Final evaluation report incorporating all edits/comments provided by IYF and USAID The Evaluation Report, which will review results and draw overall conclusions and lessons learned, should be no longer than 35 pages, excluding annexes, structured as follows: 1. Executive Summary—including concise summary of findings and recommendations (2–3 pp); 2. Introduction—purpose, audience, and synopsis of task (1 pp); 3. Background—brief overview of the activity and purpose of the evaluation (3–4 pp); 4. Methodologies—describe evaluation methods, including constraints and gaps (1–2 pp); Include summary or actual secondary data/information used 5. Findings/Conclusions/Recommendations—for each key question (18–22 pp); 6. Key Lessons Learned—provide a list of key technical and/or administrative lessons learned, if any (2–3 pp); 7. Overall recommendations for future programming (2–3 pp); 8. Annexes—annexes that document the evaluation methods (attach final evaluation SOW), schedules, interview lists, and referenced documents. 10 9. Raw data collected through surveys, focus groups and interviews should be submitted electronically in a separate file 10. The report should include sufficient graphs, tables, FGD and KII quotes to support findings The Evaluation Report must also meet the following criteria as outlined in the USAID Evaluation Policy. CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT (USAID Evaluation Policy, Annex 1)  The evaluation report should represent a thoughtful, well-researched and well organized effort to objectively evaluate what worked in the project, what did not and why.  Evaluation reports shall address all evaluation questions included in the scope of work.  The evaluation report should include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by the AOR.  Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report.  Evaluation findings will assess outcomes and impact on males and females.  Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.).  Evaluation findings should be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence.  Sources of information need to be properly identified and listed in an annex.  Recommendations need to be supported by a specific set of findings.  Recommendations should be action-oriented, practical and specific, with defined responsibility for the action. III. Evaluation Management Pragma’s home office and the evaluation team will be responsible for arranging logistical support for this assignment. However, MSTAS and IYF project staff and partners in Senegal will assist in facilitating meetings. The IYF home office personnel and local partners in Senegal will be available to answer inquiries concerning the design and implementation of the program. The evaluation team leader will report to and be under the supervision of the Washington-based MSTAS Chief of Party. He will inform AFR/SD concerning the progress of the Evaluation Team. The evaluation will be managed by the AFR/SD Senior Education Management Analyst, who is the point of contact in USAID/Washington. She will advise the AFR/DP Monitoring and Evaluation 11 Advisor of the Evaluation Team’s progress. The M&E advisor will be available for consultations concerning issues and questions that may arise during the course of the evaluation. The Evaluation Team will work closely with USAID/Senegal. At the start of the field work the Evaluation Team will meet with the mission and seek the mission’s guidance for best carrying out its responsibilities. The field work will close with a presentation to the mission which reviews findings and conclusions. 12 Annex A: Program Indicators, Senegal Intermediate Result 1: Improved access to training and knowledge/skills Intermediate Result 2: Improved prospects of employment/self￾employment of youth Intermediate Result 3: Increased sustainability and scale through public-private partnerships Outcome Indicators 1. Knowledge of entrepreneurship increased 2. Knowledge of ICT increased 3. Knowledge of life skills increased 4. Knowledge of technical skills in agribusiness increased 1. Youth supported through mentoring2 2. Youth supported through coaching 3. Youth supported through learning expeditions 4. Youth supported through retreats 5. Youth with improved capacity to start their own businesses 6. Youth participation in agribusiness value chains increased 1. Partnerships with public, private, and civil society partners established 2. Increased capacity at Synapse Output and Process Indicators 5.No. of youth participating in Youth Map training programs 6. No. and % of youth completing YouthMap training 7. No. and % of youth beneficiaries demonstrating improved capabilities upon completion of training 7. No. and % of youth receiving mentoring services 8. No and % of youth receiving coaching services 9. No. of learning expeditions organized 10. No. of retreats organized 11. No. of business plans created 12. No. and % of youth receiving business development support services 13. No. of businesses created or improved 14. No. and % of youth beneficiaries reporting greater optimism and aspirations for their future. 15. No. and % of participants with jobs 3. Resources leveraged for YouthMap activities from multi￾sector partners 4. No. of partnerships created 5. No. of systems strengthened 2 Qualitative focus on the effects of mentoring support for knowledge gain. INCEPTION REPORT PERFORMANCE EVALUATION OF THE SENEGAL YOUTHMAP JEUNES AGRICULTEURS PROJECT DATE: JULY 22, 2015 This publication was produced for review by the United States Agency for International Development. It was prepared by Drs. Eric Allemano and Alioune Dieng and the Project Team of the Management Support and Technical Analysis Services Project, implemented by the Pragma Corporation and Management Systems International. Annex 2 INCEPTION REPORT PERFORMANCE EVALUATION OF THE SENEGALYOUTH MAP- JEUNES AGRICULTEURS PROJECT Contracted under AID-OAA-M-13-00012 USAID Management Support and Technical Analysis Services Project DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL i CONTENTS Introduction.................................................................................................................................... 2 Summary of the Program and the Evaluation ..................................................................... 2 List of Evaluation Questions............................................................................................... 4 Evaluation Design .......................................................................................................................... 5 EVALUATION DESIGN & METHODOLOGY..............Error! Bookmark not defined. Data Collection Methods............................................................................................................... 7 DATA COLLECTION ....................................................................................................... 7 Data Analysis Methods................................................................................................................ 11 Annexes......................................................................................................................................... 13 Annex 1: Getting to Answers...................................................................................................... 14 Annex II: Work Plan ................................................................................................................... 23 Annex III: Tools........................................................................................................................... 26 Annex IV: References.................................................................................................................. 33 Annex V: Team Roles and Responsibilities............................................................................... 37 Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 1 List of Acronyms and Abbreviations ASESCAW Amicale Socio-Economique Sportive et Culturelle des Agriculteurs du Walo CIH Centre d’Initiation Horticole CRREJ Centre Régionale de Ressources Pour l'Emploi des Jeunes FGD Focus Group Discussion FONGIP Fonds de Garantie des Investissements Prioritaires ICT Information and Communications Technologies IYF International Youth Foundation JA Jeunes Agriculteurs KII Key Informant Interview TVET Technical and Vocational Education and Training UGB Université Gaston Berger USAID United States Agency for International Development. Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 2 INTRODUCTION Summary of the Program and the Evaluation In 2013 the International Youth Foundation (IYF) launched the YouthMap Jeunes Agriculteurs (young farmers) project with a $400,000 budget to cover surveys, training, counselling and job placement or creation among youth in the following regions of Senegal: Dakar, Thiès and Saint Louis. The Jeunes Agriculteurs (JA) project was a two-year pilot program designed to prepare participants for successful employment and/or enterprise development in Senegal’s agriculture value chains. It was built on lessons learned from YouthMap, a four-year regional program designed to assess youth circumstances and support promising youth development programs in up to eight African countries. The YouthMap initiative was designed by the International Youth Foundation (IYF), and supported by a US$7.8 million agreement with USAID. YouthMap is based on research in six sub-Saharan African countries in order to identify needs and strategies to respond holistically and sustainably to the demand for employment among youth. Findings of the YouthMap assessment conducted in five regions of Senegal in 2011 revealed priorities, approaches, and types of programs that should be supported to benefit youth. The findings affirmed that although Senegal has made great strides increasing access to and completion of education at the primary level, quality remains an issue throughout the entire education system. Youth and key stakeholders believe the curriculum at all levels needs to be more practical and better aligned to labor market needs and opportunities. Respondents to the research stated that more career guidance, counseling services, and life skills training are needed to help youth pursue productive career paths. It was found that the demand for Technical and Vocational Education and Training (TVET) programs exceeds the available supply. Unfortunately, however, inadequate infrastructure is a persistent problem across all school levels, with rural areas being especially under resourced. IYF managed the Jeunes Agriculteurs (JA) project with the Synapse Center, whose mission since its founding by a Senegalese university student in 2002, is to help young people become leaders and entrepreneurs who contribute to the development of their communities. Through all its work, Synapse promotes creative leadership and innovative entrepreneurship as well as personal, organizational, and community development. The project, which trained and aided the job placement of 288 youths in two seven-month cohorts, ended in June, 2015. The performance evaluation, which will also serve as the final evaluation of the Jeunes Agriculteurs (JA) project, will be conducted by a three-man team working at three research locations between July 20 and August 20, 2015. Besides the Synapse Center, the research team will hold focus group discussions with representatives of 19 institutions that provided training, counselling, mentoring and other services to the JA project in the three target regions. The beneficiaries of the JA project include 288 young people aged 16 to 29 from Dakar, Saint-Louis, and Thiès (urban, suburban and rural areas) who enrolled in two seven-month training cycles in the areas of life skills, entrepreneurship, coaching, mentoring, and financing platforms. Through this program 172 businesses were created. The goal of this two-year pilot program was to prepare participants for successful employment and/or enterprise development in Senegal’s agriculture value chains. IYF and its Senegalese implementing partner, Synapse Center, managed the Jeunes Agriculteurs project in 2013 – 2015 using a blended approach which included experiential learning activities such as learning expeditions and exchanges, innovation labs, training workshops, individual mentoring and coaching, networking opportunities, and the chance to learn about resources available to youth such as financial services. To support this program, YouthMap/JA established regional committees that consisted of technical training institutes, governmental bodies, civil society organizations, and private sector members. The Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 3 regional committees were charged with the daily operation of the program, including the recruitment of young people. The Jeunes Agriculteurs project process was built around the following activities: • Learning expeditions to help participants achieve a greater understanding of important societal problems, community needs and project management methods. • Two-day retreats per cohort followed the learning expeditions help participants overcome their internal boundaries and to make sense of their learning. • Personal Action Plans were drafted by each participant in order to gain greater knowledge of an issue of his or her previous training and experience. • Individualized Self Study was a process designed around self-learning using various books, websites, films, etc. Significant attention was given to processes of self-awareness, self￾understanding, and examining one’s life choices. • Skills and competence workshops (80 hours, 20 sessions of 4 hours each) Following the learning expeditions and the retreat, 20 sessions of 4 hours each (lasting 2 to 3 months) were offered to each of the 20 cohorts. In between the sessions, debriefing sessions and coaching sessions were held. All project participants were to participate in workshops featuring life skills, entrepreneurship and business creation and development skills as well as ICT skills. In addition, they received practical training on agriculture management, marketing, and agro-food processing among other skills in agribusiness. • Coaching sessions (40 hours per cohort) these sessions provided an opportunity to meet with coaches, mentors and Synapse staff to report on and receive feedback about their personal and project development goals and proposed course of action. • Debriefing sessions (six monthly sessions per cohort, 4 hours each) in-depth sessions of peer review and problem solving, the debriefing sessions helped the participants to find ways of overcoming the stumbling blocks, dilemmas and other issues besetting them. • Innovation labs (One per cohort, 30 to 45 days) — the innovation labs were aimed at developing participants’ entrepreneurial behavior and personal power to help them improve their professional and/or personal lives. Further innovative aspects of the Jeunes Agriculteurs project include the following: • Thematic sessions (Five monthly sessions per cohort, 4 hours each participants had the opportunity to listen, learn, question and network with leaders, entrepreneurs and real practitioners willing to share their experiences, expertise and advices. • Mentoring on joining the program, each of the entrepreneurs was assigned a personal mentor with appropriate skills and experience to help him or her work through any personal and project challenges that arose. • Exchange platforms served many purposes but mainly helped to broaden the interdisciplinary outlook of participants, to acquire new skills in leadership, to expand the network with other key decision makers and opinion leaders, and to build the interactions within each cohort. • Financing platforms (Two sessions per cohort, 4 hours each) To give to project beneficiaries the chance to build relationships with lead investment decision makers, regular meetings were organized Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 4 providing entrepreneurs with the chance to learn how to make their companies more attractive to investment, secure capital and begin conversations leading to later-stage investments. EVALUATION PURPOSE This performance evaluation is intended primarily to assess the effectiveness and impact on beneficiaries of the Senegal Jeunes Agriculteurs project. The evaluation also seeks to determine the sustainability and scalability of the project in order to inform similar youth training and job creation projects that may be implemented in the future. The evaluation is designed to provide a comprehensive analysis of all elements of the project, including: youth recruitment and training as well as employment creation or placement. It responds to the evaluation questions in depth, question by question. It will provides an assessment of the degree to which JA project objectives and outcomes have been achieved, with a particular focus on how the program has contributed to employability prospects for youth beneficiaries. The target audience of this evaluation includes key stakeholders ranging from USAID and the Government of Senegal to IYF, private companies, and youth-serving NGOs/donors. The evaluation of the JA project will use a mixed-methods approach to assess effectiveness, impact, sustainability and scalability. The evaluation team has received qualitative data from IYF and Synapse Center, the local implementing partner in Senegal, and will make use of Synapse’s performance monitoring and quantitative data obtained from local partners involved in providing training and facilitating job placement and creation among youth. EVALUATION QUESTIONS The evaluation team’s understanding and interpretation of the Evaluation Questions, with an emphasis on how each question will be answered are as follow: There are six evaluation questions. Four of them concern the effectiveness and impact of the project on the participating youth. The two final questions ask about the project’s sustainability and scalability. I. Evaluation Questions on Program Effectiveness and Impact 1. How effective were the approaches taken for each of the four program components in achieving the activity’s goals? i. Entrepreneurship ii. Agriculture iii. ICT iv. Life Skills Sub-questions:  How useful have internal assessments by USAID, IYF and the Synapse Center been for designing and/or improving training/employment activities?  How were youth qualifications and corresponding employer characteristics matched? In the final report, provide a profile of the youth and their qualifications and include a description of the selection process and criteria. Also propose potential improvements based on analysis, and include section on summary statistics of youth trainees. 2. What was the activity’s overall impact on youth beneficiaries? What benefits did the targeted youth receive from participating across each of the training components? Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 5 3. To what extent was the capacity of various activity partners strengthened through JA and what were the outcomes of these efforts? 4. What are identified trends in youth employment data with respect to sex, sectors of interest, employer retention, and employment quality (e.g., part-time/fulltime, salary, hours, location, and promotions)? Similar questions will be asked regarding youth in self-employment. II. Evaluation Questions on Program Sustainability and Scalability 5. To what degree was the program cost-effective? Sub-questions:  What are the activity costs? Include detailed analysis of cost components—this will impact data requests from the existing project team. If costs are high make recommendations on which cost components could be dropped or reduced.  How do activity costs compare to similar activities? (Can most effectively be done if data are made available by USAID on similar programs—publically available reports will be reviewed as well for purposes of comparative analysis).  What was the financial rate of return of the activity? Please explain the methodology used for the calculation. 6. What is the likelihood of sustainability of program activities? What are the factors facilitating or inhibiting the sustainability of activity components beyond the JA (e.g. regional councils, partnership formation, etc.)? Subquestions:  Considering the financial and institutional context, which activity components can be scaled up and which stakeholders are best placed to support and are actively interested in scaling-up (e.g., government, private sector, university system, and international donors)?  How did partnership formation support the building of local ownership and encourage innovation? EVALUATION DESIGN AND METHODOLOGY The evaluation of the JA project will use a mixed-methods approach to assess effectiveness, impact, sustainability and scalability. The evaluation team has received qualitative data from IYF and Synapse Center, the local implementing partner in Senegal, and will make use of Synapse’s performance monitoring and quantitative data obtained from local partners involved in providing training and facilitating job placement and creation among youth. While the evaluation’s focus will be primarily on the six key evaluation questions above, the design of secondary questions will provide feedback evidence as to how the targets of the JA project were achieved. The evaluation team will use both qualitative and quantitative research methods and tools (e.g. databases created by Synapse on the pre- and post-training characteristics of the target youth will be studied and key informant interviews (KIIs) and focus-group discussions (FGDs) will be held with Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 6 beneficiary youth and representatives of partner organizations. A non-experimental evaluation design is preferred for the evaluation because only one of the two cohorts of youth trainees participated in a six-month post-training survey and a control group is lacking. The documentation provided to the Pragma Evaluation Team indicates that a formal baseline study of youth who successfully applied to join the training program was conducted at the start of the JA project. A PMP for the JA project has not been provided to the team although a Youth Map Monitoring table provides targets on training but only through March, 2014. However the project’s Results Framework does not provide quantified performance indicators that would guide the monitoring of project performance beyond the activity and output level. The YouthMap Semi-Annual Report (Oct. 1 – 2014 to March, 2015, the YouthMap Monitoring Table for Senegal for the same period have been reviewed and will be used to help fill in data gaps. Further data on project performance and results, particularly for March, 2014 through June 2015 will be gathered from Synapse Center and the partner implementing organizations in Senegal. In addition, this further underscores the importance of the cost-efficiency and cost/benefit analysis to be undertaken, as a key means of establishing the overall effectiveness, cost￾effectiveness, and scalability of the projects core interventions. Moreover to further bolster the availability of relevant data related to the key evaluation questions, the following sources will be used for data about youth knowledge, and skills before and after the training program: Intermediate Result 1: Improved access to training and knowledge/skills Baseline data will be derived from the detailed questionnaires submitted by youth who applied to the program. Post-training results will be measured by the findings in the six-month follow-up survey on the first cohort of youth trained. (A follow-up survey on the second cohort was not done; however, the evaluation team will review the final evaluation questionnaires of the second cohort for relevant information. This information will be provided by Synanpse Center in Dakar). Intermediate Result 2: Improved prospects of employment/self-employment of youth The data on youth employment prior to the training is provided in a data base created by Synapse. Post-training results will be measured by the findings in the six-month follow-up survey on the first cohort of youth trained. (A follow-up survey on the second cohort was not done. To compensate for this for the second cohort, we will consult the list of graduates who a) created their own enterprises, b) found salaried employment c) are unemployed or d) have returned to formal education). Intermediate Result 3: Increased sustainability and scale through public-private partnerships The nature and number of the partnerships will be assessed in KIIs with representatives of the partners. The partners will be asked to provide information about the nature, duration and funding of the partnerships formalized during the project. Table 1: Project Indicators Intermediate Result 1: Improved access to training and knowledge/skills. Intermediate Result 2: Improved prospects of employment/self￾employment of youth Intermediate Result 3: Increased sustainability and scale through public-private partnerships Outcome Indicators The end-of-training evaluations of each cohort of youth will provide data on the changes in knowledge, attitudes and skills resulting from the 7-month training program. 1. Knowledge of entrepreneurship increased KIIs with the trainers and with Synapse staff will provide data on outcomes 1 to 4. Synapse will provide the numbers of business started by graduated youth including businesses that are part of agribusiness value chains (Outcomes 5 and 6). 1. Youth supported through mentoring1 2. Youth supported through coaching Documents provided by Synapse and its partners will provide measurable data on the first outcome. The evaluation team will assess changes in the capacity of Synapse through site visits, KIIs and interviews with its staff and partners. 1 Qualitative focus on the effects of mentoring support for knowledge gain. Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 7 Intermediate Result 1: Improved access to training and knowledge/skills. Intermediate Result 2: Improved prospects of employment/self￾employment of youth Intermediate Result 3: Increased sustainability and scale through public-private partnerships 2. Knowledge of ICT increased 3. Knowledge of life skills increased 4. Knowledge of technical skills in agribusiness increased 3. Youth supported through learning expeditions 4. Youth supported through retreats 5. Youth with improved capacity to start their own businesses 6. Youth participation in agribusiness value chains increased 1. Partnerships with public, private, and civil society partners established 2. Increased capacity at Synapse Output and Process Indicators 5.No. of youth participating in Youth Map training programs 6. No. and % of youth completing YouthMap training 7. No. and % of youth beneficiaries demonstrating improved capabilities in entrepreneurship methods and skills, agricultural value chain concepts and practices, ICT applications and life-skills upon completion of training 7. No. and % of youth receiving mentoring services 8. No and % of youth receiving coaching services 9. No. of learning expeditions organized 10. No. of retreats organized 11. No. of business plans created 12. No. and % of youth receiving business development support services 13. No. of businesses created or improved 14. No. and % of youth beneficiaries reporting greater optimism and aspirations for their future. 15. No. and % of participants with jobs 3. Resources leveraged for YouthMap activities from multi-sector partners 4. No. of partnerships created 5. No. of systems strengthened DATA COLLECTION METHODS DATA COLLECTION Both primary and secondary qualitative and quantitative data will be collected and analyzed for this evaluation. Methodological triangulation will be used to verify the quality of the data and evidence a) Secondary data collection A desk review of key relevant documents received from IYF and Synapse provide an initial knowledge base for the evaluation team which have been used to finalizing develop our research strategy. Overall this provides the foundation of the final Evaluation Methodology and the initial findings presented in the Team Planning Meeting prior to field work. A desk review instrument to codify and organize data from the documentary review for analysis as per answering the evaluation questions will be utilized. For example, review of information from IYF documents on the multi-country YouthMap initiative or the JA project will be undertaken to understand the context, design and implementation of the initiative. b) Primary data collection The evaluation team will conduct both focus group discussions (FGDs) and key informant and interviews (KIIs) with youth beneficiaries, staff from Synapse and the project partners. Both methods will be used to explore the effects of the project training on youth beneficiaries. The Synapse data base will be used to constitute three groups of youth interviewees in each of the target regions (Dakar, Thiès and Saint Louis): 1) Youth who found salaried employment, 2) Youth who created their own enterprises and Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 8 3) Youth who are still unemployed after training in:  Entrepreneurship  Agriculture  ICT  Life Skills The choice of male and female respondents will reflect the percentage of each sex among the trainees in each region. FGDs will be held with each group on the first day of field research. On the second day, KIIs will be used to explore the variables linked to salaried or self-employment as well as unemployment. Because the youth range in age from 16 to 29, KIIs will help to determine whether older or younger youth were likelier to be employed. Similarly, education levels range from primary or Koranic to post-graduate education. The KIIs will seek to find out how pre-training education helped or hindered success in finding or creating employment. All 288 youth who completed the two cycles of training will be invited to participate in the FGDs and KIIs. We will work to maximize the response rate through utilizing local institutional networks (in collaboration with Synapse) to mobilize participants; as well as through compensation for transport and lunch costs. In addition an e-survey is planned in order to reach youth with email addresses but who are unable to travel to the research locations. Synapse will email these youth the appropriate KII questionnaire to fill out and return. The questionnaire will be the same one used for the KIIs with youth at the interviewing locations. Observation during visits will be made to partner organizations, youth employers or youth-run farms as time and accessibility allow2 . The evaluation plan will be submitted to the USAID COR for approval prior to the commencement of field work. To ensure completeness and validity of the evidence, the evaluation team will use a mix of data collection methods in this evaluation. Table 2 below identifies this range of methods and associates them with the evaluation questions. Additional detail on each method and its application to the evaluation is provided below. Table 2: Data Collection Methods for the Six Evaluation Questions Methods Evaluation Questions Desk Review, including data base, budget and expenditure results 1, 5, 6 Focus Group Discussions 2, 3, 4, 6 Key Informant Interviews 2, 3, 4, 6 Observation at Site Visits 6 1. Desk Review In order to answer Questions 1, 5 and 6, the team conducted a review of documents and records to enable it to build a historical timeline of the project from its inception to the final evaluation. This will include understanding the environment in which the project was initiated, the development of work plans, the implementation and monitoring of activities, achievement of milestones and use of financial resources. Through the process of secondary data collection from various stakeholders the team will 2 The JA project is rural-oriented and some workplaces may not be easily accessible from the cities, particularly since the research coincides with the rainy season in Senegal. Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 9 develop a holistic view of whether or not the project goals have been achieved, and how the broad range of project stakeholders and beneficiaries perceive the outcome of the overall goal. Documents will be used from various sources including the Jeunes Agriculteurs project, IYF, the Government of Senegal and research on similar programs on youth employment, An illustrative list of IYF and Synapse documents sources is shown in Table 33 . A more complete bibliography is presented in Annex IV. Table 3 Illustrative List of Documents and Records to be Reviewed Documents & Records Diouf, N. ( ). La Collecte et l’Analyse de Données pour l’Evaluation du Projet Jeunes Agriculteurs. Hilmer, D. Fox, L. (2014). L’Emploi des Jeunes en Afrique subsaharienne. Banque Mondiale et Agence Française de Développement. .International Youth Foundation (IYF). (2011). Youth assessment: The road ahead. Executive Version. International Youth Foundation (IYF). (2011). Youth assessment: The road ahead. Volume 1: Main Report, July. International Youth Foundation (IYF). (2011). Youth assessment: The road ahead. Volume 2: Annexes, July. International Youth Foundation. (2013). Youth map Semi-Annual Report April 1 – September 30 2013. International Youth Foundation (IYF). (2014). YouthMap Monitoring Table Global, Annex A. International Youth Foundation (IYF). (2014). YouthMap Monitoring Table Uganda. International Youth Foundation (IYF). (2014). YouthMap Monitoring Table Senegal. International Youth Foundation (IYF). (2014). YouthActionNet Regional Leadership Institutes. International Youth Foundation (IYF). (2014). YouthMap Monitoring Table Liberia. International Youth Foundation. (2014). Youthmap Semi-Annual Report, October 1, 2013 –March 31, 2014. International Youth Foundation (IYF). (2014). Leverage Report-Leverage Commitments – Total, Annex B. International Youth Foundation. (2014). Youthmap Semi-Annual Report October 1, 2014 – March 31, 2015. International Youth Foundation. (2014). Youthmap Semi-Annual Report April 1, 2014 – September 30, 2014. International Youth Foundation. (2015). CIES: Ubuntu! Life Skills Training for Out-of-School Youth in Senegal March, Washington, DC. International Youth Foundation (IYF). Results Framework. Youth map Senegal. International Youth Foundation (IYF). Youth map Budget. Ministère de la Fonction Publique et de l‘Emploi (2010) : Session du Haut Conseil pour l’Emploi et la Formation en vue de l’adoption du Projet de Politique Nationale de l’Emploi, République du Sénégal. Synapse center (2013). Jeunes agriculteurs. Grille d’évaluation des projets. Phase intensive, kit d’orientation du jury. Synapse Center. Jeunes agriculteurs. Rapport sur les partenariats, le leveraging et les formations techniques. Synapse Center. Jeunes agriculteurs. Rapport sur le business development services. Synapse Center (2014). Jeunes agriculteurs. Rapport final, octobre. 2. Key Informant Interviews (KIIs) and Group Interviews (FGDs) 3 The complete bibliography is presented in Annex IV Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 10 The KIIs and FGDs will cover the progress and success or non-success of JA activities, management structure, sustainability, lessons learned for future programs, interactions and collaboration with key stakeholders, and answers to evaluation questions 2, 3, 4 and 6. The team will discuss the extent to which outcomes (or lack of outcomes) in targeted indicators might be due to project interventions or to other factors. Notwithstanding the obstacles posed by attribution and the counterfactual, it will be useful to have some idea – even qualitatively -- if the project itself may take responsibility for improvements, or if the improvements (or the lack thereof) are related to other factors. Specific individuals to be interviewed in partner organizations will be identified in consultation with Synapse Center at the start of the mission in Senegal. As noted above we will initially attempt to contact by phone/mail all 288 program participants. Based on the response rate achieved through this approach, we will as needed subsequently for the FGDs apply stratified random sampling within the categories of salaried, self-employed and unemployed graduates of the JA project activities, as well as for purposes of ensuring appropriate gender balance. If appropriate information is not available or incomplete for recent graduates (June 2015), random sampling will be done by age group (16- 20; 21 – 25 and 26 – 29). Purposive selection of participants who participate in the FGD’s will be utilized for the KIIs; in a manner designed to ensure that these include a reasonably robust spectrum of educational and work/entrepreneurship experience characteristics. In this regard relative emphasis for the KIIs will be placed on selection of candidates from the first cohort for each age group, since they will have more post-training experience than the more recent cohort. The process will operate as follows: We will plan 5 FGD’s per region with about 10 particpants each, and then select 4-5 from each group to participate in KIIs in the afternoon. The afternoon sessions will begin with (1) the selected KII interviewees convening as a group and spending the 1st hour filling out the questionnaires; and then (2) conducting follow-up interviews with the KII interviewees during the remainder of the afternoon. Through this approach we will attempt to ensure that at least 140-150 participants (approx. 50%) are included in the FGDs, and at least 45 KII’s with participants are conducted. In the case of partner organizations, KIIs will be conducted with at least one representative of each organization. Table 4: Categories of Institutions for key informant and group interviews Region Organization Dakar - Synapse Center - Association Exode Urbain/ Rural - Fédération Producteurs des Niayes - Fongip - Institut de Technologie Alimentaire - Mairie Guédiawaye - Maison du Consommer Sénégalais - Mamelles Jaboot - Réseau des Jeunes Entrepreneurs du Sénégal Thiès - Synapse Center Thiès - Agence Régionale du Développement - Chambre de commerce - Centre National d'Appui à la Formation Professionnelle - Ecole Polytechnique de Thiès - Ecole Nationale Supérieure d'Agriculture de Thiès - Inspection Régionale de le Jeunesse Saint Louis - Agence Régionale de Développement Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 11 - Amicale Socio-Economique Sportive et Culturelle des Agriculteurs du Walo (ASESCAW) - Centre d’Initiation Horticole (CIH) - Centre Régionale de Ressources Pour l'Emploi des Jeunes (CRREJ) - Université Gaston Berger (UGB) The information obtained through the different interviews will be categorized according to the SoW evaluation questions outlined. Data analysis by the evaluation team will be facilitated by tabulating the responses to the interview questions by using content analysis to extract findings and to draw conclusions from the tabulated responses. DATA ANALYSIS METHODS DATA ANALYSIS The Pragma research team members will use a variety of analysis methods to enable them to answer the six main evaluation questions. Analysis methods are matched to evaluation questions as well as to data collection methods shown in Table 5. Table 5: Analysis methods matched to evaluation questions Data Analysis Methods Evaluation Questions Cross-Tabulations, Frequency Distribution 2, 3,4 and 6 Content/Pattern Analysis 2, 3,4 and 6 Comparison Analysis 2, 3,4 and 6 Interpretation Analysis (Observation) 6 Discourse/Narrative Analysis All The evaluation team will use different methods for data analysis, which are described as follows: Statistical Analysis This method for analysis will be used as a tool to determine what demographic factors (sex, age, education and completion of training) are most closely associated with a) self-employment; b) salaried employment and unemployment, including a return to formal education. This method will be particularly important for answering some of the key questions and will provide a better understanding of the role that the JA project played in achieving (or not achieving) the desired outcomes and meeting the project goals. 1. Quantitative Analysis a. Descriptive statistics: This method includes cross-tabulations, frequency distribution and data disaggregation by parameters such as sex, age, level of education and location. In assessing attribution, we will include cross-tabulations, frequency distribution and data disaggregation by parameters such as sex, age, level of education and location. Multiple regression analysis will be utilized to help establish attribution for change in wages/income for program participants. In general the rigor of the results derived from these tools may be limited by a variety of factors, including the Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 12 quality of reported information on employment/earnings, massive dispersion in reported earnings/income within the participant group, lack of rigorous program outcome information on the second cohort, as well as the short period of program implementation under review. Again if this proves to be the case, this will place even greater relative importance on the rigorous application of core qualitative assessment tools; which will be designed to accurately assess both programmatic outcomes and the critical relationship (or lack of such) between project interventions and those outcomes. Finally, the application of cost-effectiveness and cost/benefit analysis will provide an additional strong quantitative indication of program impact on wage/income streams and on the financial rate of return to project activities; and of the program’s capacity to drive increased wage/income results at a reasonable per capita cost. 2. Qualitative Analysis a. Content and pattern analysis: The transcriptions of the KIIs and FGDs will be analyzed to determine patterns so that comparisons can be made between respondents and sites. b. Comparison analysis: The evaluation team will document how the training, mentoring and job-placement or job-creation work played out in the three target regions. c. Interpretation analysis: This method will use a narrative framework approach to interpret the team’s observations during the field visits to various locations. d. Discourse/narrative analysis: This method will be used to analyze the results of the in-depth interviews conducted under the qualitative evaluation. The table in Annex I shows the relationships between these analytic methods and the evaluation questions. Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 13 ANNEXES I) Getting to Answers (G2A) II) Workplan III) Tools IV) References V) Roles & Responsibilities Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 14 ANNEX 1: GETTING TO ANSWERS Evaluation Questions on Program Effectiveness and Impact Sub-questions Data sources Methods Sampling or Selection Approach (if needed) Data Analysis Methods 1. How effective were the approaches adopted for each of the four components of the program in reaching the goals of the program? 1. Entrepreneurship 2. Agriculture 3. ICT 4. Life Skills What was the experience of youth who found salaried employment after completing their training? Views of the salaried youth. Views of trainers, Synapse and partner staff as well as employers. FGD & KIIs Using lists provided by Synapse, the team will make random choices of male and female youth in this category for the focus groups (FGD) and KIIs. Synapse will suggest knowledgeable partner staff and employers for KIIs. Data will be gathered on the quality of the jobs in terms of salary & benefits, hours, promotion…. Statistical and/or descriptive analysis as appropriate. What was the experience of youth who gained self￾employment after completing the training? Views of the self￾employed youth. Views of trainers, Synapse and partner staff. FGD & KIIs Using lists provided by Synapse, the team will make random choices of male and female youth in this category for the focus groups (FGD) and KIIs. Synapse will suggest knowledgeable partner staff for KIIs. Statistical and/or descriptive analysis as appropriate. What was the experience of youth who did not find employment (or lost it) despite their training? Views of the unemployed youth. Views of trainers, Synapse staff and partner staff. FGD et KIIs Using lists provided by Synapse, the team will make random choices of male and female youth in this category for the focus groups (FGD) and KIIs. Synapse will suggest knowledgeable partner staff for KIIs. Statistical and/or descriptive analysis as appropriate. How useful were internal assessments of JA YouthMap to design and/or enhance employment activities? M&E procedures used to identify problems and develop solutions in the training process and FGD/ KIIs The persons in charge of M&E at Synapse and its partners will be interviewed. Youth beneficiaries will also respond to the question Descriptive analysis Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 15 the finding or creation of employment How were lessons learned from the first cohort applied to the second? Synapse and partners KIIs The persons in charge of M&E at Synapse and its partners will be interviewed. Descriptive analysis 2. What was the overall impact on the youth trained by age, sex, region, educational level, experience in employment? Which demographic variables are most strongly associated with : 1. Finding salaried employment? 2. Becoming self￾employed? 3. Remaining unemployed? a) Views of employers and partners. b) Data on the youth. KIIs Excel data base provided by Synapse Interviews with partner staff and trainers. a) Descriptive analysis b) Statistical analysis. Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 16 Evaluation Questions on Program Effectiveness and Impact Sub-questions Data sources Methods Sampling or Selection Approach (if needed) Data Analysis Methods 3. To what extent was the capacity of the various activity partners strengthened through JA and what was the outcome of the capacity￾building efforts? Also, what contributions were made by partners to the JA project (such as lecturers, trainers, rooms, land ...)?  How were the partners brought into the JA project?  What capacity-building or other benefit was provided by the JA Project for the partners? Interviews (KIIs) with representatives of the relevant partner organizations. . Partner questionnai re. Identificati on of the most frequently cited themes. Using lists of names, positions and contact information to be provided by Synapse, the team will interview the relevant partner representatives. Statistical and descriptive analysis.  What is the relationship among the partners, Synapse Center and IYF?  What capacity-building activities did IYF provide for the partners? Contracts, quarterly reports or evaluations of the training programs for participating youth. Documenta ry review and KIIs with partner representati ves. Descriptive analysis. What kind of contributions were made by the project partners (In-kind contributions, staff time, classrooms, etc.)? Contracts, quarterly reports or evaluations of the training programs for participating youth. Documenta ry review and KIIs with partner representati ves. One or more interviewees will be identified by Synapse, who will also provide their contact information. Statistical and/or descriptive analysis as appropriate. 4. What trends emerged from the data regarding sex, sectors of interest, employer retention, employment quality in salaried and self￾employment? FGDs and KIIs with youth and partner organizations. The employment results of all the 288 youth graduates of the JA program. Statistical and/or descriptive analysis as appropriate. Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 17 Evaluation Program Sustainability and Scalability Sub-questions Data sources Methods Sampling or Selection Approach (if needed) Data Analysis Methods 5. What is the cost effectiveness of the JA Project?  What is the value of the USAID budget to the direct costs of the project?  What is the value of the in￾kind contributions of Synapse and the partners?  What was the unit cost of: o Placing a program graduate in a salaried job? o Helping a graduate to set up his or her own enterprise?  How do activity costs compare to similar activities (can be most effectively carried out if data are made available by USAID on similar programs—publically available reports will be reviewed as well to assemble comparative data)?  What was the financial rate of return on the JA project? Budget and expenditure figures for youth training, job creation or placement, including the value of in-kind contributions from Synapse and the project partners. Information about the profitability of youth enterprises gathered in KIIs with youth. .Cost figures, including the relevant proportionate value of IYF overhead costs and cost figures from: other USAID programs internationally, and if possible from UN & other donor & NGO projects in Senegal. The salaries and/or net profits or losses of the first cohort. The unit cost to the project of producing a successful project graduate. Calculation s using Excel. Data on self-employed and salaried participants in both cohorts will be used. Calculation of unit costs. The cost of failure to find or create jobs will be factored into the unit costs of successful job placement and creation Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 18 6. What is the likelihood of sustainability of program activities?  What was the cost of each component of the JA project?  Which costs could have been reduced/cut in view of prolonging its activities with the support of Senegalese and/or foreign partners?  Which activities can be scaled up, using lessons learned from the JA experience?  To what degree are local governmental or private sector/NGO institutions interested in supporting, including financially, follow-on training/job placement/entrpreneurship interventions modeled on the JA initiative. Project budget and financial reports. Data from KII s with project partners. Calculation s using Excel. All components will be reviewed. Calculations in Excel. Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 19 DATA COLLECTION AND ANALYSIS PLAN I. Primary Data Collection The sample size consists of 288 young people divided between Dakar, Thiès and Saint Louis. Primary data about the JA project and the results of its training activities will be collected using two questionnaires. 1. The Partner Questionnaire and 2) The Youth Questionnaire - Data Collection All the youth will be contacted by telephone and/or email to find out how many would be able to participate in FGDs and KIIs at given locations on specified days in Dakar, Thiès and St. Louis. Focus Group Discussions (FGDs) Random sampling will be used to select the youth who are willing and able to participate in Focus group discussions. Assuming a robust positive response rate, the participants will be stratified into FGDs for currently employed youth, self-employed youth and unemployed youth of both sexes. The research team plans to hold six participant FGDs per location: Organization of the Youth FGS Locations Age Groups at Each Location Focus Groups by Employment Status at Each Location Dakar Thiès St. Louis 16-22 and 23 - 29 Salaried Self-employed Unemployed Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 20 In each city, focus group discussions will be held with youth participants in two age groups who are a) currently working for an employer, b) self￾employed and c) unemployed. Depending on the number of respondents available, each focus group will comprise 7 to 10 participants. The number of male and female participants chosen will reflect as closely as possible the sex-breakdown of participants by employment status in each age group. Key Informant Interviews (KIIs). KIIs will be used with both the youth respondents and individuals from partner organizations who played a key role in supporting the JA project. In the case of the youth, articulate participants in the FGDs will be invited to participate individually in KIIs in order to explore issues related to how they view the effectiveness of the training program components in terms of the youth’s employment outcomes. Region Partner Organizations Dakar 1. Association Exode Urbain/ Rural 2. Fédération Producteurs des Niayes 3. FONGIP 4. Institut de Technologie Alimentaire 5. Mairie Guédiawaye 6. Maison du Consommer Sénégalais 7. Mamelles Jaboot 8. Réseau des Jeunes Entrepreneurs du Sénégal Thiès - Agence Régionale du Développement - Chambre de commerce - Centre National d'Appui à la Formation Professionnelle - Ecole Polytechnique de Thiès - Ecole Nationale Supérieure d'Agriculture de Thiès - Inspection Régionale de le Jeunesse Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 21 Saint Louis 1. Agence Régionale de Développement 2. Amicale Socio-Economique Sportive et Culturelle des Agriculteurs du Walo (ASESCAW) 3. Centre d’Initiation Horticole (CIH) 4. Centre Régionale de Ressources Pour l'Emploi des Jeunes (CRREJ) 5. Université Gaston Berger (UGB) - Data Entry and Processing The work of computerizing of the data from the survey will be carried out through the following steps: - The collection, verification and classification of completed questionnaires; - Data input and control; - The cleaning and correction of files; - Tabulation and production of results; - Archiving and securing data in database. This organizational process provides an appropriate means of managing the questionnaires and all of the files created for the evaluation. Regarding the software to be used,  EXCEL will be used for data entry, cleaning and management;  SPSS (Statistical Package For Social Sciences) will be used to generate statistical tables and other calculations; - Data analysis Once the data have been collected, two data entry models will be developed. The procedure described above will be followed As regards the analysis, two types of data are concerned, the qualitative variables and quantitative variables For qualitative variables, descriptive statistics will be generated (frequencies, crosstabs, sums and ratios); while for quantitative variables, there will be averages, minimum, maximum and standard deviation); Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 22 A comparative analysis of averages will be done for each of the three research zones; And finally a regression analysis (linear model) will be used to target factors associated with success or failure among the categories of youth participants. LIMITATIONS OF THE PROPOSED EVALUATION DESIGN AND METHODOLGY The main limitations of the evaluation design and methodology are the following: 1. Difficulties in contacting the former youth trainees. As many as 92 of the 288 former youth trainees may not reachable either because Synapse does not have currently valid phone numbers or email addresses or some are out of reach by cell phone. As soon as possible, Synapse staff will ask the partner organizations in the three regions for updated youth contact information. In many cases, these partners hired or helped youth graduates to set up their own enterprises. Thus the partners will be useful in contacting the youth to invite them to participate in the research. 2. Challenges in attending the FGDs and KIIs For work, personal or family reasons, some youth may be reluctant to travel to the appropriate research location on the designated days to participate in the FGDs and KIIs. We will work to address this challenge rate through utilization of local institutional networks (in collaboration with Synapse) to mobilize participants; as well as through compensation for transport and lunch costs. In addition an e-survey is planned in order to reach youth with email addresses but who in the end are unable to travel to the research locations. Synapse will email these youth the appropriate KII questionnaire (the same one used in the face-to-face interviews) to fill out and return. 3. Self-Selection of FGD Participants Participation in FGD’s will inevitably be partially dependent on willingness to incur the time and travel inconvenience associated with the interview process; this could in turn lead to some bias in the weighting of participants towards those with relatively strong views (most likely those with positive views) of the program. The design of the questionnaires has been developed in a manner designed to facilitate the most accurate and objective transmission possible of empirical information regarding program experiences of the interviewees; such that the degree of enthusiasm of response can be disentangled from empirical outcomes described. Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 23 ANNEX II: WORK PLAN In-country Workplan of the JA Youth Map – Senegal Evaluation Period of July 27 to August 28, 2015 Activities # Days Planned Dates Methods Target Groups Comments Arrival of Evaluation Team Leader 1 day 7/29/2015 Evaluation Team Leader arrives in the field. Eric Allemano Preparatory Work 3 days 30/07/ to 01/08/2015 Meeting in Dakar of the research team to finalize plans for the field work. Eric Allemano, Alioune Dieng, Babacar Diouf and Synapse Center staff. Fine-tuning of roles and logistics. USAID￾Dakar Briefing and guidelines for the field work. 1 day 03 or 04/2015 Meeting with the mission to be briefed on the Project. USAID. Interview with: Ms. Michelle Barrett, POC YouthMap Ms. Sharon Carter Head Program Office Mr. Roy Geiser, Program Officer Field team and USAID counterparts Review of the research methodology and tools. Workplan presentation. Data collection in the Dakar region 6 days 03 to 04/08/2015 Focus group discussions (FGDs) with random sample of youth beneficiaries. Post-FGD Key informant interviews (KIIs) with selected youth in the sample. Youth beneficiary age groups 16-22 ; 23-29 - Employees - Self￾employed - Unemployed 05/08/2015 Key informant interviews (KIIs) with project partners. Partners - Association Exode Urbain/ Rural - Fédération producteurs des Niayes - Fongip - Institut de Technologie Alimentaire - Mairie Guédiawaye - Maison du Consommer Sénégalais - Mamelles Jaboot - Réseau des Jeunes Entrepreneurs du Sénégal Data triangulation 06/08/2015 – 07/08/2015  Visits to selected partner organizations in the Dakar region Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 24  Site visits to selected youth enterprises in the Dakar region Input and analysis of data collected in Dakar (Excel). 08/08/2015 Print-out of data and preparation of a memo of key findings. Research team Data collection in the Thiès region 6 days 10 to 11 /08/2015  FGDs with sample of youth beneficiaries.  Post-FGD KIIs with selected youth in the sample. Youth beneficiary age groups 16-22 ; 23 -29 - Employees - Self￾employed - Unemployed 12/08/2015 Interviews (KIIs) with project partners Partners - Agence Régionale du Développement - Chambre de commerce - Centre National d'Appui à la Formation Professionnelle - Ecole Polytechnique de Thiès - Ecole Nationale Supérieure d'Agriculture de Thiès - Inspection Régionale de le Jeunesse Data triangulation 13/08/2015 – 14/08/2015  Visits to selected partner organizations in the Thiès region  Site visits to selected youth enterprises in the Thiès region Input and analysis of data collected in Thiès (Excel). 15/08/2015 Print-out of data and preparation of a memo of key findings. Research Team Data triangulation Data collection in the SaintLouis region 6 days 17 to 18 /08/2015  Focus group discussions (FGDs) with sample of youth beneficiaries.  Post-FGD Key informant interviews (KIIs) with selected youth in the sample. Youth beneficiary age groups 16-22; 23-29 Selected youth from the FGDs - Employees - Self￾employed - Unemployed Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 25 19/08/2015 Key informant interviews (KIIs) with project partners. Partners - Agence Régionale de Développement - Amicale Socio-Economique Sportive et Culturelle des Agriculteurs du Walo (ASESCAW) - Centre d’Initiation Horticole (CIH) - Centre Régionale de Ressources Pour l'Emploi des Jeunes (CRREJ) - Université Gaston Berger (UGB) Data triangulation 20/08/2015 – 21/08/2015  Visits to selected partner organizations in the St. Louis region  Site visits to selected youth enterprises in the St. Louis region Input and analysis of data collected in St. Louis (Excel). 22 /08/2015 Print-out of data and preparation of a memo of key findings Research team Work in Dakar at Synapse office : Preparation of the preliminary results of the research in Dakar 4 days 24 to 27 /08/2015 The research team with inputs from Synapse Research team Integration and editing of partial reports and field memos. Debriefing 1 day 28 /08/2015 Presentation The research team and USAID Dakar Presentation of the preliminary report. Total 28 days Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 26 ANNEX III: TOOLS The draft Partner and Participant questionnaires are presented below. Interview Format for USAID-Dakar Date of interview:____________________________________________________________________ Person(s) interviewed + titles: __________________________________________________________ 1. How long have you been involved with the JA project? _______________________________ 2. Have you made site visits to the youth training? ______ If so, how often and at which sites? _____________________________________________________________________________________ _____________________________________________________________________________________ _____________________________________________________________________________________ _________________________________________________________ 3. In your opinion, what are the strong points of the project in terms of its a) objectives, b) design and c) management ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ 4. How well has the project succeeded in terms of capacity building of Synapse and the partner organizations? Please give some examples. ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ 5. What factors favor the sustainability of the youth training and job placement/creation process started by the JA project? What are the risk factors for failure? How would you assess Senegalese Government ownership of this process of youth development? Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 27 ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ 6. What synergies do you think could be developed between Synapse and its partners in the JA project and other initiatives on youth employment in Senegal? 7. If we could do a Phase II of the JA project, what aspects would you like to see - maintained? ___________________________________________________________________________ __________________________________________________________________________ - changed? ___________________________________________________________________________ ___________________________________________________________________________ - What new aspects would be added? ___________________________________________________________________________ ___________________________________________________________________________ Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 28 Draft Partner Questionnaire 1. Zone :……………………………………… !___ ! 1. Dakar, 2. Thiès, 3. Saint Louis 2. Name & title of the participant: ………………………………………………………………………………………. 3. Sex :……………………………………… !____ ! 1. Male, 2. Female 1. Contact information:  Address: .............................................................................................  Telephone No. ................................. Email address...............................  Website: ………………………..… 2. Main activity of the partner: .................................................................................. 3. Other activities: .................................................................................... 4. How was your organization selected to be part of the project? 5. What kind of relationship was there with IYF and Synapse during the JA project? (ex. training youth on entrepreneurship, ICTs or other topic), mentoring, counselling, follow-up on employment, funding, other….) 6. What was the objective of this collaboration? (ex. preparing youth for finding jobs? Providing guidance on self-employment? Other objective? 7. In terms of results, how useful was this collaboration? ! ____! 1 = Highly; 2 = Generally; 3 = Somewhat; 4 = Not very. 8. What challenges arose in developing your partnership with or providing services for: a. Synapse and IYF? b. The youth trainees? c. Other partners? 9. How were these challenges met? Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 29 10. If your organization were to work again with Synapse and IYF or a different external partner on youth employment issues, what would you do differently? 11. Which other partners should have been involved with the JA project but were not included? 12. What would these partners have been able to contribute to the success of the project? 13. In your view, which aspects of the project were:  The most successful? Why were they successful?  The least successful? Why were they not (very) successful? 14. In your view, how sustainable are the results of the project? ! ____! 1 = Highly; 2 = Fairly; 3 = Somewhat; 4 = Not very (much). 15. What are the main factors that support or mitigate the sustainability of the project results? 16. If you had to give an overall rating to the project, what grade would you give it? ! ____! 1= Excellent, 2 = Very good, 3 = Good, 4= Mediocre, 6=Poor Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 30 Draft Participant Questionnaire 1 Zone :……………………………………… !___ ! 1. Dakar, 2. Thiès, 3. Saint Louis 2 Name of the participant: ………………………………………………………………. 3 Age…………………………………………… !____ ! years 4 Sex :……………………………………… !____ ! 1. Male, 2. Female 5 Employment Status: !____ ! 1. Employed (salaried), 2. Self-employed, 3. Unemployed 6 Address:……………………………………………………………………………… 7 Tel. number: ……………………………………………………………………… 8 Email Address :………………………………………………………………………. 9 Marital status? 9.1 Do you live alone ? 9.2 Do you live with a spouse or partner? 9.3 Do you live with your parents or a guardian ? 10 What is your highest level of formal education? 11 Did you have work when you joined the JA program? 11.1 If yes, was the work temporary or permanent/long-term? 11.2 Did you receive a regular salary? If yes, give the amount in FCFA. 12 Please describe the type of work and the enterprise that hired you. 13 Are you still employed? 13.1If yes, do you receive a salary? How much? 13.2What kind of work? 13.3Is this work permanent or temporary? 14 How were you able to find this employment? 15 Are you self-employed? 15.1 If yes, what kind of business? Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 31 15.2 What major challenges are you facing? What are the major successes you have experienced since your participation in the program? 15.3 What is your monthly (or annual) profit after deducting operating costs from the revenue derived from your products or services? 16 If not employed, how long have you been without work? !___ ! 1. Less than one month, 2. 1-3 months, 3. 4-6 months, 4. 7-12 months, 5. More than a year. 17 If not currently employed, what kind of work are you looking for? What are the major challenges which are limiting your capacity to find a job? 18 What are the strengths and weaknesses of the Synapse/IYF program in terms of: 18.1 Guidance and training 18.2 Job placement 18.3 Professional development (guidance, placement services and participation in networking events with former trainees). 18.4 Entrepreneurship training. . 19 In your view, what could be done to improve each of the program components? 20 Are you satisfied with the recruitment process for the program? !__! 1= Yes, 2=No 21 If not, why not? 22 How would you evaluate the IYF and Synapse staff? IYF !__ ! Synapse !__ ! 23 What kind of relationship do you have with the technical and financial partners of the program? !__ ! 1=Professional, 2= Friendly, 3=Religious, 4=Other (please specify). 24 Are you satisfied with the support provided by the technical and financial partners? !__ ! Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 32 1= Yes 2=No 25 If dissatisfied, for what reason(s)? 26 If you were to evaluate the components of the program, how would you rate each one ? 1= Excellent, 2 = Very good, 3 = Fairly good, 4= Acceptable, 5=Mediocre, 6= Poor i. Entrepreneurship !__ ! ii. Agriculture !__ ! iii. ICT !__ ! iv. Life Skills !__ ! 27 Other comments and recommendations (please detail) ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ___________________________________ Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 33 ANNEX IV: REFERENCES BIBLIOGRAPHY Publications, including Senegalese Government documents Hilmer, D. Fox, L. (2014). L’Emploi des Jeunes en Afrique subsaharienne. Banque Mondiale et Agence Française de Développement. Ministère de la Fonction Publique et de l‘Emploi (2010) : Session du Haut Conseil pour l’Emploi et la Formation en vue de l’adoption du Projet de Politique Nationale de l’Emploi, République du Sénégal. OIT (2013): Global employment trends 2013, Recovering from a second jobs dip, Geneva. République du Sénégal (2003). Programme Décennal de l’Education et la Formation. République du Sénégal (2013). Programme d’Amélioration de la Qualité, de l’Équité et de la Transparence (PAQUET), juillet. République du Sénégal (2014). Plan Sénégal Emergent, février. République du Sénégal (2015). Loi d’Orientation de la Formation Professionnelle et technique. Articles Sène, I. (2006). L'agriculture au Sénégal. Les enjeux ruraux et agraires en Afrique - De l'agriculture familiale à l'agro-business : la transition est-elle possible ? Quel avenir pour les agricultures d'Afrique de l'Ouest ?, Séminaire, novembre. Toulmin C., Guèye B. (2003). Transformations de l'agriculture ouest africaines et rôles des exploitations familiales, Dossier IIED, n°123. Zongo, M. (2010). La dimension foncière de l'agrobusiness au Burkina Faso: étude de cas dans la province du Ziro, Cahiers du Cerleshs Tome XXV, n° 35, janvier, pp. 127-159. Research reports Banque Mondiale (2007). Sénégal : À la recherche de l’emploi le chemin vers la prospérité, en collaboration avec le CEPOD, Dakar. CEPOD (2012) : Analyse diagnostique de l’emploi au Sénégal, Ministère de l‘Economie et des Finances. 10 p. Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 34 Jeunes Agriculteurs Project reports Coulibaly, V. (2014). Jeunes agriculteurs program. Recrutement 2014, Dakar, janvier. Coulibaly, V. (2014). Rapport global de formation. Expédition pédagogique et retraite. Coulibaly, V. (2014). Rapport global de formation. Compétences de vie (Life Skills). Coulibaly, V. (2014). Rapport de formation. Entrepreneuriat, juin. Diouf, N. ( ). La Collecte et l’Analyse de Données pour l’Evaluation du Projet Jeunes Agriculteurs. International Youth Foundation (IYF). Termes de Référence. Evaluation du Programme Youth Map Sénégal International Youth Foundation (IYF). (2010). Accounting and Appropriation data. International Youth Foundation (IYF). (2011). Youth assessment : The road ahead. Executive Version. International Youth Foundation (IYF). (2011). Youth assessment : The road ahead. Volume 1 : Main Report, July. International Youth Foundation (IYF). (2011). Youth assessment : The road ahead. Volume 2: Annexes, July. International Youth Foundation (2011). Youthmap workshop report, novembre. International Youth Foundation (2011). Youthmap workshop annexes, novembre. International Youth Foundation. (2013). Youthmap Semi-Annual Report April 1 – September 30 2013. International Youth Foundation (IYF). (2014). YouthMap Monitoring Table Global, Annex A. International Youth Foundation (IYF). (2014). YouthMap Monitoring Table Uganda. International Youth Foundation (IYF). (2014). YouthMap Monitoring Table Senegal. International Youth Foundation (IYF). (2014). YouthActionNet Regional Leadership Institutes. International Youth Foundation (IYF). (2014). YouthMap Monitoring Table Liberia. International Youth Foundation. (2014). Youthmap Semi-Annual Report, October 1, 2013 –March 31, 2014. Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 35 International Youth Foundation (IYF). (2014). Leverage Report-Leverage Commitments – Total, Annex B. International Youth Foundation. (2014). Youthmap Semi-Annual Report October 1, 2014 – March 31, 2015. International Youth Foundation. (2014). Youthmap Semi-Annual Report April 1, 2014 – September 30, 2014. International Youth Foundation. (2015). CIES: Ubuntu! Life Skills Training for Out-of-School Youth in Senegal March, Washington, DC. International Youth Foundation (IYF). Results Framework. Youth map Senegal. International Youth Foundation (IYF). Youth map Budget. Échevin, D. Sylla, M.B. Ly, M.A. Seck F.G.C. (2013). Youth employment in northern Senegal: Creating Job Opportunities for Young People, march. International Youth Foundation (2014). Senegal fact sheet. Synapse center (2013). Jeunes agriculteurs program. Quaterly report, janvier-mars. Synapse center (2013). Jeunes agriculteurs program. Quaterly report, avril-juin. Synapse center (2013). Jeunes agriculteurs program. Rapport trimestriel, juillet-septembre. Synapse center (2013). Jeunes agriculteurs program. Rapport trimestriel, octobre-décembre. Synapse center (2013). Jeunes agriculteurs. Grille d’évaluation des projets. Phase intensive, kit d’orientation du jury. Synapse Center. Jeunes agriculteurs. Rapport sur les partenariats, le leveraging et les formations techniques. Synapse Center. Jeunes agriculteurs. Rapport sur le business development services. Synapse Center (2014). Jeunes agriculteurs. Rapport final, octobre. Press releases Dasylva, E. (…). Entrepreneuriat agricole. 320 jeunes s’engagent dans l’agrobusiness. Walfadjiri. Diagne, M.K. (2014). « Jeunes agriculteurs » : 172 entreprises créées à Thiès, Dakar et Saint￾Louis, Soleil du 24 juin 2014. Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 36 Dieng, S. (…). Promotion d’entreprises agricoles à fort impact social et économique. 118 jeunes entrepreneurs formés en agro-business, Thiès, n°6696, p.7. Thiam, I. (…). Promotion de l’emploi. Synapse center accompagne près de 320 jeunes. On-line documents DIOP. D. (2011). Le chômage et le sous-emploi au Sénégal : Des maux très enracinés, Université de Montréal. www.sendeveloppementlocal.com, consulté le 1 juillet 2015. International Youth Foundation. (2014). YouthMap Tanzania: Assets & Opportunities: http://library.iyfnet.org/library/youthmap-tanzania-assets-opportunities. International Youth Foundation. (2014). YouthMap Tanzania: A Cross-Sector Analysis of Youth in Zambia: http://library.iyfnet.org/library/youthmap-zambia-cross-sector-analysis-youth￾zambia. International Youth Foundation. (2014). Voices of the Next Generation: An Assessment of Youth in Mozambique http://library.iyfnet.org/library/voices-next-generation-assessment-youth￾mozambique. International Youth Foundation. (2013). YouthMap DRC–Conogolese Youth: Assets & Opportunities http://library.iyfnet.org/library/youthmap-drc-congolese-youth-assets￾opportunities. International Youth Foundation. (2011). YouthMap Senegal – Youth Assessment: The Road Ahead: http://library.iyfnet.org/library/youthmap-senegal-youth-assessment-road-ahead. International Youth Foundation. (2011). Navigating Challenges. Charting Hope. A Cross Sector Situational Analysis on Youth in Uganda: http://library.iyfnet.org/library/navigating-challenges￾charting-hope-cross-sector-situational-analysisyouth-uganda. International Youth Foundation. (2013). Promoting Youth Employment through Internships: A Guide to Best Practices. Retrieved from:http://library.iyfnet.org/library/promoting-youth￾employment-through-internships-guidebest-practices. Récupéré le 25-06-2014 : http://www.seneweb.com/news/Societe/entrepreneuriat-314-jeunes￾formes-en-agro-businessmen_n_128954.html Annex 2 INCEPTION REPORT – EVALUATION of the JEUNES AGRICULTEURS PROJECT – SENEGAL 37 ANNEX V: TEAM ROLES AND RESPONSIBILITIES The Pragma research team comprises three members: Dr. Eric Allemano, Evaluation Team Leader/USAID YouthMap Senegal Evaluation Dr. Allemano has overall responsibility for supervising the team members in the data collection and analysis process and for preparing and submitting the final report to Pragma Corporation. He will have primary responsibility for the timely delivery and overall quality of the final report. In Senegal, Dr. Allemano will explain the work of the research team to USAID and project implementing partners during an initial briefing in Dakar. During the field research he will conduct Focus Group discussions (FGDs) and key informant interviews (KIIs) with youth and project partners at the research locations. He will work with the research team on analyzing and interpreting the data gathered and brief USAID-Senegal on key findings of the field work at the end of the mission. Dr. Allemano will ensure team accountability for high-quality work during the field research and inform Pragma in a timely fashion of any issues that may arise during the data-gathering and interviewing process in Senegal. Dr. Alioune Dieng, national research expert Dr. Dieng will work closely with Dr. Allemano on collecting data for the Inception Report and the final report, as needed. In Senegal, Dr. Dieng will participate in briefings with USAID and meetings with key national project counterpart organizations. Dr. Dieng will conduct individual interviews and focus group discussions with Dr. Allemano at the three research locations in Senegal. He will also be responsible for inputting the data to an Excel data base and producing appropriate quantitative analyses of data gathered during the KIIs and FGDs. He will consult with Dr. Allemano on interpreting and refining the data collected. Dr. Babacar Diouf, national logistics and translation coordinator Dr. Diouf will manage the complex process of contacting the youth graduates of the JA skills training program and arrange for those chosen for FGDs and KIIs to travel to the Synapse offices in Dakar, Thiès and St. Louis for the interviews. He will also work with the Director of Synapse in contacting the partners of the Jeunes Agriculteurs project to delegate suitable officers or technicians to participate in KIIs with Drs. Allemano and Dieng. When needed, Dr. Dieng, who is an experienced researcher, will help to conduct FGDs and/or KIIs. Annex 2 Indicator unit of measure frequency responsible party method/source Target Training curriculum developed, adapted and implemented (ASAP adapted) # of curricular materials adapted and implemented Quarterly Synapse project records 3 No. of youth participating in Youth Map training programs∂ Quarterly Program Coordinator Partnership and Sustainability Manager project records 300 youth No. and % of youth completing YouthMap training∂ Quarterly Program Coordinator              Youth Entreprise Development Manager (BDS Manager) project records 320 youth No. and % of youth beneficiaries demonstrating improved capabilities upon completion of training Quarterly Program Coordinator pre/post test 192 youth; 60% with improved capacities No. of youth participating in Youth Map training programs∂ Quarterly Program Coordinator project records 320 youth No. and % of youth completing YouthMap training∂ Quarterly Program Coordinator project records 320 youth No. and % of youth beneficiaries demonstrating improved capabilities upon completion of training Quarterly Program Coordinator pre/post test 192 youth; 60% with improved capacities No. of youth participating in Youth Map training programs∂ Quarterly Program Coordinator project records 320 youth No. and % of youth completing YouthMap training∂ Quarterly Program Coordinator project records 320 youth No. and % of youth beneficiaries demonstrating improved capabilities upon completion of training Quarterly Program Coordinator pre/post test 192 youth; 60% with improved capacities; No. and % of youth beneficiaries gaining employment or better employment as a result of participation in USG funded workforce development programs*∂ Quarterly Program Coordinator Follow up survey 50%; 160 No. and % of youth beneficiaries transitioning to further education and training as a result of participation in USG‐funded workforce development programs*∂ Quarterly Program Coordinator Follow up survey 96 # of of business plans created Quarterly Program Coordinator project records 60% of entreperenurship trainees submitted business plans to jury No. and % of youth receiving business development support & services Quarterly Program Coordinator project records 192 youth; 60% #of business created/impoved Quarterly Program Coordinator project records 120 businesses created/improved Youth participation in agribusiness value chains increased #  and % of participants with jobs and/or businesses in the agribusiness sector Quarterly Program Coordinator Follow up survey 160 youth; 50% Youth engaged as mentors to future e/ship trainees #  of youth serving as mentors Quarterly Program Coordinator project records 50 youth Result 3: Icnreased sustainability and scale through Public Private Partnerships Youth with improved capacity to start their own businesses Knowledge of entrepreneurship skills increased Knowledge of ICT increased Goal: Enhanced participation of youth with relevant skills in the agriculture sector to stimulate growth of jobs and Knowledge of life skills increased Result 1: Improved access to training and knowledge/skills Result 2: Improved prospects of employment/self‐employment of youth Annex 3 Resources leveraged for YouthMap activities from multi‐sector partners∂ Quarterly Program Coordinator project records $125,000 # of new partnerships created Quarterly Program Coordinator project records 10 Increased capacity at Synapse Center # of systems stregthened Quarterly Program Coordinator Capacity assesment (Baseline/endline) * USAID standard indicator / ∂ Youth:Work standard indicator Partnerships with public, private and civil society partners established Annex 3 YouthMap Monitoring Table Senegal Oct 2012-March 2013 April - Sept 2013 Total Oct 2013 - March 2014 April - Sept 2014 Total Number of assessments completed 1 0 - - Number of people who attended the dissemination event and other program events 100 0 200 - 200 - Number of partnerships created in support of innovation fund program N/A 10 6 6 12 Resources leveraged for YouthMap Senegal N/A 62.500 33.256 1.480 34.736 62.500 0 4.520 4.52 Cash 31.250 - - 31.250 - In-kind 31.250 33.256 1.480 34.736 31.250 4.520 4.520 Number of systems strengthened N/A 1 - 1 - Number of youth applying for program N/A 200 413 369 782 200 827 0 827 Male 293 262 555 515 515 Female 120 107 227 312 312 Number of youth participating in training programs N/A 160 20 160 180 160 160 0 160 Male 80 5 94 99 80 92 92 Female 80 15 66 81 80 88 88 Number of youth completing the program N/A 128 0 0 - 128 135 0 135 Male - 82 82 Female - 53 53 Number of youth receiving mentoring services N/A 120 0 0 - 120 123 0 123 Male - 72 80 80 Female - 48 43 43 Number of youth receiving coaching services N/A 128 20 140 160 128 123 0 123 Male 76 5 84 89 80 80 Female 52 15 56 71 43 43 Number of learning expeditions organized N/A 8 1 7 8 8 8 8 1 7 8 8 8 Number of retreats organized N/A 8 1 7 8 8 8 8 1 7 8 8 8 Number of business plans created N/A 96 33 33 96 101 97 198 33 33 101 97 198 Number of youth receiving business development services N/A 96 20 160 180 96 101 154 255 Male 48 5 94 99 60 92 152 Female 48 15 66 81 41 62 103 Number of businesses created N/A 30 10 10 30 23 33 56 23 33 56 Number of businesses improved N/A 30 31 31 30 53 63 116 53 63 116 Number of youth gaining employment as a result of program participation N/A 100 0 0 - 100 0 0 - Male 50 - 50 - FY2014 Annual Target Actual 2011-12 Completed FY2013 Annual Target Actual Annex 4 Female 50 - 50 - Number of youth setting up businesses as a result of program participation N/A 30 0 10 10 30 23 33 56 Male 5 5 14 27 41 Female 5 5 9 6 15 Number of youth transitioning to further education as a result of program participation N/A 96 96 Male 48 48 Female 48 48 Annex 4 Cost Per Job by Project Annex 5 Cost Cost/job $$ USAID Afghanistan Small and Medium Enterprise Development project. ASMED, was a cost plus fixed fee contract with DAI. The project supported SMEs, the major drivers of Afghanistan’s economic development. Through support for investment, technology, and business development services, ASMED improved private sector productivity by expanding SMEs and increasing employment. ASMED collaborated with Afghan private sector and international partners in key sectors such as agribusiness, carpets, marble, gemstones, and business services. ASMED also facilitated public-private partnerships, grants, and practical and technical training. In its final year, ASMED concentrated on creating professional, commercial linkages among SMEs in 25 key districts. Overall, the program generated 95,000 jobs (according to the information on DAI's website) All project costs Afghanistan 2006 ‐ 2012 95000 113,993,245 1,200 DAI website and IG Report dated 2014 The USAID Education for Income Generation in Nepal Program (a contract to Winrock), was a five-year, $14.7 million program designed to help address the root causes of conflict-exclusion of disadvantaged youth from education, training and employment opportunities. The program combined literacy and life skills education; technical and vocational training linked to employment; training to increase agricultural productivity and raise rural household incomes; and targeted scholarships for disadvantaged youth to increase access to higher (10+2/college certificate) level education. EIG‘s primary beneficiaries were disadvantaged, conflict-affected and internally displaced youth throughout the Mid-Western Region of Nepal. The project final report includes "Number of people gaining employment or more remunerative employment as a result of participation in a USG-funded workforce development program - 9,568" - costs thus are inclusive of the entire contract value - award (2008-2013) Nepal 2008-2013 9568 14,681,683 1,534 Final Report USAID Akazi Kanoze (AK) Project in Rwanda (implemented via a cooperative agreement with EDC). It was a five-year, $9.8 million youth workforce development project designed to provide Rwandan youth ages 14-35 with market-relevant life and work readiness training and support, hands-on training opportunities, and link to the employment and self￾employment job market. Akazi Kanoze was designed to build capacity and create linkages between youth and the Rwandan economy; and in a manner that helps youth access enhanced opportunities for productive engagement in society. AK youth received work readiness and entrepreneurship training, in addition to internship opportunities for on-the-job learning, job placement services and/or business start-up coaching. This was a 5-year $9.8 million program. The number of jobs is reported in the study of the scale and sustainability of the USAID￾funded Akazi Kanoze (AK) youth livelihoods program in Rwanda prepared by the Aguirre Division of JBS International Rwanda 2009-2014 4421 9,800,000 2,217 A study of the scale and sustainability of the USAID-funded Akazi Kanoze (AK) Description Costs included # jobs Country Year Source of Information Cost Per Job by Project Annex 5 Cost Cost/job Description Costs included Country Year # jobs $$ Source of Information USAID Afghanistan Workforce Development Program (implemented via a contract with Creative Associates). The program sought to increase job placements and wages for 25,000 Afghan citizens – at least 25 percent were women – by strengthening the labor pool in major economic areas and addressing the twin problems of high unemployment and scarcity of technically-skilled Afghan labor and trained business managers. It provided technical and business education, training, job placement and support services. By January 2014 the program documented 5,471 placements. The entire value of the award was used for cost calculation. $22.65M divided by 48 months (project duration) and multiplied by 20 months (the point up to which job results were available for the report) Afghanistan 04/2012- 01/2014 5471 9,436,250 1,725 Mid-term evaluation report USAID Improving Access to Employment Program in El Salvador (implemented via a contract with Carana). The program focused on improving the functioning of the Salvadoran labor market to help better match the supply of skilled workers with private sector demand for labor. It had 4 components: (1) Promoting occupational skills standards and competency certification; (2) Expanding consumer based reports and providing career counseling and placement; (3) Improving the labor market information system; and (4) Creating alliances with the private sector. The total value of contract $7.4 divided by 48 months (project duration) multiped by 32 months (the point up to which job results were available at the time of evaluation) El Salvador 2009-2012 4629 4,933,333 1,066 Performance Evaluation Report USAID Youth for the Future (Y4F) program in Jordan implemented by IYF (cooperative agreement). This was originally the Youth:Work Jordan project. Three years into the program, USAID/Jordan changed the program name to Youth for the Future and the goal to “creating an enabling environment with a greater capacity to more effectively serve youth at-risk.” Mission officials said they changed the goal to focus on building the capacity of community organizations and Jordanian Government entities so they could serve at￾risk youth, teach them employable skills, and increase their involvement in community activities. Moreover, IYF faced several challenges trying to implement the program because Jordanian NGOs, community organizations, and MoSD did not have the capacity needed to implement IYF’s employability models. The final report states that 2,678 youth placed in jobs within 6 months of program completion. The entire value of the coop agreement was $33.4 m. Jordan 2010-2015 2678 33,400,000 12,472 Project Final Report The USAID-financed Business Reform & Competitiveness Project (BRCP) is a 3-year project implemented by the Pragma Corporation (contract) to transform enterprise competitiveness in the Information/ Communication Technology (ICT) sector and in other key sectors supported by ICT to improve value chains and increase growth, sales, employment, investment, and exports. BRCP is active in the following areas: Providing technical assistance to businesses; Improving business access to capital; Matching growing businesses with qualified job-seekers; providing targeted training & job placement services; and Improving the enabling environment in key areas All project costs for Year 1. Tunisia 2014-2015 3965 5,000,000 1,261 BRCP Annual report - Year 1 Cost Per Job by Project Annex 5 Cost Cost/job Description Costs included Country Year # jobs $$ Source of Information Leveraging Effective Applications of Direct Investments (LEAD) was a USAID-funded project implemented by PADF that worked to attract investments in small and medium￾sized enterprises (SMEs) based in Haiti, as well as increase the development impact of remittances. To facilitate access to capital LEAD provided matching grants to leverage investment in SMEs and SEs. LEAD also offered technical assistance and workforce training to help enterprises implement their business plans and strengthen their operations. Funds paid by AID at the time of GAO audit and the # of jobs reports at the time, about 3 years into the 4-year program Haiti 2011-2014 634 4,800,000 7,571 GAO audit report Georgia New Economic Opportunities (NEO) Initiative (a USAID contract with Chemonics). By providing assistance to local governments, enterprises, and individuals in Georgia, the New Economic Opportunities Initiative focused on raising incomes, reducing poverty levels, and improving food security for 70,000 rural households. The project was comprised of four components: local economic planning, rural economic development, assistance to strengthen highly vulnerable households and individuals, and long-term housing for internally displaced people. The project was focused on 10 municipalities in five regions of Georgia. Results on jobs came from the annual M&E report, that has figures for the first 2 years of the project. The total value of this 4-year contract is $23.8 million. So half of the total amount I am attributing to the first 2 years Georgia 2011-2013 2377 11,900,000 5,006 Annual M&E Report, results through Sep. 30 2013 Skills and Knowledge for Youth Educational Employment in Guyana Development (SKYE Guyana) implemented by EDC through a cooperative agreement. The project was tasked with: expanding employment, education, and skill-building opportunities for youth at risk; strengthening re-integration of youth offenders into society; and improving the enabling environment for youth development Accoding to the mid-term evaluation report (Dec 2014) 38% of program participants received a job due to their involvement in SKYE (there were 1,058 participants total). The total value of contract $5 divided by 48 months (project duration) multiped by 36 months (the point up to which job results were available at the time of evaluation) Guyana 2011-2015 402 3,750,000 9,328 Mid-term evaluation report and OIG audit report Young Entrepreneurs Program (YEP) in Kosovo was implemented by EDC through a cooperative agreement. The project aimed to better prepare Kosovo youth ages 18–35 for work in a growing market economy. To meet this goal, YEP: provided support and matching seed grant assistance to prepare young entrepreneurs; engaged employers and other leaders to combine resources, skills, and policies to create a sustainable system of opportunities and supports for out-of￾school and out-of-work young people; and invested in the development of a sustainable youth entrepreneurial support system. This system included more youth-inclusive financial and consulting services and a peer-support network with links to networks of established entrepreneurs. All project costs Kosovo 2010-2013 2683 3,270,000 1,219 Project Final Report 131828 214,964,511 1,631 Annex 6 Discount rate 10% Cost: participants ‐1,001,316 288 ‐326,818 94 Discounted cost Total Discounted Cost 2013 ‐ 50% ‐163,409 ‐163,409.21 year 0 ‐311,963.03 2014 ‐50% ‐163,409 ‐148,553.83 year 1 ‐311,963.03 Total Benefit: Change in income Constant Discounted 10% Increase 5% Year ‐311,963.03 ‐311,963.03 0 2013 0.00 0.00 0.00 1 2014 29,400.96 26,728.14 26,728.14 2 2015 58,801.91 48,596.62 51,026.45 3 2016 58,801.91 44,178.75 51,026.45 4 2017 58,801.91 40,162.50 46,387.69 5 2018 58,801.91 36,511.36 42,170.62 6 2019 58,801.91 33,192.15 38,336.93 7 2020 58,801.91 30,174.68 34,851.75 8 2021 58,801.91 27,431.53 31,683.41 9 2022 58,801.91 24,937.75 28,803.10 10 2023 58,801.91 22,670.68 26,184.64 11 2024 58,801.91 20,609.71 23,804.22 12 2025 58,801.91 18,736.10 21,640.20 13 2026 58,801.91 17,032.82 19,672.91 14 2027 58,801.91 15,484.38 17,884.46 15 2028 58,801.91 14,076.71 16,258.60 16 2029 58,801.91 12,797.01 14,780.55 17 2030 58,801.91 11,633.65 13,436.86 18 2031 58,801.91 10,576.04 12,215.33 19 2032 58,801.91 9,614.58 11,104.84 20 2033 58,801.91 8,740.53 10,095.31 21 2034 58,801.91 7,945.94 9,177.56 22 2035 58,801.91 7,223.58 8,343.23 23 2036 58,801.91 6,566.89 7,584.76 24 2037 58,801.91 5,969.90 6,895.23 25 2038 58,801.91 5,427.18 6,268.39 26 2039 58,801.91 4,933.80 5,698.54 27 2040 58,801.91 4,485.27 5,180.49 28 2041 58,801.91 4,077.52 4,709.54 29 2042 58,801.91 3,706.84 4,281.40 30 2043 58,801.91 3,369.85 3,892.18 NPV 215,629.44 288,160.76 IRR after 30 years 5.80% 7.39% *Note: It is assumed that costs were incurred over two years, while incomes from program participants did not reflect any gains until year 1, when half of the participants had completed the program. The change in income in year 2 reflects gains from all cohorts. The change in income for year 1 is assumed to be half of the total measured for year 2. Annex 7 Results of the Fifteen Focus Group Discussions Held with Selected JA Youth in the Dakar, Thiès and St. Louis Regions Participant characteristics One hundred thirty-nine youth participated in the FGDs including 58 women, or approximately 42 percent of the total. In the Dakar Region, there were 48 participants in 6 focus groups, including 19 women. In Thiès, the five focus groups were made up of 45 participants, including 25 women (slightly over 50 percent). The smallest number of focus groups was four, in the Saint Louis Region, where 46 youth including 14 women participated in the FGDs, Few participants (28) were less than 23 years old. The participants tended to be self-employed in the Dakar and St. Louis groups while the largest number of participants in Thies was students (17 out of 45 participants). The educational levels were fairly high. Over half the respondents had a secondary education and many had a bachelor’s degree of better. The discussion format The questionnaire used by the consultants to guide the focus group discussions was similar to the one used for the KIIs in terms of structure and themes. The interviewers (the Pragma consultants) established the group demographics by age group, sex, employment status, educational attainment and revenue. After the introductory phase of the discussion had finished, the interviewers steered the conversation to the JA training program. Key Results The FGDs confirm the findings of the individual questionnaires and the KIIs. Overall, the youth in the FGDs rated the life-skills module as the best (by far), followed by the entrepreneurship module and the technical skills module. There were certain regional differences, however. The Dakar respondents in the crop production cohorts noted difficulty in accessing land while the respondents in the Ross Bethio rice-growing location in the St. Louis Region emphasized the need for financing for equipment for rice irrigation, processing, storage and transportation to markets. The Thiès respondents regretted the lack of ICT training and the need for instruction on how to design a project. All of the FGDs were aligned on the issue of having a diploma or certificate for the training and the need for better (more professional) consultants for the technical skills module. The Ross Bethio youth said that Synapse should have organized an end-of-training seminar to share the experiences of the youth in the training program, and focus on follow-up measures such as better access to financing. Other needs cited included training on marketing of produce. The youth recommended that future “JA” programs provide instruction in Wolof for the less-educated youth; as well as money to travel to the training sites (particularly in the rural areas of St. Louis and Thiès). Annex 8 Organizations with which KIIs Were Held with the Director, Deputy Director or Senior Staff Members 1. ARD Agence Régionale de Développement (Regional Development Agency) St. Louis. Mr. Alioune Badara Fall, Program Officer. Coordinating partner in the St. Louis Region. Mr. Fall stated that the coordinating committee did not work because Synapse did not reimburse members for fuel costs. Mr. Fall felt that Synapse did not provide adequate monitoring of the youth trainees in the project. However, despite these problems Mr. Fall felt that the project responded to a need and should be renewed, with certain changes. Synapse, however, should communicate more with the project partners. 2. ASESCAW Amicale Socio-Educative, Sportive et Culturelle des Agriculteurs du Walo (Walo Farmers’ Social/Educational, Sports and Cultural Club) Ross Bethio, St. Louis Region. Mr. Abdoulaye Faye, Program Officer. Recruited youth, hosted training, supervised consultants, coordinated guest speakers, mentored youth. Mr. Faye stated that there was a convergence of objectives in terms of ASESCAW’s mission and Synapse’s approach. The equality of male and female trainees in the project was a positive dimension. Rural parents were very supportive of the project, which was particularly attractive to girls. Synapse succeeded in convincing the regional authorities that the JA project was innovative and productive. On the other hand, Mr. Faye said that a future JA project should have a modified program to take into account differences between rural and urban areas. The field trips to rice-growing areas and conferences with senior agronomists were very valuable learning events. The training consultants did a good job despite receiving only 8,000 CFA Francs an hour when they should have been getting at least 20,000 Francs an hour at going rates. The project should have provided a transportation allowance for the youth, who had long distances to travel at their own expense. 3. Chambre de Commerce de Thiès Thiès Chamber of Commerce. Ms. Oumy Thiam Sangaré, General Secretary. Coordinating and implementing partner. Provided training space for the JA project. Ms. Sangaré said that although the JA project was innovative and responded to real needs, the coordinating committee ceased to function during the first year of the project. The problem was the lack of funds (from Synapse) for its operations. 4. Chambre de Commerce de St. Louis St. Louis Chamber of Commerce. Ms. Penda Dia, General Secretary. A coordinating partner. Ms. Dia said that the committee was to play a key role in project monitoring and evaluation in the St. Louis Region. However, she felt that the operations of the committee and its members were poorly defined and lacked financial support from Synapse. She said that the Chamber of Commerce had several useful initiatives that could have been of value to the Synapse project, such as the AGOA investment partnership with the USA or the Chamber of Commerce’s own business incubator program but these opportunities were ignored. Annex 8 5. CIH Centre d’Initiation à la Horticulture (Center for the Introduction to Horticulture) Mr. Demba Diop, Director. Implementing partner to production cohorts (vegetable and fruit production) in a suburb of St. Louis. Mr. Diop found the JA project to be of great value in providing training leading to sustainable livelihoods for local youth in the agriculture sector. The CIH is unique in that it is continuing to mentor and support some of the JA project youth. It benefitted from a large plot of walled land given to it by Synapse Center and the plot is used for further training purposes. Mr. Diop said that the training process generally went smoothly during the project. The CIH, which is an agricultural training school, provided trainers specialized in vegetable gardening and fruit production. Synapse Center provided the trainers who held classes on marketing and finance. The CIH of St. Louis is now the only CIH in Senegal that offers training in entrepreneurship, thanks to the JA project. While Mr. Diop was generally very pleased with the JA project, he said that the weak points were the following: • The training on finance and accounting was too short • There was no computer training • The youth evaluated the instructors (it should have been the other way around, according to Mr. Diop) • Attendance of some youth was irregular. No money had been provided for their transportation. As a result, about 5 youth in a cohort of 15 came on an irregular basis and about 5 dropped out of most cohorts. In the future, according to Mr. Diop, literacy training should be added to a JA program since (in his experience) many youth had weak literacy skills, which made it difficult to participate in some of the lessons. Training in Wolof would also be a good idea. 6. CNAFP Centre National d’Appui à la Formation Professionnelle (National Center for Professional Training) in Thiès. Mr. Mouhamadou Fadra Sylla Director. The CNAFP provided training on food processing and restaurant services. The trainees were all female. Mr. Sylla said that there had been good coordination within the project and that the Chamber of Commerce had been helpful in providing business guidance to the trainees. One of the main challenges of the project for Mr. Sylla was the scheduling of instruction during the school holidays, when the CNAFP is closed. It was a challenge to mobilize staff to keep it open for the JA training. Mr. Sylla stated that future training that involves formal educational institutions like his need to take into account the official school year. 7. CPFPS Centre Polyvalent de Formation des Producteurs de Sangalkam (Sangalkam Multipurpose Training Center for Producers) Mr. Modou Fall Thioune, Director. Implementing partner, hosted training of production cohorts in the Dakar Region. The inclusion of the CPFP/S was the result of a discussion with Synapse Center on youth employment in agriculture. Following the discussion, the center was chosen to be a project partner. Annex 8 No protocol or contract has been signed with Synapse, whose promise to provide a computer and a printer has not been fulfilled. On the other hand, the collaboration between the two partners has been generally productive in terms of providing a suitable training location. The CPFPS provided the technical training because horticulture is the specialty of the organization. The technical advantages of the relationship is based on good logistics and specialized knowledge (management of market gardens and agricultural accounting. 8. Commune de Lam-Lam Lam-Lam municipality, Dakar Region. : Association for the Development of Lam￾Lam and Environs. Mr. Louis Jérôme Ndionlène, President. Implementing partner. Hosted training of production cohorts, provided land for youth training activities on cultivation techniques. The municipality had worked previously with Synapse Center on a previous entrepreneurship project and respects the commitment of Synapse Center. As for the JA project, Mr. Ndiolene said that the training program was a good one and that the life skills module was outstanding. On the other hand, Mr. Ndiolène found that the transportation and marketing component of the training was inadequate and that the youth in the production cohorts had problems in accessing land for cultivation as well as water sources (the water table in the area is 70 meters below the surface). Mr. Ndiolène suggested that future initiatives be decentralized to the village level and be incorporated in village development initiatives. 9. CRREJ Centre Régional de Ressources pour l'Emploi des Jeunes (Youth Employment Ressource Center), St. Louis Region. Coordinating partner. Mr. Moustapha Niang, Deputy Director. The CRREJ was part of the Coordinating Committee set up to supervise the JA project in the St. Louis Region. The Coordinating Committee ceased to function because Synapse Center did not provide any funds for its operations. Nevertheless, the CRREJ took an interest in the project and was It was a member of the jury set up to give prizes to the best business plans developed by the youth. Mr. Niang noted that the prize money was very helpful in setting up businesses after the training but only three members of each 20-person cohort received prize money. He explained that the Crédit Mutuel du Sénégal has a special youth enterprise fund and gives priority to youth projects vetted by the CRREJ. However, the JA project did not provide post-training follow-up and the lack of financing was a major obstacle to creating enterprises. The 10. ENSA Ecole Nationale Supérieure d’Agronomie (National School of Agronomy). Mr. Mamadou Ndiaye, Director of Administration and Mr. Toffène Ddione, President of the Student Union. Implementing partner. Recruitment of youth, hosting of training activities, consultant supervision, facilitation of field trips, provision of land for training on cultivation methods. The ENSA staff were pleased overall with the JA project. They found it innovative in terms of training youth in the economic and business aspects of agriculture. Consequently, ENSA will integrate the JA training (with some adaptations) into its own curriculum in the coming school year. Annex 8 11. Exode Urbain/Rurale Urban-Rural Exodus, a Senegalese NGO. Ibrahima MBaye, Président. Implementing partner. Handled recruitment of youth and hosted training in the Dakar Region. « Exodus » organized the JA youth in agricultural production groups. The revenues received by the youth served as an operating fund The main activity of the grouping of the youth was as follows: 1) the purchase of agricultural products in rural areas; 2) the processing of these products and 3) the distribution and marketing of these products in Dakar and environs. The profits were reinvested in the youth group. There were many difficulties but the main one was the lack of communication between Synapse, “Exodus” and the participants. The training site was inadequate (an asthmatic youth fell to the ground on the training site). A further problem was that the training schedule was not followed and the youth were demotivated because their transportation costs were not covered. The focal point had no financial incentive to do the work expected of him. No solution was found for these problems. The technical dimensions of the training took place only because of intervention of Urban-Rural Exodus. In theory, the role of Exodus was not to make investments but rather to monitor and support the youth during the training process. The strongest point was the training. The weakest point was the lack of financing. However, despite these challenges, “Exodus” rated the project as “Good”. 12. Mairie de Guédiawaye City Hall of Guédiawaye. Mr. Oumar Niasse, Director of the Training Center. The City Hall of Guédiawaye was an implementing partner of the JA project in a suburb of Dakar. It lent its training center facility to be used as classrooms for the training. Overall Mr. Niasse found the project to be innovative and responsive to the needs of youth. What was most lacking, however, was post-training follow-up in terms of financing or technical guidance. 13. RENCJES Réseau National de Clubs de Jeunes Entrepreneurs du Sénégal (National Network of Young Entrepreneurs’ Clubs of Senegal) El Hadji Abou Gueye, President. RENCJES Hosted training in Dakar. As a branch of the Ministry of Youth and Sports, it is a member of the Conference of Ministries of Youth and Sports. RENCJES can, among other services, facilitate access to financing for youth employment projects. Mr. Gueye said the JA project was a promising initiative but had some flaws, notably a mixture of educational levels in each cohort and gaps in the management of the training. Among the difficult issues was that no funds had been provided for the youth to travel to the training site. Mr. Gueye also found that the cash prizes for outstanding business plans developed by the youth were too small to have a significant impact. On the other hand, RENCJES is ready to collaborate on follow-up training and is in a position to certify learning results with official diplomas. 14. SDDR Service Départemental de Développement Rural – St. Louis (Departmental Service for Rural Development – St. Louis. Mr. Moussa Diouf. Annex 8 The SDDR was a member of the coordinating committee of the JA Project in the St. Louis Region. While Mr. Diouf found the project to be worthwhile, the lack of funding for the coordinating committee limited his involvement with the project. 15. Synapse Center Synapse Center. Mr. Ciré Kane, Director, El Hadji Niasse, Deputy Director, Mame Fatou Fall, Mariane Diaz. Mr. Kane briefed the Pragma research team on the background, origins and management of the JA project. Mr. Kane said that Synapse had been working on youth employment in Senegal since 2007. RTI and IYF were working on the “Making Sense” initiative in four regions of Senegal with the Ministry of Youth and Sports when IYF contacted Synapse Center to teach life skills to 4,400 youth via 166 trained facilitators. In 2012 Synapse proposed the idea of a youth agribusiness project to IYF. A grant of $508,939 dollars of USAID Innovation Fund resources was given to Synapse Center by IYF to set up and manage the JA project. Further briefings were held during the presence of the Pragma Research team in Dakar towards the end of the mission to respond to questions the consultants had about the management of the project. 16. USAID￾Senegal United States Agency for International Development-Senegal. Ms. Michelle Barrett, General Development Office Director, Roy Geiser, Program Officer and Ibrahima Top, M&E Specialist. A meeting was held at the USAID office for Eric Allemano to elicit comments from USAID counterparts about the design and management of the Jeunes Agriculteurs Project. The USAID staff had limited direct knowledge of or involvement with the project. Ms. Barrett referenced in this regard that USAID’s involvement with the project was primarily through USAID-Washington and IYF.