1 September 2015 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Steven J. Kelly of KNO Worldwide and Mariam Ghazala, Roger Nassif and Salma El Abbassy of CDC Academy under subcontract to Institute of International Education. MIDTERM PERFORMANCE EVALUATION Sustainability of Career Development Centers in Egypt Public Universities 1 MIDTERM PERFORMANCE EVALUATION SUSTAINABILITY OF CAREER DEVELOPMENT CENTERS IN EGYPT PUBLIC UNIVERSITIES October 11, 2015 AID-263-TO-14-00001 - Scholarships & Training for Egyptian Professionals (STEP) Submitted by: Joseph Ghanem, STEP Chief of Party Institute of International Education COVER PHOTO: SUEZ CANAL UNIVERSITY CDC, AUC ECDC DISCLAIMER 2 The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. TABLE OF CONTENTS Table of Contents........................................................................................................................................................... 2 List of Figures and Tables ............................................................................................................................................. 3 Acronyms ...................................................................................................................................................................... 4 1. Executive Summary ................................................................................................................................................... 5 2. Introduction .............................................................................................................................................................11 3. Background ..............................................................................................................................................................13 4. Methodology ............................................................................................................................................................14 5. Limitations ...............................................................................................................................................................16 6. Findings, Conclusions & Recommendations ...........................................................................................................17 7. References ...............................................................................................................................................................44 Annexes .......................................................................................................................................................................46 Annex A: List of Stakeholders Contacted ...................................................................................................................47 Annex B: Student Focus Group Analysis ....................................................................................................................50 Annex C: Performance Monitoring Plan Tables ..........................................................................................................57 Annex D: Gender Balance Analysis ............................................................................................................................60 Annex E: Illustrative Photos of CDC Operations ........................................................................................................66 Annex F: Evaluation Methods and Limitations ...........................................................................................................71 Annex G: Data Collection Design Matrix ...................................................................................................................80 Annex H: Data Collection Instruments ........................................................................................................................94 Annex I: Evaluation Statement of Work ....................................................................................................................135 Annex J: Disclosure of Conflicts of Interest ..............................................................................................................147 3 LIST OF FIGURES AND TABLES Figure 1: Map of Egypt highlighting CDC locations ...................................................................................................18 Figure 2: Ranking of factors influencing the success of CDCs ...................................................................................30 Figure 3: Importance ranking of successful CDCs ......................................................................................................31 Figure 4: Force ranking of Successful CDCs ..............................................................................................................31 Figure 5: Percent of students ECDC vs LEEP .............................................................................................................50 Figure 6: Students % of distribution per CDC .............................................................................................................51 Figure 7: Percent of Females vs Males ........................................................................................................................51 Figure 8: Percent of females vs males per CDC ..........................................................................................................52 Figure 9: How did you know that the CDC exists? How did they try to reach you? ..................................................53 Figure 10: Ranking for ECDC .....................................................................................................................................53 Figure 11: Ranking for LEEP ......................................................................................................................................54 Figure 12: Highest Student Participation .....................................................................................................................54 Figure 13: Effectiveness of Service ............................................................................................................................. 55 Figure 14: HU males vs females distribution ..............................................................................................................60 Figure 15: SVU males vs females distribution ............................................................................................................60 Figure 16: ASU males vs females distribution ............................................................................................................61 Figure 17: ASU-Eng males vs females distribution ....................................................................................................61 Figure 18: SCU males vs females distribution ............................................................................................................62 Figure 19: Number of students gaining access to CDCs work development programs ...............................................62 Figure 20: Number of students applying career centers content in their employment search .....................................63 Figure 21: Number of Students with improved languages skills .................................................................................63 Figure 22: Number of students who attended "Employability Skills" courses ............................................................64 Figure 23: Number of Students who attended "Career Development" workshops ......................................................64 Figure 24: Number of Students who benefited from individual career advising services ...........................................64 Table 1: SVU evaluation .............................................................................................................................................33 Table 2: Helwan University evaluation .......................................................................................................................34 Table 3: Suez Canal evaluation ...................................................................................................................................35 Table 4: Ain-Shams general evaluation .......................................................................................................................36 Table 5: Ain-Shams engineering evaluation ................................................................................................................37 Table 6: Contacts and informants list ..........................................................................................................................49 Table 7: ECDC Plan Progress To-Date .......................................................................................................................57 Table 8: LEEP Plan Progress To-Date ........................................................................................................................58 Table 9: CDC mid-term overall schedule ....................................................................................................................79 Table 10: CDC evaluation timeline of events and processes .......................................................................................79 Table 11: Overall areas of CDC research parameters ..................................................................................................93 Table 12: Ranking of factors influencing success of CDCs ......................................................................................101 Table 13: Financial flows for CDC sustainability .....................................................................................................102 Table 14: AUC-ECDC summary performance indicator table ..................................................................................103 Table 15: WL-LEEP summary performance indicator table ....................................................................................105 Photo 1 Business English language Course .................................................................................................................66 Photo 2: Build your Business training program ...........................................................................................................66 Photo 3: Job Search training program..........................................................................................................................67 Photo 4: Disability Training ........................................................................................................................................68 Photo 5: Job fair ...........................................................................................................................................................68 Photo 6: School Ambassador program visiting the faculty of dentistry ......................................................................69 Photo 7: CV Support services ......................................................................................................................................70 Photo 8: An Experiential learning exercise during employability skills track- ASU Main Campus ...........................70 Photo 9: Thinking patterns, ASU-Main campus ..........................................................................................................71 Photo 10: Power of positive attitude at work, ASU-Engineering ................................................................................71 Photo 11: Volunteers at ASU celebrating the success of the employment fair ............................................................72 4 ACRONYMS ADS Automated Directives System APS Annual Program Statement ASU Ain Shams University AUC American University Cairo CDC Career Development Center DEC Development Experience Clearinghouse ECDC Employability and Career Development EGP Egyptian Pounds FY Fiscal Year HU Helwan University IIE Institute for International Education KNO Kelly Novak Opportunities LEEP Linking Education & Employment Project MOD Modification MOU Memorandum of Understanding NCDA National Career Development Association PMP Project Monitoring Plan PROC USAID Procurement Office RFP Request for Proposal SCU Suez Canal University SOW Statement of Work STEM Science, Technology, Engineering, and Mathematics STEP Scholarships and Training for Egyptian SVU South Valley University USAID U.S. Agency for International Development USG United States Government WL World Learning 5 1. EXECUTIVE SUMMARY EVALUATION PURPOSE AND EVALUATION QUESTIONS The purpose of this mid-term performance evaluation is to review, analyze, and evaluate the effectiveness of USAID- funded CDCs in achieving program objectives and completing deliverables. The evaluation answers the following questions: 1. To what extent have the CDCs achieved their goals of this SOW? 2. If and how did the selection of the geographic location for the CDCs affect their performance? To what extent CDCs geographic location affect the performance of CDCs in achieving the results? Why? 3. To what extent are the LEEP/WL and ECDC/AUC projects on track to transitioning the management of the CDCs to the host universities? Which factors have contributed to or hindered the sustainability of the CDCs? What are the prospects for sustainability post￾project completion? 4. How do key stakeholder groups (e.g., students, faculty members, university management, employers) perceive the value and utility CDCs? How, and to what extent, do key stakeholder groups engage with the CDCs and which factors facilitate and inhibit utilization of the CDCs? How effective have the CDCs been in connecting graduates with employers and providing them with the skills that employers demand? 5. How effective have the CDCs been in achieving gender balance in the provision of CDC services (e.g., training, job placement services, etc.) during implementation? Which factors have facilitated or inhibited the CDCs from achieving gender balance in the provision of services? AUDIENCE AND INTENDED USES The audience of the evaluation report will be USAID/Egypt Mission, specifically the Office of Education and Training and the Program Office. USAID/Egypt will review and share the expanded executive summary, final report, and evaluation recommendations with the Ministry of Higher Education (MOHE), other donors in Egypt working on the Employment Alliance, and the general public via the Development Education Clearinghouse (DEC) within 3 months of the report completion. PROJECT BACKGROUND Until the 1950’s, Egypt’s higher education system was a model for the region. However, Egypt now confronts a serious knowledge and skills deficit. A major reason underlying this was the rapid expansion of the tertiary education system after 1957, which was not matched with the allocation of adequate resources, and a subsequent lack of responsiveness of education institutions to the needs of the labor market. 6 In 2012, USAID/Egypt’s Office of Education supported the establishment and management of state-of-the-art, university- based Career Development Centers (CDCs) through a three-year activity aimed at improving the long-term capacity of Egyptian universities to assist students and recent graduates in their transition from education to employment. The CDC project’s development hypothesis is that the CDCs will enhance a student’s employability, stability and prosperity through the provision of training courses, mentorship opportunities, workshops, internships and job opportunities, which will result in an educated workforce that responds to the labor market needs. USAID made awards to two implementers, namely: American University Cairo (AUC) Employability and Career Development Centers (ECDC) and World Learning (WL) Linking Education & Employment Project (LEEP). Together, they established 8 CDCs at 7 public universities of which 5 remain open. In early 2013, as a result of events in Egypt and the U.S. foreign policy response, USAID partially suspended the programs as they wound down nearly all education program implementation. After 6 months, USAID authorized re-start of the programs once again. The five CDCs that are currently operational have been expected to become an integral part of each respective university campus by the completion date of the projects, and are expected to be sustained by corporate funding and fee-for- service programs. Both completion dates were scheduled for May 31, 2015, but USAID provided a no-cost extension of the modified award to AUC to until September 30, 2016 and a no-cost extension to WL through December 31, 2015. USAID asked Scholarships and Training for Egyptian Professionals (STEP), a five-year program that supports Egypt’s economic and social development, to undertake a mid-term performance evaluation of the two programs prior to the end of FY15. FINDINGS AND CONCLUSIONS World Learning’s Linking Education & Employment Project (LEEP) Founded in 1932, World Learning/SIT Graduate Institute in Vermont has decades of experience delivering training and capacity globally in over 70 countries. Currently WL is managing two CDCs, one at Helwan University and the other at South Valley University. Two others were started in Fayoum and Port Said Universities, but closed due to senior management changes. The contracts went through several modifications due to the short-term suspension and budget re-alignments due to changes in university partners. Based on the data provided to the evaluation team, the LEEP performance on meeting Performance Monitoring Plan indicator targets is relatively strong based on numbers through the period of the evaluation. Major achievements include outputs involved in providing service to students such as access to career development programs and soft skill training where participation surpass targets by 140-160% despite only having two operating CDCs. As well, reported data from Year 2 surveys indicate high use of skills in employment searches. This is supported by the student focus group data. Indicators tracking staff actions and committee meetings are mixed, but do not seem to have impacted operations significantly (see individual CDC evaluations below). As well, the employment fairs and youth ambassador programs were 7 delayed in starting with resulting lower results (Below target by 50% or more at the time of the evaluation). Best practices The LEEP initiative has a number of actions that the evaluators highlight as ―best practices‖ in building sustainable CDCs:  Initiating youth ambassadors introduces the center and university to graduating secondary students.  Building internal staff training capabilities allows for better quality monitoring and lower cost over time than external commercial sources.  Ongoing active student / faculty volunteer networks help build enthusiasm for the program, while spreading word of services and successes.  Bringing specialized English language business curriculums and training existing teaching assistants to deliver them while very challenging and time consuming, over the long term this builds further university capacity to support career center needs rather than out-sourcing.  Conducting employment surveys as these are refined, they can provided data on jobs realized, key skill sets used from training, and provide links between employer needs and CDC service.  A program addressing special needs for the vision impaired is done in collaboration with a nonprofit. This can become a model for other special needs services over time.  Use of expanded advisory committees for centers that includes employers, faculty and students can allow for diverse inputs and feedback for program design and monitoring. As well, the LEEP program has a number of challenges in the areas of MOU content, lack of revenue generation, sustainability planning, and documenting center procedures. American University Cairo Employability and Career Development Centers (ECDC) The American University in Cairo (AUC), Egypt is a premier English-language institution of higher learning. The Career Center, formerly known as office of career advising and placement services (CAPS), was established 20 years ago as the first university career center in the region and is implementing the program. Currently, AUC is managing three CDCs, one at Suez Canal University and the other two at Ain Shams University. One other was started in Assiut University, but closed due to a shift in senior management attitude. The grants went through several modifications dealing with the temporary wind-up and a reduction of the budget due to the trimming in numbers of university partners. Based on the data provided to the evaluation team, the ECDC performance on meeting their Performance Monitoring Plan indicator targets is quite strong in several areas based on reported numbers through the period of the evaluation. Other areas are weak or inconclusive. Major achievements include surpassing the reported output targets for providing service to students such as short-term career development training, individual counseling sessions and attendance at employment fairs where participation surpass targets in some cases by 300 – 500% based on 8 very high demand by students. Numbers for staff certification and longer training programs were within 10% of actual targets. For the indicator percentage of graduates reporting themselves as employed, no data was available for this category. Student surveys regarding employment success or application of skills in job searches have not been completed to-date. The ECDC initiative has several actions that the evaluators highlight as ―best practices‖ in building sustainable CDCs:  The generation of revenues from employment fairs has been an important element is helping to build the financial sustainability of the CDCs. This both begins to build a bank balance to support the transition to the university, but also accustoms employers to expect to pay for these types of services.  In partnership with ASPIRE Experiential Consulting Solutions, AUC has designed and customized an innovative intensive soft skills training program, especially customized for student audiences  ECDCs have built a strong volunteer of student volunteers and allowed the office to function also as a ―community center‖ for students for informal encounters and engagement.  AUC has championed the certification of all ECDC staff as career development facilitators. This has been instrumental in insuring high quality individual assessment and counseling services. As well, the ECDC program has a number of challenges in the areas of MOU content, faculty involvement, out-sourcing of soft skills training, and employability surveying. The successes of the five operating CDCs were evaluated according to senior university support, director ability, staff competence, quality of facility, faculty involvement, employer network, service delivery and satisfaction, job opportunities created and financial status. Based on the status of these factors the sustainability of the CDC was predicted. Generally it was found that the a CDC established over 2 years has a good chance of survival, although none of the centers have created joint sustainability plans including budget requirements with the university management. COMMON LESSONS LEARNED 1. The five CDCs operating under the grants provide a variety of services to students, based on the original project scope of work. Currently there is no common agreement of an optimal CDC model: what should be core services, what size and make-up of staff is best, what topics of supplemental soft skill training are most important, whether trainers should be a part of staff or outsourced, what are the parameters of an employers’ network, what should be the nature of faculty involvement, etc. A ―best practices‖ model of a CDC would be helpful to guide future implementation in partner universities. This can be included in future Requests for Proposals if funding is made available to expand the program. 9 2. The findings clearly point to the importance of senior university management support as the critical factor in building sustainable CDCs. When evaluating potential partners, this should be the paramount issue to be vetted, as opposed to geographic location, size of student body, or nearby employers. The support should be demonstrated by university provision of facilities, identification of potential internal staff, and agreement to find a mechanism to include the center in the university budget and payroll within a few years. 3. The evaluation reviewed the operations of the current CDCs, with several relatively new and others operating for several years. The findings demonstrate that it takes several years to establish a physical facility, hire and train staff, rollout training, and build an employers’ network. Financial support for 30 months, with the last six being overtaken by CDC revenue generation and university budget support is necessary when considering timetables and funding. 4. Although there was consideration of elements, none of the five CDCs have prepared a sustainability plan. Without this, it is unclear the milestones necessary, joint role and responsibilities, financial requirements and progress indicators. Preparing a framework jointly with the university within the first six months of the project, that is finalized at the beginning of the last year of support, will allow both partners to insure actions are taken to build sustainability. 5. The memorandum of agreement (MOU) signed with the university is a critical document to establish roles and responsibilities of both parties, not just the universities. These should require the joint preparation of a sustainability plan, key milestones toward sustainability, and indicators to measure progress. The potential instability of management support in some institutions might be controlled by MOUs with phases and milestones to be achieved; if stipulated conditions are not met, USAID funding could be suspended. 6. The project staff, CDC management and university management did not appear to have explored very deeply the financial requirements and source of funds for sustainability after the conclusion of the project. While the evaluation team had prepared simple revenue/cost tables for data collection, it was very difficult to find data to analyze. No financial projections have been prepared (tied to lack of sustainability planning above) and parties assumed the universities would find the budget or CDC revenues would suddenly spike, and all will work out. Financial planning, with realistic targets and expense ratios, must be a highlighted element for both planning and monitoring. 7. The PMPs, which contains indicators for monitoring performance, are weak in identifying specific outcomes/outputs focused on sustainability. While hiring and training staff are important, along with delivery of services, other indicators as mentioned above focusing on financial progress and actions for the university to move the CDC administratively internally should also be included. PMPs with a heavy emphasis on service delivery targets and outputs large numbers can sometimes distort the focus of the small CDC staff. Actions that take time, such as developing deeper relationships with employers (beyond job fair attendance), or surveying graduates, may lose staff attention while they pursue the more visible event attendance targets. 10 8. Although all CDC staff have been trained and in some cases certified, there is a uniform lack of written policies and procedures manuals (other than some communication strategies) in place within the center. When staff leaves, the knowledge leaves with them. Although time￾consuming to develop, written operation manuals that document best practices, procedural steps, and approved policies can be very helpful to orient new hires. If a common model of the CDC is adopted, basic administrative software can be developed and provided to the new CDCs to automate data collection and improve the ability to monitor and evaluate efforts. 9. While it is agreed that English language is an important qualification for many jobs, the addition of this component to the CDC services has been highly problematic. ECDC has attempted to address this by outsourcing, while LEEP pursued a strategy to introduce business curriculums into existing language programs. Becoming proficient in English language is a multi-year effort and might best be removed from the scope of CDCs. 10. While there have been some efforts to track graduate success with employment, and the link between CDC career training, this must be expanded. Surveys of both graduates and employers should use common templates for easy comparison, and may benefit with the contributions of education/labor market specialists in their design. CONCLUSIONS Question #1: Generally, despite the recent instability and mid-term suspension of activity, the ECDC and LEEP projects have achieved the goals of their modified scopes of work. Questions #2: Geography is not an important factor in CDC performance. Most critical is university manager support, followed closely by quality of services delivered by competent staff. Questions #3: Those CDCs with over two years of operation are well on their way to sustainability. As covered throughout the analysis, a variety of factors influence sustainability. Other than conducting necessary sustainability planning, to include financial elements, the longer term CDCs have embedded these factors Questions #4: The engagement with key stakeholders has been variable depending on the program and individual CDC. Generally students are highly engaged at all centers, while active faculty involvement and deeper relationships with employer networks should be strengthened at some locations. The effectiveness of CDC services providing the right mix of skill development and connections with employers requires more attention. Question #5: Gender balance is excellent in the programs as illustrated in participation in services. When considering the development hypothesis … that the Career Development Centers will enhance a student’s employability, stability and prosperity through the provision of training courses, mentorship opportunities, workshops, internships and job opportunities, which will result in an educated workforce that responds to the labor market needs. 11 The evaluators conclude that while this is partially supported by the findings, there still needs to be further research as regards the impact of the CDCs as directly linked to student employability and prosperity. Main Recommendation: The Career Development Center concept has value toward achievement of matching workforce education to labor market needs and should be scaled up under best practice model if priorities and funds permit. A variety of actionable recommendations are made in the full report, addressing the overall programmatic issues, establishment of new CDCs, and more effective operations. 2. INTRODUCTION EVALUATION PURPOSE The purpose of this mid-term performance evaluation is to review, analyze, and evaluate the effectiveness of USAID- funded CDCs in achieving program objectives and completing deliverables. An evaluation team lead by Kelly Novak Opportunities (KNO), and assisted by Career Development & Consultants Academy (CDC), conducted the mid-term performance evaluation of the ECDC/AUC and LEEP/WL components of the CDCs project. OBJECTIVES OF THE EVALUATION INCLUDE: 1. Identification of lessons learned with regard to project implementation and building relationships with counterparts in the Ministry of Higher Education and public universities around CDC establishment; 2. Assessment of the sustainability of the CDCs at the Egyptian public universities in which they have been established; and 3. Provide recommendations that can be used to inform the design and expansion of future CDC-related activities in Egypt’s public universities. AUDIENCE AND INTENDED USES The audience of the evaluation report will be USAID/Egypt Mission, specifically the Office of Education and Training and the Program Office. USAID/Egypt will review and share the expanded executive summary, final report, and evaluation recommendations with the Ministry of Higher Education (MOHE), other donors in Egypt working on the Employment Alliance, and the general public via the Development Education Clearinghouse (DEC) within 3 months of report completion. EVALUATION QUESTIONS The evaluation answers the following questions: 12 1. To what extent have the CDCs achieved their goals of this SOW? 2. If and how did the selection of the geographic location for the CDCs affect their performance? To what extent CDCs geographic location affect the performance of CDCs in achieving the results? Why? 3. To what extent are the LEEP/WL and ECDC/AUC projects on track to transitioning the management of the CDCs to the host universities? Which factors have contributed to or hindered the sustainability of the CDCs? What are the prospects for sustainability post￾project completion? 4. How do key stakeholder groups (e.g., students, faculty members, university management, employers) perceive the value and utility CDCs? How, and to what extent, do key stakeholder groups engage with the CDCs and which factors facilitate and inhibit utilization of the CDCs? How effective have the CDCs been in connecting graduates with employers and providing them with the skills that employers demand? 5. How effective have the CDCs been in achieving gender balance in the provision of CDC services (e.g., training, job placement services, etc.) during implementation? Which factors have facilitated or inhibited the CDCs from achieving gender balance in the provision of services? The CDCs project’s critical assumptions include:  The bilateral relationship between the Government of Egypt (GOE) and the USG remains positive and fully collaborative;  The GOE presented in the public universities’ management cooperates fully with the implementation of the programs and activities; and  Funding/resources are available to implement planned activities. Based on evaluation findings, the evaluation team has developed actionable recommendations that will help USAID to better design and implement effective CDCs programming going forward. In particular, the recommendations are based on lessons learned from implementation to date that USAID should take into consideration in future designs, and key considerations have been identified for replicating CDCs in other geographic locations? 13 3. BACKGROUND The Egyptian education system is the largest in the Middle East and North Africa. It is made up of 19 public universities, 19 private universities, and another 139 public and private technical colleges and institutes. As of 2011/2012, 1.6 million students were enrolled in public universities, 87,000 in private universities, and another 428,000 students in private and public technical colleges and institutes. Until the 1950’s, Egypt’s higher education system was a model for the region. However, Egypt now confronts a serious knowledge and skills deficit. A major reason underlying this was the rapid expansion of the tertiary education system after 1957, which was not matched with the allocation of adequate resources, and a subsequent lack of responsiveness of education institutions to the needs of the labor market. Currently, the university system is not positioned to provide high quality education that will meet the needs of the labor market. The lack of balance and fit in graduate supply to the labor market is at the core of Egypt’s challenge. Egypt’s higher education institutions struggle to produce graduates with the skills employers seek, thus posing constraints for growth opportunities. Moreover, they lack a sustainable financing system, sufficient autonomy, adequate faculty compensation levels, and merit‐based systems of recruitment and promotion. In 2004, the GOE embarked on an aggressive regulatory and institutional reform program. However, some structural issues prevented many of the benefits from being realized by most Egyptians, and the percentage of the population in poverty increased. As a result, over 23.5% of Egypt’s population in 2010 was between 18 and 29 years with 90% of them being either unemployed or underemployed. In an attempt to address those issues, several public Egyptian universities have attempted to create Career Development Centers (CDCs) charged with preparing students and graduates with skills needed to seek employment and succeed in the workplace. The most recognized of those efforts being previously USAID supported Nahdet El Mahrousa sponsored Center at the University of Cairo and the Egypt Junior Business Association sponsored Key Career Center at Ain Shams University. However, few exist outside of major urban areas where unemployment is especially high. Moreover, most CDCs are managed by outside providers and are not integrated into university systems. Hence, the universities do not develop the capacity to manage and expand the CDC concept independently across faculties, and hence ensuring long term sustainability. In conclusion, there is an urgent need, to support CDCs that are integrated into university structures. In 2012, USAID/Egypt’s Office of Education supported the establishment and management of state-of-the-art, university- based Career Development Centers (CDCs) through a three-year activity aimed at improving the long-term capacity of Egyptian universities to assist students and recent graduates in their transition from education to employment. USAID made awards to two implementers, namely: American University Cairo (AUC) Employability and Career Development Centers (ECDC) for $ 2,728,700 and World Learning (WL) Linking Education & 14 Employment Project (LEEP) for $ 1,678,768. Together, they established 8 CDCs at 7 public universities of which 5 remain open. In early 2013, as a result of events in Egypt and the U.S. foreign policy response, USAID partially suspended the programs as they wound down nearly all education program implementation. Almost 6 months, USAID authorized re-start of the programs once again. Consequently, both the LEEP/WL and ECDC/AUC programs experienced major challenges during the second year of operations in 2013, and both awards went through several modifications as a result of the suspension and changes in university partners. The five CDCs that are currently operational have been expected to become an integral part of each respective university campus by the completion date of the projects, and are expected to be sustained by corporate funding and fee-for- service programs. Both completion dates were scheduled on May 31, 2015, but USAID provided a no-cost extension of the modified award to AUC to until September 30, 2016 and a no-cost extension to WL through December 31, 2015. USAID asked Scholarships and Training for Egyptian Professionals (STEP), a five-year program that supports Egypt’s economic and social development, to undertake a mid-term performance evaluation of the two programs prior to the end of FY15. STEP subcontracted with the international firm KNO and Egyptian researchers from CDC Academy to conduct the evaluation during the period July-September 2015. The cost of the evaluation was $112,000. 4. METHODOLOGY EVALUATION DESIGN As a mid-term performance evaluation, the design focuses on how the CDCs projects have been implemented, what the projects have achieved, whether expected results were attained according to the projects’ design and the underlying development hypothesis, how activities were perceived and valued by stakeholders, and whether the projects are sustainable. The evaluation team used a mix of quantitative and qualitative data collection and analysis methods to answer the questions of interest in this evaluation. The evaluation follows the principles and guidelines for high quality evaluations outlined in the USAID Evaluation Policy (January 2011), as well as addresses criteria from the Checklist for Assessing USAID Evaluation Reports (V1.0). EVALUATION QUESTIONS The evaluation team developed data collection tools that are consistent with the evaluation questions to ensure high quality analysis. The primary design tool is the Evaluation Design Matrix (Annex G). This matrix breaks out each of the major research areas into more specific sub-questions. The matrix describes for each question the indicator area, source of information, type of evaluation design, type of sampling if used, and the data collection instrument to be used. Prior to the finalizing of the Design Matrix, initial meetings were held with the LEEP and ECDC project staff, as well as USAID Education Office representatives, to insure the design was aligned with expectations. 15 Based on the design matrix, the team developed detailed interview guidelines including questionnaires covering the specific evaluation issues and concerns for all levels of the CDC project implementation (annex H). These questionnaires include specific questions for each stakeholder group based on the Design Matrix. Given the extremely short project timeline, full pilot testing could not be done. However, the team conducted the first set of interviews in the form of a pilot. No adjustment to the questionnaires was found necessary. The detailed interview questionnaires were used as discussion guidelines and allowed the evaluators to obtain similar types of qualitative information at the various levels of the project implementation. The research team was trained on the use of the guides to conduct interviews. KNO submitted to USAID through STEP the Evaluation Methodology and Design Matrix for their review. IIE submitted the document to the USAID Evaluation Program Manager and organized a meeting with USAID for review, feedback and/or discussion prior to use in the field. A list of the institutions, individuals, and groups interviewed is included as an annex to the final report. Over 40 interviews and 9 focus groups with 182 student participants (55% female) were undertaken by the evaluation team. EVALUATION DATA SOURCES The evaluation will be based on the following sources of information: a) Project reports and other documents listed in the bibliography b) Secondary data sources identified at appropriate points in the report c) Interviews and meetings with LEEP and ECDC project staff d) Field visits in 4 universities: Helwan, South Valley, Suez Canal and Ain Shams (main school and engineering faculty). EVALUATION STEPS The evaluation process consisted of the following eleven tasks over 6 weeks of effort (see annex F for more detail and schedule). The survey team was led by a senior international performance expert specializing in the design and implementation of performance assessments and evaluations, with experience in the higher education sector. The local survey team includes a senior lead researcher and two researchers, an analyst and administrator. All researchers speak the local languages. DATA ANALYSIS The evaluation team conducted site visits to each of the five active CDCs, as well as made contacts with closed ones when possible. Strict guidelines were prepared and followed for analysis as detailed in the methodology annex. Notes were written during or immediately after interviews. The international team leader received completed research data regularly and provided ongoing direction to the local team. Quantitative data was analyzed using spreadsheets with an aim to provide percentages and ratios for presentation. As appropriate, charts and graphs were prepared to summarize or highlight data. Data was triangulated between different informants to assess validity and reliability. All beneficiary data is disaggregated by gender and geographic location as available. The international team leader returned to Cairo at the conclusion of the data collection effort to lead the team in the final data analysis phase. An out-brief presentation was provided to USAID 16 on draft findings, lessons learned and recommendations for review and feedback prior to submitting this draft report. 5. LIMITATIONS DATA QUALITY STANDARDS The evaluation team ensured that the data collected clearly and adequately represents answers to the evaluation questions, is sufficiently precise to present a fair picture of performance, and is at an appropriate level of detail. DATA LIMITATIONS The evaluation team insured that issues potentially affecting the quality (including validity, integrity, and reliability) of evaluation data were discussed and documented in the evaluation planning stage and assessed on an ongoing basis during evaluation implementation, including during data collection and analysis. The limitations and identified measures to address or overcome limitations were discussed by the evaluation team and USAID during the out-briefing at the conclusion of the implementation phase. The list below includes all notable limitations and if and how they may affect the evaluation’s findings, conclusions, and recommendations. There are known limitations to the design and implementation of the evaluation:  Due to the delay in approval for project start and the fixed deadline, the time spent at each university was limited, as well as the time spent interviewing stakeholders and CDC beneficiaries.  Instructions by USAID prohibited the team access to financial data concerning the grantees budgets to only total grant amounts (identified in the SOW). This limited the analysis of various funds provided to support the individual CDCs and reduced the ability to fully identify actual required funding for CDC sustainability in some cases.  Given that the evaluation takes place on university campuses during the August summer break, there was a risk that adequate samples of student beneficiaries are not available. Thus, in some cases, random selections of students were not possible to establish. Each CDC identified and invited students for focus groups and this may result in a positive bias in their selection.  Contacts for involved employer representatives were of necessity provided by each project. There is a risk of bias in the selection of interviewees and in some cases too small samples were provided by the CDC despite several requests.  Contacts for involved faculty representatives were of necessity provided by each project. There is a risk of bias in the selection of interviewees, and in the case of ECDC, too small samples were provided.  The team relied on data provided from surveys designed and administered by project staff. There is a risk that the survey design might not be valid or reliable, with the potential of incorrect results.  Not all requested documents and/or data necessary for desk review were provided in a timely manner; or if provided, not complete or accurate. As well, certain records 17 requested for observation as evidence while onsite were not provided or were incomplete.  In some cases student and alumni data were not made available by Project or CDC management, or defined by preferred categories. Despite these limitations, every attempt has been made to review all available data and information gathered from all key informants to ensure the data matches the various findings. The evaluation team has insured that the generalized as well as the specific findings are valid and accurate for the project being evaluated, considering the time, effort, and resources expended, as well as the efforts made to verify all the collected data with a wide array of secondary information. 6. FINDINGS, CONCLUSIONS & RECOMMENDATIONS FINDINGS Given the concerns about labor market links to higher education noted in the background above, USAID designed an Annual Program Statement (APS) with the following objectives (CDC Support APS-263-11-000001): Establish sustainable career development centers in Egyptian public universities; Provide career guidance to students and university graduates and build employability skills of students and alumni; Develop participants’ skills in disciplines needed for the labor market; and Bridge the gap between current university graduate skills and job market requirements. In 2012, USAID’s Office of Education supported the establishment and management of state-of￾the-art, university- based Career Development Centers (CDCs) through a three-year activity aimed at improving the long-term capacity of Egyptian universities to assist students and recent graduates in their transition from education to employment. Given the instability in Egypt during the life of the project since the 2012 start, the implementers of the CDCs have had a very challenging job. Of the eight CDCs that were started, three of them were closed, in some cases, after significant investment due to university changes in senior management that were coupled with political issues. For over a 6 month period following the June 30th revolution (October 2013 to March 2014), USAID was required to work with the implementers to wind-up certain existing activities consistent with U.S. law and policy. On October 13, 2013, the Procurement Office informed the two career development programs: LEEP and ECDC which directly benefit and support public universities in Egypt that they must precede with an orderly wind-up of certain activities. WL and AUC developed detailed wind-up plans and the contracts office modified their awards accordingly. The CDC project’s development hypothesis: … is that the Career Development Centers will enhance a student’s employability, stability and prosperity through the provision of training courses, mentorship opportunities, workshops, internships and job opportunities, which will result in an educated workforce that responds to the labor market needs. 18 At that time, USAID agreed with AUC and WL to complete ongoing activities on campus by the end of January 2014. It was also agreed that both partners could continue providing necessary services to Egyptian students off the university campuses, in a community based context. This arrangement was planned to allow for project activities/services greatly needed by Egyptian youth to meet the job market needs while complying with assistance restrictions. On March 4, 2014, before proceeding with moving the CDC activities off campus, both programs were informed that the wind-up imposed on the CDC activities was relieved and activities could go back to its originally planned and approved program description, work plan and budget. This resulted both in loss of momentum with the established CDCs and also several modifications to both the LEEP and ECDC contracts for the renewal of activity. Without a doubt, the instability and suspension had impact on the ability to build operating career development centers to a level where they can be sustainable. Nevertheless, both ECDC and LEEP have had success with at least one of the CDCs that have a strong potential to be sustainable after the end of USAID funding. The narrative below is organized according to the five primary evaluation questions and separates the two implemented programs since they have different work scopes. It focuses only on the LEEP / ECDC scope of work performance following the last contract modification for each implementer as identified below. Given this focus, the evaluation focuses only on the scope of work (SOW) and project management plan (PMP) from the time of the last modifications. As well, the evaluation team only studied the operations of the 5 remaining open CDCs:  Ain Shams University (two CDCs) – managed by ECDC/AUC  Helwan University – managed by LEEP/WL  South Valley University – managed by LEEP/ WL  Suez Canal University– managed by ECDC/AUC The three other university CDCs that were established but were closed include:  Fayoum University (closed 9/2013) – managed by LEEP/WL  Assiut University (closed 4/2014) – managed by ECDC/AUC  Port Said University (closed 11/2014) – managed by LEEP/WL As previously mentioned, for the three CDCs that closed, this was primarily due to issues with university management who are very independent in decisions affecting their institutions. In September 2013, following the June 30 events, the President of Fayoum University requested the cessation of the CDC’s activities, as he individually decided he did not want any USG supported-programs at his Figure 1: Map of Egypt highlighting CDC locations 19 university. CDC-Assiut and CDC-Port Said closed in April and November 2014, respectively, due to changes in university management. Assiut University’s new administration was not cooperative, and ultimately support to Assiut ended. Due to changes in the university’s senior management, the CDC’s activities at Port Said University were suspended in November 2014. A variety of photos illustrating CDC operations are grouped together in annex E because of report length considerations. 1. TO WHAT EXTENT HAVE THE CDCS ACHIEVED THEIR GOALS OF THIS SOW? In researching the achievements of the implementers of their work scopes under the current award modification, the focus of evaluation is on:  General achievement of goals  Success in meeting Performance Management Plan (PMP) indicator targets  Implementation of Other Strategies Identified from SOW (non-PMP indicators) As the two implementers have differing specific objectives and work scopes, their performance is reviewed separately. Since much of the success of the work scope is illustrated by the operation and potential sustainability of their sponsored university CDCs, these will each be explored in-depth separately in the analysis of research question #3. World Learning’s Linking Education & Employment Project (LEEP) Overview of Goals Founded in 1932, World Learning/SIT Graduate Institute in Vermont has decades of experience delivering training and capacity development courses to students and participants around the world. World Learning works globally in over 70 countries, with over 18 country offices. World Learning implements the Linking Education and Employment Program with the goal to establish self-sustaining Career Development Centers at target universities in regions of high youth unemployment. Objective 1: To establish self-sustaining career centers at target universities in regions of high youth unemployment: Each participating university will host a fully integrated career development center that meets the needs of students and alumni by providing specialized trainings and career mentoring and that links students with networking and employment opportunities. Objective 2: To develop students’ and graduates’ job seeking skills and to strengthen their ability to obtain employment: LEEP will provide university students and recent graduates with critical skills and access to employment opportunities needed to successfully secure professional employment. Objective 3: Foster Collaboration between Universities, Industry, Business and Civil Society: CDCs will nurture sustainable relationships between universities and external stakeholders in 20 industry, business and the non-profit sector. This will enhance relevance of program by providing students’ knowledge of and access to career opportunities and job training, and create dialogue between universities and the outside world. Through internships with NGOs, students will also gain employment skills while strengthening these organizations. The primary direct beneficiaries of the program include:  The students and recent graduates who gain job search and workplace skills through the career development center programs.  Current high school students who may benefit from the program’s Youth Career Ambassador Program.  Employers who benefit from support in recruiting qualified, motivated candidates and from increased access to the university and improved communication with administration and faculty.  The partner universities benefit from strengthened and deepened links with local and national industry and business generated through the work of the career development centers within the program. Currently WL is managing two CDCs, one at Helwan University and the other at South Valley University. Two others were started in Fayoum and Port Said Universities, but closed as noted above. The awards went through several modifications as noted below:  MOD #1 on 2/24/2014 to fully fund the agreement and include the approved wind-up plan;  MOD #2 on 3/19/2014 to remove the wind-up plan;  MOD #3 on 7/30/2014 to change the Program Description to substitute Fayoum University CDC with Helwan University and re-align the budget; and  MOD #4 on 4/23/2015 to extend the completion date until December 31, 2015 and revise the Program Description to remove Port Said University CDC and re-align the budget. The award to World Learning was $1,678,768, and the budget was re-aligned twice due to changes in university CDC support. As well, the project completion date was given a no-cost extension through the end of 2015. Success in Meeting Performance Management Plan Indicator Targets The LEEP Performance Management Plan (revised April 2015) generally responds to the goals and objectives outlined in the CDC Support APS-263-11-000001. It includes indicators that monitor the success for addressing the mismatch of relevant skills to the job market and low employability among current university students and recent college graduates; improving employability and interpersonal skills of young Egyptians; and strengthening their ability to create and capitalize on job opportunities in various sectors. There is no baseline data as the project started from a zero base. However, given the overriding goal to establish sustainable operating CDCs that may be transferred when funding ends to the host university, indicators and targets to track specific 21 progress in this area such as growth of revenue generation, transfer of CDC staff to university payrolls, and progress toward meeting sustainability plan actions are missing. While staff recruitment and committee meetings, as well as service delivery are important, they could use supplemental indicators for measuring potential sustainability. This is especially critical in the financial area, where the ability of the CDCs to generate funds close to matching expense levels is very critical for survival in the host university. As well, although there is some information concerning students applying career development skills to their employment search, there are no data tracking successful employment numbers or linking the application of trained skills to job search success. PMP Indicators LEEP has a total of three outcome level indicators, one for each objective. In addition, they define output level indicators that support these outcomes. LEEP has a total of 16 indicators. ______________________________________________________________________________ Objective 1: Self-sustaining Career Development Centers established at partner universities Outcome 1.1: # of CDCs embedded in partner universities Output 1.1.1: # of CDC personnel recruited Output 1.1.2: #of Management Committee meetings conducted Output 1.1.3: # of Advisory Board meetings conducted Output 1.1.4: # of staff participating in training and development Output 1.1.5: # of staff participating in language skills training Output 1.1.6: # of training sessions held under USAID funded LEEP project Output 1.1.7: # of students attending 16 hours (or more) of training at the CDCs Objective 2: To improve job seeking skills among students’ and graduates’ and improve their ability to obtain employment Outcome 2.1: % of graduates from USG supported tertiary education reporting themselves as employed Output 2.1.1: # of individual persons gaining access to CDC work development programs supported by USG Output 2.1.2: % of student clients demonstrating improved skills after CDC training Output 2.1.3: % of students applying career center content in their employment search Output 2.1.4: # of student with improved language skills Objective 3: Universities, industry, business, and civil society are collaborating with CDCs Outcome 3.1: # of partnerships developed with industry, business, and non-profits Output 3.1.1: # of employers using career center services to search for employees or interns Output 3.1.2: # of career fairs held at each partner university ______________________________________________________________________________ Based on the data provided to the evaluation team, triangulated by interviews and random samples verified during the CDC visits, the LEEP performance on meeting indicator targets is relatively strong based on numbers through the period of the evaluation. Major achievements include outputs involved in providing service to students such as access to career development programs and soft skill training (i.e. Outputs 1.1.6, 1.1.7, and 2.1.1.) where 22 participation surpass targets by 140-160% despite only having two operating CDCs. As well, reported data from year 2 surveys indicate high use of skills in employment searches. This is supported by the student focus group data. Indicators tracking staff actions and committee meetings are mixed, but do not seem to have impacted operations significantly (see individual CDC evaluations below). As well, the employment fairs and youth ambassador programs were delayed in starting with resulting lower results (below target by 50% or more at the time of the evaluation). A complete table of LEEP PMP indicators including targets and accomplishments is contained in annex C. Implementation of Other Strategies Identified from SOW (non-PMP indicators) The LEEP modified scope of work includes other strategies and actions that are not tracked by PMP indicators. Generally the evidence indicates that these have been accomplished. Given the mid-term suspension, some actions took longer to initiate such as the student ambassador program, the career fairs and the initiation of English language training. But, these are now on track. Communication plans are in place and a variety of media are employed to promote the centers (as evidenced by data collected during the student focus groups). LEEP undertook to train the CDC staffs using experts from the WL home office. This has resulted in larger staffs (5-8) that include trainers. While carrying more office overhead cost, this allows for full internal capability to execute all programs (rather than relying on outsourcing). The faculties in both LEEP CDCs have been highly engaged. They have become volunteer trainers and engage students for counseling session outside the centers in offices and sometimes homes. The network of employers is growing and strengthening, resulting in sponsorship/delivery of technical training such as Build Your Business in collaboration with Masr Ta3mal-Microsoft. While grant parameters did not allow for generation of revenue for the centers, during employment fairs they were able to obtain cost share contributions of materials and services. LEEP/WL partners further include a network of NGOs and private sector entities including Etisalat, Vodafone, Microsoft, Intel, The Marketer.Net, and AWTAD to support training workshops on career development and employability skills programs. The practice of student surveys, including alumni who participated in center services, is providing some data on employment opportunities realized and application of new skills in job search. A special program addressing needs for the sight impaired is a success. Best practices The LEEP initiative has several actions that the evaluators highlight as ―best practices‖ in building sustainable CDCs: 23  Initiating youth ambassadors: introduces the center and university to graduating secondary students.  Building internal staff training capabilities: allows for better quality monitoring and lower cost over time than external commercial sources.  Ongoing active student / faculty volunteer networks: help build enthusiasm for the program, while spreading word of services and successes.  Bringing specialized English language business curriculums and training existing teaching assistants to deliver them: while very challenging and time consuming, over the long term this builds further university capacity to support career center needs rather than out-sourcing.  Conducting employment surveys: as these are refined, they can provide data on jobs realized, key skill sets used from training, and provide links between employer needs and CDC service.  Addressing special needs: the program for the vision impaired is done in collaboration with a nonprofit. This can become a model for other special needs services over time.  Use of expanded advisory committees for centers: while these can be cumbersome to initiate (beyond just a management board), an expanded advisory group that includes employers, faculty and students can allow for diverse inputs and feedback for program design and monitoring. Challenges  Given the effort needed from both parties to build a sustainable center in a few years, there is a need for the development of a joint sustainability plan. This was not prepared with the LEEP university partners. A draft of this plan should be developed within the first 6 months of initiation that includes key milestones and revenue targets. This should be finalized at the start of the final year of USAID support and extend at least a year past transition of the center to the university.  The Memorandums of Understanding were signed with universities prior to award of the grants, more in the form of a traditional teaming agreement. This resulted in fairly general stipulations based on lack of operating experience. The later MOU signed with Helwan University built in further requirements (although primarily on the university side, not WL). These still can be strengthened to address more sustainability actions that include the requirement for a joint sustainability plan and clear indictors for monitoring progress.  The requirements of the MOU did not allow for revenue generation (beyond in-kind cost sharing) for the centers. This practice sets up unrealistic perceptions among stakeholders that services will remain free. Without the opportunity for growing income generation, it becomes very difficult to predict likely self-funding levels after the conclusion of the LEEP. As well, financial data on exact costs (beyond estimated payroll) were not readily available or easily calculated. These should be calculated and included as a portion of sustainability planning.  While the PMP included indicators tied to center operations and service delivery, it lacks more specific indicators to measure progress toward sustainability. This could include financial targets, timetables to include the center within the university budget and payroll, and other jointly identified milestones. 24  The CDC staff is provided training and coaching in all aspects of center operation and service delivery. However, there is a lack of written policies / procedures for routine administration and operations. Since knowledge is not written and primarily in the minds of the staff, when employees leave there is little left behind to guide and orient new replacements.  The distribution of annual surveys still needs to be refined and expanded. Tracking of employment success is still weak. It is especially necessary to attempt to link the key CDC services that support student ability to acquire jobs.  English language skills are clearly identified as a primary employer demand for hire in many cases. The ability of students to learn necessary fluency of language can take years. Providing support to university language programs, while useful is often problematic and delayed. Given the limited resources of CDC funding, this may be a service best left to language faculties. American University Cairo Employability and Career Development Centers (ECDC) Overview of Goals The American University in Cairo (AUC), Egypt is a premier English-language institution of higher learning. The Career Center, formerly known as office of career advising and placement services (CAPS), was established 20 years ago as the first university career center in the region. Since then, it has been providing comprehensive career guidance and recruitment services to students and alumni and has become a leading model for comprehensive career services in the region. American University Cairo’s project goal, implemented through CAPS, is to provide career guidance and employability skills to public university students to equip them with the right skills for the world of work and bridge the gap between current university graduate skills and job market requirements. Program objectives include: Objective 1: Establish 4 Employability and Career Development Centers (ECDCs) - ―One-stop￾shop‖ – in 3 public universities in upper Egypt, Suez Canal area and Cairo that will become an integral part of the universities after the life-span of the project Objective 2: Hire 16 ECDC staff; of which 15 will be trained and licensed as career development facilitators (CDF) to provide required career education, connect with employees and generate internship/employment opportunities to students in selected universities. Objective 3: Provide an array of job search and soft/employability skills to a selected subset of students from each university; offering a technical track when needed via a network of partnerships Objective 4: Ensure the functionality of a sustainability model to secure availing required employability/soft skills to university students after life-span of project The primary beneficiaries of the program include: 25  Professionally license 15 career development facilitators (CDFs) at two public universities: Suez Canal University and Ain Shams University  Over 200,000 students from 3 different governorates will annually have access to professional career services (around 130,000 students from Ain Shams University including their faculty of Engineering, 50,000 students from Assiut University, and 30,000 students from Suez Canal University).  A subset of approximately 3000 pre-final and final year students will additionally benefit from a specially tailored module that cover career management, employability skills and a specialized technical training track  15-20 professionally trained and coached volunteer trainers for employability skills model who are equipped to sustain the model and re-breed others through ―training of trainer‖ approach  NGOs, corporation and training providers partnering in the project Currently AUC is managing three CDCs, one at Suez Canal University and the other two at Ain Shams University. One other was started in Assiut University, but closed as noted above. The awards went through several modifications as noted below:  MOD #1 on 2/26/2014 to fully fund the agreement and include the approved wind-up plan;  MOD #2 on 3/19/2014 to remove the wind-up plan; and  MOD #3 on 3/1/2015 to extend the completion date till September 30, 2016, change the Program Description to remove Assiut University CDC and decrease the total estimated cost. The award to American University Cairo was originally for $ 2,728,700. The budget was decreased due to reduction in the number of universitiessupported by $426,982 to $2,301,718. The project has a no-cost extension through the end of September 2016. Success in Meeting Performance Management Plan Indicator Targets ECDC has a total of three intermediate result indicators, each with one or more output indicators. ECDC has a total of 8 indicators. These indicators focus primarily on delivery of career development services to students. There are several indicators directed toward measuring employment opportunities, however the linking of these to actual employment is unclear without further surveys. Given the overriding goal to establish sustainable operating CDCs that may be transferred when funding ends to the host university, indicators and targets to track specific progress in this area such as growth of revenue generation, transfer of CDC staff to university payrolls, and progress toward meeting sustainability plan actions are missing. While staff recruitment and training / certification, as well as service delivery are important, other supplemental indicators would be useful for measuring potential sustainability. This is especially critical in the financial area, where the ability of the CDCs to generate funds close to matching expense levels is very critical for survival in the host university. 26 As well, although there is some information concerning students applying career development skills to their employment search, there is no indicator linking the application of trained skills to job search success, and there is no data as regards the key indicator for tracking successful employment numbers (see annex C). There is no baseline data as the project started from a zero base. PMP Indicators ECDC has a total of eight indicators that seem to be related to the four objectives. They are all output oriented and do not show relationship to outcomes. In some cases the specific definition of the output indicator is unclear (such as in the case of 2.5 and 3.2). ______________________________________________________________________________ Output 1: # of ECDC staff receiving training and are certified as Career Development Facilitators (CDFs). Output 2.1: # of students attending career development workshops, and/or corporate information sessions of duration less than 16 hours Output 2.2: # of individuals attending any individual career advising service of duration less than 16 hours Output 2.3: # of students/alumni attending employment fairs Output 2.4: # of internships generated Output 2.5: # of employment opportunities generated Output 3.1: # of persons completing a USG-supported Career Development Center workforce development programs of 16 hours or more Output 3.2: % of graduates from USG-supported tertiary education programs reporting themselves as employed. ______________________________________________________________________________ Based on the data provided to the evaluation team, triangulated by interviews and random samples verified during the CDC visits, the ECDC performance on meeting these indicator targets is quite strong in several areas based on reported numbers through the period of the evaluation. Other areas are weak or inconclusive. Major achievements include surpassing the reported output targets for providing service to students such as short-term career development training, individual counseling sessions and attendance at employment fairs (i.e. Outputs 2.1, 2.2 and 2.3) where participation surpass targets in some cases by 300 – 500%. Numbers for staff certification and longer training programs were near to targets. The indicators 2.5 and 3.2 tracking career opportunities and percentage employed are critical indictors. The numbers for tracking employment opportunities generated seemed to be based on publicizing job announcements to students, however no evidence could be provided to support the number provided. For the indicator percentage of graduates reporting themselves as employed, no data was available for this category. Student surveys regarding employment success or application of 27 skills in job searches have not been completed, or if so, were not shared, although it was understood that these were to be done for a report in the 2nd quarter of FY 2015. A table of ECDC performance indicators extracted from provided quarterly reports that include targets and reported accomplishments is contained in annex C. Implementation of Other Strategies Identified from SOW (non-PMP indicators) The ECDC modified scope of work includes other strategies and actions that are not tracked by the indicators referenced above. Generally the evidence indicates that these have been accomplished in an ad hoc manner as described below. ECDC undertook to train and certify the CDC staffs using the National Career Development Association (NCDA) curriculum that AUC is authorized to conduct. Thus, the ECDC staff is certified as career development facilitators (CDF). The ECDC staff are quite small, usually 3-4 people. However, this staff primarily focuses on core services – individual career counseling, job fairs and employer network. The heavy workload of the delivery of the soft skills training programs is not done by the CDC staff, but rather by ASPIRE and INJAZ. These are two subcontracts on the grant and so these services are paid directly from AUC. The two firms are well known on the Egyptian market and provide high quality programs. However, the costs for these types of services are not inexpensive. The ECDC work scope strategy indicates the transfer of these training skills through training of volunteer trainers. This has not started as of the time of the evaluation. Project staff indicated that this effort will be initiated during the last year of the project. English language training is outsourced to the AUC language center, and this is also paid directly by the grant. No efforts to build capacity within the university are planned. Similar to the soft skills training, this type of training, that requires extended durations for fluency, is expensive. Based on USAID/Egypt Education Office guidance, no financial data regarding the costs of these services were made available. An important strategy from the ECDC work scope was the focus on recruiting 1-2 faculty members per faculty to supplement the small staff with of the CDC. Unfortunately, there is no evidence that the faculty in three ECDC CDCs have been highly engaged. The team was unable to find or speak to anyone who is actively involved with the CDC as regards referring students or acting as a volunteer trainer. At the same time, ECDC centers have been highly successful to build strong student volunteer networks to support the employment fairs. The database of employers is quite large for the CDCs, seemingly supported by the long standing AUC career advisory center’s contacts in business and industry. The ECDC has partnered with a network of NGOs and private sector entities, including ASPIRE Experiential Business Solutions, INJAZ, Vodafone, the International Labor Organization in Cairo, MIT Company, and Mobinil, to deliver workshops on career development and employability skills. While the employer database seems very useful for promoting the employment fairs, in most cases there is no evidence that very deep relationships have been developed between the CDC staff and employer human resource functions. These types of relationships are very important 28 for understanding employer needs and supporting student for specific job opportunities as they are open. The centers were able to provide only a few business contacts to interview, and some of these were only familiar with the job fair, rather than CDC services. The generation of revenue from employers from employment fairs is a success for the ECDC programs. Between the three centers, this income has surpassed 400,000 EGP (~$ 52,000) in the past few years, and is available for the center use within university controlled bank accounts. As well, the CDCs were able to obtain other sponsorship in-kind of materials and services. . The scope of work states that ECDC staff will be trained on strategies for conducting market studies such as conducting bi-annual online employer surveys and employer focus groups, to stay current with labor market demands. As well, faculty members were to be appointed as ECDC liaisons to business. While there has been some surveying of employer satisfaction following the fairs, no hard data was available concerning labor market needs, employment opportunities realized and application of new skills in job search. Generally, faculty has very limited involvement with the ECDCs. Best practices The ECDC initiative has several actions that the evaluators highlight as ―best practices‖ in building sustainable CDCs:  The generation of revenues from employment fairs has been an important element is helping to build the financial sustainability of the CDCs. This both begins to build a bank balance to support the transition to the university, but also accustoms employers to expect to pay for these types of services.  In partnership with ASPIRE, AUC has designed and customized an innovative intensive soft skills training program, especially customized for student audiences  ECDCs have built a strong volunteer of student volunteers and allowed the office to function also as a ―community center‖ for students for informal encounters and engagement.  AUC has championed the certification of all ECDC staff as career development facilitators. This has been instrumental in insuring high quality individual assessment and counseling services. Challenges  Given the effort needed from both ECDC and the universities to build a sustainable center in a few years, there is a need for the development of a joint sustainability plan. This was not prepared with the ECDC university partners. A draft of this plan should be developed within the first 6 months of initiation that includes key milestones and revenue targets. This should be finalized at the start of the final year of USAID support and extend at least a year past transition of the center to the university.  The Memorandums of Understanding with the university partners are quite brief and while outlining goals, provide no details as to roles and responsibilities, requirements for sustainability planning or indicators to measure success. 29  While the soft skills training program is highly acclaimed by the students, this is being delivered by professional experienced trainers from established companies. Although the curriculum has been developed and the plans are to provide training-of-trainers, this effort has not started yet. The concept is to recruit volunteers to learn to teach these programs and provide them at no charge to the center. This model has not been tested and evaluator past experience questions the likelihood of success both to maintain program quality with inexperienced trainers and to keep them engaged without compensation.  While the indicators from the PMP focused on center operations and service delivery, there is a lack of more specific indicators to measure progress toward sustainability. This could include financial targets, timetables to include the center within the university budget and payroll, and other jointly identified milestones.  The CDC staff has been certified and provided training and coaching in all aspects of center operation and service delivery. However, there is a lack of written policies / procedures for routine administration and operations. Given the knowledge is primarily in the minds of the staff, when employees leave there is little left behind to guide and orient new replacements.  The engagement of university faculty was an important element for building sustainable services as regards to liaisons with employers, volunteers to conduct training, and referral points for students. This is very weak area based on the research.  The distribution of periodic employer and student surveys still needs to be started. Tracking of labor market needs and employment success is informal at best. It is especially necessary to attempt to link the key CDC services that are supposed to improve student’s ability to acquire jobs with actual identified labor needs.  English language skills are clearly identified as a primary demand by hiring employers in many cases. In the case of the ECDCs, the language training is provided by the AUC language center and the costs will likely not be sustainable. No capacity building to provide this capability was developed within the university. 30 2. IF AND HOW DID THE SELECTION OF THE GEOGRAPHIC LOCATION FOR THE CDCS AFFECT THEIR PERFORMANCE? TO WHAT EXTENT CDCS GEOGRAPHIC LOCATION AFFECT THE PERFORMANCE OF CDCS IN ACHIEVING THE RESULTS? WHY? The findings in this area looked at a comparison of a number of factors that might influence the successful performance of a CDC. These are rated using a survey instrument that both asked for forced ranking of the factors, as well as scoring them on importance. Data was collected using the instrument directly from both the LEEP and ECDC project staff (8) plus all the managers of the CDCs (6 - including former closed Port Said Manager) for a total of 14. The factors are as noted in the sidebar. After reviewing the results of the survey data, follow-up interviews were held with the LEEP/ECDC Project Directors to further review and prioritize the data. The instrument is below: Figure 2: Ranking of factors influencing the success of CDCs The importance of the 12 factors was consistently rated between both comparisons. On the ranking of importance from 6 – 1, quality of the center services along with good relations with university senior leadership was top, closely followed by competence of CDC director and facilitators. Geography was rated of lowest importance. (Figure 3 below) Competence of CDC Director Geographic location of university Financial support of university administration Competence and certification of facilitators Reputation for matching students with job opportunities Serviceable office and technical equipment Strength of network with employers Financial sponsorship by employers Good relations with university senior leadership Number of employers in the area Strength of communication campaign with students Quality of career development services 31 Figure 3: Importance ranking of successful CDCs Looking at the same 12 factors forced ranked, the same key factors came highest. Once again, geography was quite low. (Figure 4 below) Figure 4: Force ranking of Successful CDCs Based on this survey data, the primary factors necessary for success indicated are:  Quality of career development services  Good relations with senior university counterparts  Strength of CDC staff and director In the follow-up conversations with the Project Directors, as well as triangulated with interviews with CDC staff and university managers, two conclusions were reached and verified: 1. The most important factor for sustainability success is the strong support and ongoing relationships with senior university management. If you have a strong team in place, and do not have support with the university administration, the CDC will likely fail. 2. Geography is seen as a minimal factor influencing success. There is high student demand for these types of services throughout the country. With proper senior university support 32 as a basis, building on a well trained staff and high quality services, over several years’ sustainability can be established. 3. TO WHAT EXTENT ARE THE LEEP/WL AND ECDC/AUC PROJECTS ON TRACK TO TRANSITIONING THE MANAGEMENT OF THE CDCS TO THE HOST UNIVERSITIES? WHICH FACTORS HAVE CONTRIBUTED TO OR HINDERED THE SUSTAINABILITY OF THE CDCS? WHAT ARE THE PROSPECTS FOR SUSTAINABILITY POST￾PROJECT COMPLETION? Findings in for this question are presented based on a set of criteria overlapping with the success factors above. They include the 9 areas below:  University Support: Ongoing support by senior university officials such as President or Vice-Presidents  Director Ability: Knowledge and skills of the CDC Manager to lead staff, provide services, and operate independently without external project support  Staff Competence: Technical knowledge and skills of staff (counselor and trainers) to deliver high quality services  Quality of Facility: Offices, rooms and equipment dedicated for use of the CDC  Faculty Involvement: Volunteer active support in referring students, providing counseling, and training programs  Employer Network: Engagement with companies and nonprofits as demonstrated by attendance at job fairs, sponsorship of technical programs, and ongoing placement of students  Service Delivery: Providing quality services according to established programs and schedules  Student Satisfaction: Perceptions of beneficiaries as to value of services received  Financial Status: Currently level of revenues generated related to ongoing costs  Sustainable: Likelihood of sustainability (financing, staff competences, and service provision) under the university umbrella following end of USAID funding Using these areas of focus, each of the five operating CDCs are reviewed below. The criteria are highlighted by color to depict the overall strength at the time of the evaluation. STRONG NEEDS IMPROVEMENT WEAK Evaluation of South Valley University (LEEP) Small rural area university. Total number of students: 36,000 (UG); post-graduate 11,000 President: Dr. Abbas Mansour VP for Environment and Community Service: Dr. Sayed Taha. 33 The CDC at South Valley is one of the longer operating centers. It has been established by LEEP in 1st quarter 2013 and open for 2 ½ years (with mid-term suspension of activities). University Support: Very strong; graduation ready. University president mentioned that he cannot wait for LEEP to finish their project for him to start generating funds. The project sustainability ideas are well formulated from his point of view, but not shared. Director Ability: Very strong. A faculty member in the university. He is already a master trainer as well. Very apt at fostering good relationships with the university, and staff, Staff Competence: Experienced (5 / 4 trainers) Quality of Facility: High level; 340k EGP (~ $44k) invested by university Faculty Involvement: Very strong. Faculty involved from first days. They volunteer even in their homes to give career counseling. Employer Network: Strong 100+; Big fair; relationships with employers are well developed given the rural area they are working in. Service Delivery & Satisfaction Exceed numbers and targets for training in all areas High student demand; Highly satisfied students Several success stories. One student received a 50K EGP (~ $6.5k) gift from the Nile University for her project that was supported by center advisory. Job Opportunities: Exceed Call Center company created their upper Egypt branch in Qena as a result of the employment fair. Employed 225 persons, as well as the HR rep from the fair Visually impaired persons as supported with employability skills as well. Financial Status: No generation of funds so far. Annual expenses exceed 260K EGP (~ $34k) per year. The plan to generate funds is not documented but the university president is confident that he can do so. Sustainable: Very likely, given the commitment of the university president and the active faculty and the management. Table 1: SVU evaluation Evaluation of Helwan University (Leep) Large Greater Cairo University. Perceived to be second level university. Total number of students: 100,000 (UG) President: Dr. Yasser Sakr Senior Management Rep: Dr. Mayada Belal The CDC at Helwan University is quite new and established by LEEP in 4 th quarter 2014 and operating under one year. 34 University Support: The university perceives the CDC to be a part of the larger Capacity Building Center. Using the CDC as a marketing tool for the university. Director Ability: Currently the Acting Director seems reactive with an unclear vision for building sustainability. LEEP project management is currently looking to recruit a more skilled CDC Director. Staff Competence: Enthusiastic however they lack experience and close supervision. Currently mostly reactive without plans. A total of 7 staff member, with 5 trainers. Quality of Facility: Adequate premises, provided by the university with 3 rooms. Currently 1 room is an office and the other 2 as training rooms. Have no problem in getting more rooms to use when the need arises. Faculty Involvement: Very engaged faculty; They volunteer with the CDC. Several of them happy with the services the CDC is offering, would prefer to add a larger range of services to the CDC to cover the current gap between academia and applied competencies of the student in their fields of study. Employer Network: Moderate, the roster was incomplete and not computerized. However, employers met were enthusiastic about the CDC. Received in-kind donations to their fair from several sponsors. Service Delivery & Satisfaction Exceed numbers and targets for training in all areas High student demand; Highly satisfied students. Job Opportunities: No surveys currently or aggregate data; Bibars school employed more than 15 teachers from the employment fair. Financial Status: No generation of funds so far; Annual expenses exceed 440K GBP per year. The plan to generate funds is not established. Several scenarios are being discussed, but none are well formulated. Sustainable: Not currently as the center is too new, but likely if funded longer through 30 months. Table 2: Helwan University evaluation Evaluation of Suez Canal University (ECDC) A middle-sized urban University. Total number of students: 45,000 (UG) VP Environment & Community Service (Outgoing for retirement): Dr. Kamal Sharobim VP Research and post grad studies: Dr. Nahed Ally (Incoming) The CDC of Suez Canal University is quite established from 2nd quarter 2013, and operating over 2 ½ years. University Support: New VP to supervise the CDC as of June 2015. Medium support will be given until ―AUC is out‖; CDC perceived as AUC satellite. The new VP is interested in changes to the program operation. Director Ability: Very strong, active, and proactive overall; Very good relationship management with the university senior manager. 35 Staff Competence: Strong active staff with a large waiting list of student volunteers to help the CDC. 3 full time, 1 part time staff Quality of Facility: Adequate considering the overall infrastructure of the university. 3 offices + 2 small training rooms provided; Work in corridors, and in the open air when need arises. Faculty Involvement: Limited faculty involvement. And students’ complains from that. The CDC is working as a stand-alone island. Only creating ties with the senior management at the UNI. Employer Network: Large list of employers; first fair successful; two per year planned; Deeper relationships with employers beyond the employment fair could not be verified. Service Delivery & Satisfaction Exceed numbers and targets for training in all areas High student demand; Highly satisfied students. Job Opportunities: Job opportunity surveys currently not carried out; One success story is two CDC students were employed by Henkel. Financial Status: Funds generated from fairs: 67K EGP (~ $9k) in total Annual expenses are 200K GBP plus a undisclosed cost for outsourced training. Sustainable: Likely, but depends primarily on 2 factors: 1) the new VP commitment 2) question how to fund costs of current outsourced training Table 3: Suez Canal evaluation Evaluation of Ain Shams University – General (ECDC) Largest University in the Middle East. 180,000 students (UG) Geographically spread out over several cities and campuses President: Dr. Hussein Eissa VP for Student Affairs: Dr. Mohammed El Toukhy The CDC in Ain Shams – General is quite new and established by ECDC in 4 th quarter 2014, thus operating under one year. University Support: Medium university management support, it is shared with the Engineering CDC. VP mentioned the need to open at least 3 more CDCs: Faculty of commerce, Zamalek area faculties , and faculties of arts and their ―neighboring disciplines‖ Director Ability: Unclear from visit; guarded throughout the conversations. Staff Competence: Very competent staff; Overworked due to heavy demand and unstructured in their authorities and responsibilities; 4 fulltime staff. Quality of Facility: Good; well-equipped premises; More rooms are given when the need arises. Faculty Involvement: Very weak faculty involvement; No faculty members were referred for comments. 36 Employer Network: Large database; Strong response in the employment fair; No depth in areas in other than the employment fair could be verified. Fair generated 88K GBP Service Delivery & Satisfaction Exceed numbers and targets for training in all areas; Highly satisfied students; Very high demand - CDC has difficulties coping with current number of students in need of the service. Job Opportunities: No data could be verified; No surveys currently as to numbers of opportunities generated were available. Financial Status: Fair generates some revenues Annual expenses are 225K EGP (~ $30k) plus a undisclosed cost for outsourced training. Sustainable: Not currently as the center is too new, but likely if funded longer for a full 30 months. Table 4: Ain-Shams general evaluation Evaluation of Ain Shams University – Engineering (ECDC) Largest University in the Middle East. 10,000 students in faculty (UG) Geographically spread out over several cities and campuses President: Dr. Hussein Eissa Dean of Engineering: Dr. Ajman Ashour. The Engineering Faculty is long established from 3 rd quarter 2013, and operating over 2 years. University Support: Very strong, the VP in charge of the CDC is an engineering professor and previously the Dean of Engineering who signed the original MOU. Director Ability: Very strong, proactive; Highly experienced. Staff Competence: Effective, knowledgeable, and active during and after working hours. Good number of student volunteers; 3 full time staff. Quality of Facility: Good; office & 2 training rooms well equipped. More training rooms available when required Faculty Involvement: Unknown, likely weak as faculty members could not be identified for comment. Employer Network: Large database of100+; 2 Fairs generated revenues of over 240k EGP (~ $31k). Service Delivery & Satisfaction Exceed numbers and targets for training in all areas; High student demand; Highly satisfied students. Job Opportunities: No data could be verified. No current surveys monitoring employment opportunities have been conducted. Many corporate events are conducted that may lead to job opportunities. Financial Status: Fair generates some revenues Annual expenses are 210K GBP plus a undisclosed cost for outsourced training. Sustainable: Very likely. Given the senior management support. 37 Table 5: Ain-Shams engineering evaluation 4A. HOW DO KEY STAKEHOLDER GROUPS (E.G, STUDENTS, FACULTY MEMBERS, UNIVERSITY MANAGEMENT, EMPLOYERS) PERCEIVE THE VALUE AND UTILITY CDCS? HOW, AND TO WHAT EXTENT, DO KEY STAKEHOLDER GROUPS ENGAGE WITH THE CDCS AND WHICH FACTORS FACILITATE AND INHIBIT UTILIZATION OF THE CDCS? The evaluation looked at these questions in detail and much information has already been provided in the earlier narrative. Although there are some trends common to all CDCs, each individual center and university also have unique characteristics. The key findings for each of these stakeholder groups are summarized below: A. University management As previously explored in the analysis of factors influencing the success of CDCs, the support of the senior university management is the most important factor for success. Based on team interviews with these officials, the following findings have been identified: Senior university officials become engaged for several reasons (usually combined):  The CDC is seen as an important supplement to a university curriculum that is not linked to business needs and an important way to strengthen graduate abilities to get jobs  The CDC is hoped to become an important nucleus linking faculty, employers, and students through a variety of activities to build the university effectiveness in the community  Publicity from high profile events and day-to-day successes of the CDC are a way to build the university image in both traditional and social media. The most successful CDCs engage university management including the President. Typically, they are within the university structure under the Vice-President of Student Affairs (or similar role). When management is approached for CDC sponsorship based on the apparent values above, officials are willing to provide resources such as space, equipment and utilities. The LEEP/ECDC project director must engage them in a spirit of joint partnership and through the establishment of management/advisory committees transfer the relationship to the CDC manager as they reach competence. B. Faculty There have been mixed results with engaging faculty in support of the CDCs. Motivations for faculty to become active with the CDC are several and must be established individually for each member.  Personal desire to support individual students in ongoing development and career success  Interest to learn new knowledge and skills in areas of training and counseling 38  Potential for visibility within the university by being linked to CDC successes Prior to attempting to engage faculty members, it is important to secure the visible support of senior university management to include the President if possible and the Dean of the faculty. With this support, contacts with potential faculty volunteers can be initiated. Faculty can support the CDC operations in several ways:  Promoting the CDC during their classes  Actively referring students to the centers  Volunteering as a liaison to employers or in support of employment fairs  Undertake training as a counselor or instructor and volunteer to provide of career development programs Only small numbers of faculty are required as instructors or counselors. These can often be recruited over time as they witness the success of the CDC outcomes. C. Employers Businesses and nonprofits are continually looking for candidates to replace turnover in their workforce or support increases in jobs due to growth. There are several motivations that help to engage employers, usually through their human resource functions:  Forward job openings to CDCs to distribute to students  Participate in an employment fair for visibility and possible leads for jobs  Participate in an employer survey or focus group to provide inputs to requirements for student skills that can be addressed in CDC career development services  Sponsor technical training to both educate potential workforce and identify high potential candidates  Work closely with the CDC for specific job opportunities that result in vetted referrals Developing a close relationship with human resources specialist is a time-consuming process, but once established can be maintained for years even as the person moves to other firms. The CDC staff must evaluate potential partner companies very closely as regards to likelihood of providing value to the employer in filling jobs with qualified graduates. D. Students Demand by students in all the universities where CDCs have been established is very high. Often there are waiting lists for programs. In the focus groups that were conducted with 182 students (9 groups with all CDCs) during the evaluation satisfaction was very high with the services provided. Many students took advantage of a variety of programs. As well, the centers have found it easy to recruit highly motivated and engaged students as volunteers to support employment fairs and other activities. 39 The overall students’ perception of the CDC services is very satisfactory. It is agreed that the range of services offered had a positive impact on the lives and futures of the students who had the opportunity to participate. The CDC services offered, whether by LEEP or ECDC, represented an opportunity for positive individual change and skill improvement. Students generally suggested that the center be expanded, and have offices to address specific faculties or disciplines. Especially of interest would be offers of core technical courses in certain highly technical disciplines, in order to help them target their employer opportunities more effectively. A separate report on the findings of the student surveys and focus groups is contained in Annex B. 4B. HOW EFFECTIVE HAVE THE CDCS BEEN IN CONNECTING GRADUATES WITH EMPLOYERS AND PROVIDING THEM WITH THE SKILLS THAT EMPLOYERS DEMAND? This is an area of weakness in some of the CDC programs. For surveys to be useful, the CDCs need to be in operation for over a year in order to have graduates for research. The Helwan and Ain Shams - General CDCs are still too young. There is data from two of the LEEP CDCs that initiated student surveys looking at both employment success and perceptions of the value of skill training in supporting the job search. The Suez Canal University and Ain Shams Engineering ECDCs have yet to begin this research, so data is not available at this time. The evaluation team took survey results from LEEP as provided and there may be risks with validity or reliability. Of the research completed by the LEEP program with Port Said (prior to closing) graduates, fully 50% of the 84 respondents had found employment at the time of the survey. Sixty-two percent were women. After the South Valley University first employment fair, with a survey of almost 800 attendees, almost 10% reported employment following the event. Other SVU surveys followed several training courses. In one, close to 25% responding to the survey reported finding employment in one batch of 74 graduates. In another batch, 12% of 54 graduates were successful. In all cases, graduates reported high benefit from the CDC training whether or not they obtained employment. As Helwan is a relatively new program, no surveys have been completed yet. This is an area that clearly needs ongoing focus from all CDCs. However, the initial data supports the value of the CDC services in supporting graduates in their job searches. However, these are quite low numbers of respondent based on high participation in the training programs. As regards surveys of employers evaluating the value of skills developed in graduates for satisfying job requirements; this is a research area that has yet to be investigated. 5. HOW EFFECTIVE HAVE THE CDCS BEEN IN ACHIEVING GENDER BALANCE IN THE PROVISION OF CDC SERVICES (E.G, TRAINING, JOB PLACEMENT SERVICES, ETC.) 40 DURING IMPLEMENTATION? WHICH FACTORS HAVE FACILITATED OR INHIBITED THE CDCS FROM ACHIEVING GENDER BALANCE IN THE PROVISION OF SERVICES? The CDCs have all been very effective in achieving gender balance, with high levels of female students involved in all facets of career services. Factors that have influenced this success are explored in Annex D in detail. They include multiple strategies for promoting the centers, high student referrals to friend, and use of social media. The career services have been designed as gender neutral in content and the counselor / trainers are very effective with both gender groups. CONCLUSIONS Based on the findings explored previously, there are a number of conclusions to be drawn from analysis of the integrated set of data from desk research, interviews, and focus groups.. Prior to exploring the conclusions, it can be helpful to review lessons learned over the period of the project. Although the LEEP and ECDC projects have different work scopes and approaches, they both have a common goal to establish operating career development centers within their partner universities that will be sustainable once USAID funding concludes. LESSONS LEARNED 1. The five CDCs operating under the grants provide a variety of services to students, based on the original project scope of work. Currently there is no common agreement of an optimal CDC model: what should be core services, what size and make-up of staff is best, what topics of supplemental soft skill training are most important, whether trainers should be a part of staff or outsourced, what are the parameters of an employers’ network, what should be the nature of faculty involvement, etc. A ―best practices‖ model of a CDC would be helpful to guide future implementation in partner universities. This can be included in future Requests for Proposals if funding is made available to expand the program. 2. The findings clearly point to the importance of senior university management support as the critical factor in building sustainable CDCs. When evaluating potential partners, this should be the paramount issue to be vetted, as opposed to geographic location, size of student body, or nearby employers. The support should be demonstrated by university provision of facilities, identification of potential internal staff, and agreement to find a mechanism to include the center in the university budget and payroll within a few years. 3. The evaluation reviewed the operations of the current CDCs, with several relatively new and others operating for several years. The findings demonstrate that it takes several years to establish a physical facility, hire and train staff, rollout training, and build an employers’ network. Financial support for 30 months, with the last six being overtaken by CDC revenue generation and university budget support is necessary when considering timetables and funding. 4. Although there was consideration of elements, none of the five CDCs have prepared a sustainability plan. Without this, it is unclear the milestones necessary, joint role and responsibilities, financial requirements and progress indicators. Preparing a framework jointly with the university within the first six months of the project, that is finalized at the beginning of 41 the last year of support, will allow both partners to insure actions are taken to build sustainability. 5. The memorandum of agreement (MOU) signed with the university is a critical document to establish roles and responsibilities of both parties, not just the universities. These should require the joint preparation of a sustainability plan, key milestones toward sustainability, and indicators to measure progress. The potential instability of management support in some institutions might be controlled by MOUs with phases and milestones to be achieved; if stipulated conditions are not met, USAID funding could be suspended. 6. The project staff, CDC management and university management did not appear to have explored very deeply the financial requirements and source of funds for sustainability after the conclusion of the project. While the evaluation team had prepared simple revenue/cost tables for data collection, it was very difficult to find data to analyze. No financial projections have been prepared (tied to lack of sustainability planning above) and parties assumed the universities would find the budget or CDC revenues would suddenly spike, and all will work out. Financial planning, with realistic targets and expense ratios, must be a highlighted element for both planning and monitoring. 7. The PMPs, which contains indicators for monitoring performance, are weak in identifying specific outcomes/outputs focused on sustainability. While hiring and training staff are important, along with delivery of services, other indicators as mentioned above focusing on financial progress and actions for the university to move the CDC administratively internally should also be included. PMPs with a heavy emphasis on service delivery targets and outputs large numbers can sometimes distort the focus of the small CDC staff. Actions that take time, such as developing deeper relationships with employers (beyond job fair attendance), or surveying graduates, may lose staff attention while they pursue the more visible event attendance targets. 8. Although all CDC staff have been trained and in some cases certified, there is a uniform lack of written policies and procedures manuals (other than some communication strategies) in place within the center. When staff leaves, the knowledge leaves with them. Although time￾consuming to develop, written operation manuals that document best practices, procedural steps, and approved policies can be very helpful to orient new hires. If a common model of the CDC is adopted, basic administrative software can be developed and provided to the new CDCs to automate data collection and improve the ability to monitor and evaluate efforts. 9. While it is agreed that English language is an important qualification for many jobs, the addition of this component to the CDC services has been highly problematic. ECDC has attempted to address this by outsourcing, while LEEP pursued a strategy to introduce business curriculums into existing language programs. Becoming proficient in English language is a multi-year effort and might best be removed from the scope of CDCs. 10. While there have been some efforts to track graduate success with employment, and the link between CDC career training, this must be expanded. Surveys of both graduates and employers 42 should use common templates for easy comparison, and may benefit with the contributions of education/labor market specialists in their design. CONCLUSIONS Question #1: Generally, despite the recent instability and mid-term suspension of activity, the ECDC and LEEP projects have achieved the goals of their modified scopes of work. Although both had operating CDCs closed with partner universities, investigation concludes that this was based on political issues that influenced senior university management support. Performance indicators that were monitored were surpassed in some cases, especially service delivery. Other weaker activity still has the opportunity to be advanced during the no-cost extension period. Questions #2: Geography is not an important factor in CDC performance. Most critical is university manager support, followed closely by quality of services delivered by competent staff. Questions #3: Those CDCs with over two years of operation are well on their way to sustainability. As covered throughout the analysis, a variety of factor influence sustainability. Other than necessary sustainability planning, to include, financial elements, the longer term CDCs have embedded these factors. While individual CDCs need to strengthen faculty involvement, employment networks, and regular impact surveys, these can be improved with management emphasis. Questions #4: The engagement with key stakeholders has been variable depending on the program and individual CDC. Generally students are highly engaged at all centers, while active faculty involvement and deeper relationships with employer networks should be strengthened at some locations. All are engaged with and have support from university management, although this can be problematic and requires close attention when managers change. The effectiveness of CDC services providing the right mix of skill development and connections with employers requires more attention. While tracking has started with limited populations in some universities, and show promise, more extensive evaluation needs to be done. Question #5: Gender balance is excellent in the programs as illustrated in participation in services. When considering the development hypothesis … that the Career Development Centers will enhance a student’s employability, stability and prosperity through the provision of training courses, mentorship opportunities, workshops, internships and job opportunities, which will result in an educated workforce that responds to the labor market needs. The evaluators conclude that while this is partially supported by the findings, there still needs to be further research as regards the impact of the CDCs as directly linked to student employability and prosperity. If this proves to have a high correlation, there will still be a need to scale up the program dramatically given the hundreds of thousands of students graduating annually. As well, for the most impact, it is recommended that new CDCs be implemented along best practices outlined in the report. RECOMMENDATIONS 43 Based on the previous findings, lessons learned and conclusions, the following recommendations are made: Main Recommendation: The Career Development Center concept has value toward achievement of matching workforce education to labor market needs and should be scaled up under best practice model if priorities and funds permit. Programmatic Issues for Future CDCs  USAID should adopt a successful model with best practice criteria and indicators to include in future RFPs  The key factor for selecting public universities for CDC expansion should be strong senior leadership support  MOUs with public universities should include detailed responsibilities and indicators to address sustainable transfer of staff and services. In the case of unclear commitment, the MOU should define several phases with targets to monitor the ongoing relationship.  The Project Management Program for the CDCs should include additional indicators for monitoring sustainability to include financial targets  A sustainability transfer plan should be established jointly with university management within 6 months of project start and finalized one year prior to end of USAID funding (required in MOU)  A Ministry of Higher Education or Council of University policy issued supporting CDCs in public universities will be helpful to promote understanding and interest Establishment of Future CDCs  Project support and funding for a new CDC should be for a minimum 30 months; income generation by the CDC should be allowed, encouraged and tracked  In larger universities, CDCs should focus either on specific faculties and/or physical locations for specialization in the job market and easy student access to services  A priority should be placed on hiring and training existing staff for CDC positions when possible  As soon as possible, the CDC should become a special project unit with a budget line and staff on the university payroll  CDCs should develop internal trainers for core courses  CDCs should maximize sponsored technical training from employers to supplement training offered by the center  Initial CDC focus should be on core services (interviews/cv preparation/counseling, employer network and fairs, and finding placement opportunities)  Soft skills training should be introduced once the core services are operational or be optional for small centers  While efforts should be made to link students with existing language programs, the CDC should not offer this service 44 Operation of Future CDCs  Policy and procedure manuals should be developed and in place within 6 months of the start of the CDC.  Regular graduate / student surveys should conducted on employment success and application of trained behavior/skills (at least annually); survey design experts should be engaged to create valid and reliable instruments  For training programs, pre/post testing should be implemented to gain immediate feedback on effectiveness  Employers should be surveyed periodically for specific needs to provide as input into service design, as well as evaluating the value of skills developed through CDC services in graduates for satisfying job requirements  Advisory committees should be required and consist of stakeholders such as faculty, employers, and students  CDC staffs in different universities should network closely to trade lessons learned and share best practices; consideration should be given to bi-annual meetings that also include skill development  Standard software for CDC administration should be developed and provided to start-up CDCs to improve efficiency, monitoring and evaluation 7. REFERENCES The following documents were reviewed: 45 46 ANNEXES 47 ANNEX A: LIST OF STAKEHOLDERS CONTACTED The following stakeholders were key informants interviewed during the evaluation along with 182 students in 9 focus groups (see annex B). Stakeholder Group Name Job Title Project staff Yasmine El Bendary Project Director, LEEP Project staff Ms. Maha Guindy Director, CAPS, Main overseers of project, ECDC Project staff Ms. Dina El Gohary Deputy Director CAPS,Co-Investigator of Project (Suez Canal & Ex.Assiut) , ECDC Project staff Ms. Maha Fakhry Co Investigator of the project ,Ains Sham CDCs (2) , ECDC Project staff Ms. Rania New Hire, Financial Admin, ECDC Project Staff Dalia Awad AIN Shams CDCs Project staff Ms. Norhan El Badry Project Manager / Project manager Suez Canal CDC , ECDC CDC Mngt. Staff Ms. Seddiqa Ashraf Senior career developer, SCU CDC Mngt. Staff Mr. Ahmad Staff member, SCU CDC Mngt. Staff Ms. Eman Staff member, SCU CDC Mngt. Staff Ms. Aya Ahmed IT & Communication Officer, SCU CDC Mngt. Staff Eng. Islam Naaeem Director, ASU-ENG CDC Mngt. Staff Ms. Mira Hani Career Devel Spec., ASU-ENG CDC Mngt. Staff Ms. Noha Ahmed Senior Recruitment Officer ASU-ENG CDC Mngt. Staff Eng. Shady Shafik Acting Director, Senior Career Development Specialised, ASU CDC Mngt. Staff Dr. Mohamed Omran CDC Manager, SVU CDC Mngt. Staff Mr. Mohamed Mohsen Trainer, SVU CDC Mngt. Staff Mr. Ahmed Shahin Master Trainer LEEP, SVU CDC Mngt. Staff Ms. Karima Gad Trainer , SVU 48 CDC Mngt. Staff Ms. Hend Magdy Admin Assistant, SVU CDC Mngt. Staff Dr. Fairouz Acting Director, HU CDC Mngt. Staff DR. AMR EL NAGAR MASTER TRAINER & FORMER CDC MANAGER AT CDC- PORT SAID, HU CDC Mngt. Staff MR. AHMED SAMY, TRAINER, HU CDC Mngt. Staff MS. MAI EL KATATNY, TRAINER (NEW), HU CDC Mngt. Staff MS. ASMAA EL SHERIF, TRAINER (NEW), HU CDC Mngt. Staff MS. AMINA EMAM, TRAINER (NEW), HU CDC Mngt. Staff MS. SANEYA Adminstratvie Assitant, HU CDC Mngt. Staff MS. FATMA Adminstratvie Assitant, HU Senior Uni. Mgr Dr. Kamal Sharobim VP of SCU Senior Uni. Mgr Ms. Nahed Mostafa Mohamed Aly VP post grad studies and Research, SCU Senior Uni. Mgr Dr. Mohammed El Toukhy VP Student Affairs, ASU Senior Uni. Mgr Professor Ayman Ashour Dean of the Faculty of Engineering , ASU-ENG Senior Uni. Mgr Dr. Ayman Wahba – Head of Computer and Systems Department, Faculty of Engineering.ASU Senior Uni. Mgr Dr. Yasser Sakr HU President Senior Uni. Mgr Dr. Mayada Belal CBC director, direct CDC supervisor, HU Senior Uni. Mgr Dr. Abbas Mansour President, South Valley university Senior Uni. Mgr DR. SAYED TAHA, VICE PRESIDENT OF ENVIRONMENT AFFAIRS AND COMMUNITY SERVICES (UNIVERSITY LIAISON) Senior Uni. Mgr Dr. Nahed Aly Senior Management, SCU Uni. Faculty DR. MOHAMED YOUNIS,VICE DEAN, FACULTY OF LAW, SVU Uni. Faculty DR. SALAH SALEEM, VICE DEAN, FACULTY OF ARTS SVU Uni. Faculty DR. RAFAAT SHIPAT, FACULTY OF AGRICULTURE SVU 49 Uni. Faculty DR. MONA SHAHAT, FACULTY OF ARTS SVU Uni. Faculty DR. MAHMOUD ABOU EL MAGD, FACULTY OF EDUCATION SVU Uni. Faculty DR. MOHAMED HEMDAN, FACULTY OF EDUCATION SVU Uni. Faculty DR. ALAA TAG, FACULTY OF COMMERCE SVU Uni. Faculty DR. RANA TOGHAN AHMED, FACULTY OF MEDICINE SVU Uni. Faculty DR. AHMED EZZAT, FACULTY OF VETERINARY SVU Uni. Faculty Dr. Nabiha Kotb Faculty of Education, HU Uni. Faculty Ms. Rasha Kholy Faculty of Commerce member, HU Uni. Faculty Dr. Ashraf Meraey Head of Disability Unit, HU Uni. Faculty Dr. Mayda Mostafa Faculty of Science member, HU Uni. Faculty Dr. Mohamed Hassan Faculty of Education member, HU Uni. Faculty Dr. Kamal Abdel kader Faculty members, SVU Uni. Faculty Dr. Sherine Mazloum English Professor, Faculty of Arts, ASU Employers Mr. Ahmed Shawky Sourcing Manager, ETISALAT Employers Ms. Ola El Kady HR Head, PWC Employers Mr. Mohamed Ramadan HR & PR Director, BIBARS SCHOOLS Employers Ms. Amal Mohamed Assistant Manager, QALB KEBEER Employers Mr. Abou Hassan Ibrahim (Sherif) Training & Development dept. Manager, ORIENT Employers Mr. Adel Shaaban HR Manager, EXCEED Employers Ms. Eman Tayseer HR Assistant, EXCEED Employers Mr. Ahmed General Manager, EGYTRUST Employers Ms. Neehal Dawood HR Executive, HSBC Table 6: Contacts and informants list 50 ANNEX B: STUDENT FOCUS GROUP ANALYSIS The following annex outlines the results of the student focus group meetings. The students are the primary service beneficiaries. Their perception about the quality of service in the CDC is an integral part of the success factors of the CDCs as well as the potential future for each CDC. INTRODUCTION, AGGREGATE AND GENDER STRATIFICATION: The CDCs visited were:  LEEP Active CDCs: o Helwan University (1 focus group) o South Valley University (1 focus group)  ECDC Active CDCs: o Ain Shams General (2 focus groups) o Ain Shams Engineering (2 focus groups) o Suez Canal University (3 focus groups) STUDENT AGGREGATE AND DISTRIBUTED DATA: As per the following table, the total aggregate number of students interviewed within the focus groups is 182, divided per service provider as follows:  ECDC: 126 students  LEEP: 56 students Figure 5: % of students ECDC vs LEEP As per the following table, the number of students interviewed per CDC is  LEEP Active CDCs: o Helwan University 25 students o South Valley University 31 students  ECDC Active CDCs: 69% 31% Number of students ECDC LEEP 51 o Ain Shams General 31 students o Ain Shams Engineering 47 students o Suez Canal University 48 students T The following table represents the percentage distribution of students per CDC visited. Figure 6: Students % of distribution per CDC GENDER STRATIFICATION: As per the below tables, the ratio between females and males is  55% females  45% males Figure 7: % of Females vs Males It is noteworthy to mention, as per the below table, that the female percentage was higher in Suez Canal University & South Valley University at 71%. However, these percentages were reduced in the Ain Shams University Engineering section, where males reached 74%. The Faculty of Engineering in Ain Shams University is mostly attended by male students, and the overall ratio of females in these faculties is much lower. ASU 17% ASU-Eng HU 26% 14% SCU 26% SVU 17% Students % distribution per CDC Female 55% Male 45% Females vs Male (overall) 52 Figure 8: % of females vs males per CDC STUDENTS PERCEPTION OF THE CDC SERVICES: During the focus group meetings, the facilitator followed the Student Focus Group guide exhibited in annex H. The questions asked of the students ranged from: knowledge of the CDC existence, knowledge of the spectrum of services offered, services used by the students, the usefulness of having a CDC in the university or faculty, the satisfaction of the employment services, the satisfaction from the training services offered, and their range; ideas and recommendations for improving the CDC services, and the perception of the future. Furthermore, individually the students answered a biodata and survey sheet that was aimed at measuring the individual students perception of the value and quality of CDC services, as well as recommendations for improvement. DATA COMPILATION & ANALYSIS: In general, most students are enthusiastic and grateful for having a CDC in their university or faculty. Most students who came to the focus groups even volunteered to be part of team and henceforth, ―give back‖ to the center. In Suez Canal University, the CDC has a waiting list of students wishing to volunteer their time to help the CDC staff. When prompted to answer they found out about the CDC existence, (26%) said through Facebook and websites, another (26%) said that they know the CDC by word of mouth from other colleagues, The lowest (3%) mentioned that they came across the CDC through the general exploration of the university buildings, and the other lower 3% mentioned that they know of the CDC existence through the employment fair. It is noteworthy to mention that most of the students interviewed at ASU general mentioned that they were not informed that the CDC offers services other than English language and soft skills courses. 0% 10% 20% 30% 40% 50% 60% 70% 80% ASU ASU-Eng HU SCU SVU 55% 26% 64% 71% 71% 45% 74% 36% 29% 29% Females vs Males per CDC Female Male 53 Figure 9: How did you know that the CDC exists? How did they try to reach you? When asked to rank the CDC services and training, the student’s answers were mostly as follows:  ECDC-managed CDCs: Figure 10: Ranking for ECDC *It is noteworthy to mention that, AUC had their highest rankings in CV writing and lowest for attending the employment fair. Facebook / website 26% Friends / word of mouth 26% cooperation between student activity 3% seeing the office (curiosity about the new building) 3% Direct contact (mail, messages) 6% employment fair 6% online applications 6% flyers / banners 12% thtrough faculty members 3% TA promotion 9% How did you know that the CDC exists? How did they try to reach you? 7 3 5 6 4 1 8 2 Attending Technical Training programs Attending Language classes Placement services with employers Career advising services with a counselor Internship programs during the summer Attending an Employment fair Help with writing your CV Attending Soft skills training programs Ranking for ECDC ECDC 54  LEEP-managed CDCs: As for LEEP, their highest and lowest rankings were the same as the overall rankings. Figure 11: Ranking for LEEP Regarding overall participation, the highest was attending soft skills training, followed by CV writing. The lowest participation was for technical training and language classes. Figure 12: Highest Student Participation When asked to comment about the effectiveness of past services used within the CDC, most students answered that the job search, resume critique, and presentation skills were the most effective and satisfactory services they received. Furthermore, the students mentioned that the environment of the CDC overall was inviting and that the center offered student support in various areas that reach beyond its current mandate and range of services. 8 6 3 4 5 2 7 1 Attending Technical Training programs Attending Language classes Placement services with employers Career advising services with a counselor Internship programs during the summer Attending an Employment fair Help with writing your CV Attending Soft skills training programs Ranking for LEEP LEEP Attending Technical Training programs Attending Language classes Placement services with employers Career advising services with a counselor Internship programs during the summer Attending an Employment fair Help with writing your CV Attending Soft skills training programs 360 367 429 440 443 530 678 717 Highest Student Participation Overall Points 55 Figure 13: Effectiveness of Service Asked to comment about the offering of employment opportunities and the strength of the employers network, most students mentioned that while there was a network, it needed more strengthening. They expressed that they needed to see more internship opportunities being generated as well as employment opportunities. It is noteworthy to mention that ASU and HU students have not yet experienced any employment services. They are currently unaware that the CDCs even have an employers’ network. When asked to suggest future recommendations for the CDC, most students did not have specific recommendation and answered as follows  The CDC offered them a life changing experience.  The CDC helped them overall to develop quality employability skills  The CDC offered them good opportunities to build team building skills When prompted to mention unsatisfactory CDC services received, most students answered by saying that:  The course scheduling is too intensive (in terms of hours per day) and should be modified  The English courses were mostly delayed and problematic  While the employment fairs offered vacancies for graduates, there were minimal internship opportunities for junior and senior students. When asked about specific courses and trainer capabilities the students’ answers were: Trainer capabilities Overall, the students were happy with the trainer’s rapport building capabilities, and that felt empathy with them. The trainers were receptive to their individual needs, and that they felt that the courses were tailored to their requirements. Attending Technical Training programs Attending Language classes Placement services with employers Career advising services with a counselor Internship programs during the summer Attending an Employment fair Help with writing your CV Attending Soft skills training programs 1 2 3 4 5 6 7 8 Effectiveness of Service Rank 56 1) ECDC CDCs: Most satisfactory course:  Teambuilding skills  Presentation skills  Communication skills  Job search courses. Least satisfactory course: ;  English language course: specifically that the ―courses on offer were conversation General English courses and not business targeted course that help them with the world of work.‖ 2) LEEP CDCs,: Most satisfactory courses:  Employability Skills Least satisfactory courses:  English language specifically ―the delays in the opening of new classes.‖ In conclusion, the overall students’ perception of the CDC services is very satisfactory. It is agreed that the range of services offered had a positive impact on the lives and futures of the students who had the opportunity to participate. The CDC services offered, whether by LEEP or ECDC, represented an opportunity for positive individual change and skill improvement for them. Students generally suggested that the center be expanded, and have offices to address specific faculties or disciplines. Especially of interest would be offers of core technical courses in certain highly technical disciplines, in order to help them target their employer opportunities more effectively. One student1 mentioned that he was so impressed by the services on offer at the CDC, he valued them as more worthy than a full year at the University. 1 Suez Canal university (AUC-managed CDC) 57 ANNEX C: PERFORMANCE MONITORING PLAN TABLES ECDC Plan Progress To-Date Table 7: ECDC Plan Progress To-Date 58 LEEP Plan Progress To-Date Table 8: LEEP Plan Progress To-Date 59 LEEP Plan Progress To-Date (Continued) 60 ANNEX D: GENDER BALANCE ANALYSIS Gender Data Analysis The following annex outlines the results gender distribution throughout the CDCs visited, and the evaluation of the services performed since the onset of the project to date. The data was extracted from the PMP documents received from the implementers. GENDER STRATIFICATION A. LEEP-managed CDCs: 1. Helwan University Figure 14: HU males vs females distribution 2. South Valley University (1 focus group) Figure 15: SVU males vs females distribution Male 37% Female 63% HU Male 36% Female 64% SVU 61 As per the able above, the gender distribution for females is by far higher than males in the overall service delivery, for both LEEP-managed CDCs. B. ECDC-managed CDCs: 1. Ain Shams General Figure 16: ASU males vs females distribution 2. Ain Shams Engineering Figure 17: ASU-Eng males vs females distribution 3. Suez Canal University (3 focus groups) ASU Gen Males 42% ASU Gen Females 58% ASU Gen ASU Eng Males 63% ASU Eng Females 37% ASU-Eng 62 Figure 18: SCU males vs females distribution As per the three tables above, it is evident that the gender distribution, overall, in the ECDC managed CDCs also higher for females; except for ASU engineering. The reason for that is, the overall student body in the faculty of engineering has by far higher numbers of male students than female students. GENDER RATIOS EXTRACTED FROM LEEP REPORTS Figure 19: # of students gaining access to CDCs work development programs From the table above we can infer that the overall number of students who has access to the CDC development programs exceeds for females in both LEEP managed projects. SCU Males SCU Females 44% 56% SCU HU SVU 678 1179 1156 2245 gaining access to CDCs work development programs Male Female 63 Figure 20: # of students applying career centers content in their employment search With regards to students applying career center content in their employment search, the Data received from LEEP outlines that the overall number of females still exceeds the number of males who used the service. Figure 21: # of Students with improved languages skills With regards to students attending work development programs; it is noteworthy to mention that this was the only area outlined where the number of makes exceeds the number of females. This is the in the case of SVU. The reason for this could be: the lowered possibility for female students to attend extra curricular courses in Upper Egypt rural areas. GENDER RATIOS EXTRACTED FROM ECDC REPORTS HU SVU 0 60 0 67 students applying career centers content in their employment search Male Female HU SVU 19 133 23 103 students with improved language skills Male Female 64 Figure 22: # of students who attended "Employability Skills" courses From the table above we can infer that the overall number of females who attended the employability skills training course is generally higher except for ASU Engineering whose nature is to have a larger male student body. Figure 23: # of Students who attended "Career Development" workshops From the able above we can still deduce that in general the numbers of females attending career development workshops is generally higher, except also for ASU engineering. Figure 24: # of Students who benefited from individual career advising services From the table above, we can also deduce the same deductions that the overall numbers of females using the service is generally higher except for the filed of engineering. ASU Eng SCU ASU Gen 370 355 241 197 458 367 EMPLOYABILITY SKILLS COURSES M F SCU ASU Eng ASU Gen 534 42 25 748 21 148 CAREER DEVELOPMENT WORKSHOPS M F SCU ASU ENG 231 43 249 20 INDIVIDUAL CAREER ADVISING SERVICES M F 65 In conclusion, it is safe to assume that the gender distribution overall is higher for females, and lower for males, across all CDCs except for the following:  The possibility of females in rural areas to attend extracurricular activities such as in the SVU, and  The students of engineering in ASU, bearing in mind the nature of the faculty, and the natural distribution of gender in that faculty. 66 ANNEX E: ILLUSTRATIVE PHOTOS OF CDC OPERATIONS 1. HELWAN UNIVERSITY- LEEP Photo 1 Business English language Course Photo 2: Build your Business training program 67 Photo 3: Job Search training program 68 2. SOUTH VALLEY UNIVERSITY- LEEP Photo 4: Disability Training Photo 5: Job fair 69 Photo 6: School Ambassador program visiting the faculty of dentistry 70 3. AIN SHAMS UNIVERSTY - ECDC Photo 7: CV Support services Photo 8: An Experiential learning exercise during employability skills track- ASU Main Campus 71 Photo 9: Thinking patterns, ASU-Main campus Photo 10: Power of positive attitude at work, ASU-Engineering 72 Photo 11: Volunteers at ASU celebrating the success of the employment fair ANNEX F: EVALUATION METHODS AND LIMITATIONS 73 Purpose This evaluation is a mid-term performance evaluation intended to provide USAID/Egypt with an independent assessment of the two Career Development Center (CDC) APS awards to World Learning (WL) and the American University Cairo (AUC). The purpose of this performance evaluation is to review, analyze, and evaluate the effectiveness of USAID- funded CDCs in achieving program objectives and completing deliverables. In line with the SOW approved by USAID/Egypt, the evaluation objectives focus on: 1) Identification of lessons learned with regard to project implementation and building relationships with counterparts in the Ministry of Higher Education and public universities around CDC establishment; 2) Assessment of the sustainability of the CDCs at the Egyptian public universities in which they have been established; and 3) Provide recommendations that can be used to inform the design and expansion of future CDC￾related activities in Egypt’s public universities. The evaluation focuses on the period from 2012–2015, while keeping in perspective the suspension of work during the period October 2013 through March 2014 as a result of USAID’s decision to cease all programs’ implementation during that time. Evaluation Questions The evaluation answers the following questions: 1. To what extent have the CDCs achieved their goals of this SOW? 2. If and how did the selection of the geographic location for the CDCs affect their performance? To what extent CDCs geographic location affect the performance of CDCs in achieving the results? Why? 3. To what extent are the LEEP/WL and ECDC/AUC projects on track to transitioning the management of the CDCs to the host universities? Which factors have contributed to or hindered the sustainability of the CDCs? What are the prospects for sustainability post-project completion? 4. How do key stakeholder groups (e.g, students, faculty members, university management, employers’) perceive the value and utility CDCs? How, and to what extent, do key stakeholder groups engage with the CDCs and which factors facilitate and inhibit utilization of the CDCs? How effective have the CDCs been in connecting graduates with employers and providing them with the skills that employers demand? 5. How effective have the CDCs been in achieving gender balance in the provision of CDC services (e.g, training, job placement services, etc.) during implementation? Which factors have facilitated or inhibited the CDCs from achieving gender balance in the provision of services? 74 Based on evaluation findings, the evaluation team will identify actionable recommendations that will help USAID to better design and implement effective CDCs programming going forward. In particular, identify what are the lessons learned from implementation to date that USAID should take into consideration in future designs, and what are key considerations for replicating CDCs in other geographic locations? Evaluation Steps The evaluation process consisted of the following main tasks of work (see Evaluation Plan for more detail and schedule): 1. Complete initial desk review 2. Planning / training meetings with IIE and CDC research team 3. Planning meeting with USAID 4. Finalize research planning and submit to USAID 5. Conduct data-gathering, compiling and analysis 6. Debrief session for USAID review and comment 7. Submit draft report (in English) for USAID review and comment 8. USAID provides written comments on draft report 9. Submit final report (in English) 10. Executive summary translated into Arabic 11. Final report translated into Arabic The survey team was led by a senior international performance expert specializing in the design and implementation of performance assessments and evaluations, with experience in the higher education sector. The local survey team includes a senior and two researchers, an analyst and administrator. All researchers speak the local languages. Evaluation Design As a mid-term performance evaluation, the design focused on how the CDCs projects have been implemented, what the projects have achieved, whether expected results were attained according to the projects’ design and the underlying development hypothesis, how activities were perceived and valued by stakeholders, and whether the projects are sustainable. The evaluation team used a mix of quantitative and qualitative data collection and analysis methods to answer the questions of interest in this evaluation. The evaluation follows the principles and guidelines for high quality evaluations outlined in the USAID Evaluation Policy (January 2011), as well as addresses criteria from the Checklist for Assessing USAID Evaluation Reports (V1.0). Evaluation Data Sources 75 The evaluation was based on the following sources of information: e) Project reports and other documents listed in the bibliography  Relevant parts of Projects agreement and amendments  Relevant parts of Projects’ quarterly and annual reports  Budget information as relevant and appropriate to be shared.  Projects’ Performance Monitoring Plan  Other appropriate documents as available f) Secondary data sources identified at appropriate points in the report g) Interviews and meetings with LEEP and ECDC project staff h) Field visits in 4 universities: Helwan, South Valley, Suez Canal and Ain Shams (main school and engineering faculty). During these visits, data was gathered from key informants by:  Interviews with CDC management and trainer/facilitator staff;  Interviews with other stakeholders including senior university managers and involved faculty  Focus group meetings with a sample of CDC student beneficiaries  Interviews with local employers  Interview with appropriate manager within the Ministry of Higher Education (not completed based on USAID preference) i) Tables and text prepared by the project staff at the request of the evaluation team for information on specific aspects of the project. The team also attempted to gain interviews with former stakeholders at universities with closed CDCs: Fayoum, Assiut and Port Said, although this did not prove possible. Evaluation Questions The evaluation questions are defined in the Design Matrix (Annex G) and are formulated using three standard evaluation criteria (relevance, effectiveness, and sustainability) suitable for performance evaluations, and overarching criteria such as gender mainstreaming, replication, communications, coordination, and management. The three standard criteria are defined as follows:  Relevance: This evaluation examined the continued relevance of the intervention’s objectives, components/activities, and approaches in light of changing development problems, policies, and priorities. The political, economic, and institutional environments in which a project is designed and implemented tend to change over time. Some of the changes can have major consequences for a project. 76  Effectiveness: refers to the extent to which an intervention’s objectives are achieved or are likely to be achieved. Evaluations seek to determine whether interventions’ services and products are reaching the targeted populations; whether the intended beneficiaries are using them; whether the coverage of the target population is as planned in the project design; and whether the intervention is likely to achieve its targets.  Sustainability: refers to the continuation of an intervention’s services and benefits after foreign assistance ends. Three dimensions of sustainability—financial, institutional, and environmental— will be examined in evaluations. Financial sustainability indicates the capacity of an agency or organization assisted by a project or program to be financially self￾sufficient, either through revenue-generating activities or through substitution of other public, private, or donor sources of funding. Institutional sustainability refers to the supported organization’s capacity to manage its operations independently. Finally, environmental sustainability refers to the capacity of an intervention’s services and benefits to survive in the changed or changing environment. The definitions of relevance, effectiveness, and sustainability have been taken from “Evaluation Guidelines for Foreign Assistance,” Planning and Performance Management Unit, Office of the Director of U.S. Foreign Assistance (Final Version: March 25, 2009). The evaluation team developed data collection tools that are consistent with the evaluation questions to ensure high quality analysis. The KNO performance expert led multiday workshops with the local CDC evaluation team members to facilitate the development of the data collection tools. The local CDC team members contributed to the development of materials based on their experience in this sector. The primary design tool is the Evaluation Design Matrix (Annex G). This matrix breaks out each of the major research areas into more specific sub-questions. The matrix describes for each question the indicator area, source of information, type of evaluation design, type of sampling if used, and the data collection instrument to be used. Prior to the finalizing of the Design Matrix, initial meetings were held with the LEEP and ECDC project staff, as well as USAID Education Office representatives, to insure the design was aligned with expectations. Based on the design matrix, the team developed detailed interview guidelines including questionnaires covering the specific evaluation issues and concerns for all levels of the CDC project implementation. The tools include:  Interview guide for LEEP and ECDC program staff  Interview guide for CDC managers and staff  Interview guide for Senior University Managers  Interview guide for Faculty  Interview guide for Employers  Focus Group Guide for Students  Survey for Students 77 These questionnaires (see annex H) include specific questions for each stakeholder group based on the Design Matrix. However, the team conducted the first set of interviews in the form of a pilot. No adjustment to the questionnaires was found necessary. The detailed interview questionnaires will be used as discussion guidelines and allowed the evaluators to obtain similar types of qualitative information at the various levels of the project implementation. Using these flexible key informant questionnaires, extensive interviews will be carried out with key project staff and stakeholders. The research team has been trained on the use of the guides to conduct interviews. At the conclusion of the planning process, KNO submitted to IIE the Evaluation Methodology and Design Matrix following for their review. IIE submitted the document to the USAID Evaluation Program Manager and organized a meeting with USAID for review, feedback and/or discussion prior to use in the field. A list of the institutions, individuals, and groups interviewed are included as an appendix to the final report. Over 182 interviews and 9 focus group discussions were undertaken by the evaluation team. Data Analysis The evaluation team conducted site visits to each of the five active CDCs, although they were unable to visit the closed ones. The data analysis plan encompasses the following elements:  During the interviews and site visits, the evaluation team will write up the findings immediately using the interview guidelines to insure no information is lost. In case where two team members are present in the interview, one is assigned the primary recording duty and the other edits the notes. Appropriate photos will be taken during the site visits.  Within 48 hours of the data collection, the interview findings will be provided to the research analyst who will input into a customized database, follow-up on any unclear notes, and begin analysis of trends in findings, any gaps in information, potential lessons learned and recommendations.  Initially analyzed data will be provided to the international team leader (distance support) at the conclusion of each CDC collection effort.  Quantitative data will be analyzed using spreadsheets with an aim to provide percentages and ratios for presentation. As appropriate, charts and graphs will be prepared to summarize or highlight data.  Qualitative data will be analyzed looking to derive specific recurring factors, or unique indicators. Data will be triangulated between different informants to assess validity and reliability.  All beneficiary data will disaggregated by gender and geographic location as available.  At the conclusion of the data collection effort, the international team leader will return to Cairo to lead the team in the final data analysis phase. The team leader, with participation of the local team, will weigh and integrate data from qualitative sources with quantitative performance monitoring 78 data to determine conclusions about the effectiveness of both projects and the five established CDCs.  A out-brief presentation will be prepared by the team and provided to USAID on draft findings, lessons learned and recommendations for review and comments prior to submitting the draft report. CDC Mid-Term Evaluation: Overall Schedule Steps Activity Purpose Timetable 1 Complete initial desk review Gain general background on CDC project requirements and formal progress reporting by 26 July 2 Planning / training meetings with IIE and CDC research team a) confirmation of research objectives and key questions b) clarify roles and responsibilities c) identification of key research stakeholders and audiences d) preparation of data collection tools and training e) draft of data collection plan and timetable by 30 July 3 Planning meeting with USAID a) review draft methodology and data analysis plan b) review draft schedule of data collection interviews By 3 Aug 4 Finalize research planning and submit to USAID a) submit final methodology and data analysis plan b) submit final schedule of data collection interviews By 6 Aug 5 Conduct data￾gathering, compiling and analysis Note: further detail provided in attached schedule Gather data concerning WL-LEEP and AUC ECDC performance along the areas of focus:  Identify lessons learned with regard to project implementation and building relationships with counterparts in MHE and the public universities hosting CDC establishment.  Assess sustainability of CDCs in Egyptian public universities where they are established.  Provide actionable recommendations that can be used to inform the design and expansion of future CDC-related activities in Egypt’s public universities. by 10 Sept 5.1 Data gathering, compiling & analysis Suez Canal University By 11 Aug 5.2 Ai Sham University By 18 Aug 5.3 Helwan University By 26 Aug 5.4 South Valley University By 2 Sept 6 Debrief session for USAID review and comment Provide briefing on findings and take feedback on:  CDC achievement and issues  Lessons learned  Recommendations for future designs By 17 Sept 7 Submit draft report (in English) for USAID review and a) Complete draft report considering USAID comments and feedback from briefing clearly describing findings and conclusions. b) Complete draft internal memo concerning recommendations for future By 24 Sept 79 comment programming 8 USAID provides written comments on draft report USAID provides feedback and comments from draft report peer review By 4 Oct 9 Submit final report (in English) Incorporate comments from USAID and finalize the report By 13 Oct 10 Executive summary translated into Arabic Provide executive summary for Arabic audiences By 15 Oct 11 Final report translated into Arabic Provide final report for Arabic audiences By 15 Nov Table 9: CDC mid-term overall schedule Table 10: CDC evaluation timeline of events and processes 80 ANNEX G: DATA COLLECTION DESIGN MATRIX OVERALL FRAMEWORK IS ALWAYS: 1. Identify lessons learned with regard to project implementation and building relationships with counterparts in MHE and the public universities hosting CDC establishment. 2. Assess sustainability of CDCs in Egyptian public universities where they are established. 3. Provide actionable recommendations that can be used to inform the design and expansion of future CDC-related activities in Egypt’s public universities. KEY AUDIENCES:  USAID Education Office: Grant Manager  LEEP/ECDC Directors and Senior Staff  Senior University Management  Employer Human Resource Managers  University Faculty  Students Note: The interview guidelines for each audience are formulated from these key questions. (see annex H)  81 Overall Areas of CDC Research RESEARCH QUESTIONS (Indicator) DATA TYPE SOURCES OF INFO (sample if appropriate) DATA COLLECTION INSTRUMENT I. To what extent have the CDCs achieved their goals and objectives? How did the implementation approach contribute or hinder the achievement of project objectives and planned results? 1. What are the goals / targets for each CDC?  Quantitative targets  Desk research  Management  Document review  Interview Guide 2. Have the defined indicators with targets been met? (ex. Hiring staff, certifying staff, premises, conducting training, building a communication network with employers, compiling a database…. )  Actual number of deliverables vs. targeted per indicator  Desk research  Evidence of achievement (lists, numbers, job descriptions, etc…)  Document review  Interview Guide 3. How does actual performance match the annual work plan?  Quantitative actual results  Progress reports  PMP updates  Document review  Interview Guide 4. What are the main strengths of the implementation over the last couple of years? What are some of the real success stories that you can share with us?  Qualitative Data  Evidence  Photos  Progress reports  Managers  Document review  Interview Guide 5. What are the biggest challenges you faced in the implementation of the CDCs over the last couple of years. How do you think this has affected the achievement of your goals and  Qualitative Data  Evidence  Photos  Progress reports  Managers  Document review  Interview Guide 82 RESEARCH QUESTIONS (Indicator) DATA TYPE SOURCES OF INFO (sample if appropriate) DATA COLLECTION INSTRUMENT objectives? 1) Establishing the MOU with the university, and strengthening relations with University Mngmt.?  Evidence  Progress reports  Managers  Document review  Interview Guide 2) Obtaining facilities, and hiring staff, and setting up the office.  Evidence, Photos  Progress reports  Managers  Site visit  Document review  Interview Guide 3) Delivering training and other services to students  Evidence  Photos  Progress reports  Managers  Students  Document review  Interview Guide 4) Establishing a network with the employers.  List/ Database  Progress reports  Managers  Employers  Document review  Interview Guide 6. Knowing what you know now, if you could change anything in the basic implementation process, what would it be?  Qualitative  Progress reports  Managers  Document review  Interview Guide 7. If you would create a new CDC now with a new university, what would you do to increase the likelihood of  Qualitative  Progress reports  Managers  Document review  Interview Guide 83 RESEARCH QUESTIONS (Indicator) DATA TYPE SOURCES OF INFO (sample if appropriate) DATA COLLECTION INSTRUMENT success of the new CDC? 8. What do you think your main impact is on the students and the surrounding community that you are operating in?  Qualitative  Progress reports  Managers  Students  Employers  Document review  Interview Guide 9. What has been the success of matching students with jobs?  Quantitative survey  Qualitative  Progress reports  Managers  Students  Employers  Document review  Interview Guide 10. Is the concept of having a CDC a successful one? Do you think it actually makes a difference? Do you think it should be in all universities?  Qualitative  Progress reports  Managers  Sr University Mngt  Students  Document review  Interview Guide II. If, and how, did the selection of the geographic location for the CDCs affect their performance? How did the geographic locations of the CDCs affect results? Why? 1. How was a university specifically selected for establishing a CDC? And why? (criteria)  Qualitative  Criteria  Evidence  Progress reports  Managers  Document review  Interview Guide 84 RESEARCH QUESTIONS (Indicator) DATA TYPE SOURCES OF INFO (sample if appropriate) DATA COLLECTION INSTRUMENT 2. How has the number of employers located in the governorate affect CDC results?  Qualitative  Quantitative  Progress reports  Managers  Document review  Interview Guide 3. How do you think the size of the university or population of the governorate affects success?  Qualitative  Quantitative  Progress reports  Managers  Document review  Interview Guide 4. How does “geography” compare to the other factors in the success list compare to build a successful CDC? (*refer to Table 1)  Quantitative; Use forced ranking and scales to compare importance  Managers  Interview Guide  Table 1 III. To what extent are the CDCs on track to transitioning the management of the centers to the universities? What factors helped or hindered the sustainability of the CDCs? What are the prospects for sustainable post-project completion? 1. How independent are the CDCs now?  Qualitative  Progress reports  Managers  Interview Guides  Document Review 2. Who does the CDC director report to now?  Evidence  Progress reports  Managers  Document review  Interview Guide 3. What is the nature of the relationship currently, between the CDC and the University Mngt?  Qualitative  Evidence  Progress reports  Managers  Sr University Mgr  Document review  Interview Guide 4. Where is the CDC office within the overall university structure? Is this  Qualitative  Progress reports  Document review 85 RESEARCH QUESTIONS (Indicator) DATA TYPE SOURCES OF INFO (sample if appropriate) DATA COLLECTION INSTRUMENT appropriate for optimal operation?  Organizational structure  Managers  Sr University Mgr  Interview Guide 5. Is there a written transition plan that includes tasks and timeline for CDC turnover to the university? Who will be involved in the transition? How is progress being measured?  Qualitative  Plan evidence  Progress reports  Managers  Document review  Interview Guide 6. Please use the provided table to detail both the revenue sources and expenses for the CDC operation.  Quantitative  Managers  Table 2 7. Is there currently a line item in the university budget for CDC financing? If yes how much per annum? Is the budget line item financed? If yes by whom?  Qualitative  Quantitative  Budget numbers  Progress reports  Managers  Sr University Mngrs  Document review  Interview Guide 8. What is the current competence level of the staff? What is their background? How long have they been employed at the CDC?  Qualitative  Quantitative - Evaluations of performance  Progress reports  Managers  Document review  Interview Guide 9. How is the CDC staff compensated? Is the compensation aligned with the university salaries? Are they currently paid directly by the contractor or the university?  Qualitative  Quantitative - Financial data  Progress reports  Managers  Document review  Interview Guide 86 RESEARCH QUESTIONS (Indicator) DATA TYPE SOURCES OF INFO (sample if appropriate) DATA COLLECTION INSTRUMENT 10. What is the % and frequency of the CDC staff turnover? And how is the transfer of knowledge managed?  Qualitative  Quantitative  Progress reports  Managers  Document review  Interview Guide 11. Is there a policies and procedures manual that is established and followed by the staff members?  Qualitative - Evidence  Procedures manual  Progress reports  Managers  Staff  Document review  Interview Guide 12. What is the role of the trainer/facilitators (CDFs) within the CDCs and how are they paid?  Qualitative  Progress reports  Managers  Staff  Document review  Interview Guide 13. What are the factors that are most useful in ensuring the sustainability of the CDC?  Qualitative  Progress reports  Managers  Staff  Document review  Interview Guide 14. What are the factors that challenged your efforts to ensure the sustainability of the CDC in the future?  Qualitative  Progress reports  Managers  Staff  Document review  Interview Guide 15. Five years from now, what do you think will happen to the CDC?  Qualitative  Progress reports  Managers  Document review  Interview Guide 87 RESEARCH QUESTIONS (Indicator) DATA TYPE SOURCES OF INFO (sample if appropriate) DATA COLLECTION INSTRUMENT  Staff  Sr University Mngrs IV. How do key stakeholder groups perceive the value and utility of CDCs? (students, faculty, Sr University Mngrs, employers) How and to what extent do the stakeholders engage with CDCs? What factors facilitate or inhibit use of the CDCs? How effective have CDCs been in connecting graduates with employers and providing them will skills that employers demand? 1. Communications & Marketing: Do you have a written marketing and communications strategy? Do you have an ongoing campaign? Please describe it. Who is responsible for it? Who are you targeting and how? How do you measure the success of the campaign?  Qualitative - evidence  Progress reports  Managers  Staff  Document review  Interview Guide 2. University Senior Management Team: 1) Who in in the university senior management team do you communicate with? 2) Who supervises the CDC work in the university? What is the type of reporting? 3) What is the type of relationship you have with the university senior management? 4) Do you have a formal management committee that includes senior university  Qualitative - evidence  Progress reports  Managers  Staff  Sr University Mgrs  Document review  Interview Guide 88 RESEARCH QUESTIONS (Indicator) DATA TYPE SOURCES OF INFO (sample if appropriate) DATA COLLECTION INSTRUMENT management? If so, how is it working? Are there regular meetings? Do they approve a strategy or policies/procedures? 5) Do you have access to student lists and other data? 6) Do you have access to bulletin boards? Publicity tools? Access to premises, lecture halls, facilities? 7) If USAID funding was suspended, do you think the Uni. SMT would continue to fund the CDC? 8) What are the possible means that can be used to strengthen the relationship of the CDC with the University senior management? 9) Is the CDC position in the university strong enough to survive if there is a change in management? 3. Students: 1) What kind of communications strategy do you use to target/reach students? 2) Please list and rank your services by priority with regards to students’ interest and needs? 3) How do most students find out about the CDC and your services?  Qualitative – evidence  Quantitative  Quantitative – student survey  Progress reports  Managers  Staff  Student sample  Document review  Focus Group Guide  Student Survey 89 RESEARCH QUESTIONS (Indicator) DATA TYPE SOURCES OF INFO (sample if appropriate) DATA COLLECTION INSTRUMENT 4) Who do you target from the student body? Year 1 4? Please list them in percentages 5) Do you measure engagement of students in your services? If so, how do you rate it? 6) Are you satisfied with this level of engagement or is there more to do? 7) How is the level of demand comparing to the available resources, with regards to the type of service you offer? 8) What type of placement services do you offer currently? How do you measure the placement rate? 9) What level of fees are you currently charging students for services? How does this impact demand? 4. Faculty: Is the faculty currently engaged with CDC? If yes 1) What form does the engagement take? 2) What kind of communications strategy do you use to target/reach faculty members?  Qualitative  Quantitative  Progress reports  Managers  Staff  Faculty sample  Document review  Interview Guide 90 RESEARCH QUESTIONS (Indicator) DATA TYPE SOURCES OF INFO (sample if appropriate) DATA COLLECTION INSTRUMENT 3) How do the faculty members help project and market the CDC? 4) What % of students, who come to the CDC, are referred by faculty? 4.1) What is the optimal target % of students to come to the CDC referred by faculty members? 5) How does the faculty perceive the CDC program? Do they comment about what the CDC is doing? 6) How do you plan to increase faculty engagement in the future with the CDC? 5. Employers: 1) Do you have a formal communications strategy to reach employers? 2) Do you have a network of employers? If so, How many companies do you have in your network? How does the network work? 3) Who are your contacts at the employer’s offices? Who are the management staff that you deal with at the employers? 4) Do you survey the needs of the job market to identify the priority employment areas? How is this used to deliver services? 5) What kind of agreements (if any) do you have with the employers?  Qualitative  Quantitative  Progress reports  Managers  Staff  Employer Sample  Document review  Interview Guide 91 RESEARCH QUESTIONS (Indicator) DATA TYPE SOURCES OF INFO (sample if appropriate) DATA COLLECTION INSTRUMENT 6) What types of activities do the employers provide you with? (i.e. placement, internship programs, Guest speakers, money sponsorship, other?) 7) Do you organize employment fairs? If yes how often? How many members of the network attend the fair? 8) What level of funding are you receiving from employers? What form does this take (i.e. event sponsorship, fees, publicity, etc)? 9) How many jobs have been filled with employers in the network by students referred from the CDC? 10) How do you find out about employment opportunities in companies? How do you inform the students about the opportunities? 11) What have been the biggest challenges that limit the strength of the employers network? 12) What are the strategies that have proved to be most useful in strengthening the employer’s network? 13) If the CDC project ends, what effect would this have on the employers in finding job candidates? V. How effective have CDCs been in achieving gender balance and addressing special needs audiences in provision of services during implementation? 92 RESEARCH QUESTIONS (Indicator) DATA TYPE SOURCES OF INFO (sample if appropriate) DATA COLLECTION INSTRUMENT What factors have facilitated or hindered CDCs from achieving gender balance and addressing special needs audiences in the provision of services? Gender: 1) What is the ratio of gender participation in your CDC Services?  Quantitative  Progress reports  Managers  Document review  Interview Guide 2) Is this a reflection of the gender balance in the university?  Quantitative  Document review  Interview Guide 3) How do you use communications to encourage both genders to participate in services?  Qualitative  Document review  Interview Guide 4) Do you target special services to ladies/women? If so what are they?  Qualitative - Evidence  Document review  Interview Guide 5) What do you think are the biggest challenges facing young women students in using your services?  Qualitative  Document review  Interview Guide 6) What actions have been most successful to provide services to young women students?  Qualitative  Document review  Interview Guide Special Needs: 1) What is the ratio of Students with Special Needs (SWSN) participation in your CDC Services?  Qualititive  Progress reports  Managers  Document review  Interview Guide 93 RESEARCH QUESTIONS (Indicator) DATA TYPE SOURCES OF INFO (sample if appropriate) DATA COLLECTION INSTRUMENT 2) Is this a reflection of SWSN in the university?  Qualitative - Evidence  Document review  Interview Guide 3) How do you use communications to encourage SWSN to participate in services?  Qualitative  Document review  Interview Guide 4) Do you target special services to SWSN? If so what are they?  Qualitative  Document review  Interview Guide 5) What do you think are the biggest challenges facing SWSN in using your services?  Qualitative  Document review  Interview Guide 6) What actions have been most successful to provide services to SWSN?  Qualitative  Document review  Interview Guide Table 11: Overall areas of CDC research parameters 94 ANNEX H: DATA COLLECTION INSTRUMENTS INTERVIEW GUIDE FOR LEEP / ECDC PROJECT STAFF PURPOSE To gather data from LEEP and ECDC project team concerning performance, lessons learned, and recommendations to support evaluation analysis.  PARTICIPATION  Senior representatives of project management TIME  Approximately 2 Hours (may require several meetings) LOCATION  LEEP, World Learning, Cairo  ECDC, AUC, Cairo DEMOGRAPHIC DATA Date: ___/___/_______ CDC Name: ________________________________________________ Location: ________________________________________________ City: ________________________________________________ Name of interviewee: ______________________________________ Job Title: ______________________________________ Years within the job: ______________________________________ 95 PRIMARY DISCUSSION QUESTIONS (use appropriate probing and follow-up questions) I. To what extent have the CDCs achieved their goals and objectives? How did the implementation approach contribute or hinder the achievement of project objectives and planned results? 1. What are the goals / targets for this grant? (use approved project SOW and PMP as basis for discussion – confirm on Table 3) 2. Have the targets within the PMP using the defined indicators been met to-date? What is projected through the end of the project? (look for evidence of claims, record on Table 3) 3. Have independent strategies and goals / targets been established for each CDC? If so, can you share these? 4. Have the defined indicators with targets been met? (ex. Hiring staff, certifying staff, premises, training participation, job placement, employer network, compiling a database) 5. We have seen the annual project work plans. Does each CDC have one? How well does actual performance match the annual work plan? 6. What are the main strengths of the implementation over the last couple of years? What are some of the real success stories that you can share with us? 7. What are the biggest challenges you faced in the implementation of the CDCs over the last couple of years. How do you think this has affected the achievement of your goals and objectives? 96 1) Establishing MOU with the university, and strengthening relations with Uni.Mngmt.? 2) Obtaining facilities, and hiring staff, and setting up the office. 3) Delivering training and other services to students 4) Establishing a network with the employers. 8. Knowing what you know now, if you could change anything in the basic implementation process, what would it be? 9. If you would create a new CDC now with a new university, what would you do to increase the likelihood of success of the new CDC? II. If and how did the selection of the geographic location for the CDCs affect their performance? How did the geographic locations of the CDCs affect results? Why? 1. How were the universities selected for establishing a CDC? And why? (any criteria?) 2. How does ―geography‖ compare to the other factors in the provided success list for building a successful CDC? (ask them to complete Table 1) III. To what extent are the CDCs on track to transitioning the management of the centers to the universities? What factors helped or hindered the sustainability of the CDCs? What are the prospects for sustainable post-project completion? 97 1. How independent are the CDCs now? Where does the CDC fit within the university structure and to whom does the director report? 2. Is there a written transition plan in place (That indicates transition implementation steps and timelines)? Who will be involved in the transition? How is progress going to be measured? 3. Please detail for us the revenue sources and expenses for each CDC in percentages. What % is USAID money and what are the other revenue sources in %? ( See Table 2) 4. Is there a budget line item in the Uni. budget for CDC financing? How are expenses paid currently? 5. What is the current competence level of the staff? How long have they been employed at the CDC and experience level 6. How is the CDC staff compensated? Is the compensation aligned with the university salaries? Are they currently paid directly by the contractor or the university? 7. What are the factors that are most useful/helpful in ensuring the sustainability of the CDC? 8. What are the factors that challenge your efforts to ensure the sustainability of the CDC in the future? 9. Five years from now, what do you think will happen to each CDC? Will it exist, be bigger or smaller? Will it be providing the same services as it is doing today? IV. How do key stakeholder groups perceive the value and utility of CDCs? (students, faculty, univ. mngt, employers)? How and to what extent do the stakeholders engage with CDCs? 98 What factors facilitate or inhibit use of the CDCs? How effective have CDCs been in connecting graduates with employers and providing them will skills that employers demand? Communications & Marketing: 1. Does each CDC have a marketing and communications strategy? 2. Do they have an ongoing campaign? Who is responsible for it? Who are you targeting and how? 3. How do you measure the success of campaigns? University Senior Management Team: 4. Who in in the university senior management team do you communicate with? 5. What is the type of relationship you have with the university senior management? 6. Do you have a formal management committee or is it an informal relationship? 7. Do you have regular meetings with them? Do you have Joint Plans? 8. Who supervises the CDC work in the Uni. SMT? If yes what is the type of reporting? 9. Where is your position in the organizational structure of the Uni.? Do you think this the correct place for the CDC? If not… what do you think your correct place would be? 10. If USAID funding was suspended do you think the Uni. SMT would continue to fund the 99 CDC? 11. What are the possible means that can be used to strengthen the relationship of the CDC with the Uni. SMT? 12. Is the CDC Position strong enough in the Uni. to survive if there is a change in management? Employers: 13. Do you have a formal communications strategy to reach employers? 14. What have been the biggest challenges standing against having a strong employers network? 15. What are the strategies that proved to be most useful in strengthening the employers network? 16. If the CDC project ends, what effect would this have on the employers in finding job candidates? V. How effective have CDCs been in achieving gender balance and services to persons with special needs in provision of services during implementation? What factors have facilitated or hindered CDCs from achieving gender balance and provision of services for persons with special needs? 1. Gender: What is the ratio of gender participation in your CDC services?  Is this a reflection of the gender balance in the university?  How do you use communications to encourage both genders to participate in services?  Do you target special services to ladies/women? If so what are they?  What do you think are the biggest challenges facing young women students in using your services? 100  What actions have been most successful to provide services to young women students? 2. Disability: What is the ratio of Students with Special Needs (SWSN) participation in your CDC Services?  Is this a reflection of SWSN in the university?  How do you use communications to encourage SWSN to participate in services?  Do you target special services to SWSN? If so what are they?  What do you think are the biggest challenges facing SWSN in using your services?  What actions have been most successful to provide services to SWSN? Table 12: Ranking of factors influencing success of CDCs 1 2 3 4 5 6 Competence of CDC Director Geographic location of university Financial support of university administration Competence and certification of facilitators Reputation for matching students with job opportunities Serviceable office and technical equipment Strength of network with employers Financial sponsorship by employers Good relations with university senior leadership Number of employers in area Strength of communication campaign with students Quality of career development services For Each Factor, Check Importance Very Important <-----------------> Not Important What is the most important factor influencing the success of a CDC? Rank from 1 to 12 2014 Actual 2015 Projected 2014 Actual 2015 Projected 2014 Actual 2015 Projected 2014 Actual 2015 Projected 2014 Actual 2015 Projected Revenue Funds from USAID -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Funds from University -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Student Fees -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ External Sponsorships -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Other -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Totals * -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Expense Personnel costs -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Office and equipment lease -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Supplies -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Event logistics -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Other -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Totals * -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Deficit w/o USAID Funding -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Note: Totals must match Financial Flows for CDC Sustainability Ain Shams - Main Ain Shams - Eng Helwan South Valley Suez Canal Table 13: Financial flows for CDC sustainability AUC: ECDC SUMMARY PERFORMANCE INDICATOR TABLE Table 14: AUC-ECDC summary performance indicator table WL: LEEP SUMMARY PERFORMANCE INDICATOR TABLE Objective 1. Self-Sustaining CDCs: Establish in 3 target universities Outcome 1.1 CDCs embedded & sustained in partner universities HU SVU TOTAL HU SVU TOTAL HU SVU TOTAL 1.1.1 # of CDC personnel recruited 1.1.2** # of management committees meetings conducted 1.1.3 # of advisory boards meetings conducted 1.1.4 # of staff participating in training and development 1.1.5 # of staff participating in language skills & training 1.1.6 # of training sessions held under USAID funded LEEP project 1.1.7** # of persons completing a USG supported CDC, work force development programs of 16 hours or more Planned Base Years Year 1 Notes Target 3;Actual 3 Year 2 Target 3;Actual 2 Year 3 Target 2 Actual Output Planned Actual Planned Actual Table 15: WL-LEEP summary performance indicator table Objective 2. To improve job seeking among students’ and graduates’ skills and improve their ability to obtain employment Outcome 2.1** % of graduates from USG supported tertiary education reported themselves as employed (Target 30%) HU SVU TOTAL HU SVU TOTAL HU SVU TOTAL 2.1.2 % of student clients demonstrating improved skills after CDC training 2.1.3 % of students applying career centers content in their employment search 2.1.4 # of students with improved language skills Actual Planned Base Years Year 1 Notes Target and actual N/A Year 2 Target 20%; Actual 30.3% Year 3 Target 30% Actual Output Planned Actual Planned 2.1.1** # of individual persons gaining access to CDCs work development programs supported by USG Objective 3. Universities, Industry, Business and civil society are collaborating HU SVU TOTAL HU SVU TOTAL HU SVU TOTAL Outcome 3.1 # of partnerships developed with industry, business and non-profits Output 3.1.1 # employers using career centers services to search for employees or interns 3.1.2 # of career Fairs held at each partner university Outcome 3.2 # of secondary school students reached Base Years Notes Year 1 Year 2 Year 3 Planned Actual Planned Actual Planned Actual 108 INTERVIEW GUIDE FOR CDC MANAGEMENT AND STAFF PURPOSE To gather data from LEEP and ECDC CDC Management & team members concerning performance, lessons learned, and recommendations to support evaluation analysis.  PARTICIPATION  CDC Managers & Team members TIME  Approximately 2 Hours (may require several meetings) LOCATION  CDC locations DEMOGRAPHIC DATA Date: ___/___/_______ CDC Name: ________________________________________________ Location: ________________________________________________ City: ________________________________________________ Name of interviewee: ______________________________________ Job Title: ______________________________________ Years within the job: ______________________________________ 109 PRIMARY DISCUSSION QUESTIONS (use appropriate probing and follow-up questions) I. To what extent have the CDCs achieved their goals and objectives? How did the implementation approach contribute or hinder the achievement of project objectives and planned results? 1. What are the goals / targets for each CDC? (look for written strategy or operational documents) 2. Have the defined indicators with targets for each CDC been met? (ex. Hiring staff, certifying staff, premises, conducting training, building a communication network with employers, compiling a database…. ) 3. How does actual performance match the annual work plan? 4. What are the main strengths of the implementation over the last couple of years? What are some of the real success stories that you can share with us? 5. What are the biggest challenges you faced in the implementation of the CDCs over the last couple of years. How do you think this has affected the achievement of your goals and objectives? 1) Establishing MOU with the university, and strengthening relations with Uni.Mngmt.? 2) Obtaining facilities, and hiring staff, and setting up the office. 110 3) Delivering training and other services to students 4) Keeping training curricula up-to-date and trainers competent 5) Establishing a network with the employers. 6. Knowing what you know now, if you could change anything in the basic implementation process, what would it be? 7. If you would create a new CDC now with a new university, what would you do to increase the likelihood of success of the new CDC? 8. What do you think your main impact is on the students and the surrounding community that your are operating in? How do you track and measure matching students with jobs or opportunities? 111 9. Is the concept of having a CDC a successful one? Do you think it actually makes a difference? Do you think it should be in all universities? II. If and how did the selection of the geographic location for the CDCs affect their performance? How did the geographic locations of the CDCs affect results? Why? 1. How has the number of employers located in the governorate affected CDC results? 2. How does ―geography‖ compare to the other factors in the provided success list for building a successful CDC? (ask them to complete Table 1) 112 III. To what extent are the CDCs on track to transitioning the management of the centers to the universities? What factors helped or hindered the sustainability of the CDCs? What are the prospects for sustainable post-project completion? 1. How independent is your CDC currently? 2. Who does the CDC director report to now in the University? 3. What is the nature of the relationship currently, between the CDC and the Uni. Mngmnt? 4. Where does the CDC office lie in the structure of the university? 5. Is there a written transition plan (That shows how the transition will be implemented)? And who will be involved in the transition? And how is progress going to be measured? 6. Please detail the revenue sources / expenses using the provided template (Table 2) 113 7. Is there currently a budget line item in the Uni. budget for CDC financing? If yes how much per annum? Is the budget line item financed? If yes by whom? 8. What is the current competence level of the staff? How were they recruited? How long have they been employed at the CDC, what is their career path? Do you have succession planning for them? 9. How are the CDC staff compensated? Is the compensation aligned with the university salaries? Are they currently paid directly by the contractor or the univ? 10. What is the % and frequency of the staff turnover? And how is the transfer of knowledge managed? 11. Is there a policies and procedures manual that is established and followed by the staff members? 12. What is the role of the trainer/facilitators (CDFs) within the CDCs and how are they paid 114 ? 13. What are the factors that are most useful/helpful in ensuring the sustainability of the CDC? 14. What are the factors that challenged your efforts to ensure the sustainability of the CDC in the future? 15. 5 years from now, what do you think will happen to the CDC? Will it exist? Will it be bigger? Will it be doing the same thing it is doing today? IV. How do key stakeholder groups perceive the value and utility of CDCs? (students, faculty, univ. mngt, employers) How and to what extent do the stakeholders engage with CDCs? What factors facilitate or inhibit use of the CDCs? How effective have CDCs been in connecting graduates with employers and providing them will skills that employers demand? 1. Communications & Marketing: 1) Do you have marketing and communications strategy? 2) Do you have an ongoing campaign? Where is it? Who is responsible for it? Who are you targeting and how? 3) How do you measure the success of the campaign? 115 2. University Senior Management Team: 1) Who in in the university senior management team do you communicate with? 2) What is the type of relationship you have with the university senior management? 3) Do you have a formal management committee or is it an informal relationship? 4) Do you have regular meetings with them? Do you have Joint Plans? 5) Who supervises the CDC work in the Uni. SMT? If yes what is the type of reporting? 6) Where is you position in the organizational structure of the Uni.? Do you think this the correct place for the CDC? If not… what do you think your correct place would be? 7) Do you have Access to Student lists? 8) Do you have access to bulletin boards? Publicity tools? Access to premises, lecture halls? facilities? 9) If USAID funding was suspended do you think the Uni. SMT would continue to fund the CDC? 10) What are the possible means that can be used to strengthen the relationship of the CDC with the Uni. SMT? 11) Is the CDC Position strong enough in the Uni. to survive if there is a change in management? 3. Students: 10) What kind of communications strategy do you use to target/reach students? 11) Please list and rank you services in priorities with regards to students’ needs? 12) How do most students find out about you? And your services? 13) Who do you target from the student body? Year 1 4? Please list them in percentages 14) How do you measure engagement of students in your services? 15) are you satisfied with this level of engagement or is there more to do? 116 16) How is the level of demand comparing to the available resources, with regards to every type of service you offer? 17) Student placement services: what is the type of placement services do you offer currently? How do you measure the placement rate? 4. Faculty: What is the extent of faculty engagement currently? If yes 1) What form does the engagement have? 2) What kind of communications strategy do you use to target/reach faculty members? 3) How do the faculty members help project and market the CDC? 4) What % of students, who come to the CDC, are referred by faculty? 4.1) What is the optimal target % of students to come to the CDC referred by faculty members? 5) How does the faculty perceive the CDC program? Do they comment about what the CDC is doing? 6) How do you plan to increase faculty engagement in the future with the CDC? 5. Employers: 1) Do you have a formal communications strategy to reach employers? 2) Do you have a network of employers? If so, How many companies do you have in your network? 3) Who are your contacts at the employers offices? What are the positions that you deal with at the employers? (Job title) 4) Do you survey the needs of the job market identify the priority employment areas? How is 117 this used to deliver services? 5) What kind of agreements (if any) do you have with the employers? 6) What types of activities do the employers provide you with? Placement, internship programs? Guest speakers? Money? 7) Do you organize employment fairs? If yes how often? How many members of the network attend the fair? 8) How many jobs have been filled with students referred from the CDC? 9) How do you find out about employment opportunities in companies? How do you inform the students about the opportunities? 10) what have been the biggest challenges standing against having a strong employers network? 11) what are the strategies that proved to be most useful in strengthening the employers network? 12) If the CDC project ends, what effect would this have on the employers in finding job candidates? V. How effective have CDCs been in achieving gender balance in provision of services during implementation? What factors have facilitated or hindered CDCs from achieving gender balance in the provision of services? meet their sustainability plans? Gender: 4) What is the ratio of gender / SWSN participation in your CDC Services? 5) Is this a reflection of the gender /SWSN balance in the university? 118 6) How do you use communications to encourage both genders/ SWSN to participate in services? 7) Do you target special services to ladies/women// SWSN ? If so what are they? 8) What do you think are the biggest challenges facing young women / SWSN students in using your services? 9) What actions have been most successful to provide services to young women / SWSN students? Table 1 Table 2 2014 Actual 2015 Projected 2014 Actual 2015 Projected 2014 Actual 2015 Projected 2014 Actual 2015 Projected 2014 Actual 2015 Projected Revenue Funds from USAID -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Funds from University -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Student Fees -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ External Sponsorships -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Other -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Totals * -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Expense Personnel costs -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Office and equipment lease -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Supplies -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Event logistics -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Other -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Totals * -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Deficit w/o USAID Funding -$ -$ -$ -$ -$ -$ -$ -$ -$ -$ Note: Totals must match Financial Flows for CDC Sustainability Ain Shams - Main Ain Shams - Eng Helwan South Valley Suez Canal INTERVIEW GUIDE FOR SENIOR UNIVERSITY MANAGERS PURPOSE To gather data from senior university managers concerning performance and relationships, lessons learned, and recommendations to support evaluation analysis. 1. PARTICIPATION  Senior Management Team Members of the University TIME  Approximately 1 Hours (may require several meetings) LOCATION  University Headquarters DEMOGRAPHIC DATA Date: ___/___/_______ CDC Name: ________________________________________________ Location: ________________________________________________ City: ________________________________________________ Name of interviewee: ______________________________________ Job Title: ______________________________________ Years within the job: ______________________________________ PRIMARY DISCUSSION QUESTIONS (use appropriate probing and follow-up questions) IV. How do key stakeholder groups perceive the value and utility of CDCs? (students, faculty, univ. mngt, employers) How and to what extent do the stakeholders engage with CDCs? What factors facilitate or inhibit use of the CDCs? How effective have CDCs been in connecting graduates with employers and providing them will skills that employers demand? 1. Who in in CDC do you communicate with? 2. Do you as a university supervise the CDC work? If yes, who is the person/department responsible for the supervision? And what is the type and extend of supervision that you do? 3. What is the type of relationship you have with the CDC management? 4. Are you a member of a formal management committee? If yes, how is the committee working? Are there regular meetings? Do you approve strategies or policies/procedures? 5. Do you allow the CDC access to student lists and other data? 6. Do you allow the CDC access to bulletin boards, publicity tools, premises, lecture halls, facilities? 7. If USAID funding was suspended, would the university continue to fund the CDC? To what level of operation? 8. What are the possible means that can be used to strengthen the relationship of the CDC with the University senior management? 9. Is the CDC position in the university strong enough to survive if there is a change in management? 10. Where do you see the CDC in 5 or 10 years from now? INTERVIEW GUIDE FOR UNIVERSITY FACULTY PURPOSE To gather data from university faculty members concerning performance and relationships, lessons learned, and recommendations to support evaluation analysis.  PARTICIPATION  University Faculty Members TIME  Approximately 1 Hour (may require several meetings) LOCATION  University headquarters DEMOGRAPHIC DATA Date: ___/___/_______ CDC Name: ________________________________________________ Location: ________________________________________________ City: ________________________________________________ Name of interviewee: ______________________________________ Job Title: ______________________________________ Years within the job: ______________________________________ PRIMARY DISCUSSION QUESTIONS (use appropriate probing and follow-up questions) IV. How do key stakeholder groups perceive the value and utility of CDCs? (students, faculty, univ. mngt, employers) How and to what extent do the stakeholders engage with CDCs? What factors facilitate or inhibit use of the CDCs? How effective have CDCs been in connecting graduates with employers and providing them will skills that employers demand? Are you personally currently engaged with the CDC? If yes 1. What form does your engagement take? 2. What kind of communications does the CDC Management and staff use to target/reach you and other faculty members? 3. How do you and other faculty members help promote and market the CDC? 4. How many students ( % of ) have you referred to the CDC? And for what purposes? 5. How do you and other faculty members perceive the CDC program? 6. Do you have plans to increase your personal engagement in the future with the CDC? In what form? 7. What would be your recommendations to improve CDC value to both students and the community? INTERVIEW GUIDE FOR EMPLOYERS PURPOSE To gather data from employers concerning performance and relationships, lessons learned, and recommendations to support evaluation analysis. 2. PARTICIPATION  Employer HR staff engaged with the CDC TIME  Approximately 1 Hour LOCATION  Phone interview, if possible on site DEMOGRAPHIC DATA Date: ___/___/_______ CDC Name: ________________________________________________ Location: ________________________________________________ City: ________________________________________________ Name of interviewee: ______________________________________ Job Title: ______________________________________ Organization Name: ______________________________________ Years within the job: ______________________________________ PRIMARY DISCUSSION QUESTIONS (use appropriate probing and follow-up questions) IV. How do key stakeholder groups perceive the value and utility of CDCs? (students, faculty, univ. mngt, employers) How and to what extent do the stakeholders engage with CDCs? What factors facilitate or inhibit use of the CDCs? How effective have CDCs been in connecting graduates with employers and providing them will skills that employers demand? 1. How did the CDC reach out to you? In what areas have they offered or do you cooperate? 2. Are you a member of a network of employers working with the CDC? If yes, who are the other companies that are in your network? How does the network work? 3. Who from the CDC is your liaison? How often are you in contact? 4. Does the CDC management survey your staffing needs? Do they offer you candidates with the competencies that match the job openings you have? 5. Do you have any type of formal agreement with the CDC? If yes, what type of agreement is that? What are the key elements? 6. In what types of activities does the CDC ask you to participate? Do you participate in all of them or just some? Which are they? (Ex. placement, internship programs, Guest speakers, money sponsorship, other? 7. Do you attend any of their employment fairs? If yes how often? How many members of the network attend the fair? How many successful candidates have you recruited as a result of the fair? 8. Do you offer any type of funding to the CDC? If yes, how much and what form does this funding take? (i.e. event sponsorship, fees, publicity, etc)? 9. When you have vacancies do you inform the CDC? If yes, how? And what is their general response? 10. What have been the biggest challenges you have faced during your cooperation with the CDC? 11. What have been the biggest advantages of being engaged with the CDC and using their services? 12. If the CDC project ends, would this have an effect on your recruitment efforts? 13. What would you recommend for CDC do to improve their services to support your needs as an employer? Student Focus Group Moderator Guide PURPOSE To gather data from Students, as stakeholders, concerning performance, lessons learned, and recommendations to support the CDC evaluation analysis. PARTICIPATION  Students who used the CDC service. TIME  Approximately 1 Hour LOCATION  CDC __________________________________________________ Participants: see attached list Date: _________________ Attachments: Please attach all individual questionnaires to this form Moderator Script I. INTRODUCTION My name is _______________ and I’m the moderator today. The purpose of this discussion is to talk about the effectiveness of your universities career development center performance and value to you. I’ll be asking your opinions and your experiences. I am an independent research moderator and am only here to work with the evaluation team to help collect information. II. GROUND RULES (5 minutes) 1. This session will last about 1.5 hours. 2. This session is very informal and although I may be using a flipchart to capture some main points, my colleagues will be taking notes on key issues. 3. My colleagues (introduce them) are spending some weeks on this evaluation collecting information that will be helpful to the university CDC and also USAID who is currently funding the CDC. 4. There are no wrong answers in this discussion; we are looking for different points of view. I want to know what your opinions are, but they should be based on facts as you understand them. 5. Everyone needs to talk during our session but each person doesn’t have to answer each question. 6. Notes are only being taken in a general fashion. No one’s name is associated with any comments. 7. Please talk one at a time and in a clear voice, avoid side conversations. It is distracting to the group and I don’t want to miss any of your comments. 8. Exchange points of view with each other – you don’t need to address all answers to me. 9. We will also ask each of you to complete a short questionnaire at the end of the session. This includes basic demographic information as well as some questions for which we are interested in individual answers. It should take less than 10 minutes to complete. 10. Does anyone have any questions before we begin? 11. This is a non-smoking session. 12. Please turn off all cell phones. III. BACKGROUND (8 minutes) 13. Please take the first few minutes to get to know the person seated next to you. 14. I’m going to ask you to introduce him/her to the group. I’d like you to tell us something about that person such as their study focus, areas of specialty or job interest. IV. Focus Group Questions (60 minutes) Group members should be made up of random students who have used CDC services. Note: use appropriate probing and follow-up questions KEY AREAS OF INTEREST How do key stakeholder groups perceive the value and utility of CDCs? (students, faculty, univ. mngt, employers)? How and to what extent do the stakeholders engage with CDCs? What factors facilitate or inhibit use of the CDCs? How effective have CDCs been in connecting graduates with employers and providing them will skills that employers demand? 1) How did you know that the CDC exists? How did they try to reach you? 2) How did you know about the various services that they offer? 3) What services have you used in the past? And were they effective and why? 4) What do you think of the whole idea of having a CDC is it a useful idea? Would you recommend it to other colleagues? 5) What do you think of the employment services offered by the CDC are they successful at placing students in jobs? Do you think they have a strong network of employers? 6) What about the training services of the CDC ? Have you used them? And how satisfied are you with the training programs? And the facilitators? Has the training made a difference in you personally? If yes how? 7) What can the CDC management do more to improve their level of service? Are they doing everything you expect from a CDC, or are there more activities that they can do ? 8) If the CDC would stop operating would you mind it? Are you getting a value from it now? Would you like to see it continue? V. CLOSE & SUMMARY (5 minutes) Thank you for your comments and your time. This has been a valuable session. Information from a knowledgeable group such as this will be very helpful to inform our analysis going forward. Now I am going to distribute the short questionnaire. Please enter the information requested and answer the questions based on your personal knowledge and/or opinion. As I mentioned previously, no one’s name will be identified with any answers. We will aggregate the data to analyze for trends. Let me know if you have any questions as you work through the questionnaire. Thank you again. STUDENT QUESTIONNAIRE (to be filled by Individual Students) 1. BIO DATA: Name: _______________________________________________________________ Age: ______________ Year of Study/ Graduation Date: ___________________________ Faculty: _____________________________________________________________________________ University: ____________________________________________________________________________ Current Job (if applicable) ______________________________________________________________ 2. CDC SERVICES: Please tick and rank the service/s that you have personally used from the CDC: Where 1 means really satisfied with the service and 5 means totally dissatisfied with the service. Service Ranking 1 2 3 4 5 Help with writing your CV Internship programs during the summer Placement services with employers Attending an Employment fair Career advising services with a counselor Attending Technical Training programs Attending Soft skills training programs Attending Language classes 3. OPINIONS: What was the most satisfactory service you have received from the CDC and why? What was the least satisfactory service you have received from the CDC and why? What are your recommendations to make the CDC services better 1.__________________________________________________________________________________________ 2.__________________________________________________________________________________________ 3.__________________________________________________________________________________________ 4.__________________________________________________________________________________________ 5.__________________________________________________________________________________________ ANNEX I: EVALUATION STATEMENT OF WORK BACKGROUND Scholarships and Training for Egyptian Professionals (STEP) is a five-year program that supports Egypt’s economic and social development by improving organizational capacity with Egyptian institutions and developing a stronger workforce through the provision of scholarships, training, and technical assistance. STEP will create a cadre of highly trained female leaders through scholarships for 160 Egyptian women over the course of five years to obtain master of business administration or bachelor’s degrees in science, technology, engineering, and mathematics (STEM) fields from U.S. universities. STEP will also provide thousands of Egyptian professionals with long and short-term training in the fields of Democracy and Governance, Economic Growth, Education and Training, and Health. In Egypt, there is no formal system for career guidance and instruction in public universities, and initiatives to develop youth beyond the classroom curriculum are limited. In addition, there is no focus on university graduates’ soft skills and career development even though these are critical in today’s competitive job market. The current gap between university graduates’ skills and private sector needs heavily contributes to Egypt’s unemployment rates. Given these concerns, USAID designed an Annual Program Statement (APS) with the following objectives:  Establish sustainable career development centers in Egyptian public universities;  Provide career guidance to students and university graduates and build employability skills of students and alumni;  Develop participants’ skills in disciplines needed for the labor market; and  Bridge the gap between current university graduate skills and job market requirements. In 2012, USAID’s Office of Education supported the establishment and management of state-of￾the-art, university- based Career Development Centers (CDCs) through a three-year activity aimed at improving the long-term capacity of Egyptian universities to assist students and recent graduates in their transition from education to employment. In June 2012, USAID made awards to two implementers pursuant to the APS to establish Career Development Centers at Public Universities at underserved and geographically diverse regions in Egypt: (1) the American University in Cairo (AUC) for $ 2,728,700 and (2) World Learning (WL) for $ 1,678,768. Both completion dates were scheduled on May 31, 2015, but USAID extended the award to AUC to until September 30, 2016 and the award to WL until December 31, 2015. Since June 2012, AUC and WL have established 8 CDCs at 7 public universities. Of these CDCs, 5 currently remain open and 3 have closed: 1.Ain Shams (two CDCs) – managed by AUC 2.Helwan – managed by WL 3.South Valley – managed by WL 4.Suez Canal – managed by AUC 5.Fayoum (closed 9/2013) – managed by WL 6.Assiut (closed 4/2014) – managed by AUC 7.Port Said (closed 11/2014) – managed by WL The Employability and Career Development Centers implemented by the American University in Cairo (ECDC/AUC) have a total of 15 licensed career development facilitators certified through the National Career Development Association in the U.S. The CDCs deliver capacity building courses, ensure quality of services, and provide sustainability training and set strategies to generate income at Suez Canal and Ain Shams universities. To date, the ECDC/AUC have provided 1,770 students with career development services. At the present time, the ECDC/AUC partners with a network of NGOs and private sector entities, including ASPIRE Experiential Business Solutions, INJAZ, Vodafone, the International Labor Organization in Cairo, MIT company, and Mobinil, to deliver workshops on career development and employability skills. Currently, AUC is managing three CDCs, two at Ain Shams University and one at Suez Canal University. The CDCs implemented through World Learning’s Linking Education & Employment project (LEEP/WL) have 27 certified career counselors who deliver capacity building programs, ensure quality of services, and provide training to generate sustainable income at South Valley and Helwan universities. To date, the LEEP/WL has provided 1,285 students and graduates with career guidance and employability skills programs. LEEP/WL partners with a network of NGOs and private sector entities including Etisalat, Vodafone, Microsoft, Intel, The Marketer.Net, and AWTAD to conduct workshops on career development and employability skills programs. Currently WL is managing two CDCs, one at Helwan University and the other at South Valley University. The five CDCs that are currently operational are expected to become an integral part of each respective university campus by the completion date of the projects, and are expected to be sustained by corporate funding and fee-for- service programs. Of the three CDCs that closed, this was primarily due to issues with university management. In September 2013, following the June 30 events, the President of Fayoum University requested the cessation of the CDC’s activities, as he individually decided he did not want any USG supported￾programs at his university. CDC-Helwan was established as a substitute. CDC-Assiut and CDC￾Port Said closed in April 2014 November 2014, respectively, due to changes in university management. Assiut University’s new administration was not cooperative, which threatened project sustainability. Following negotiations with AUC, Assiut University requested the closure of the center in November 2013 and the center officially closed in April 2014. Due to changes in the university’s senior management, the CDC’s activities at Port Said University were suspended in November 2014. World Learning, in consultation with the USAID Procurement Office, decided to close down the CDC since there was no final decision on the status of the center by the university management. A. Development Hypothesis Egypt suffers from a serious youth unemployment problem in large part because many Egyptian youth lack the basic education and sufficient skill set to enter the work force. One especially challenged cohort of Egyptian youth is current university students and recent college graduates. As employer demands increase for well-trained employees, improved core employability skills become ever more important. These include information technology (IT) proficiency, career￾specific technical/vocational skills, entrepreneurial skills, and interpersonal/soft skills, as well as strong Arabic and English language ability. Two other important factors contributing to youth unemployment are student access to available employment opportunities through well￾structured databases and lack of job-search knowledge. The CDCs project’s development hypothesis is that the Career Development Centers will enhance a student’s employability, stability and prosperity through the provision of training courses, mentorship opportunities, workshops, internships and job opportunities, which will result in an educated workforce that responds to the labor market needs. The CDCs project’s critical assumptions include:  The bilateral relationship between the Government of Egypt (GOE) and the USG remains positive and fully collaborative;  The GOE presented in the public universities’ management cooperates fully with the implementation of the programs and activities; and  Funding/resources are available to implement planned activities. B. Goals, Target Beneficiaries, and Achievements to Date The goals, beneficiaries, and results of American University in Cairo-managed CDCs include: Goals  Establish four Employability and Career Development Centers that will serve as a ―one￾stop shop‖ for students and which will become an integral part of public universities in the following areas after the life-span of the project: Upper Egypt, Suez Canal, and Cairo;  Train and license 15 career development facilitators and ECDC management to provide required career education, connect with employees, and generate internship/employment opportunities to students in selected universities;  Provide an array of job search and soft/employability skills to a selected subset of students from each university, offering a technical track when needed via a network of partnerships; and  Ensure the functionality of a sustainability model that secures availing required employability/soft skills to university students after the life-span of the project. Target Beneficiaries  15 professionally licensed career development facilitators at two public universities: Suez Canal University and Ain Shams University;  Over 160,000 students from two different governorates, Cairo and Ismailia, will annually have access to professional career services (approximately 130,000 students from Ain Shams University and 30,000 students from Suez Canal University);  A subset of approximately 3,000 pre-final and final year students will additionally benefit from a tailored module that covers career management, employability skills, and a specialized technical training track;  15-20 professionally trained and coached volunteer trainers who are prepared to sustain the employability skills model and train others through a ―training of trainers‖ approach; and  NGOs, corporations and training providers partner in the project. Achievements to Date AUC’s achievements to date include:  1,770 students equipped with employability skills;  1,503 students equipped with motivational skills in career development;  1,085 students and alumni attended an Ain Shams University Employment Fair in May 2014, where students applied for 88 job vacancies and 3 internship positions and the CDC raised $12,857 for future operations;  859 students provided with information about corporate business development;  578 students provided with career advising sessions;  532 students provided with English placement testing;  132 students employed through vacancies and internships; and  125 students provided with improved English language skills. The goals, beneficiaries, and results of World Learning-managed CDCs include: Goals  Establish self-sustaining career centers at three universities—Fayoum, South Valley and Port Said Universities—in regions of high youth unemployment;  Develop students’ and graduates’ job seeking skills and strengthen their employability; and.  Foster collaboration between universities, industry, business and civil society. Target Beneficiaries  Students and recent graduates who gain job search and workplace skills through the career development center programs;  Current high school students who benefit from the program’s Youth Career Ambassador Program;  Employers benefit from the support in recruiting qualified, motivated candidates and from increased access to the university and improved communication with administration and faculty; and  The partner universities benefit from strengthened and deepened links with local and national industry and businesses identified through the work of the career development centers within the program. Achievements to Date WL’s achievements to date include:  2,584 participants attended courses in career counseling as well as in job search skills, business English, entrepreneurship, and presentation skills;  1,285 students and graduates attended Employability Skills programs;  639 students and graduates attended an Employment Fair at South Valley University;  397 students and graduates attended Career Advising sessions;  295 students and graduates attended English language training;  162 students and graduates attended motivational workshops in career development;  94 students attended Education For Employment (EFE) seminars to learn about corporate systems;  67 graduates (CDC alumni) employed through vacancies and internships; and  66 students and graduates at South Valley University took English placement testing. C. Project Modifications As a result of events in Egypt in 2013 and the U.S. foreign policy response, USAID was required to wind down nearly all education program implementation. Consequently, both the LEEP/WL and ECDC/AUC programs experienced major challenges during the second year of operations in 2013. The USAID Procurement Office (PROC) informed LEEP/WL and ECDC/AUC that they must proceed with an orderly wind-up of certain activities. In response, WL and AUC developed detailed wind-up plans and PROC modified the awards accordingly. In short, USAID agreed with WL and AUC to complete ongoing activities on the respective campuses by the end of January 2014. However, it was also agreed that both partners would continue providing necessary services to Egyptian students off the university campuses, in a community￾based context. This arrangement was planned to allow for some project activities and services, which are greatly needed by Egyptian youth to meet the job market needs, to continue while complying with U.S. foreign policy requirements. Six months later, in March 2014, policy and legislative changes cleared the way for USAID to request that both projects resume operations on university campuses. On March 4, 2014, before proceeding with moving the CDC activities off-campus, both projects were informed that the wind-up imposed on the CDC activities was relieved and activities could go back to their originally planned and approved program description, work plan and budget. This letter was followed with another modification removing the wind-up from the award. Overall, thewind￾up process affected the momentum, activities and morale of both projects, but both LEEP/WL and ECDC/AUC are currently operational and back on track. A summary of project modifications is as follows: ECDC/AUC  MOD #1 on 2/26/2014 to fully fund the agreement and include the approved wind- up plan;  MOD #2 on 3/19/2014 to remove the wind-up plan; and  MOD #3 on 3/1/2015 to extend the completion date till September 30, 2016 and change the Program Description to remove Assiut University CDC and decrease the Total Estimated Cost by $426,982 to $2,301,718. LEEP/WL  MOD #1 on 2/24/2014 to fully fund the agreement and include the approved wind￾up plan;  MOD #2 on 3/19/2014 to remove the wind-up plan;  MOD #3 on 7/30/2014 to change the Program Description to substitute Fayoum University CDC with Helwan University and re-align the budget; and  MOD #4 on 4/23/2015 to extend the completion date until December 31, 2015 and revise the Program Description to remove Port Said University CDC and re-align the budget. Evaluation Rationale A. Purpose The purpose of this mid-term performance evaluation is to review, analyze, and evaluate the effectiveness of USAID- funded CDCs in achieving program objectives and completing deliverables. An evaluation team lead by Kelly Novak Opportunities (KNO), and assisted by Career Development & Consultants Academy (CDC), will conduct the mid-term performance evaluation of the ECDC/AUC and LEEP/WL components of the CDCs project. Objectives of the evaluation include: 1.Identification of lessons learned with regard to project implementation and building relationships with counterparts in the Ministry of Higher Education and public universities around CDC establishment; 2.Assessment of the sustainability of the CDCs at the Egyptian public universities in which they have been established; and 3.Provide recommendations that can be used to inform the design and expansion of futureCDC￾related activities in Egypt’s public universities. B. Audience and Intended Uses The audience of the evaluation report will be USAID/Egypt Mission, specifically the Office of Education and Training and the Program Office. USAID/Egypt will review and share the expanded executive summary, final report, and evaluation recommendations with the Ministry of Higher Education (MOHE), other donors in Egypt working on the Employment Alliance, and the general public via the Development Education Clearinghouse (DEC). Evaluation Questions The evaluation will answer the following questions: 1. To what extent have the CDCs achieved their goals of this SOW? 2.If and how did the selection of the geographic location for the CDCs affect their performance? To what extent CDCs geographic location affect the performance of CDCs in achieving the results? Why? 3.To what extent are the LEEP/WL and ECDC/AUC projects on track to transitioning the management of the CDCs to the host universities? Which factors have contributed to or hindered the sustainability of the CDCs? What are the prospects for sustainability post￾project completion? 4.How do key stakeholder groups (e.g, students, faculty members, university management, employers) perceive the value and utility CDCs? How, and to what extent, do key stakeholder groups engage with the CDCs and which factors facilitate and inhibit utilization of the CDCs? How effective have the CDCs been in connecting graduates with employers and providing them with the skills that employers demand? 5.How effective have the CDCs been in achieving gender balance in the provision of CDC services (e.g, training, job placement services, etc.) during implementation? Which factors have facilitated or inhibited the CDCs from achieving gender balance in the provision of services? Based on evaluation findings, the evaluation team should identify actionable recommendations that will help USAID to better design and implement effective CDCs programming going forward. In particular, what are the lessons learned from implementation to date that USAID should take into consideration in future designs, and what are key considerations for replicating CDCs in other geographic locations? Evaluation Design and Methodology A. Evaluation Design This is a mid-term performance evaluation and is intended to focus on how the CDCs projects were implemented, what the projects have achieved, whether expected results were attained according to the projects’ design and underlying development hypothesis, and how activities were perceived and valued by stakeholders, and whether the projects are sustainable. The evaluation team will use a mix of quantitative and qualitative data collection and analysis methods to answer the questions of interest in this evaluation. The evaluation must follow the principles and guidelines for high quality evaluations outlined in the USAID Evaluation Policy (January 2011). B. Data Collection and Analysis The evaluation team must develop data collection tools that are consistent with the evaluation questions to ensure high quality analysis. KNO will lead a training with the CDC evaluation team members to facilitate the development of the data collection tools. Data collection methods must include, but are not limited to, a combination of the following:  Desk review of relevant documentation (e.g. quarterly reports, output from the project monitoring system, etc.);  Site visits to CDCs;  Key informants interviews (KIIs) (which should include, but are not limited to, KIIs with USAID/Egypt Education Team, including the Activity Manager; LEEP/WL staff; ECDC/AUC staff; CDCs’ Management; and university management ); and  Focus group discussions with beneficiaries (e.g., students, faculty members, employers) and other counterparts and stakeholders. All beneficiary data must disaggregated by gender and geographic location. The evaluation team is expected to conduct site visits to each of the five active CDCs, as well as to each of the closed ones if possible. Prior to beginning field work, KNO will submit to IIE the following for their review. IIE will then submit to the USAID Evaluation Program Manager for review, feedback and/or discussion, with sufficient time for USAID’s review, before they are used in the field. CDC will contribute to the development of these materials and submit to KNO for their review before KNO submits to IIE.  A list of interviewees and key stakeholders  An evaluation design matrix that details the proposed methodology for data collection and analysis  Data collection tools  A data analysis plan that encompasses the following:  How focus groups and key informant interviews will be transcribed and analyzed  What procedures will be used to analyze qualitative and quantitative data from key informant and other stakeholder interviews  How the evaluation will weigh and integrate qualitative data from these sources with quantitative data from performance indicators and project performance monitoring records to reach conclusions about the effectiveness and efficiency of both projects C. Relevant Documentation USAID and the implementing partners will provide the evaluation team with soft copies of a package of briefing materials, including:  Relevant parts of Projects agreement and amendments  Relevant parts of Projects’ quarterly and annual reports  Budget information as relevant and appropriate to be shared.  Projects’ Performance Monitoring Plan  Other Documents, as available, appropriate to be shared and requested by the evaluation team. The evaluation team (KNO and CDC) should complete the document review prior to KNO’s arrival in Egypt. The evaluation team may also request and review additional resources to the extent necessary to perform its work. D. Data Quality Standards The evaluation team must ensure that the data they collect clearly and adequately represents answers to the evaluation questions, is sufficiently precise to present a fair picture of performance, and is at an appropriate level of details. E. Data Limitations USAID requires that issues potentially affecting the quality (including validity, integrity, and reliability) of evaluation data be discussed and documented in the evaluation planning stage and assessed on an ongoing basis during evaluation implementation, including during data collection and analysis. All limitations and identified measures to address or overcome limitations should be discussed by the evaluation team and USAID in the implementation phase and detailed in the final report. The final evaluation report should be clear and transparent about any notable limitations and if and how they may affect the evaluation’s findings, conclusions, and recommendations. The evaluation team should be aware that student and alumni data may not be made available by university management. Evaluation Products A. Deliverables Evaluation Team Planning Meeting: A kickoff meeting between IIE, USAID, and the evaluation team must be held in Egypt at the onset of the evaluation. This meeting will allow USAID/Egypt to discuss the purpose, expectations, and work plan of the assignment with the evaluation team. In addition, USAID/Egypt and the evaluation team must:  Finalize team members’ roles and responsibilities;  Review and finalize evaluation questions;  Review and finalize the evaluation timeline;  Present and discuss data collection methods, instruments and tools, analysis, and guidelines; and  Review and clarify any logistical and administrative procedures for the assignment. USAID will provide the evaluation team with a stakeholder analysis that includes an initial list of interviewees, from which the evaluation team can work to create a more comprehensive list. KNO will take the lead, with CDC input, in the construction of a preliminary interview schedule that includes different stakeholders. KNO will submit to IIE, who will then share with USAID the updated lists of interviewees and schedule as meetings/interviews take place and informants are added to/deleted from the schedule. The finalized list must be sent to IIE no later than six working days after submission of the preliminary interviewees’ schedule. Evaluation De-Brief. After conducting the field work, the evaluation team must present its preliminary findings to USAID/Egypt. The debriefing must include a discussion of findings, including project achievements and challenges, as well as preliminary recommendations for the future activity designs and implementation. The team must consider any USAID/Egypt comments, as appropriate, when revising the draft evaluation report. Draft Evaluation Report. Prior to departing Egypt, the evaluation team must submit a draft report of the findings and recommendations to IIE. IIE will review and then submit to the USAID Evaluation Manager. The written report must clearly describe findings, conclusions and recommendations for future programming. USAID will provide written comments on the draft report within seven working days of receiving the document. Final Evaluation Report. The final evaluation report must be submitted by IIE to USAID within seven working days of receiving USAID’s comments. In order to meet this deadline KNO will solicit feedback from CDC and incorporate any edits into the final evaluation report for IIE’s review within five working days of receiving USAID’s comments to allow IIE time to review before final submission. The final report must not exceed 30 pages in length (not including appendices, lists of contacts, etc.). The format must include an executive summary, table of contents, glossary, methodology, findings, and conclusions. The report must be submitted initially in English, electronically, and later, an Arabic translation must be submitted. KNO, with CDC’s input, must submit a flash drive to IIE that includes the survey instruments, interviews, and data sets for IIE’s submission to the USAID Evaluation Program Manager with submission of the final English report. KNO, with CDC’s input, must submit to IIE an expanded executive summary to accompany the final report. The expanded executive summary must include a background summary on the evaluation purpose and methodology, and an overview of the main data points, limitations, findings, and conclusions. The expanded executive summary must be easy to read for wide distribution to local audiences. The expanded executive summary must be written in English and Arabic in hard copy and electronically. KNO will be responsible for the English version and CDC will be responsible for the Arabic translation USAID/Egypt intends to disseminate the evaluation report and expanded executive summary within USAID and to stakeholders. B. Evaluation Report Requirements The format for the evaluation report is as follows: 1.Executive Summary 2.Table of Contents 3.Introduction 4.Background 5.Methodology 6.Limitations 7.Findings/Conclusions/Recommendations 8.References 9.Annexes The final report will be reviewed using the Checklist for Assessing USAID Evaluation Reports (http://www.usaid.gov/results-and-data/information-resources/program-evaluations). The final evaluation report must conform to the Criteria to Ensure the Quality of the Evaluation Report found in Appendix I of the USAID Evaluation Policy. The Evaluation Program Manager will determine if the criteria are met. This evaluation will not be considered ―final‖ until the Evaluation Program Manager has confirmed, in writing, that the report has met all of the quality criteria. General evaluation report guidelines include:  The evaluation report must represent a thoughtful, well-researched and well organized effort to objectively evaluate what worked in the project, what did not, and why;  The report must include the evaluation Scope of Work as an annex. All modifications, whether in technical requirements, evaluation questions, evaluation team composition, methodology, budget, or timeline must be agreed upon in writing by the AOR;  Evaluation methodology must be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides must be included in an Annex in the final report;  Limitations to the evaluation must be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparison groups, etc.) and what is being done to mitigate the threats to validity;  Evaluation findings must be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings must be specific, concise and supported by strong quantitative or qualitative evidence;  Sources of information must be properly identified and listed in an annex;  Recommendations must be supported by a specific set of findings; and  Recommendations must be action-oriented – organized according to whether recommendations are short-term or long-term, practical, and specific, with defined responsibility for the action. The final evaluation report in English must be submitted by KNO to IIE in electronic format (Microsoft Word) no later than 2 working days after the receipt of the acceptance decision. The entire report must be no longer than 30 pages, single-spaced in Times New Roman font, size 12 type fonts. All data and materials are to be surrendered to and will remain the property of USAID. Evaluation Management A. Logistics IIE will provide overall direction to the evaluation team, identify key documents, and assist in facilitating a work plan. USAID, through IIE, will assist in arranging meetings with key stakeholders identified by USAID prior to the initiation of field-work.CDC is responsible for arranging other meetings as identified during the course of this evaluation and advising KNO and IIE prior to each of those meetings. IIE will update USAID/Egypt regarding all meetings. CDC is responsible for arranging transportation as needed for site visits in and around Cairo and other governorates. USAID, through IIE, can assist with hotel arrangements if necessary but CDC will be responsible for arranging the work/office space, computers, internet access, printing, and photocopying for the entire evaluation team while in Egypt. CDC also responsible for procuring and paying for translation services for interviews, reports and any other evaluation related tasks. Evaluation team members will be required to make their own lodging and travel payments. USAID personnel, through IIE, will be made available to the team for consultations regarding sources and technical issues, before and during the evaluation process. Additional considerations:  WL and AUC will work with the evaluation team to facilitate access to the CDCs and university management; and  U.S. citizens (or foreign non- Egyptian nationals) will require GOE security clearance to enter public universities. In addition, to be considered is the sensitivity on the part of the university management for non- Egyptian nationals conducting interviews with university management and beneficiaries. ANNEX J: DISCLOSURE OF ANY CONFLICTS OF INTEREST U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523