September 2015 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by: Robert Miller, Joanna Khater, Wassim Katerji, Najwa Andraos, Kamil Wanna and Harvey Herr of Social Impact. USAID LEBANON UNIVERSITY SCHOLARSHIP PROGRAM MID-TERM EVALUATION Cover Photo: A tranquil university setting at one of the three USP implementing partners. Photo by Robert Miller. Acknowledgements: The authors wish to express their appreciation to the USP and university staff at AUB, LAU and Haigazian University who were generous with their time, answered a myriad of questions, facilitated appointments, and supplied data. We also are indebted to the many students who participated enthusiastically in our focus group discussions, interviews and Survey Monkey exercise. A special thanks to the staff of Information International who conducted fieldwork in remote areas, and to Stephanie Deep, who served as reporter for our focus group discussions. Finally, we wish to express our appreciation to staff of USAID Lebanon for providing this opportunity to carry out the evaluation and for providing excellent guidance and assistance in carrying out the task. USAID LEBANON UNIVERSITY SCHOLARSHIP PROGRAM MID-TERM EVALUATION September 2015 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. TABLE OF CONTENTS Acronyms ............................................................................................................................................................ i Executive Summary ..........................................................................................................................................ii Background and Purpose ............................................................................................................................ii Evaluation Questions ...................................................................................................................................ii Findings and Conclusions ..........................................................................................................................iii Evaluation Purpose & Questions ................................................................................................................... 1 Evaluation Purpose ....................................................................................................................................... 1 Evaluation Questions ................................................................................................................................... 1 Project Background .......................................................................................................................................... 3 Summary of Task .......................................................................................................................................... 4 Evaluation Methods & Limitations ................................................................................................................. 5 Findings, Conclusions, and Recommendations........................................................................................... 8 Annexes ........................................................................................................................................................... 33 Annex 1: The Evaluation Statement of Work ................................................................................... 34 Annex 1A: List And Description of USP Programs Included in Scope of the Evaluation ..... 34 Annex 2: Evaluation Matrix ..................................................................................................................... 35 Annex 3: Data Collection Tools ........................................................................................................... 39 Annex 4: Distribution of USP Enrolled Students by Districts......................................................... 71 Annex 4A: Distribution of USP Enrolled Students by Districts and Sex .................................. 72 Annex 4B: Distribution of USP Enrolled Students by Districts and by USP............................ 74 Annex 5: Maps ........................................................................................................................................... 75 Annex 6: Distribution of Applications by USP and Geographical Area ........................................ 80 Annex 7: Distribution of Public High Schools by District ................................................................ 82 Annex 8: Distribution of Students by Major and Sex ....................................................................... 90 Annex 9: MEPI and USP Budget Breakdown ....................................................................................... 98 University Scholarship Program Formative Evaluation – September 2015 i ACRONYMS AO Assistance Objective AOR Agreement Officer Representative AUB American University of Beirut AUC American University of Cairo CAAP Collegiate Assessment of Academic Proficiency CAS Central Administration of Statistics CDCS Country Development Cooperative Strategy COS College Outcome Survey CV Curriculum Vitae DO Development Objective FGD Focus Group Discussion GIS Geographic Information System GPA Grade Point Average GPS Geographic Positioning System HU Haigazian University IM Implementing Mechanism IP Implementing Partner IR Intermediate Result KII Key Informant Interview LAU Lebanese American University M&E Monitoring and Evaluation MENA Middle East and North Africa MEPI Middle East Partnership Initiative OIRA Office of Institutional Research and Assessment PMSPL Performance Management and Strategy Program for Lebanon PPR Annual Performance Plan and Report UNRWA United Nations Relief and Works Agency USP University Scholarship Program USAID United States Agency for International Development University Scholarship Program Formative Evaluation – September 2015 ii EXECUTIVE SUMMARY BACKGROUND AND PURPOSE USAID Lebanon initiated the University Scholarship Program (USP) in 2010 at Lebanese American University (LAU) and Haigazian University (HU) and later added American University of Beirut (AUB). The aim of this $63.9 million program is to provide very bright Lebanese public school students who have high financial need with the opportunity to attain a quality higher education in order to maximize their potential to support Lebanon’s democratic and economic development. A total of 496 students were admitted to programs at the three elite universities. The USP scholarship provided each student with full tuition and fees, a book allowance, a living stipend, a computer and all other related education expenses. In 2015, the USAID Agreement Officer’s representative requested a mid-term evaluation of USP (USP I scheduled to end in 2017 to USP V which scheduled to end in 2019) in order to obtain concise, actionable recommendations to inform program implementation for the remainder of the agreements. A list and a description of USP programs that are included in the scope of the evaluation is available in Annex 1.a. EVALUATION QUESTIONS In meetings with the Mission and university staff, six broad questions were provided to the evaluation team to guide the program evaluation, along with numerous sub-questions. These questions covered: national outreach and program coverage; the scholars’ overall performance; programmatic performance; sustainability; appropriateness of the USAID mechanism in use; and a comparison of USP to a similar Middle East Partnership Initiative (MEPI) program. In addition to answering the overall guiding questions which form the main body of this report, the evaluation team reported additional information uncovered in the evaluation that they thought would provide additional understanding of the program’s functioning. METHODS AND LIMITATIONS The evaluation methodology included a desk review of reports, key informant interviews (KIIs) with 28 key informants, five focus group discussions (FGDs) with students, anonymous online survey (215 responses), a review of university records, interviews with five (out of 16) students who were dropped from the program, a field survey with 45 individuals in five remote districts that had fewer students benefiting from the program, and a detailed USAID questionnaire received from 89 newly graduated students conducted in 2014. With this extensive data University Scholarship Program Formative Evaluation – September 2015 iii collected, the evaluation team used a combination of qualitative and quantitative analyses in a process of triangulation to arrive at the most empirically defensible answers to the evaluation questions. Certain limitations of the data and analysis include possible lack of representativeness of respondents, some possible biases among KIIs, and some disparity in the results from different methodologies. FINDINGS AND CONCLUSIONS National Coverage  Outreach: The three universities succeeded in achieving broad national outreach. Students were recruited from all of Lebanon’s 26 districts, with select underrepresentation in some programs from specific remote areas (Annex 4.B – Distribution of USP enrolled Students by districts and by USP). Visits by university staff to the districts and high school principals were instrumental in spreading knowledge about USP to the successful applicants. USP student enrollment is gender balanced at AUB, female biased at LAU and was even further female biased at HU. Overall, USP students have a similar gender distribution as that of public high school students.  Meeting Criteria: The findings support the conclusion that the applicants who are accepted into USP meet the criteria of high academic achievement and financial need for scholarship awards.  Enrolled Cohorts Diverse and Nationally Distributed: The student cohort is nationally geographically distributed with some underrepresentation of some remote districts, and contains a majority (60%) of female students.  Impediments to Recruitment in Any Areas: Impediments appear to include: lack of community knowledge about USP in the communities that have supplied few students for the program, and the small schools with lower number of students to select from in some remote areas. Fieldwork was not done in areas regularly supplying students, which would have been a valuable comparison. A similar study was deemed unnecessary because the team expected such information to be available in FGDs, Survey Monkey data, and the USAID Survey of Graduates. Scholars Overall Performance:  Performance: Program managers report USP students are among their best performers academically. USP students also perceive themselves as doing well in the program. Graduates achieved high Grade Point Average (GPAs around 3.3 over 4), and the majority of computer science students on the Dean’s Honor List at LAU are USP students. LAU reports 56% of USP students have GPAs above 3.2, compared to 29% of non-USP students. The Office of Institutional Research and Assessment (OIRA) at the AUB ran an analysis over a random sample of 15 students comparing their academic performance and ranking in a total of 30 exams (2 exams/student). The results showed that USP students scored higher than the average of the non-USP counterparts in 27/30 University Scholarship Program Formative Evaluation – September 2015 iv exams (90%). In 20/30 exams, the USP students were ranked among the first five of their classes (class size average ± stdev = 30±17 students).  Integration/Abuse, Disrespect: Students report some initial issues with their integration into the university community, but these tended to decrease in time. However, concerns were expressed by AUB students that USP dormitory students are being moved to another dorm where they will be segregated. FGD participants at one of the universities reported several instances of what they interpreted as disrespectful treatment by university appointed program managers, and indeed reported an undercurrent of substantial tension between USP program managers and USP students. Much more positive relationships between program managers and students were reported by FGD participants at the other two universities who were reported by USP students to be more encouraging and responsive. Another important issue was raised by a dropped student who was advised to seek counseling and testing at the counseling center. The student reported that the counseling center was not set up to maintain auditory privacy and, without his/her permission, shared his/her confidential information with faculty, staff and the student’s parents.  Active Student Leadership: The USP program encouraged, supported and mandated an educational program that emphasized community and campus involvement, and leadership training and experience. USP achieved active student leadership among many USP students. It is uncertain whether graduates will assume active community and professional leadership positions in the future although they are optimistic on this subject.  Preparation for Entry into the Labor Market: Almost all (94%) of the respondents to the online survey are planning to enter the labor market at some time in the future. Workshops were organized by the universities to prepare the students to be successful at jobs interviews (for example, AUB held a Blom Bank Curriculum Vitae (CV) Writing and Interview Skills lecture for USP II students. Haigazian held an Interview Boot Camp Training and workshops on career skills; provided one-to-one support to students on writing a CV, cover letter and interview; and provided support in job searches). The Career Guidance Office at LAU has held a number of activities and workshops (e.g. the Communication Skills Workshop, which was aimed at engaging USP scholars in exploring their career paths and enhancing their communications skills to improve both personal and professional aspects of their lives).  The vast majority (90%) of students reported wanting to obtain graduate education either immediately after graduation or after they gain a few years of employment. However, a minority of graduates (22%) have found employment quickly (within three months). Programmatic Performance:  Achievement of Planned Outputs and Outcomes: Appropriate outputs and outcomes are poorly defined in program documents supplied by USAID; however, the graduation rate is on track with 3.5% (17/496) dropout rate by the date August 2015. Higher level outcomes will require longer term evaluation. University Scholarship Program Formative Evaluation – September 2015 v  Achievement of Value for Money: While “value for investment” is difficult to assess, especially in the short-term, and where outcomes are not well articulated, all stakeholders interviewed in KIIs enthusiastically expressed the view that USP provides value for USAID’s investment by bringing transformational change in the life of individuals, families and communities. Moreover, USAID has been supporting the Lebanese public school system with more than $100M of funds invested since 2006 in the public primary education sector. While offering those same public high school students scholarships that allow them to obtain a quality higher education and thus maximizing their potential to support Lebanon’s democratic and economic development, the USP program complements the efforts and leverages the funds invested in the primary education Sustainability  Prospects: USP is likely not sustainable as a whole because the universities afford to financially support the program, but many of its program features including workshops, community involvement, leadership training, capstone projects, etc. are recognized by university staff as adding value for all students, and could strengthen the overall universities’ educational programs if the staff is motivated and finds resources to support these activities. Additionally, the alumnae program may add sustainable components into the future by keeping lines of communication and influence open with USP graduates. Mechanism  Cost and Management Burden: The present system of separate contracts for each USP places a substantial increase on the management burden on USAID and the implementing partners by requiring separate reports from the universities for each USP. Comparison to MEPI  Potentially Useful MEPI Components for USP: MEPI differs from USP in that it sends students abroad for short-term training and brings regional students to Lebanon. However two components of MEPI may prove useful to USP: encouraging continuing contact with civic engagement in the home area; and providing an intensive group training for all MEPI scholars. USP has a lower cost per student than MEPI. A higher percentage of the USP budget is allocated for educational and student support costs. RECOMMENDATIONS: 1. Expand Outreach: Continue and expand outreach visits to all public high schools with a special focus on areas which have been under-represented. Expand these visits to provide an orientation about the program to the community. Consider encouraging student and graduate involvement in the outreach program. University Scholarship Program Formative Evaluation – September 2015 vi 2. Ensure that the Management Style of all USP University Implementers Reflect USP Program Values: USAID staff should ensure that USP Implementing Partners address and adhere to compliance concerns in order to effectively respond to all student issues of concern including respect, abuse, rigidity, confidentiality and other issues. University managers should ensure that, to the extent possible, their own management styles reflect USP values including: democratic participation in decision-making; negotiated settlements of outstanding issues; absolute confidentiality during counseling regarding student problems, issues and personality tests; and integration of USP students into all aspects of University life, including common spaces such as dormitories. 3. Prevent Dropouts: USAID should review its policy on the possibility of granting a semester’s break to students in good academic standing who experience a serious psychological or physical health problem rather than dropping them from the program. 4. Provide More Help in Selection of Majors: Recognizing the importance of the selection of a major to the entire university educational experience, universities should provide more information and assistance regarding the implications of choosing different majors in securing an employment, and strive to empower students to make an informed decision among a substantial number of choices. USP staff and faculty should take further steps to encourage students to avoid gender stereotyping in the selection of their majors, while still allowing students considerable freedom to choose. University Scholarship Program Formative Evaluation – September 2015 1 EVALUATION PURPOSE & QUESTIONS EVALUATION PURPOSE In 2015, 5 years after the start of the USP in 2010, and the approaching conclusion of USP I and II in 2016, the continuation of USP III, IV and V, and the commencement of USP 6, the Agreement Officer’s Representative (AOR), under the office of Education/Democracy, Rights and Governance, requested a performance evaluation for the University Scholarship Program. This evaluation covered all implementing mechanisms (USP I-V), and activities implemented by the three implementing partners. This evaluation is intended to be used by USAID/Lebanon and others at the discretion of the Mission, such as the USP directors and officers at the three universities. The results from the evaluation will be available for use by USAID/Lebanon during its annual Portfolio Review. EVALUATION QUESTIONS Meetings with the Mission and other stakeholders were held prior to initiating the evaluation in order to determine priorities and to add clarity to the evaluation questions based on the evaluation’s intended use of the evaluation. Four broad categories were selected to guide the evaluation process. After reviewing these with the USAID AOR, two additional ones were added. 1. National Coverage  To what extent is USP achieving transparent national outreach?  To what extent do applicants meet the criteria for scholarship awards?  To what extent is the enrolled cohort diverse and nationally distributed?  What are the impediments (real or perceived ones) affecting the recruitment of scholars from any particular areas, if any? 2. Scholars Overall Performance  How well are students performing in their host universities?  To what extent are students integrated in the university with the regular student body or are there challenges to such integration?  To what extent are scholars mistreated, disrespected or abused by management for being scholarship recipients?  On average, are USP scholars performing academically like their peers (non USP scholars) in the same class or not? University Scholarship Program Formative Evaluation – September 2015 2  To what extent did USP enable scholars to become active student leadership and potential leading change agents (looking at preparing individuals as change agents rather than actual transformation)?  To what extent is USP preparing scholars for entry to the labor market? 3. Programmatic Performance  To what extent have all three implementing partners achieved the planned outputs and outcomes as clearly stated in their award documents?  To what extent can it be said that USP provides value for this USAID investment? 4. Sustainability  What are the prospects for the sustainability of USP activity results?  Which results show the most prospect of being sustained and why?  Provide recommendations on how the activity design could be enhanced to improve the sustainability of results, and any additional programming or support in the upcoming years that would improve USP results sustainability? 5. Mechanism  What is the mechanism currently used for USP and determine if this is appropriate with respect to cost and management burden. 6. Comparison to MEPI  Determine if USP and MEPI are comparable and if features of the MEPI program would be useful to USP. University Scholarship Program Formative Evaluation – September 2015 3 PROJECT BACKGROUND Many underprivileged families in Lebanon are forced to make economic sacrifices to put their children through school (Text Box 1). Through the USP program, USAID provides merit-based scholarships for bright Lebanese public school students of high academic standing and high financial need. These scholarships are intended to facilitate access to quality higher education in order to maximize their potential to support Lebanon’s democratic and economic development. The University Scholarship Program (USP) started in 2010 with the Lebanese American University (LAU) and Haigazian University (HU) with a portfolio size estimated at $13.5 million resulting from a US Congressional earmark for the program. By 2014, the USP had implemented a total of 10 programs across three universities, (adding American University of Beirut (AUB) in 2011, for a total portfolio size of $63.8 million. The program complements other USAID assistance in the Lebanese public school system, which focus on repairs and equipment, community involvement, and teacher training. USP directly supports USAID/Lebanon’s Assistance Objective (AO) #3: Student Achievement Improved, as it expands access to quality higher education for students across Lebanon. The program further supports Intermediate Result 3.3: Increase people’s access to education programs with USAID assistance. Under the Country Development and Cooperation Strategy (CDCS), the USP program fits under IR2.1 Increased Private Sector Competitiveness contributing to Development Objective 2 (DO2) Inclusive Economic Growth Enhanced. The stated aim is to be achieved by selecting approximately 50 to 90 promising Lebanese public school students per cohort (USP I, II, III, IV, and V) per university. Best efforts are to be used to ensure equal representation between males and females and wide geographical representation from all 26 Lebanese Districts. BRIEF INTRODUCTORY OVERVIEW OF THE EDUCATIONAL SYSTEM IN LEBANON According to the Center for Educational Research and Development (2014) 31% of the students in Lebanon attend Public schools. The rest are distributed among the private (52.8%), semi-private (13.1%) and UNRWA schools (3.2%). Private schools are popular because they are perceived to be of higher quality. If there is a quality difference, it is more apparent in primary than secondary schools. The semi￾private schools are sponsored by religious or political parties. Often the same teachers teach in both public and private systems, where public school teachers are allowed to earn extra income from tutoring. According to CAS (2012), 74% of teachers at public schools are women; women teachers predominate in primary schools. There is a wide range to the fees charged by private and semi-private schools. Semi-private religious schools may charge only about $1,000/year, while some private schools charge in the range of $8,000. Some families with substantial dedication to the education of their children may devote a large portion of their income to education despite being in deprived circumstances, or even go into debt to support their children’s education in private schools. Thus there is an overlapping, relatively economically disadvantaged population served by public, private and semi-private schools. Sources: 1- CRDP Statistical Bulletin, 2013-2014. 2- Central Administration of Statistics, 2012. Statistics in Focus, Issue 3 - Education in Lebanon. University Scholarship Program Formative Evaluation – September 2015 4 Under USP, students from economically disadvantaged families graduating from public high schools are awarded full scholarships to attend the three elite, private universities in Beirut – AUB, LAU, or HU. Scholarships include all relevant expenses such as tuition, fees, housing or transportation allowance, books, computers, and stipends. Continuation of awards are contingent upon the students remaining in good academic standing and maintaining a satisfactory grade point average (GPA). Students are required to study full time and to complete their undergraduate degree within the academic time frame of their respective programs, including one year of preparatory English language training, if needed. Other components of the program include focuses on leadership training, multiple special workshops, and community service projects, tutors, and mentors. These intensive learning experiences were designed to enhance learning and prepare students to serve as change agents in their communities and professions. Audience and intended uses This evaluation is intended to be used by USAID/Lebanon and other stakeholders at the discretion of the Mission, such as the USP directors and officers at the three universities. The results from the evaluation will be available for use by USAID/Lebanon during its annual Portfolio Review. A version of the Evaluation Report will become publicly available on the Development Experience Clearinghouse (DEC) in accordance of ADS 540. Summary of Task This report provides a mid-term evaluation of USP which include process questions that are usually conducted as part of a formative evaluation1 , and a number of recommendations which the evaluation team hopes will improve and strengthen USP implementation in the future. 1 USAID Performance Monitoring & Evaluation Tips - Introduction to Evaluation at USAID, NUMBER 11 2ND EDITION University Scholarship Program Formative Evaluation – September 2015 5 EVALUATION METHODS & LIMITATIONS The methodology for this evaluation drew upon quantitative and qualitative methods, including a desk review, KIIs, FGDs, a survey with current students and alumni, and program data. Methods used for data collection, including constraints and gaps, are briefly described below. A. Desk review of Reports: Each university participating in the USP programs produces a large number of quarterly and annual reports. USAID provided the evaluation team with a complete package of thirty-one reports which were reviewed by the evaluation team. Unfortunately, the time allotted to this activity was insufficient to allow an in-depth study of every report. B. Interviews of Key University Staff (and USAID staff): The team met and interviewed 28 key leaders, such as USP managers, recruitment and financial aid, and grants and contract staff at each of the Universities (see interview schedule in Annex 3A). Discussions focused on how the programs were proceeding, procedures, the integration of USP scholars, and suggestions for strengthening USP in the future. C. Focus Group Discussions with Students at Each of the Universities: The Evaluation Team conducted five focus group discussions (two at AUB and LAU and one at HU) with 40 presently enrolled and graduated students. (Discussion guide attached as Annex 3B.) Participants in two of the focus groups were invited by the Universities; members in three others were selected on a targeted semi-random basis by the evaluation team. FGD responses were compared with one another as well as with other data sources described below. D. Anonymous Survey of all USP Scholars and Graduates: The team used an online survey platform to distribute an anonymous questionnaire to gain insights into program issues from a large group of students and graduates (see questions in Annex 3C). All students (approximately 500) in all programs in the three Universities were invited to respond to the questionnaire, which was distributed by university staff to ensure that email invitations to participants did not end up in spam. Individuals were given one week to respond. The response rate by university in a one week period was 60.90% (95/156) for AUB, 58.46% (38/65) for HU and 29.82% (82/275) for LAU. The overall response rate was 43.35% (215/496). The response University Scholarship Program Formative Evaluation – September 2015 6 rate may have been reduced by the relatively short time provided for responding. Additionally, response may have been reduced because it was distributed at the beginning of the academic year, a busy time for students. However, 215 responses are adequate for conclusions (Annex 3.C.2 – Justification of the size of the Survey Monkey Respondent Population). E. Review of Student Records and Individual Interviews with Students who Dropped Out of the Program: Although the drop-out rate from USP as of 2015 was only 3% (16/496), each drop out represents an unfortunate waste of opportunity and funds. Therefore the team reviewed university records and conducted three in-person interviews with LAU dropped students, as well as two telephone interviews with AUB students who had dropped out. These interviews focused on the factors that led the individuals to drop out, the types of university actions taken to assist them, and their suggestions for other steps the universities could have taken that might have assisted them further or prevented that outcome (interview questions in Annex 3D). F. Field Visits to Five Geographic Areas where Enrolled Students were Underrepresented: The evaluation team used an interview protocol designed to explore whether parents of school age students and other community members in five underrepresented districts were aware of the program and whether high school principals recommended/assisted students to apply (Annex 3E). The methodology for selecting respondents was to start with the high school principal, request a recommendation for a parent to interview, and then use snowball sampling to expand the sample by adding recommendations from respondents. The ultimate sample included 45 respondents. (A similar study of areas that successfully recruited scholars was deemed unnecessary because the team expected such information to be available in FGDs, Survey Monkey data, and the USAID Survey of Graduates (below)). G. Results of a USAID Detailed On-line Survey Completed by 89 Graduates of USP: The USAID AOR conducted a broad survey of USP graduates of May of 2015 that contained detailed questions about the graduates, their experiences and views of USP, and supplied the questions and results to the evaluation team (Annex 3F). This provided an opportunity to further triangulate data. For example survey data made it possible to attach an estimate of the percentage of graduates who reportedly agreed with issues raised in focus group discussions. In addition it was possible to University Scholarship Program Formative Evaluation – September 2015 7 compare results of the responses from the online survey questionnaire with responses of graduates to similar questions. H. Geographic Information System (GIS) Data on the Location, Type and Size of Enrollment of All Public Secondary Schools in Lebanon: GIS data on the location, type and size of enrollment of all public secondary schools was used to produce a variety of maps which helped to pinpoint regions from which USP students were underrepresented, and possible background factors, such as populations and size of school enrollments influencing the number of applications/selections. While the answers to USAID’s evaluation questions form the main section of this report, the evaluation team expanded discussions to cover a number of issues that were not specifically requested but which arose from various sources of data during the course of the evaluation. Team members believed that including these issues in the report would complement USAID’s questions and add to overall understanding of the functioning of USP. These issues were discussed with the Mission and included: community reaction to USP; factors influencing management/faculty-student relations; dropouts from USP; selection of majors; students’ future plans; changes in applications over time; “rippling” effects of program implementation in the schools and communities; and the cross-cutting issue of gender. Data Analysis The evaluation team prepared a data source matrix (also called an evaluation matrix) to clarify how the data was collected (Annex 2). It includes the evaluation questions, and the evaluation tools, data source(s) and analysis plan for each question. This matrix ensured that all the data sources were taken into consideration and the team was able to use triangulation to answer questions, when feasible and appropriate. Triangulation is an approach to help improve the validity of survey findings. The data collected from the different sources was reviewed for reliability and validity. Differentiation between the findings, interpretations, judgments, and recommendations, and the specific sources of evidence supporting these, is included in this evaluation report. LIMITATIONS Certain limitations of the data and analysis include possible lack of representativeness of respondents, some possible biases among KIIs, and some disparity in the results from different methodologies. Another limitation is due to the unavailability of comparable academic performance data for USP and non-USP students. Thus the data was compared to the overall student cohort which include USP and Non-USP students. University Scholarship Program Formative Evaluation – September 2015 8 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS NATIONAL COVERAGE To what extent is USP achieving transparent national outreach? Findings: USP recruited students from all 26 districts (cazas) of Lebanon (see Annex 5 Distribution of Students by Sex, District and USP and Maps). However, students were not selected from all districts in all USPs. Six districts tended to be substantially underrepresented (Bcharreh, Hermel, Minieh Dinnieh, Jezzine, Marjayoun and Rachaya) (see Annex 4 – Distribution of USP Enrolled Students by District). Although the program aimed to select one male and one female from each of the 26 districts (3.84% from each), less than 2% of the USP students are from each of these districts. These districts are remote, with fewer towns (Annex 6, Map 2. USP Beneficiaries per Town), a smaller number of high schools and a lower secondary school student population (Annex 7-List of Public High Schools by Geographic Area). All USPs received a total of 5,947 applications (see graph 1 Annex 6). The largest number of applicants was received from Mt. Lebanon (1,757) and North Lebanon (956). As can be clearly noted in Graph 1 (in Annex 6), a spike in applications for USP II and USP V occurred in all governorates. Although reasons for this are unclear and deserve further investigation, we hypothesize that the spike in USP II was due both to the expansion of participating partners, and the communities learning that the program was real. The spike in USP V might be due to the fact that the eligible applicant pool included all graduating high school students, because in this particular year teachers refused to grade the students for the official exams and the ministry granted ALL the students the certificate of success, which increased the number of eligible applicants. Each of the universities reported that they sent recruitment staff to visit approximately all of the 261 high schools in Lebanon (see complete list Annex 7) (as per Program Reports and Interviews with university recruiters) and implemented advertising and media campaigns with posters, newspaper and radio announcements and downloadable application forms. Nearly all respondents in the online survey and the USAID Survey reported that the recruitment visit from the university, and or information from their high school principal or other school staff most likely generated by the recruiter visits, was the source of their knowledge about the scholarship. Newspaper advertisements, radio, TV were cited by less than 2% of the online survey respondents as sources of information about USP. Field researchers indicated that relatively few (27% of community members contacted, including parents of secondary school students), were aware of USP in districts which under-supplied USP students, and most of the individuals who knew of the program were school principals or other officials. Two of the 12 University Scholarship Program Formative Evaluation – September 2015 9 community members who knew of the USP program incorrectly cited the source of funding as NGOs and the universities. Students from earlier USP groups reported in FGDs that, although they had encouragement from their parents to apply, other community members were less supportive. Community members, especially older ones, believed that such scholarship benefits either did not exist or could not be offered without expecting something specific in return, like intelligence information. These negative and incorrect views reported by FGD participants were believed to have dissipated as students periodically visited their homes and communities and discussed their program experiences. Indeed, some students reported in FGDs that their USP success has had “rippling effects” in their schools and communities. FGD participants reported that the program has encouraged other students to study harder to become eligible for the USP, fostered increased competition between students, encouraged siblings and friends of accepted students, and even led to some transfers from private to public schools in their communities so as to be eligible for USP in the future. USP serves a majority of females. Because it provides them with the possibility to live in the city of Beirut, to access private universities and to have the freedom to spend their stipends, it is reported by female FGD participants to substantially increase their feelings of equality and empowerment, which is concurrent with USAID policy.2 Female FGD participants indicated that the stipend was especially important to them as this was their first life experience of having their own money which they could spend however they wished. As one female FGD participant said, “after gaining the empowerment I have from this program, I am concerned that getting married might be like being in jail.” As can be noted in Table 1.a, the overall gender distribution of USP students (60% female) is very similar to the gender distribution in public high schools across the years 2011-2014 (61% female). Table 1.b shows that the selection of students at HU was female biased (72% female vs. 28% male) even when compared to the gender distribution in public high schools. However, the gender distribution of applicants (69% female vs. 31% male) does not show an important discrepancy with the selected students. 2 USAID, 2012. USAID policy on gender equality and female empowerment. March 2012. University Scholarship Program Formative Evaluation – September 2015 10 Table 1.a Distribution of students by Sex in USP and All Secondary Public School Students3 USP Students Public Schools – All Secondary Cycle Students 2010-2014 (2011-2012) (2012-2013) (2013-2014) Sex N % N % N % N % Male 199 40.1 21,738 38.7 21,391 38.4 21,006 38.3 Female 297 59.9 34,496 61.3 34,247 61.6 33,805 61.7 Total 496 100.0 56,234 100.0 55,638 100.0 54,811 100.0 Table 1.b Distribution of the USP students by Sex and by USP Sex Total Male Female HU/USP1 N 18 47 65 % 27.7% 72.3% 100.0% LAU/USP1 N 21 38 59 % 35.6% 64.4% 100.0% LAU/USP2 N 23 41 64 % 35.9% 64.1% 100.0% LAU/USP3 N 36 57 93 % 38.7% 61.3% 100.0% LAU/USP5 N 25 34 59 % 42.4% 57.6% 100.0% AUB/USP2 N 24 26 50 % 48.0% 52.0% 100.0% AUB/USP4 N 26 27 53 % 49.1% 50.9% 100.0% AUB/USP5 N 26 27 53 % 49.1% 50.9% 100.0% Total N 199 297 496 % 40.1% 59.9% 100.0% 3 official data found on the CAS and on the MEHE website http://www.cas.gov.lb/images/PDFs/SIF/CAS_Education_In_Lebanon_SIF3.pdf http://www.crdp.org/sites/default/files/%D8%AA%D9%88%D8%B2%D8%B9%20%D8%A7%D9%84%D8%AA%D9%8 4%D8%A7%D9%85%D9%8A%D8%B0%20%D8%B9%D9%84%D9%89%20%D8%A7%D9%84%D9%85%20%D8%A7%2 0%D8%B1%D8%AD%D9%84%20%D8%A7%D9%84%D8%AA%D8%B9%D9%84%D9%8A%D9%85%D9%8A%D8%A9% 20%D9%88%D8%A7%D9%84%D8%AC%D9%86%D8%B3%20%D8%A8%D8%AD%D8%B3%D8%A8%20%D9%82%D8 %B7%D8%A7%D8%B9%D8%A7%D8%AA%20%D8%A7%D9%84%D8%AA%D8%B9%D9%84%D9%8A%D9%85.pdf University Scholarship Program Formative Evaluation – September 2015 11 Moreover, the distribution of students by sex at the Lebanese University, according to the Central Administration of Statistics (CAS – 2012), is 65% vs. 35% for female and male, respectively. Thus, the distribution of USP students by sex also is similar to the distribution of the students of the public Lebanese University. However, a possible weakness of USP is some possible gender stereotyping in the selection of majors at all universities where the female students are self-selected into traditionally female￾oriented occupations and male students into male-oriented occupations. While differences exist at each of the universities and each USP, the Tables on distribution of students by major and sex (Annex 8) indicate that, overall, USP females are overrepresented, compared to USP males, in business administration, communication sciences, medical laboratory sciences/biology, nursing, and education related majors. Females are underrepresented in computer science and some engineering majors. Data on majors by gender also show some reductions of traditional gender stereotyping in the selection of majors, to the benefit of both genders. According to universities’ staff, efforts were made to orient the male and female students to select majors that were traditionally regarded as strictly for their opposite sex. For example, areas which traditionally were regarded strictly as male specializations received significant female enrollment. These specialties are civil engineering (46% Female), chemical engineering (50% F), electrical engineering (42% F), construction engineering (50% F) and petroleum studies (43% F). Similarly, areas which traditionally were regarded strictly female specializations received some male enrollment for example nursing (27% Male) and nutrition (11% M). Other “more gender neutral”4 majors, like psychology, human resources , advertising, graphic design and architecture, received enrollment from both genders, though with higher female enrollment. USP II incorporated a rule that students must be committed full time to the program. This was expected to mean no outside employment and no marriage for students, both of which were expected to decrease student commitment to the program. The non-marriage rule applied equally to males and females and therefore, while questionable from a rights perspective, was not discriminatory towards female students.5 In fact, this rule was not enforced among the many students who did marry during their program but yet pursued the full terms of their scholarship. Conclusion: The universities succeeded in achieving broad national outreach; students were recruited from all of Lebanon’s 26 districts, with some underrepresentation in some USP programs from some remote areas with less population and where the USP is not well known in the community. Visits by university staff and high school principals were instrumental in spreading knowledge about USP to the successful applicants. Other sources of information about USP were very rarely cited by the online survey respondents. Staff visits to public schools 4 It is the Lebanese context it is known to the evaluation team and supported by data on real distribution of majors from CAS (CAS, 2010. Gender Statistics in Lebanon, Current situation and Future needs – EDUCATION, p.22) 5 UNFPA, 2013. Protecting girls rights, http://www.unfpa.org/resources/protecting-girls-rights University Scholarship Program Formative Evaluation – September 2015 12 were a successful approach to gaining applicants from most districts. USP recruitment is gender balanced at AUB (51%Female/49%Male), female biased at LAU (60%Female/40% Male) and was further female biased at HU (72%Female/28%Male). While female students report feeling empowered by the program, some gender stereotyping appears in a review of student majors along with some evolutionary progress away from traditional gender stereotyping. The need for further information among some students and communities may require more intensive outreach where there is less history of the USP participation. Fuller geographic participation may be possible through a creative expansion of the outreach program. Recommendations: 1. Universities should continue to utilize visits to all public high schools. 2. In districts that have less history of supplying students to USP, the university outreach orientation meetings should consider an expansion of their role and objectives. Expanded objectives might include: to orient the community to the program, create greater understanding of the purpose of USP, answer community questions, and alleviate possible parental/community concerns. 3. Newspaper, radio and TV advertisements used were not found to be a productive source of information for admitted students and probably can be dispensed with in the future. 4. Universities should explore the possibility of having enthusiastic students/graduates from the district, or a neighboring district, accompany program staff in the outreach program, especially in underrepresented communities. Some community members and high school students may find USP students to be especially credible and enthusiastic sources of program information. The Universities should also explore opportunities for encouraging students to be engaged in planning and outreach of USP and challenged to design a program to recruit successful applicants from underrepresented districts as part of their community service and leadership programs. 5. The recruitment of gender balanced cohorts may require some further study of factors influencing recruitment/selection of females and males, especially by LAU, and especially in certain identified districts. The issue of gender balance should be considered in the wide context of Lebanon’s society, social expectations for males and females in rural areas, gender of teachers, and many other social factors. This broad social perspective should be incorporated in future USP planning. 6. During the orientation sessions, USP staff and faculty should take further steps to encourage students to avoid gender stereotyping in the selection of their majors, while still allowing students considerable freedom to choose. University Scholarship Program Formative Evaluation – September 2015 13 7. As smart phones are very popular with youth in Lebanon (and most everywhere), Universities should explore opportunities for outreach through social media. For example a community project could be a blog on USP experiences delivered to friends in their home communities. To what extent do applicants meet the criteria for scholarship awards? Findings: USP staff at all of the partner universities indicated in interviews that they have clear and transparent criteria for awarding scholarships and elaborate procedures for ensuring that awardees meet all scholarship criteria – high academic standing at a public high school reflected in school transcripts, attendance at the public school for the previous three years, significant financial need as reported by the family, and documented by descriptions of income, employer reports, bank statements, and at HU, confirmation of financial information from two knowledgeable references. Each of the universities described their elaborate procedures, including computer programs, and intensive selection committees who review all applicants, and ensure, to the extent possible, that applicants meet all criteria. The committee include a representative of USAID/Lebanon. Criteria and application forms differ between universities; which might give more chances for some applicant to be accepted at one university rather than at the other University staff recognized that when parents report that they are “self-employed” it is impossible to be 100% certain that the financial information supplied is correct. However, through cross checking the size of home, renting vs. ownership, type and year of automobile, number of family members, bank records, monthly living expenditures, etc., and following-up to investigate any discrepancies which appear in the review, they believe that all applicants meet the strict USP requirements. The team reviewed financial aid and application forms, discussed all procedures in use with university and USP staff, and reviewed 5 to 15 randomly selected complete admission files at each university. The team determined that the procedures described by staff were followed in all these cases, although some files were missing tax information which was not required in many cases. (The team did not investigate the question of whether all applicants met the criteria but only whether accepted applicants met them.) Conclusion: The findings support the conclusion that the applicants who are accepted into USP meet the criteria for scholarship awards. Recommendations: 1. Since the selected USP students meet the criteria for scholarship awards, the evaluation team does not recommend any changes in the present procedures for checking program criteria which appears to work well. 2. USP is working well recruiting from public schools. However, in view of the overlapping economic conditions of some public and private school families, USAID should consider the University Scholarship Program Formative Evaluation – September 2015 14 possibility of extending eligibility for some USP scholarships to students from private schools, if their families meet the same financial need criteria used for public school applicants. 3. If USAID wishes to investigate this question further, it should engage the services of a professional auditing firm. To what extent is the enrolled cohort diverse and nationally distributed? Findings: As has been discussed above, the enrolled cohort is, to a large extent, geographically distributed representing all 26 of Lebanon’s districts, but with fewer students from six of the 26 districts (Annex 4 – Distribution of USP students by Districts). While program planners indicated that they thought it would be necessary to take steps to ensure that females were recruited, and established a minimum target of 40% female enrollment, in fact the majority of students and graduates (60% for USP as a whole) are female. However, this overall figure for the USP program masks significant difference in gender biases in recruitment/acceptance at the three universities as indicated in Table 2, below. Table 2. Percentage of Female Students in USP by University Percentage of Female USP Students (Female / total number of USP students) AUB LAU HU Total 51% (80/156) 62% (170/275) 72% (47/65) 60% (297/496) As can be noted in Table 2, 496 USP students have been enrolled at the three Universities. LAU enrolled the largest number (275), and HU the fewest (65). AUB enrolled an almost equal number of males (49%) and females (51%). LAU enrolled about 50% more females than males (reflecting the public high school gender distribution of students from which applicants were drawn – Refer to Table 1.a.), and HU enrolled more than twice as many females as males. The USP male-female demographic is reflective of the broader public school demographic but it is against the terms of the CA that require a gender balanced (50Male/50Female) selection of students. As shown in Graph 2 below, considering gender and district together, we note in the Distribution by Sex and District (Annex 4.a), that in seven of the 26 districts, more than 70% of students enrolled are female. These districts are: Aley, Baabda, Baalbeck, Chouf, Jezzine, Koura and Tripoli. These districts are quite diverse in terms of location, urbanity-rurality, confession, etc. and are not easily characterized. University Scholarship Program Formative Evaluation – September 2015 15 Graph 2: Distribution of USP students by district and by sex Several possible factors may help explain why female students predominate in USP. Females represent the majority of public high school students (see Annex 5, and Table 1.a). In FGDs and KII, some students indicated that males are expected to work rather than study in their home areas and some families prefer to invest scarce economic resources in son’s rather than daughter’s educations at private schools. Conclusions: The student cohort is nationally geographically distributed with some underrepresentation of smaller more remote districts and contains a majority of female students (as does the population of pubic high school students). 17 15 10 9 13 16 7 6 7 11 12 9 6 7 8 9 6 5 5 3 5 5 1 3 2 2 22 23 23 22 18 14 22 21 19 14 8 11 10 9 8 6 7 7 7 7 4 3 5 3 3 1 0 5 10 15 20 25 30 35 40 Sour Saida Aley Baabda Metn Beirut Tripoli Baalbeck Chouf West Bekaa Akkar Nabatieh Hasbaiya Kesrwan Zahle Jbeil Zgharta Batroun Bent Jbeil Koura Rachaiya Marjayoun Jezzine Minieh-Dannieh Bcharreh Hermel Male Female University Scholarship Program Formative Evaluation – September 2015 16 Recommendations: 1. Given that the distribution of the public school students is not gender balanced (with female predominance) similarly to the USP overall population, USAID should decide how important or valuable it is to recruit a more equally gender balanced group of USP students and take steps in accord with that decision. Should USAID decide to continue having a gender balanced recruitment, factors underlying the female biases in recruitment at LAU and HU should be further explored and USAID should determine if there are ethical procedures that could strengthen applications and acceptances of males so as to have a more gender balanced program at LAU. What are the impediments (real or perceived ones) affecting the recruitment of scholars from any particular areas, if any? Findings: As has been noted, a possible impediment to the recruitment of students from smaller, more isolated and rural areas appears to be lack of knowledge in the community about USP. In five communities underrepresented in USP, only 12 of 45 individuals contacted knew of the program, and most of those who knew of USP were school officials. As has been noted, these areas also have fewer schools and a smaller enrollment. Earlier in the life of USP, negative rumors and disbelief that the program was real were factors. These issues reportedly have dissipated as USP students have brought back information to their community and demonstrated that the program is real. Conclusion: Impediments appears to be lack of community knowledge about USP in communities which have supplied few students, as well as fewer schools with fewer students enrolled. Recommendations: 1. Expand visits from university staff to orient community members as well as public school students to USP. 2. Explore opportunities for involving students and graduates in USP outreach, both in planning and in making visits. SCHOLARS’ OVERALL PERFORMANCE How well are students performing in their host universities? On average, are USP scholars performing academically like their peers (non USP scholars) in the same class or not? Findings: University staff report that the USP students are among their best performing students academically, and that the special features of the USP, including extensive training in University Scholarship Program Formative Evaluation – September 2015 17 leadership, community service, and entrepreneurship provides them somewhat of an edge in overall capacity in comparison to regular, non-USP students. In FGDs, some students and many graduates also expressed this perspective, although at AUB, several students were quite critical of the leadership training provided in a lecture format claiming: “It was a waste of time.” FGD participants expressed the view that, despite their more humble social backgrounds, they excel in their academic programs. A computer science student in an AUB FGD reported that: “90% of computer science students on the Dean’s Honor List are USP students.” The university confirmed this statement. FGD participants reported that their instructors tend to expect more from them, in part because they are so well selected and because of their scholarship status. A long history of educational research in the U.S. has shown that faculty expectations are highly related to student success.6 Among graduates of the program, the mean self-reported GPA on the USAID survey was approximately 3.3/4. 7 According to AUB Fact book 2013-2014, the overall GPA of all AUB spring 2012-2013 registered students is 3.15/4 (78.79/100). Moreover, the USP Program Director at AUB stated that 75% of the USP students who graduated did it with distinguish, and 2 of the biology graduate students received a merit scholarship from AUB to pursue Medical studies. Also, in the scope of this evaluation, the Office of Institutional Research and Assessment (OIRA) at the AUB ran an analysis over a random sample of 15 USP students comparing their academic performance and ranking in a total of 30 exams (2 exams/student). The results showed that USP students scored higher than the overall average of their classes in 27 exams (90%). In over 20/30 exams, the USP students were ranked among the first (3 exams), second (4 exams), third (6 exams), fourth (3 exams) and fifth (4 exams) of their classes. The Collegiate Assessment of Academic Proficiency (CAAP) and the College Outcome Survey (COS)8 are two other tests managed by the OIRA at AUB. They are performed on a yearly basis for all USP students and to a sample of AUB students. Analysis of the results of both tests conducted in 2014 indicated that the USP students are out performing and showing better results and improvement than non-USP students. Finally, LAU reported that 56% of USP students have a GPA above 3.2 over 4 compared to 29% of non-USP students. Students were asked to rate the influence of various factors (considered as sources of stress for students) affecting their performance in the USP program on a five point scale, with “1 = didn’t affect at all and 5 = highly affected. Seventeen percent (36/215) of the online survey respondents provided the highest or second highest rating (rating of 4 or 5 on a five point scale of how issues are affecting their USP performance) to most of the issues raised – being away 6 Kuh, George, Jillian Kinzie, and John Schuh, 2005. Success in college: Conditions that matter,” Jossey Bas. 7 GPAs on the 100 point scale were informally estimated on the 4.0 scale to allow use of all data. 8 CAAP is a set of standardized tests developed to measure student’s achievement levels in six areas: Writing Skills, Reading, Mathematics, Critical Thinking, Science and Writing Essay. The CAAP test also helps the counselor evaluate the individual skills levels of the students, and determine how best enable them to achieve academic success. The Collage Outcome Survey (COS) is an assessment in non-academic areas that focuses on students’ growth in intellectual, social and interpersonal skills. University Scholarship Program Formative Evaluation – September 2015 18 from home, adjusting to another culture, insufficient money to live, and having to make new friends. However, the factors cited by the largest proportion of students as affecting their performance were: English language (26%; 56/215), and integration into the program and the academic rigor both cited by 23% (50/215) of the online survey respondents. In USP programs, drop-out rates for all causes are reported by staff to be quite low – 3.23% (16 out of 496) compared to approximately 10-13% among regular non-USP students.9 For instance, the AUB Fact Book 2013-2014 reported that the graduation rate (percentage of the initial cohort who completed the program in six years or fewer) was 87% of the 2007 student cohort at AUB. USP students achieve academic success, in spite of the wide variety of requirements in the program that take a substantial portion of the students’ time. In FGDs, students reported that these extensive requirements lead to annoying scheduling problems. Conclusions: USP students perform well in their host universities, outperforming their non-USP peers in the same classes. The most important sources of stress as reported by the online survey respondents were English language, integration into the program, and the academic rigor of the university. Recommendations: 1. USAID and the universities should critically review the extent of extracurricular activities required of USP students, and consider increasing choice, especially for activities labeled “volunteer activities.” 2. USP management staff should provide greater attention to scheduling conflicts, and possibly excessive demands on student time to ensure that students can remain academically strong and to improve relations with USP students. To what extent are students integrated in the university with the regular student body or are there challenges to such integration? Findings: Students at AUB reported in FGDs that there were some initial challenges to their integration. About a quarter of respondents (24%; 52/215) identified integration with the other/regular/non-USP scholars as a problem in the online survey. In FGDs, students reported that a nasty article in the LAU Tribune accused the USP students of being “uncivilized poor villagers.” This was also reported in the USAID questionnaire where an additional student 9 Estimated based on reported university graduation rates. University Scholarship Program Formative Evaluation – September 2015 19 reported: “At first we were treated like foreigners or aliens and not ordinary students.” After a few initial negative encounters, FGD participants indicated that their integration improved, but results indicated it was still a source of concern. AUB students living in dormitories reported that they now feel their successful integration is being challenged by a program management decision. USP students presently in dorms integrated with regular students reported in FGDs that they are all being moved to the same dorm where they will be segregated from regular non-USP students. FGD participants reported: “The University says they need their dorm for workshops but we do not feel this is advisable and are concerned about our pending segregation.” When asked “to which extent your integration with the program affected/are affecting your performance in the USP program” almost half of the respondents (45%, 97/215) rated their integration with the program in the top two response categories on a five point fixed response scale (1= highly affected and 5 didn’t affect at all), suggesting that this issue deserves continuing attention. Data showed that female respondents were slightly more affected by the integration to the program than male respondents (47.62% F vs. 41.57% M rated their integration in the top two response categories). A small number of random interviews with non-USP students on the AUB campus indicated that the regular students are familiar with the USP program and students, but not under its formal USP title. Rather, when prompted, they report knowing it as the “program for public school students.” Indeed, the USP students are the only public school students in these elite universities, except for a very small minority, i.e. 0.5% of the students at LAU are from public schools (as reported by the LAU Vice President). An FGD participant reported that: “there is frequent reference by both program managers and faculty to our public school backgrounds,” and that “Program Administrators have accused us of not being used to being disciplined because of our public school backgrounds.” Conclusions: The continuing reference to public school background of the USP students by management and faculty does not strengthen such integration, and may promote stigmatization. Re-segregation of USP students into a single USP dormitory at AUB is not desired by USP students and will hinder integration with non-USP students. Recommendations: 1. AUB program management and faculty should be sensitized to the possible ongoing reference to the USP public school backgrounds and the fact that students have varying levels of concerns about their integration into the regular student population. University Scholarship Program Formative Evaluation – September 2015 20 2. AUB program managers should look for a different solution to their dormitory issues rather than burden the USP students with segregation from other students in their living situation. 3. More university effort to deal with student stress should be encouraged. Accepted students and their parents should undergo a more complete orientation session at the onset of the USP enrollment rather than the university tour offered to parents. This should be followed by regular coaching/ confidential counseling sessions for the students, intensified in the event of unusual stress or trauma during the first months of their USP and exam periods when stress is likely to be more intense. To what extent are scholars mistreated, disrespected or abused by management for being scholarship recipients? A small number of students (14%, 28/201) provided an affirmative answer to the question “Did you ever witness or were yourself subject to mistreatment, disrespect or abuse based on your status as a USP scholar?” in the online survey. However, FGD participants at one of the universities reported several instances of what they interpreted as disrespectful treatment, and indeed reported an undercurrent of substantial tension between program managers and students. They also reported: “We feel that the program administrators struggle to control us.” One student mentioned in the FGD that a USP colleague who was late in responding to an email from management had the delivery of her/his stipend threatened. Students expressed the view that: “In response to most every suggestion to program management to please plan to take our schedule into account when planning meetings, or allowing us to plan for another activity during vacation or weekends, we are told by program administrators: ‘You should be grateful and always obey.’” Other complaints from students are that the expectations for them change frequently from their initial contracts signed as part of their USP entrance without explanation or prior notice. They also state that the help and guidance promised to them in solving problems in their university programs frequently is not provided, and management is unreasonably rigid. Such tensions may reflect wider social class struggles between program managers and students, and the student activism that is common among students on elite university campuses both in Lebanon and around the world.10 The wide ranging complaints with some intense emotional undertones were found in FGDs at one of the three USP university implementers whereas FGD participants from the two other universities highlighted the extent of the support they feel from program management. As one student, with agreement from others, stated: “We love the 10 Osseiran, Nazih, 2015. “AUB’s president wants to teach students to fly,” The Daily Star, September 8. University Scholarship Program Formative Evaluation – September 2015 21 faculty (and program managers) and they love us. When we have problems, they care for us. They are our second family.” Despite the fact that the USAID survey of graduates, online survey and the FGD at one of the USP university implementers provided somewhat different results on this question of mistreatment, disrespect and abuse, the team believes that answers to questions requiring critical responses may have greater validity when drawn out in the more intimate personal setting of the FGD, after rapport is developed by the FGD leader, and details of the disrespect are made public in the group. It would appear to be a minority of students (around 14%) who feel strongly enough to indicate in a questionnaire (online survey) that they feel they were treated unfairly or abused, but more students agree that there are problems in this area when discussed in FGDs. Interviews with dropouts mirror these same sentiments about USP management at one of the universities . While university staff cite “personal and/or academic problems” as reasons for dropping out, the dropped students, in each case, tell a much more complex story. One dropped student referred to not being able to take his/her desired major, not being allowed to take the courses he/she wanted, and mentioned that USP management introduced a mandatory leadership training program the day before an important exam when students were supposed to be studying. The dropped student’s conclusion was: “USP management at the university was not flexible. The transition from high school to university life was difficult; USP management made it more difficult with their lack of flexibility.” This dropped student had a high GPA and reported that s/he is now first in his class at a Lebanese university. S/he summarized the experience: “I felt like I was in prison and now I am out.” Two other students, who had health problems in the year they were dropped, cited the same inflexibility. One reported: “instead of trying to assist me they wanted to push me out because of personal prejudice. Others, with lower grades and who did not experience health problems, were not dropped.” Three of the five dropped students interviewed experienced significant health/psychological problems in the year they were dropped from USP. Of some significance, these three are all continuing in their university programs with financial aid. They believed they would have been able to continue in USP if some break were allowed due to health reasons. One asked: “If someone is in an automobile accident and is hospitalized for three months, should they be dropped from USP or given a semester’s break?” This student who sought a semester break for health reasons reported that s/he was told by USP management: “The rules of the program regarding breaks are so strong that even the U.S. Ambassador cannot help you.” An important issue was raised by another dropped student. S/he was advised to seek counseling and testing at the counseling center. The student reported that the counseling University Scholarship Program Formative Evaluation – September 2015 22 center was not set up to maintain auditory privacy and, without his/her permission, shared his/her confidential information with faculty, staff and the student’s parents. Other students heard the results of his/her counselling and testing, much to his/her embarrassment. Conclusions: There is an undercurrent of tension between program managers and USP students at one of the universities but substantially less tension at the other two universities implementing the USP program. Several explanations for the difference between the first university and the other two university responses may be considered. The selection of participants by management at one of the universities that had a better feedback may have focused on selecting those likely to report only positive interactions. Both of the FGDs at the third university contained participants who were selected randomly, with mostly positive comments. The personal styles of management at both universities with positive feedback could encourage more face to face discussions and problem solving between students and managers. The results are that students at these universities feel more fully supported. Finally, it may be of some importance that the program managers at one of those two universities were younger, did not have wider university responsibilities, did not carry lofty titles, and therefore had less social distance from their students. Two reasons cited by universities for drop outs are psychological difficulties and academic difficulties, but some dropped students blame USP management rigidity and USAID regulations for their negative experience. Recommendations: 1. USAID staff should ensure that USP Implementing Partners address and adhere to compliance concerns in order to effectively respond to all student issues of concern including respect, abuse, rigidity, confidentiality and other issues. University managers should ensure that, to the extent possible, their own management styles reflect USP values including: democratic participation in decision-making; negotiated settlements of outstanding issues; absolute confidentiality during counseling regarding student problems, issues and personality tests; and integration of USP students into all aspects of University life, including common spaces such as dormitories. 3. USAID should review its policy on the possibility of granting a semester’s break to students in good academic standing who experience a serious psychological or physical health problem rather than dropping them from the program. To what extent did USP enable scholars to become active student leadership and potential leading change agents (looking at preparing individuals as change agents rather than actual transformation)? University Scholarship Program Formative Evaluation – September 2015 23 Findings: USP required all students to participate in extracurricular activities on campus, such as clubs, sports, drama, etc., and to promote active leadership positions in those campus organizations (which is part of the scholarship terms). Graduates in the USAID Survey indicated that about half (49%) achieved an active leadership position. The program also provided workshops/lectures on leadership, opportunities to perform group work on class projects, and implement capstone projects. The USP tracked and reported high levels of such program participation. Importantly, in FGDs, HU students reported that they felt the program training in leadership was very good and led them to personal transformation. Graduates reported similar views. A male student stated in the HU FGD: “Before this program, I was a very shy person. If the phone rang, I ran away because I did not want to answer it. Now, if I am asked to dance on stage before 300 people, I am at ease doing it.” However, student FGD participants at all of the universities were less than fully enthusiastic about all the community service and project requirements. The team felt that students understood the value and impact of the leadership training and practice, especially after graduation, were pleased by the personal transformations they recognized in themselves, but nevertheless had reservations about the amount of time and scheduling problems associated with these activities, especially when they were called “volunteer activities” but were mandatory. FGD participants reported that before USP, they expected to attend a public Lebanese University. Now, the vast majority of participants (90%) plan on a graduate education, often after they gain some work experience and strengthen their financial position. Asked about returning to their communities, only two students in FGDs from distant areas planned on returning to live in their community to give back. Most indicated that they would return on weekends or in the distant future but not to live in the near future. However, students from Beirut and Mount Lebanon reported that they may return to their home area to live. FGD participants expressed optimism about their being able to play leadership roles in their futures. However, they did not attribute their ability to do this to university leadership training but rather to their personality and entire life experience. Graduates who participated in FGDs gave greater credit to their training experiences. Nearly all FGD participants indicated that they plan on playing a leadership role in Lebanese society, but not so much in the smaller communities from which many were recruited. Conclusion: The USP program encouraged, supported and achieved active student leadership among many students. We cannot know in the short-term if the program is successful in creating professional and community change agents. Recommendations: University Scholarship Program Formative Evaluation – September 2015 24 1. The USP program may wish to consider, if it is logistically possible, encouraging students to plan community service programs in their home communities so as to strengthen their continuing community ties. (However, it is difficult to conceive of any program that would encourage remotely recruited students to return home, after they spend four years in the nation’s urban capital, and they develop new friendship groups and broad opportunities.) To what extent is USP preparing scholars for entry to the labor market? Findings: Online survey responses indicate that 98% of female respondents and 91% of male respondents intend to work in the future, including after getting married and having children, and 80% of female respondents and 85% of male respondents indicate that they think they are well trained to serve as a change agents in their community. Indeed, 22% of graduates (28/126 graduates; 29% or 22/75 female graduates and 11.76% or 6/51 male graduates) found employment relatively quickly (within three months of graduation); mostly in their field of study (only 11% or 3/28 graduates who found a job said it wasn’t related to the same field of studies). Further, the USAID survey of graduates indicated that graduates generally feel well prepared for employment, and are optimistic about their futures. However, planning to obtain a job may be a longer term proposition as has been noted, as 85-90% of respondents to the online survey and USAID questionnaire indicate that they intend to apply for further graduate education, with about half planning to enroll in graduate education within the next six months. Choice of major is a critical decision affecting level of satisfaction with university education and employment prospects. In FGDs, many students expressed the opinion that their choice of major was affected by factors besides their true interests. These included, what was offered in each university, whether the enrollment in their choice major was open or not, the amount of time and academic credits required, and others. As a result, a significant number of graduates indicated less than complete satisfaction with their choice of major, 22% of graduates indicated their major was forced upon them, and indeed, two thirds of graduates reported that they were not provided any information upon enrollment about their choice of major to help refine their selection. Workshops were organized by the universities to prepare the students to be successful at job interviews. Examples include the Blom Bank CV Writing and Interview Skills lecture organized for AUB USP II students; and the Interview Boot Camp Training and workshops on career skills, one-to-one support to students on writing a CV, cover letter and interview, and support in job search at HU. A number of activities and workshops are held by the Career Guidance Office at LAU including the Communication Skills Workshop was aimed at engaging USP scholars in exploring their career paths and enhancing their communications skills to improve both personal and professional aspects of their lives. University Scholarship Program Formative Evaluation – September 2015 25 Conclusion: Nearly all students/graduates are planning to enter the labor market. Many report they feel well prepared and are optimistic. However, the vast majority (90%) wish to obtain graduate education either immediately after graduation or after they gain a few years of employment. A minority of graduates have found employment quickly. A significant portion (22%) of graduates reported that their major was forced upon them. Recommendations: 1. In view of the importance of the selection of a major to the entire university educational experience, universities should provide more information and assistance regarding the implications of choosing different majors, strive to empower students to make an informed decision and offer the maximum choice possible. PROGRAMMATIC PERFORMANCE To what extent have all three implementing partners achieved the planned outputs and outcomes as clearly stated in their award documents? Findings: The award documents supplied by USAID mostly lists planned USP inputs from the universities in place of outputs and outcomes. For example, HU lists the following as outputs and outcomes: “a carefully screened and gender balanced cohort of promising students… will receive (benefits);” “the cohort will receive numerous academic supports…solid academic, critical thinking and hands on training..” Similarly, AUB lists: “support qualified and deserving students with scholarship;” “opportunities for male and female students from under privileged communities … to complete their undergraduate education;” ”train and equip qualified and deserving students with the skills to become leaders in their fields..;” “provide a safe environment for students to experience…” And similarly, LAU lists: “screening…; interviewing…, an orientation program… and close mentoring.” However HU and LAU do list a few appropriate outputs: “At least 90% of students entering with a SP scholarship from USAID in fall 2011 will have completed their studies and graduated within four years (or five years if the additional one year English language training is needed).” Further, LAU indicated that “the graduating students will find jobs in Lebanon…” AUB lists an appropriate, if difficult to measure outcome: “improve mutual understanding and cooperation between the people of the United States and the people of Lebanon.” The evaluation team considers that the program outputs are the number of well-educated and trained graduates capable of gaining employment in their fields of study, and serving as change agents in their profession and society. The high level outcomes are a more educated, democratically oriented population and improved mutual understanding and cooperation between the people of the United States and the people of Lebanon. With a very low drop-out University Scholarship Program Formative Evaluation – September 2015 26 rate (3%) and an expected high graduation rate in the specified time period, HU is on track to surpass its planned program output of 90% graduation rate. LAU is also on track. The outcome listed by AUB is difficult to measure in the short-term, but the statements of gratitude, and personal transformation related by FGD participants, and the rippling impact of the program in home communities, offers the suggestion that at least some amount of AUB’s planned outcome is being achieved. Conclusion: Appropriate outputs and outcomes are poorly defined in program documents received. The graduation rate appears to be on track. Outcomes will require improved specification and longer term evaluation. Recommendations: 1. USAID should plan a workshop or the distribution of reference material to help clear up confusion among program planners regarding the proper planning of inputs, outputs, and outcomes so as to strengthen program planning and evaluation in future USP programs. 2. USAID should implement both a final program evaluation, and creatively plan how the long-term program outcomes might be evaluated, keeping in mind their political sensitivity. To what extent can it be said that USP provides value for this USAID investment? Findings: Hard data is not available to answer this question which remains somewhat subjective. Over the past years USAID has invested in primary and tertiary education in Lebanon. The table below lists the different implementing mechanisms funded by USAID in the primary and tertiary education in Lebanon since 2006. The USAID has supported the Lebanese public school system with rehabilitation works and provision of equipment as well as through teachers training and extra-curricular activities, and increased community involvement in public schools. The USP program complements the efforts and leverages the funds invested in the primary education. It offers those same public high school students scholarships to allow them attaining a quality higher education, thus maximizing their potential to support Lebanon’s democratic and economic development. Excerpts from Thomas Friedman, NY Times Op-Ed, 1 My, 2012 Tanks and Jets or Scholarships ..If America wants to connect with the real aspirations of these revolutions, it will expand to other Arab awakening countries the $13.5 million U.S. Agency for International Development scholarship program begun in Lebanon. …They still line up for American scholarships, though — one requirement of which is that winners have to do community service, so we are also helping to build better citizens. (A student) told me: “This whole program is helping to make the youth capable of transforming this country into what it should be and can be. …. The U.S. is giving us a chance to make a difference. I do believe if we are given the chance, we can excel. ... We will not be underestimated anymore. It is really sad when you see a whole generation in Lebanese villages — hundreds of guys doing nothing — no work, not going to college.” After getting the U.S. scholarship, said (the student), “my family and my community feel differently about America. Why would they hate someone who is helping them?” University Scholarship Program Formative Evaluation – September 2015 27 The vast majority of students supplying data in questionnaires and FGDs indicate that they think the program provides value. Faculty and program managers of the three elite universities believe the program provides value, not only for USP but for the universities as a whole. A knowledgeable informant working in the education sector reported s/he believed USP was far more successful in helping the people of Lebanon than other public sector programs which have achieved limited success. Conclusion: While “value for investment” is difficult to assess, the USP leverages the efforts made into the primary education. All stakeholders enthusiastically express the view that USP provides value for USAID’s investment. Based on the total evaluative experience, the evaluation team has concluded that USP provides value for the USAID investment. The higher education opportunities provided to Lebanese citizens create the workforce potential which government and private actors must seize as opportunities. Recommendation: USAID should further investigate this question over time, with some comparisons of achievements between various USAID investments, continue implementing USP and if funds become available, expand USP to include other universities and a higher number of students. SUSTAINABILITY What are the prospects for the sustainability of USP activity results and which results show the most prospect of being sustained and why? Findings: None of the USP program managers or university officials expressed the view that USP was a sustainable program in the absence of USAID or the equivalent large-scale funding. However, USP has gained publicity and praise from an influential journalist and N.Y. Times Columnist, Thomas Friedman who expressed his admiration for the program to his readers in glowing terms and his desire to expand the program to other countries of the region (see side text box for excerpts of his Op.-Ed.)11 . This may help generate US public support. However, even if the entire program proves unsustainable, program managers all expressed the view that components of the program -- such as workshops, community involvement, leadership training, capstone projects, etc. -- had substantial value for all students, and could strengthen the overall universities’ educational programs. University staff reported that they appreciated the overall impact of USP program experiences on students, the resulting quality of graduates, leadership and employability of the USP students. 11 Friedman, Thomas, 2012. Tanks, Jets or Scholarships. Op-Ed, NY Times, 1 May 2012. University Scholarship Program Formative Evaluation – September 2015 28 As these elite universities gain experience with their first public school students, they appear to be gaining awareness of how their universities’ community service and educational function benefit through an increase in student diversity. LAU staff specifically stated that the university’s experience with bright USP students encouraged the university to establish its own program of 50% scholarship support for the many bright public school applicants who could not be accommodated in the USP program. The university recognized the serious financial burden on families that would result from only a 50% scholarship, but was able to admit an additional five applicants under these terms. University staff report that they have many other scholarship programs and these could be reviewed in the light of USP experience and lessons. Conclusions: USP is probably not sustainable as a whole, but some of its program features may be leveraged or upheld by the universities. Provide recommendations on how the activity design could be enhanced to improve the sustainability of results, and any additional programming or support in the upcoming years that would improve USP results sustainability 1. USAID should encourage university managers to attend to the issue of sustainability and include their related thinking and experience in their annual program reports. 2. USAID could encourage the implementation of operations research studies to test the impact of USP components on regular non-USP students in different departments. A two-arm, randomized, experiment with before and after measure, using experimental and control groups to test the impact of USP special program components on in the non-USP university environment. 12 , 13 Each of the Universities could implement this research study on their students using a common research protocol. 12 Ibid. 13 Such an experiment can be diagrammed in usual operations research terms as follows: O1 X O2 R-> O3 O4 _____Time____> Where: R represents random allocation to experimental and control groups such as Classes in a major. O1 and O3 represent before measures of the dependent variable(s), such as Leadership ability, score on leadership test O2 and O4 represent after measures of the dependent variable(s) X represents the experimental conditions from USP program used with regular U Students in selected majors Tests include, is O2 greater than O1? Is the extent of change in the dependent variable, i.e. leadership, greater in the O2 vs O1, vs. the O4, O3 comparison? University Scholarship Program Formative Evaluation – September 2015 29 3. Universities should encourage students to work on the issue of sustainability of USP components in their community service requirements. 4. USAID should plan to hold a workshop with USP managers, university administrators, and selected faculty to:  Review the issue of sustainability;  Explore the potential of USP program components to transform university education programs into more practical programs which increase employability of graduates;  Determine which USP components might be elevated to best practices in university teaching;  Explore the potential for incorporating USP program components in the universities regular teaching program and other scholarship programs;  Plan research to evaluate the impact of incorporating USP program components;  Develop recommendations for improving sustainability of various USP program components; and  Explore partnerships to sustain the funding necessary, perhaps through alumni or diaspora philanthropy or the private sector. MECHANISM Determine if the mechanism currently used for USP is appropriate with respect to cost and management burden (this question was added a week following the start of the evaluation field work). Findings: For the purpose of answering this question, the team assumed that a “management burden” results from conditions that lead to an excess of management, beyond that amount necessary. USP was initiated as a grant program but was changed to a cooperative agreement. According to the AOR: “The change in the mechanism from grant to cooperative agreement is related to a desire by USAID to have substantial involvement in the program which a grant cannot provide. Substantial involvement is expanding from phase to phase to include selection guidance, interrelation across USPs, approving work plans and monitoring and evaluation documentation, coordinating and facilitating exchange semester between AUB and LAU; exploring collaboration in joint community service projects, possible joint receptions. New phases of USP will include also substantial involvement to undertake joint and unified dissemination including possibly joint application for the winners of the new procurement. The grant does not have a platform University Scholarship Program Formative Evaluation – September 2015 30 for substantial involvement as per USAID policies and a contract is not desired as we do not want to micro-manage university-based scholarships.”14 All USP programs are managed by separate cooperative agreements. Each USP cooperative agreement is managed and reported on separately leading to a plethora of reports that tend to obscure rather than clarify overall program progress. With different university staff managing different USP reporting requirements, costs are increased. At present, separate contracts are required for each USP. USAID indicated that cost-sharing increased per award from as low as nearly 8% under USP I-HU to nearly 30% in later phases, which was a requirement to maximize leveraging of university resources including direct upfront tuition reductions of 15%, freeze of annual increases on tuition (nearly 4% per year) and cap on management costs. Management costs on the USAID share were constrained to nearly 7-8% which was a requirement as of USP3 onwards to maximize funding to the scholarships rather than to the management of the scholarship.15 The Table 3, below, shows the percentage increase over time of the universities cost share. Table 3: Universities Cost-share Percentage by USP and by University Haigazian LAU AUB USAID University Cost Share USAID University Cost Share USAID University Cost Share USP I Cost USD 6,631,150 607,335 6,915,267 1,697,098 % University Cost-Share 8.39% 19.71% USP II Cost USD 7,452,814 1,921,786 6,524,811 1,057,989 % University Cost-Share 20.50% 13.95% USP III Cost USD 9,544,647 4,772,576 % University Cost-Share 33.33% USP IV Cost USD 5,721,350 2,413,449 % University Cost-Share 29.67% USP V Cost USD 5,131,797 2,205,059 5,936,144 2,499,913 % University Cost-Share 30.05% 29.63% 14 Email from AOR Mario Rebeiz, 9 September 2015. 15 Ibid. University Scholarship Program Formative Evaluation – September 2015 31 Conclusion: The present system of separate contracts for each USP every year places a substantial increase on the management burden of USAID and the implementing partners. A longer term contract covering more than one USP would substantially reduce management burden. Recommendation: USAID should extend cooperative agreements to cover more than one USP over multiple years. This will substantially reduce the management burden and create a more efficient program. COMPARISON TO MEPI Determine if USP and MEPI are comparable and if features of the MEPI program would be useful to USP Based on the request of USAID Lebanon, this question was added a week following the start of the evaluation field work. Meetings with the MEPI Coordinator at the Office the Department of States and with AMIDEAST team were held to understand the scope of the MEPI and documents related to the MEPI budgeting were reviewed. Findings: MEPI encompasses two programs: one brings undergraduate students from throughout the Middle East and North Africa (MENA) region to study in Beirut, and the other sends undergraduates for an intensive 6-week program of study in the U.S. The four year scholarships for regional students in Lebanon (plus English language training if needed) provide all expenses for the education including one round trip ticket per year to maintain civic engagement in their home area. A short-term (6 week) intensive leadership program is held at six different Universities which bring together students from 14 countries (i.e. Morocco, Algeria, Tunisia, Libya, Egypt, Lebanon, West Bank, Jordan, Kuweit, Qatar, Bahrain, Saudi Arabia and Yemen). The program that sends students for intensive study to the US is not strictly comparable to USP, which supports university education in Lebanon for Lebanese. However, two features of MEPI deserve consideration for use in USP: 1. Encouraging contact with civic engagement in the home area; and 2. Providing an intensive training for all MEPI scholars. At present, all USP students are exposed only to their own university programs, and while many positive activities in leadership training are included in each university program, there may be benefits to bringing all USP students together to share perspectives on leadership University Scholarship Program Formative Evaluation – September 2015 32 development across the different USP implementing partners. Such an activity might also set the stage for further collaboration between implementing partners which has so far been absent. In addition, as has been mentioned, the prospects of encouraging community projects in home areas probably serves the function of maintaining civic engagement. In addition, as has been mentioned, the prospects of encouraging community projects in home areas probably serves the function of maintaining civic engagement. Finally, each of the MEPI Tomorrow’s Leaders Scholarship award is a $ 3M to $3.2M contract with AUB, LAU or American University of Cairo. It is issued for a period of five years and provides a four to five-year academic matriculation, civic education and community service opportunities to 20 students from the MENA region. The budget breakdown of four MEPI awards (Annex 9) shows that on average 16% of the total budget amount covers the personnel, fringe benefits and contractual. On average, 70% of the total budget amount is to cover direct costs, i.e. educational and student support costs. On average, 23% of the MEPI budget is cost-shared by the Universities. Compared to the budget breakdown of four recent USP awards with the same universities (LAU and AUB), 14% is to cover the personnel, fringe benefits and contractual, and 78% is to cover direct costs. On average, 31% of the USP budget is cost-shared by the Universities for those recent awards. Conclusion: MEPI differs from USP in that it sends students abroad for short-term training and brings regional students to Lebanon. However two components of MEPI may prove useful to USP. These two are: encouraging continuing contact with civic engagement in the home area; and providing an intensive, group training for all MEPI scholars. The USP is lower in cost per student than MEPI. Additionally, In comparison to the MEPI, the budget breakdown of the USP allocates a larger share of funding to educational and student support costs and a lower share to personnel and fringe benefits and contractual expenses. Recommendation: The USP should encourage continuing contact with civic engagement in the home area and should consider providing an intensive group training for all USP scholars. University Scholarship Program Formative Evaluation – September 2015 33 ANNEXES University Scholarship Program Formative Evaluation – September 2015 34 ANNEX 1: THE EVALUATION STATEMENT OF WORK USP draft eval SOW final revision.docx ANNEX 1.A: LIST AND DESCRIPTION OF USP PROGRAMS INCLUDED IN THE SCOPE OF THE EVALUATION Partner Name Program Area/Title AOR Award Value Start Date Completion Date LAU USP I Mario Rebeiz $6,915,267 29-Sep-10 31-Aug-17 HU USP I Mario Rebeiz $6,631,150 29-Sep-10 28-Feb-17 LAU USP II Mario Rebeiz $7,452,814 01-Oct-11 30-Sep-16 AUB USP II Mario Rebeiz $6,524,811 30-Sep-11 29-Sep-16 AUB USP IV Mario Rebeiz $5,721,350 24-Sep-12 23-Sep-18 LAU USP III Mario Rebeiz $9,544,647 19-Sep-12 18-Sep-18 AUB USP V Mario Rebeiz $5,936,144 30-Sep-13 29-Sep-19 LAU USP V Mario Rebeiz $5,131,797 30-Sep-13 29-Sep-19 University Scholarship Program Formative Evaluation – September 2015 35 ANNEX 2: EVALUATION MATRIX Categories Evaluation Questions Data sources Data collection method sample and tools Analysis Plans 1. National Coverage To what extent is USP achieving transparent national outreach? To what extent do applicants meet the criteria for scholarship awards? To what extent is the enrolled cohort diverse and nationally distributed? What are the impediments (real or perceived ones) affecting the recruitment of scholars from any particular areas, if any? Application and acceptance records, GPS info, list of all public high schools, FGD responses from students, information on majors, Survey Monkey University records, KII Same as Above Field interviews in underrepresented districts KII interview guide, FGD discussion Guide, Survey Monkey Questionnaire Random sample of 30 records Lists of schools, Districts, Maps Field Interviews in 5 underrepresented districts using Field Interview Schedule, Snowball sampling Qualitative Quantitative Quantitative Quantitative Quantitative 2. Scholars’ Overall Performance How well are students KIIs, FGDs, USAID Survey of grads KII interview schedule, FGD discussion guide, Qualitative Quantitative University Scholarship Program Formative Evaluation – September 2015 36 performing in their host universities? To what extent are students integrated in the university with the regular student body or are there challenges to such integration? To what extent are scholars mistreated, disrespected or abused by management for being scholarship recipients? On average, are USP scholars performing academically like their peers (non USP scholars) in the same class or not? To what extent did USP enable scholars to become active student leaders and potential leading change agents (looking at preparing individuals as change agents rather than actual transformation)? KIIs, FGDs, Survey Monkey, USAID Survey of Grads FGDs, Survey Monkey, USAID survey of grads KIIs, FGDs, USAID survey of grads KIIs, FGDs, USAID survey of grads KIIs, FGDs, USAID survey of grads USAID survey questionnaire KII interview schedule, FGD discussion guide, USAID survey questionnaire FGD discussion guide, Survey Monkey questionnaire, USAID survey guide KII interview guide, FGD discussion guide, USAID survey guide KII interview guide, FGD discussion guide, USAID survey guide Qualitative Quantitative Qualitative Quantitative Qualitative Quantitative Qualitative Quantitative Qualitative University Scholarship Program Formative Evaluation – September 2015 37 To what extent is USP preparing scholars for entry to the labor market? KII interview guide, FGD discussion guide, USAID survey guide Quantitative 3. Programmatic Performance To what extent have all three implementing partners achieved the planned outputs and outcomes as clearly stated in their award documents? To what extent can it be said that USP provides value for this USAID investment? KIIs, Desk review, USAID survey of grads KIIs KII interview guide, Desk review of reports, KII interview guide Qualitative Quantitative Qualitative 4. Sustainability What are the prospects for the sustainability of USP activity results? Which results show the most prospect of being sustained and why? Provide recommendations on how the activity design could be enhanced to improve the sustainability of KIIs KIIs KII, FGDs KII interview guide KII interview guide KII interview guide, FGD discussion guide Qualitative Qualitative Qualitative University Scholarship Program Formative Evaluation – September 2015 38 results, and any additional programming or support in the upcoming years that would improve USP results sustainability? 5. Mechanism Is the mechanism currently used for USP and determine if this is appropriate with respect to cost and management burden. KII KII interview guide Qualitative 6. Comparison to MEPI Determine if USP and MEPI are comparable and if features of the MEPI program would be useful to USP. KII KII interview guide Qualitative 7. Gender (crosscutting) To what extent is gender equality and female empowerment integrated into USP? FGDs, Survey Monkey, USAID survey of grads FGD discussion guide, USAID survey guide Qualitative Quantitative University Scholarship Program Formative Evaluation – September 2015 39 ANNEX 3: DATA COLLECTION TOOLS A. Individual KII Interview for Use /Adjustment with a Wide Variety of KIIs Name(s) Title(s) Position in USP? I. General Questions: 1. What roles do you play in implementing USP? 2. How do you think the USP is progressing/succeeding? What is the most important evidence you can cite to support that view? What made the program achieve this level of success? 3. Were the monitoring and evaluation arrangements appropriate? Are you able to use the monitoring and tracking information effectively in your job? How do you use it? Do you think the monitoring data is accurate? 4. Were there any unexpected outcomes to the project that you have noted? What do you think is the reaction of the family and community to the admission of their student (you) on full scholarship funded by the US Government/American people? 5. Have the partnerships worked effectively? Are partners committed to the program? Is there anything in the USP model that can be adapted to other sources of funding (at the conclusion of USAID funding? 6. Did the USP provide equal opportunity for both males and females to participate in the program? Why was gender balance not more equal in some of the programs? 7. What are the key lessons from project planning and implementation to be taken forward? Re: Outreach for recruitment: Acceptance criteria: Leadership Development: Community Service: 8. If you could change one element of USP to strengthen it, what would you change? What else might you change? Why? University Scholarship Program Formative Evaluation – September 2015 40 II. Additional Questions for Students: (Above 1-3, 5-7) 1. How did you earn about this scholarship opportunity? 2. How well do you feel you were integrated with the other/regular/non-USP scholars? Do you feel you ever witnessed or were subject to mistreatment, disrespect or abuse by instructors, other student, or University management based on the victim’s (your) status as a USP scholar? Do you feel you were treated special by anyone? 3. Were men and women given the same opportunities, support benefits, choice? Did either receive special treatment? Who? What type? 4. How well do you feel that you are being trained to serve as a change agent in your profession and community? Are you confident? Why? What could be done to help strengthen your confidence? 5. How well did the scholarship benefits fit your needs? Do you think the scholarship was fair? Why or why not? 6. Where do you plan to live and work when you finish your education? III. Additional Questions for USAID and Embassy staff: 1. How did you conduct your outreach program to attract scholars to your university? What worked? What did not work? What was the major challenge? 2. What do you think were the impediments to recruitment from certain areas? What was done to attempt to overcome these? Does this require more efforts? 3. Were the objectives, expected results and outputs sufficiently defined and operationalized in this program? Do you think the data reported by the universities are valid and correct? How might reporting be strengthened? 4. How did the universities and implementing partners contribute to the design and management of the project? What was the selection process to choose Universities? 5 What are the possible avenues for PPP to leverage USP? 6. What from these programs, if anything, may be sustainable at the conclusion of USAID funding? How might sustainability be strengthened? University Scholarship Program Formative Evaluation – September 2015 41 7. Do you think USP provided value for the amount of funds spent? Please explain. IV. Additional Questions for University Management and Staff: 1. How did you conduct your outreach program to attract scholars to your university? What worked? What did not work? What were your major challenges? 2. In what way did the universities succeed in involving program beneficiaries in the set-up and management of the USP programs? 3. Were any of the USP programs more successful than others? Why? 4. To what degree do you think the USP adequately addressed the needs faced by students receiving scholarship assistance? Are all of the benefits needed? How might services to beneficiaries be improved? 5. Did the different networks and their management function efficiently? Were you provided sufficient resources to carry out the program? What proportion of total staff time would you estimate was committed to tracking, monitoring and reporting? 6. Did the USP provide equal opportunity for both males and females to participate in the program? Why was gender balance not greater in some of the programs? 7. What is the level of university ownership of the program – a lot, a little, or some other amount? Why do you say that? 8. What is the total level of Scholarship support from all sources at this University? What % of this is USP funding? What are some of the other scholarship Programs? 9. What are your views on the possibility of strengthening collaboration between the three Universities in the USP? What are the challenges to collaboration? 10. What do you think were the impediments to recruitment from certain areas? What was done to attempt to overcome these? Does this require more effort? V: Additional Questions for Partners: 1. In what ways if any were you involved in the planning of USP? 2. What are the specific needs your organization sees in assisting the USP scholars? In what way is your organization prepared to respond to the specific needs? Does your organization benefit from involvement with the USP? How so? 3. How could coordination with the universities be strengthened? University Scholarship Program Formative Evaluation – September 2015 42 4. Are you familiar with and does your agency participate with any other student programs? (If yes) How does USP compare with the others? University Scholarship Program Formative Evaluation – September 2015 43 B. Focus Group Discussion Guide For Students 1. How did you learn about this scholarship opportunity? 2. What do you think is the reaction of the family and community to your admission on full scholarship funded by the US Government/American people? (Probe: Was the funding from the U.S. an issue? Could the outreach by the Universities to your high school or the orientation program have helped with the community issue?) 3. How do you think the USP is progressing/succeeding? (Probes: What is the most important evidence you can cite to support that view? What made the program achieve this level of success?) 4. How well do you feel you were integrated with the other/regular/non-USP scholars? 5. Were USP scholars ever abused or disrespected by other students or faculty or University administrators because of their scholarship status? 6. Were USP scholars treated special by anyone? 7. Were men and women given the same opportunities, support benefits, choice? (Probes: Did either receive special treatment? Who? What type?) 8. How well do you feel that you are being trained to serve as a change agent in your profession and community? (Probes: Are you confident? What could be done to help strengthen your confidence?) 9. Were there any unexpected outcomes to USP that you have noted? 10. How well did the scholarship benefits fit your needs? 11. What were your plans for your future before you received the USP scholarship? And now what are your plans for after you graduate? 12. Anything else you wish to say about USP? University Scholarship Program Formative Evaluation – September 2015 44 C. 1. Survey Monkey Questionnaire for Students and Graduates Questions for Survey Monkey This survey is a part of a USAID evaluation of the USP program being carried out by Social Impact, Beirut. The survey is completely anonymous. We hope you will answer truthfully. Our objective of the evaluation is to gain insights into how to improve the program. It should only take a few minutes. Pease complete and submit the survey as soon as possible and definitely before 10 September. Thank you very much for assisting the evaluation. 1. You are □ Male or □ Female 2. Do you have a disability yes no 3. You were selected from a school in  Beirut  Baalbek and Hermel  Mount Lebanon  South Lebanon  Nabatiyeh  North Lebanon  Akkar 4. You were enrolled at  American University of Beirut  Haigazian University  Lebanese American University Year of enrollment: _______________ 5. How did you learn about this scholarship opportunity? (mark all that apply)  Outreach staff from USP or Universities  Principal of my high school  Other high school staff  Newspaper advertisements  TV advertisement  Radio  Other (explain)____________________ 6. Please use the table to answer on a scale of 1 to 5 the following questions: (1= lowest rate, 5=highest rate) a) How well were you integrated with the other/regular/non-USP scholars ? b) How fairly were you treated in the USP program? University Scholarship Program Formative Evaluation – September 2015 45 7. Did you ever witness or were yourself subject to mistreatment, disrespect or abuse based on your status as a USP scholar?  Yes  No If yes by whom  Instructors  Other student  University management  Others entities 8. Did you ever witness or were yourself subject to mistreatment, disrespect or abuse based on your gender?  Yes  No If yes by whom  Instructors  Other student  University management  Others entities 9. Please use the table to answer on a scale of 1 to 5 the following questions (1= lowest rate, 5=highest rate): a) Are you satisfied with your overall academic performance so far? b) Do you think that you are being well trained to serve as a change agent in your community? c) To what extent do the benefits of the scholarship cover your needs? 10. Please rate from 1 to 5, to which extent the following issues affected/are affecting your performance in the USP program? (1= highly affected and 5 didn’t affect at all) a) Being away from home and family b) Adjusting to another culture c) Insufficient money to live d) Making new friends e) Integration in the program f) English language g) Academic level of the university 11. What do you plan to do when you finish your USP education?  Apply for further graduate education  Go back to my home area  Get a job  Other: ____________________________________________________________ University Scholarship Program Formative Evaluation – September 2015 46 12. Do you intend to work after getting married and having children? Yes no If no why? 13. Where do you plan to work when you finish your education? - Home area where my family is located  Another Lebanese region  Another country  Undecided 14. In your opnion, did the USP provide equal opportunity for both males and females to participate in the program?  Yes  No If no, Who received more opportunity Males of Females? ________ 15. Would you recommend the USP program to a good friend or sibling?  Definitely yes  Probably yes  Probably not  Definitely not 16. If you could change one element of USP to strengthen it, what would you change? ________________________________________________________________ 17. Is there anything else you would like to say about USP? (If Yes, use the space below) _____________________________________________________________________ 18. If you have finished your USP education, Are you presently graduated and employed?  Yes -> 18A  No ->finish thank you Additional Redirected Questions for graduates and employed only: A. Is your employment in the field in which you studied?  Yes  No University Scholarship Program Formative Evaluation – September 2015 47  Partly B. How long after you graduated did you find employment?  0-3 months  4- 6 months  7- 12 months  More than 12 months C. Please indicate your approximate monthly salary.  Less than $500  $500 -- $999  $1000 -- $1499  more than $1500  Do not wish to answer D. Are you involved in alumnae activities?  Yes  No Thank you very much for participating in the USP Evaluation! Good luck in your future. University Scholarship Program Formative Evaluation – September 2015 48 C.2. Justification of the size of the Survey Monkey Respondent Population: The minimum size of the respondent population needed based on the Cochran’s formula is: t 2 x p x q t 2 x p x (1-p) n0= --------------- = ----------------------- α 2 α 2 where : t=1.96 (value for selected alpha level of 0.025 in each tail = 1.96). p=0.5 (the estimate of the proportion – assumed maximum) α=0.05 (acceptable margin of error for proportion being estimated) (1.96)2 x 0.5x(1-0.5) n0= -------------------------------------- = 384 (0.05)2 Cochran’s correction formula used to calculate the final sample size where the total population size is N=496: n0 n= --------------------- (1+ n0/N) 384 n = ------------------ =217 (1 + 384 /496) University Scholarship Program Formative Evaluation – September 2015 49 Moreover, there is no statistical difference between the respondent population and the total population when it comes to Gender Distribution (α> 0.05), thus the gender distribution of the respondent population is representative of the gender distribution when it comes to the total population. The analysis of answers by sex was justified. Sex * Respondent sex Crosstabulation Respondent sex Respondent Non respondent Total Sex Male Count 89 110 199 % within Respondent sex 41.4% 39.1% 40.1% Female Count 126 171 297 % within Respondent sex 58.6% 60.9% 59.9% Total Count 215 281 496 % within Respondent sex 100.0% 100.0% 100.0% Chi-Square Tests Value df Asymp. Sig. (2- sided) Exact Sig. (2- sided) Exact Sig. (1- sided) Pearson Chi-Square .257a 1 .613 Continuity Correctionb .171 1 .679 Likelihood Ratio .256 1 .613 Fisher's Exact Test .644 .339 Linear-by-Linear Association .256 1 .613 N of Valid Cases 496 a. 0 cells (.0%) have expected count less than 5. The minimum expected count is 86.26. b. Computed only for a 2x2 table University Scholarship Program Formative Evaluation – September 2015 50 D. Interview Questions for USP Dropouts 1. What factors or issues led you to drop out of your USP program? 2. Do you think that the University provided you as much assistance as you needed with these factors or issues before you dropped out? 2A. What more might they have done to assist you further? 3. In retrospect, how do you now feel about the USP? 4. What are you doing now? (Employment, Living place?) 5. Do you have any suggestions for strengthening USP? 6. Is there anything further you would like to say about the USP? University Scholarship Program Formative Evaluation – September 2015 51 E. Field Interview in Communities Supplying Few Students Community Study as a Component of the University Scholarship Program Mid-term Evaluation Background: As discussed in our meeting 25 August at Information International, this community study is a part of the mid-term evaluation of the USP (University Scholarship Program). USP is a USAID funded grants program at AUB, LAU and Hagazian University. In this program, accepted applicants, who are graduates of public high schools, from most of Lebanon’s Districts, are supplied a full tuition package. Students are expected to have shown leadership qualities, include one male and one female from each district, and to come from poor/needy families. Objectives: The overall purpose of this exercise is to gain some insights about how the outreach program is seen to work, how it might be improved and what local people know and think about the program. Specifically: I. From Communities which did not submit a scholarship application: (Names of the communities are:) A. From the Principal of the High School: To gain information on 1. Knowledge of University Scholarship Program? 2. If known, how was the knowledge gained? 3. Does he/she know the source of the funding? 4. If the source is known, do they have any positive or negative feelings about the source? 5. If the Principal, disseminated knowledge of the program to students, community leaders, others? 6. What was the response of students, community leaders, others? 7. Why was no application submitted? 8. Any suggestions for improving the scholarship program? B. From the community leadership: To gain information on: 1. Knowledge of University Scholarship Program? 2. Knowledge that a scholar was selected from this community and given a full scholarship? 3. If known, how was the knowledge gained? 4. Does he/she know the source of the funding? 5. If the source is known, do they have any positive or negative feelings about the source? 6. If the Principal, disseminated knowledge of the program to students, community leaders, others? 7. Why was no application submitted? University Scholarship Program Formative Evaluation – September 2015 52 8. Any suggestions for improving the scholarship program? Research Procedures: We would like to visit three communities each of two different types -- 1. From where no application was submitted; and 2. From where application(s) were submitted but none were selected. We will supply the names of the communities of each type, the names of families of accepted scholars to be interviewed (head of household if available). It will be necessary for fieldworkers to find the school principals and community leaders to gain the above information. When found, all intended interviewees should be read an informed consent statement (to be supplied) which indicates that we are seeking information to help evaluate and improve the scholarship program, that participation is entirely voluntary, the interviewee does not need to answer any question he/she does not wish to answer, and the interviewee can stop the interview at any time. Fieldworkers should use separate instrument forms for each interview and write respondent’s answers legibly and submit completed forms in English. If possible, fieldworkers should pay attention to information not a part of the formal interview. Make a note on how you were welcomed or not, any relevant non-verbal gestures you encounter/observe and anything said about the Scholarship program which is spontaneous and not an answer to a particular question. Fieldworkers may, if they believe the situation demands it, adjust the questions asked to the situations they encounter, including dropping questions that appear to be inappropriate or adding questions they think add to the achievement of objectives. If questions are dropped or added, these should be noted on the instruments. IV. Interview Instruments: II. Interview of Public High School Principals Informed Consent Statement: We are here to talk about and gain information about your opinions on a scholarship program at three Universities. We would like to ask you some questions. You are free to answer or not answer and to stop the discussion at any time. We hope the information you provide can be used to strengthen the scholarship programs. If you would like further information about this, please contact Social Impact in Beirut. 1. What is the name of your high school? _____________________________________________ 2. Are you the principal of this high school? ___ Yes ___ No (End interview and find out how to reach the actual principal) 3. Do you know anything about the University Scholarship Program? ___ Yes  Go to3b ___ No End interview University Scholarship Program Formative Evaluation – September 2015 53 3b. What do you know about the program? (mark all items mentioned) ___ for bright students ___ for poor students ___ provides full scholarships ___ involves three prestigious universities in Beirut ___ for students who show leadership ___ funded by USAID/American Government/American people ___ other: ________________________________________________________________________ 3c. How did you learn about the University Scholarship Program? ___ Visitor from Beirut discussed ___ Advertisement on TV ___ Advertisement in newspaper ___ Personal acquaintance 3d. Do you know the source of the funding for the scholarship? ____Yes  3e ____No  4 3e. (if Yes in 3d) What is the source of the funding for the University Scholarship Program ___ USAID/American Government/American people _____another source mentioned _________________________________________ 3f. (If the source is known) Do you have any positive or negative feelings about the source? What are your feelings about this? 4. Did you ever nominate a student for a scholarship? ___ Yes - 5 ___ No - 4b 4b. Why didn’t you nominate a student for a scholarship? (Mark all answers mentioned) ___ didn’t have a qualified student University Scholarship Program Formative Evaluation – September 2015 54 ___ too much trouble ___ didn’t want to play favorites ___ other: ___________________________________________________________________________ > go to 7 5. How did you select the student you nominated? 6. Was the student you nominated accepted into the program? ___ Yes -> 7 ___ No  6b. 6b. Why do you think the student you nominated was not accepted? (open ended answer below) 7. Do you have any suggestions for improving the University Scholarship Program? ___ Yes _________________________________________________________________ ___ No 8. Is there anything else you would like to say about the University Scholarship Program? ___ Yes _________________________________________________________________ ___ No Express your appreciation for the respondent’s time and thank him/her in a culturally appropriate way. If you have any observations about whether you were welcomed, how the respondent behaved, non￾verbal information, etc. , please make notes here: II. Interview with Community members Informed Consent Statement: We are here to talk about and gain information about your opinions on a scholarship program at three Universities. We would like to ask you some questions. You are free to answer or not answer and to stop the discussion at any time. We hope the information you provide can be used to strengthen the scholarship programs. If you would like further information about this, please contact Social Impact in Beirut. 1. Do you know anything about the University Scholarship Program? ___ Yes - Go to1b University Scholarship Program Formative Evaluation – September 2015 55 ___ No -End interview 1b. What do you know about the program? (mark all appropriate items mentioned) ___ for bright students ___ for poor students ___ provides full scholarships ___ involves three prestigious universities in Beirut ___ for students who show leadership ___ funded by USAID/American Government/American people ___ other: ________________________________________________________________________ 1c. How did you learn about the University Scholarship Program? ___ Visitor from Beirut discussed ___ Advertisement on TV ___ Advertisement in newspaper ___ Personal acquaintance ___ School staff 1d. Do you know the source of the funding for the scholarship? ____Yes ---- 1e ____No -- 2 1e. (if Yes in 1d) What do you think is the source? ____from USAID/American Government/American people  1f ____ don’t know source 2 _____ another source mentioned_________________________________________ 2 1f. (If the source is known) Do you have any positive or negative feelings about the source? What are your feelings about this? ___ Postive _______________________________________________________________________ University Scholarship Program Formative Evaluation – September 2015 56 ___ Negative _______________________________________________________________________ ___ Mixed view, or other ______________________________________________________________ 2. Do you know whether or not a student was nominated for the University Scholarship Program from your community? ___ Yes  2b ___ No  3 2b. Was the student accepted for the scholarship program? ___ Yes ->3 ___ No  2c ___ Don’t know 2c. Why do you think the student was not accepted for the scholarship program? 3. Do you think the people of this community know about the University Scholarship Program? ___Yes 4 ___ No  5 4. What do you think people feel about this program? 5. Please tell me any suggestions you have for improving the University Scholarship Program. 6. Is there anything else you would like to say about the University Scholarship Program? ___ Yes _________________________________________________________________ ___ No Express your appreciation for the respondent’s time and thank him/her in a culturally appropriate way. If you have any observations about whether you were welcomed, how the respondent behaved, non￾verbal information, etc. , please make notes here: University Scholarship Program Formative Evaluation – September 2015 57 F. USAID Survey of Graduates University Scholarship Program Exit Questionnaire USP and scholarship are used interchangeably across the questionnaire. Please answer all questions and try to elaborate in open ended questions. Section1: Background 1. Name & Family Name: (may remain anonymous) 2. Sex: a. Male b. Female 3. University: a. American University of Beirut b. Haigazian c. Lebanese American University 4. Village of Origin )المحلة / القرية ): 5. District of Origin )القضاء ): 6. Governorate of Origin )المحافظة): 7. Town of Current Residence )المحلة ): 8. Full Official Name of school coming from in Arabic: 9. Initial Major upon enrollment in USP: 10. Final Major upon graduation under USP: 11. Final average upon graduation (CGPA): Section 2: Basic Scholarship Benefits 12. Select the benefits you were receiving (directly or indirectly) under USP (you may select more than one): a. Full tuition coverage b. Intensive English preparation c. Medical insurance d. Stipend e. Book allowance f. Dormitory University Scholarship Program Formative Evaluation – September 2015 58 g. Transportation allowance h. Laptop i. Other 13. Was the monthly stipend given to you enough to cover your living expenses? a. Yes b. No c. If no, why not? 14. Was the book allowance provided to you enough to cover the cost of your required books? a. Yes b. No If no, why not? 15. If you were staying in dorms assigned to you under USP, were you assigned to a: a. Single room? b. Shared room? 16. If you were in a single room, did you have to cover any housing expenses yourself? a. Yes, why? b. No, why? 17. If you were provided with a laptop, did it include original software installed on the device (original Windows for example)? a. Yes b. No 18. What percentage of your monthly stipend did you spend on average on your essential life support (food only and your direct student subsistence)? 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 19. What percentage on average of your monthly stipend did you spend on other direct family subsistence? 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Section3: Scholarship Application 20. How did you know about the scholarship before you applied? University Scholarship Program Formative Evaluation – September 2015 59 21. Who encouraged you to apply to the scholarship? 22. Was your family supportive of your decision to apply to the scholarship? 23. Was it difficult to apply to the scholarship? a. Yes b. No c. If yes, why? 24. Was it difficult to earn the scholarship? a. Yes b. No c. If yes, why? 25. What were the different selection criteria and application process to get the scholarship? 26. At the stage of applying to the scholarship, did you apply to multiple universities hosting USP? a. No, why not? b. Yes, why? 27. Do you have any recommendations for USAID to consider at the selection phase when offering scholarships at multiple hosting universities? 28. Were you an Access program or Yes program student? a. Access student b. Yes student c. None University Scholarship Program Formative Evaluation – September 2015 60 29. If you were an Access or Yes student, was that beneficial to you in any way under USP? a. No, why not? b. Yes, why? 30. If you did not earn the USP scholarship, would you have been still able to enroll at the university you just graduated from? a. Yes b. No 31. If yes, how would you have financed your studies? a. Other scholarship, which? b. Family savings c. Student loan d. Part-time job e. Other, explain 32. In your opinion, what are the objectives of USP? 33. Do you think USP is providing fair access to both men and women to apply to the scholarship? a. Yes b. No, why? 34. Do you think USP is providing fair opportunity to both men and women to earn the scholarship during the shortlisting and final selection phases? a. Yes b. No, why? 35. Do you think USP is providing fair opportunity to both men and women to enroll in specific stereotyped majors such as engineering, nursing, or other fields of studies? a. Yes b. No, why? Section 4: Scholarship Academics 36. Were you provided with any information upon enrollment about your choice of major to help you refine your selection? a. Yes b. No 37. What was the main driving factor behind your choice of major? University Scholarship Program Formative Evaluation – September 2015 61 38. Did you change majors during your studies? a. Yes b. No 39. If you changed majors, what was the main driving factor behind this change? 40. How satisfied are you with your final choice of major? Very Dissatisfied Somewhat dissatisfied No real opinion Somewhat satisfied Very satisfied 41. Explain the reasons behind your satisfaction/dissatisfaction (you may select more than one): a. Meeting personal ambitions b. Difficulty/ease finding a job c. Salary different than expected d. Social prestige different than expected e. Relevance of skills taught to real life f. Employer’s perception of this degree 42. If you were not satisfied, what other major would you have chosen instead? 43. Was the choice of major imposed on your under USP upon final enrollment? Not at all, this was a major I selected wholeheartedly Somewhat as I was given few options upon acceptance, and none of these options were my first priority Totally, this was not a major I even considered in my application 44. If the choice of major was somewhat or totally imposed on you, why did you accept it? 45. If the choice of major was somewhat or totally imposed on you, what are some good ways you recommend to avoid imposing majors on students? 46. If you changed major during your studies, was the new major imposed on you? Not at all, this was an alternative major I selected wholeheartedly during my studies Somewhat as I was given few options to consider as alternate possibilities Totally, I was told that this is the only major I can change to 47. What were the main academic challenges you faced, if any? University Scholarship Program Formative Evaluation – September 2015 62 48. Are you planning within the next 6 months to enroll in graduate studies? a. Yes b. No 49. Did the scholarship inspire you to seek further graduate education? a. Yes b. No 50. If it were not for USP, would you have enrolled in graduate studies related to your choice of major anyway? a. Yes b. No Section 5: Leadership and Civic Engagement 51. Did you participate in leadership and civic engagement activities? a. Yes b. No 52. What were the main categories of activities you were engaged in (you may select more than one): a. University club b. University society c. Other university based activities (fairs, special events) d. Community service projects e. Structured and instructor-led leadership workshops f. Other, please define 53. How essential was the leadership component to achieve the objectives of USP? Not essential at all Mostly not essential except for very few activities Don’t care Mostly essential except for some activities Very essential 54. How do you define yourself as a participant in leadership activities (pick only one answer): a. Bystander, you were forced to participate but you simply attended without any real contribution from your end. b. “Voluntold”, requested by management to participate, but you contributed at the end of the day positively to the activity c. Volunteer, offered your assistance by yourself out of commitment and interest and fully engaged in the activity d. Leader, self-initiated the activity and requested others to join. University Scholarship Program Formative Evaluation – September 2015 63 55. Define your overall type of participation namely in university based activities (events, fairs, model representative organizations…); definitions are in the lower row. By overall level of participation, we mean how you feel about your participation in at least 50% of all activities you engaged in: Manipulation Decoration Tokenism Assigned but informed Consulted and informed Adult￾initiated, shared decisions with you Youth initiated and directed Youth initiated, shared decision with adults You were invited to participate but you have no influence at all in the decision and/or outcome You were invited, put in a visible position so the university looks good, but you have no meaningful role You are given some role in what you were assigned to, but really you have no choice in what you were doing and how Activities are initiated and run by managers, you were invited with a specific role and a clear expression that you have limited influence Activities are initiated and run by managers, you were invited with a specific role and you were provided advice how you would contribute to the end result Managers initiate the activities and but you were invited to share the decision equally You initiated the activity, and you could have invited the managers to support you though their presence is not essential to the continuation of your activity You have initiated the activity and you have invited the managers to share decision equally with you. 56. Did you achieve any “prominent position” in your university-based activities such as club president, distinguished member of a society? a. Yes, what was the position? b. No 57. Was USP a contributing factor by any means for you to achieve this prominent position? Not at all Probably not Neutral Maybe yes Definitely 58. If you were not a USP scholar, would you have engaged anyway to run for this prominent position (such as participating in elections to become club president)? Not at all Probably not Neutral Maybe yes Definitely 59. What was the main driving factor behind you running for these prominent positions? 60. Did the scholarship assist you in helping your community? a. Yes, how b. No 61. Were you able to develop a specific personal skill out of your participation in these leadership activities? University Scholarship Program Formative Evaluation – September 2015 64 a. Yes, what b. No c. Do not know 62. If you were to enroll today under USP as a fresh new student, what would you have done personally any different under the leadership component? 63. If you were to enroll today under USP as a new student, would you have participated in leadership activities the same way you did prior to your current graduation? a. Yes b. No, I would have participated less c. No, I would have participated more d. Do not know 64. What would you recommend to USAID to improve USP student engagement in leadership and civic engagement? 65. Do you believe your sex affected your ability to participate in specific leadership activities under USP? a. Yes, why? b. No 66. Do you believe that your university treated men and women differently when implementing leadership and civic engagement activities? a. Yes, why? b. No Section 6: Career Guidance / Job Readiness 67. Did you participate in any career guidance/ job readiness activities? a. Yes b. No 68. What were the main categories of activities you were engaged in (you may select more than one): a. Career preparation workshops (interviews preparation, resume writing…) b. Soft skills workshops c. Information sessions about career prospects in your field of study d. Job fairs e. Internships f. Other, please define University Scholarship Program Formative Evaluation – September 2015 65 69. How essential was the career guidance/job readiness to achieve the objectives of USP? Not essential at all Mostly not essential except for very few activities Don’t care Mostly essential except for some activities Very essential 70. What is your understanding of job readiness? 71. Was USP able to help you fulfill your expectations in job readiness based on your understanding above (question 68)? Not at all Mostly no, except for very few activities that were helpful Don’t know Mostly yes, except for very few activities that were meaningless Absolutely yes 72. Do you feel better prepared for the labor market currently after participating in the USP career guidance activities? Not at all Mostly no, except for very few skills I gained Don’t know Mostly yes, except for very few skills that I believe still need improvement Absolutely yes 73. Were you provided with any career guidance upon enrollment in your major as part of the USP Orientation? a. Yes b. No 74. If yes, was it beneficial? a. Yes b. No 75. When were you provided with career guidance during the course of your studies post￾orientation (you may select more than one answer): a. Intensive language preparation year b. Sophomore year c. Junior year d. Senior year e. Fourth year of professional studies (last year of engineering for example) f. Other: 76. Do you have any recommendations to USAID to improve the career guidance/job readiness component for the benefit of USP students? 77. Did you secure employment post-graduation (i.e. a work contract for a fixed duration of at least six months with at least a 50% part-time allocation)? a. Yes, and already working University Scholarship Program Formative Evaluation – September 2015 66 b. Yes, and will start working shortly c. No, but Still looking d. No and not looking for employment at all 78. If you secured employment, how did you secure this employment? 79. If you are not looking for employment, please state your reasons for not looking? 80. If you secured employment, please select the type of organization you are working for: a. Self-employed b. Government, public organization c. Private sector (Bank, Insurance, goods and/or Services Company, industry…) d. NGO sector e. UN agencies and/or other multilateral/bilateral agencies f. Other, explain 81. If you secured employment, is this related to your field of study (university major)? a. Yes b. Somewhat c. Not at all 82. If it is not related to your field of study, is it because: a. You do not like your field of study and opted to work in a different field; b. You could not find a job in your field of study; c. You did not look for a job in your field of study; d. This is a temporary fix until you find a job in your field of study e. Other 83. If you did not secure employment yet, how optimistic are you in finding a job related to your field of study? a. Very optimistic b. Somewhat optimistic c. Not optimistic at all 84. Was USP/your university of any assistance in securing your job? a. Yes b. No 85. How beneficial was your overall USP experience to your job readiness? Not beneficial at all Somewhat beneficial Very beneficial Do not know Section 7: USP and the University University Scholarship Program Formative Evaluation – September 2015 67 86. How do you characterize yourself first and foremost (select only one): a. Just a student b. An AUB, HU or LAU student c. A USP scholar d. None of the above e. Other, then what? 87. Do you feel you were treated negatively in anyway by the University (Management/ Faculty/ Staff) for being a USP student? a. Yes, how? b. No 88. Do you feel you were treated negatively in anyway by the non-USP students for being a USP scholar? a. Yes, how? b. No 89. If the answer is “Yes” to any of the above (87-88), do you recommend any means to facilitate better scholar integration at the university? Section 8: Finally!  90. Would you recommend USP to your community and colleagues if they have the chance to apply? a. Yes b. No 91. Do you have any recommendation to USAID to improve USP? 92. Do you have any story to share with USAID that you are proud of as your accomplishment under USP (besides successfully graduating) 93. How did the scholarship benefit you? 94. Do you know that you may access additional United States Government (USG) resources through the State Alumni Network? University Scholarship Program Formative Evaluation – September 2015 68 a. Yes b. No University Scholarship Program Formative Evaluation – September 2015 69 G. LIST OF KEY INFORMANT INTERVIEWS Date Organization/ University Person interviewed 24-Aug-15 USAID/Lebanon USP AOR Senior Program Officer Senior program advisor/ acting mission director Program Officer/ PMSPL COR 26-Aug-15 Haigazian University USAID University Scholarship Program Manager USAID University Scholarship Program Assistant 27-Aug-15 AUB Associate Dean, FAS & USP Director Director UPP, Associate Director USP Director OIRA OGC Department, Director Director career and Placement Services Director for Development and Community Projects Office of admissions - Assistant Director 28-Aug-15 LAU Assistant Vice President - Outreach and Civic Engagement Vice President for Student Development and Enrolment Management Assistant Vice President for Finance, budget and Grants 31-Aug-15 LAU Assistant Vice President for Enrollment Management USAID/Lebanon USP AOR 2-Sep-15 AUB Students and Graduate from USP/AUB 3-Sep-15 Haigazian University Director of student life Office of Financial Aid - Director Students and Graduate from USP/HU 4-Sep-15 USAID/Lebanon Program Office Director USAID/Lebanon Program Officer/ PMSPL COR University Scholarship Program Formative Evaluation – September 2015 70 US Department of State Director - Office of the MEPI US Department of State Grants Program Manager 7-Sep-15 AUB USAID USP Coordinator USAID USP Coordinator 8-Sep-15 D-RASATI II Chief of Party 10-Sep-15 AUB USP scholar dropout USP students/graduate 11-Sep-15 LAU USP scholar dropout USP student/graduate USP student/graduate 14-Sep-15 AMIDEAST Country Director Educational and Scholarship Programs Manager AUB Office of admissions - Assistant Director Office of admissions - Assistant Director 15-Sep-15 USAID/Lebanon USP AOR University Scholarship Program Formative Evaluation – September 2015 71 ANNEX 4: DISTRIBUTION OF USP ENROLLED STUDENTS BY DISTRICTS District N % 1 Sour 39 7.86 2 Saida 38 7.66 3 Aley 33 6.65 4 Baabda 31 6.25 5 Metn 31 6.25 6 Beirut 30 6.05 7 Tripoli 29 5.85 8 Baalbeck 27 5.44 9 Chouf 26 5.24 10 West Bekaa 25 5.04 11 Akkar 20 4.03 12 Nabatieh 20 4.03 13 Hasbaiya 16 3.23 14 Kesrwan 16 3.23 15 Zahle 16 3.23 16 Jbeil 15 3.02 17 Zgharta 13 2.62 18 Batroun 12 2.42 19 Bent Jbeil 12 2.42 20 Koura 10 2.02 21 Rachaiya 9 1.81 22 Marjayoun 8 1.61 23 Jezzine 6 1.21 24 Minieh-Dannieh 6 1.21 25 Bcharreh 5 1.01 26 Hermel 3 0.6 Total 496 100 University Scholarship Program Formative Evaluation – September 2015 72 ANNEX 4.A: DISTRIBUTION OF USP ENROLLED STUDENTS BY DISTRICTS AND SEX District Sex Total Male Female Akkar N 12 8 20 % 60.00% 40.00% 100.00% Aley N 10 23 33 % 30.30% 69.70% 100.00% Baabda N 9 22 31 % 29.03% 70.97% 100.00% Baalbeck N 6 21 27 % 22.22% 77.78% 100.00% Batroun N 5 7 12 % 41.67% 58.33% 100.00% Bcharreh N 2 3 5 % 40.00% 60.00% 100.00% Beirut N 16 14 30 % 53.33% 46.67% 100.00% Bent Jbeil N 5 7 12 % 41.67% 58.33% 100.00% Chouf N 7 19 26 % 26.92% 73.08% 100.00% Hasbaiya N 6 10 16 % 37.50% 62.50% 100.00% Hermel N 2 1 3 % 66.67% 33.33% 100.00% Jbeil N 9 6 15 % 60.00% 40.00% 100.00% Jezzine N 1 5 6 % 16.67% 83.33% 100.00% Kesrwan N 7 9 16 % 43.75% 56.25% 100.00% Koura N 3 7 10 % 30.00% 70.00% 100.00% Marjayoun N 5 3 8 % 62.50% 37.50% 100.00% Metn N 13 18 31 % 41.94% 58.06% 100.00% Minieh-Dannieh N 3 3 6 % 50.00% 50.00% 100.00% Nabatieh N 9 11 20 % 45.00% 55.00% 100.00% Rachaiya N 5 4 9 % 55.56% 44.44% 100.00% University Scholarship Program Formative Evaluation – September 2015 73 Saida N 15 23 38 % 39.47% 60.53% 100.00% Sour N 17 22 39 % 43.59% 56.41% 100.00% Tripoli N 7 22 29 % 24.14% 75.86% 100.00% West Bekaa N 11 14 25 % 44.00% 56.00% 100.00% Zahle N 8 8 16 % 50.00% 50.00% 100.00% Zgharta N 6 7 13 % 46.15% 53.85% 100.00% Total N 199 297 496 % 40.12% 59.88% 100.00% University Scholarship Program Formative Evaluation – September 2015 74 ANNEX 4.B: DISTRIBUTION OF USP ENROLLED STUDENTS BY DISTRICTS AND BY USP District USP Akkar Aley Baabda Baalbeck Batroun Bcharreh Beirut Bent Jbeil Chouf Hasbaiya Hermel Jbeil Jezzine Kesrwan Koura Marjayoun Metn Minieh￾Dannieh Nabatieh Rachaiya Saida Sour Tripoli West Bekaa Zahle Zgharta Total AUB/USP2 2 5 1 3 2 2 3 2 2 3 1 1 1 2 3 3 2 1 3 3 1 2 1 1 50 AUB/USP4 3 2 2 3 2 1 2 2 3 1 2 1 1 2 2 4 1 2 2 2 2 2 3 2 4 53 AUB/USP5 3 1 5 2 1 1 3 2 2 2 2 2 2 1 2 2 1 3 3 3 2 1 4 1 2 53 HU/USP1 1 7 10 4 8 1 3 1 1 3 1 2 3 2 6 2 4 3 2 1 65 LAU/USP1 2 5 4 6 1 2 4 1 1 2 2 4 2 4 1 3 5 2 3 3 2 59 LAU/USP2 5 4 2 3 1 9 7 3 1 2 1 4 2 8 2 5 4 1 64 LAU/USP3 7 4 1 4 3 3 2 8 3 2 1 3 3 7 5 10 12 5 3 5 2 93 LAU/USP5 2 4 4 3 1 1 1 2 2 1 1 5 3 1 3 11 9 3 2 59 Total 20 33 31 27 12 5 30 12 26 16 3 15 6 16 10 8 31 6 20 9 38 39 29 25 16 13 496 University Scholarship Program Formative Evaluation – September 2015 75 ANNEX 5: MAPS University Scholarship Program Formative Evaluation – September 2015 76 University Scholarship Program Formative Evaluation – September 2015 77 University Scholarship Program Formative Evaluation – September 2015 78 University Scholarship Program Formative Evaluation – September 2015 79 University Scholarship Program Formative Evaluation – September 2015 80 ANNEX 6: DISTRIBUTION OF APPLICATIONS BY USP AND GEOGRAPHICAL AREA USP I USP II USP III USP IV USP V USP VI Grand Total Akkar 21 83 54 91 110 75 434 Akkar 21 83 54 91 110 75 434 Baalbak-Hermel 15 106 45 112 87 74 439 Baalbak 14 95 44 109 77 66 405 Hermel 1 11 1 3 10 8 34 Beirut 17 124 63 80 69 68 421 Beirut 17 124 63 80 69 68 421 Bekaa 19 120 56 111 163 122 591 Rachaya 2 9 8 14 22 33 88 West Bekaa 3 57 20 56 83 53 272 Zahle 14 54 28 41 58 36 231 Mount Lebanon 67 423 184 259 500 324 1757 Aley 20 105 39 48 83 57 352 Baabda 15 114 24 62 109 85 409 Chouf 6 53 28 67 133 62 349 Jbeil 7 31 28 12 29 38 145 Keserwan 11 29 31 16 37 29 153 Metn 8 91 34 54 109 53 349 Nabatieh 17 105 57 101 164 120 564 Bent Jbeil 9 29 13 26 37 32 146 Hasbaya 4 17 17 19 46 43 146 Marjeyoun 1 3 5 3 7 10 29 Nabatieh 3 56 22 53 74 35 243 North Lebanon 59 193 125 123 298 158 956 Batroun 5 24 17 23 32 13 114 Bcharreh 3 6 2 3 4 1 19 Koura 3 27 9 13 28 15 95 Minieh - Dinnieh 7 23 11 11 29 12 93 Tripoli 28 102 71 62 167 87 517 Zgharta 13 11 15 11 38 30 118 South Lebanon 30 175 82 120 252 126 785 Jezzine 3 16 8 5 18 17 67 Saida 3 105 41 64 102 48 363 Sour 24 54 33 51 132 61 355 Grand Total 245 1329 666 997 1643 1067 5947 University Scholarship Program Formative Evaluation – September 2015 81 Graph 1: Distribution of applicants by USP and by Governorate 0 200 400 600 800 1000 1200 1400 1600 1800 USP1 USP2 USP3 USP4 USP5 USP6 Distribution of the application by USP and Governorate Akkar Baalbak-Hermel Beirut Bekaa Mount Lebanon Nabatieh North Lebanon South Lebanon University Scholarship Program Formative Evaluation – September 2015 82 ANNEX 7: DISTRIBUTION OF PUBLIC HIGH SCHOOLS BY DISTRICT Governorate Caza School name Beirut Beirut Al Achrafieh First Secondary Public School for Boys Beirut Beirut Al Achrafieh Second Secondary Public School Beirut Beirut Laure Moghayzel Secondary Public School for Girls Beirut Beirut Zahia Salman Secondary Public School (formerly Wata Al Mousaytbeh) Beirut Beirut Fakhreddine Al Maani Secondary Public School for Girls Beirut Beirut Prime Minister Riad Al Soloh Secondary public School for Boys Beirut Beirut Jameel Al Rawass Secondary Public school for Boys Beirut Beirut Omar Farroukh Secondary Public School for Girls Beirut Beirut Zahia Qaddoura Secondary Public School for Girls (formerly Ras Beirut for Girls) Beirut Beirut President Rene Mouawad Secondary Public School Beirut Beirut Hasan Saab Mixed Secondary Public School Mount Lebanon Metn Sin Al Fil Secondary Public School Mount Lebanon Baabda Furn Al Shebbak for Girls Secondary Public School Mount Lebanon Baabda Shafik Said Al Chiah Secondary Public School Mount Lebanon Baabda Ghubairi First Secondary Public School for Girls - a.m. Mount Lebanon Metn Al Akhtal Assaghir Secondary Public School for Boys Mount Lebanon Metn Beit Meiri Secondary Mount Lebanon Baabda Al Hadath Secondary Public School Mount Lebanon Baabda Kfarshima mixed secondary public school Mount Lebanon Baabda Borj Al Barajneh Secondary Public School for Girls Mount Lebanon Baabda Hussein Ali Nasser secondary public school for boys Mount Lebanon Baabda Haret Herayk Secondary Mixed public school Mount Lebanon Kesrwan Al Safra Secondary public school - p.m. Mount Lebanon Kesrwan Jounieh Secondary public school Mount Lebanon Kesrwan Elias Abou Chabki Secondary Public School Mount Lebanon Metn Antelias Secondary Public School Mount Lebanon Metn Jal Al Dib Secondary Public School Mount Lebanon Metn Zalka Secondary Public School- p.m. Mount Lebanon Aley Aramoun Secondary Public School Mount Lebanon Aley Shoueifat Mixed Secondary Public School Mount Lebanon Chouf Martyr Rafiq Hariri Secondary Public School Mount Lebanon Jbeil Jbeil Secondary Public School Mount Lebanon Jbeil Amshit secondary public school Mount Lebanon Jbeil Mayfouk Secondary Public School Mount Lebanon Kesrwan Ghazir Mixed Secondary Public School Mount Lebanon Kesrwan Al Kfour Mixed Secondary Public School Mount Lebanon Metn Bekfaya Secondary Public School University Scholarship Program Formative Evaluation – September 2015 83 Mount Lebanon Metn Bteghreen Mixed Secondary Mount Lebanon Metn Baskenta Secondary Public School Mount Lebanon Metn Dhour Al Choueir Secondary Public School Mount Lebanon Metn Liwa Jamil Lahoud Secondary Mount Lebanon Baabda Qarnayel Secondary Public School Mount Lebanon Baabda Hamana Secondary Public School Mount Lebanon Aley Maroun Abboud Secondary Public School - Previousely Aley Mount Lebanon Aley Baysour Secondary Public School Mount Lebanon Aley Hussein Massoud Mixed Secondary Public School Mount Lebanon Aley Majdel Baana Secondary Public School Mount Lebanon Aley Ain Dara Secondary Public School Mount Lebanon Chouf Barja Secondary Public School Mount Lebanon Chouf Kfarhim Secondary Public School Mount Lebanon Chouf Historian Joseph Nehme Public School Mount Lebanon Chouf Baaklin Secondary Public School Mount Lebanon Chouf Al Mokhtara Secondary Public School Mount Lebanon Chouf Ain Zhalta Secondary Public School Mount Lebanon Chouf Chehim Secondary Public School Mount Lebanon Chouf Aanout Secondary Public School North Lebanon Tripoli Tripoli Secondary Public School For Girls North Lebanon Tripoli Educator Hassan Al Hajjah Secondary Public School - a.m. North Lebanon Tripoli Educator Fadel Al Mokaddem Secondary public School For Girls North Lebanon Tripoli Andre Nahhas Secondary Public School - For Girls Al Mina North Lebanon Tripoli Saba Zreik Secondary Public School For Boys North Lebanon Tripoli Educator Mawaheb Osta Secondary Public School For Boys - Prevouisely Tripoli Al Haddadin North Lebanon Tripoli Tripoli Al Haddadin Secondary Public School For Girls North Lebanon Tripoli Al Kobeh Mixed Secondary public school North Lebanon Minieh￾Dannieh Al Minyeh Secondary public school North Lebanon Tripoli Al Qalamoun Mixed Secondary Public School North Lebanon Minieh￾Dannieh Bakhoun Secondary public school North Lebanon Minieh￾Dannieh Sir Al Dinniyeh Secondary Public School- Kfarhabou branch North Lebanon Minieh￾Dannieh Beit Al Faks Secondary public school North Lebanon Akkar Halba Secondary public school North Lebanon Akkar Minyarah Secondary public school North Lebanon Akkar Martyr Rafic Hariri Secondary Public School- Bebnine Akkar North Lebanon Akkar Berqayel Secondary public school North Lebanon Akkar Al Ouyoun Secondary Public School University Scholarship Program Formative Evaluation – September 2015 84 North Lebanon Akkar Akkar Al Aatika Secondary Public School North Lebanon Akkar Al Kobayat Secondary public school North Lebanon Akkar Mashta Hasan Secondary public school North Lebanon Akkar Al Hisha Secondary Public School North Lebanon Zgharta Meryata Secondary Public School North Lebanon Zgharta Zgharta Secondary Public School North Lebanon Zgharta Zgharta Secondary Public School - Sebhel Branch (Assaad Sebaali) North Lebanon Koura Khalil Salem Bterram Secondary Public School North Lebanon Koura Kfarhata Secondary Public School North Lebanon Bcharreh Bsharre Secondary Public School North Lebanon Batroun Batroun Secondary Public School North Lebanon Batroun Ebreen Secondary Public School North Lebanon Batroun Batroun Secondary Public School - Douma Branch North Lebanon Batroun Tannourine Secondary Public School Beqaa Zahleh Zahle Secondary Public School for Girls Beqaa Zahleh Zahle Mixed Secondary Public School Beqaa Zahleh Hosh Al Oumara Secondary Beqaa Zahleh Taalbaya Secondary Beqaa Zahleh Riyaq Secondary Beqaa Zahleh Bar Elias Secondary public school Beqaa Zahleh Majdal Anjar Secondary Public School Beqaa Zahleh Kab Elias Secondary public school Beqaa Hermel Al Hermel Secondary public school Beqaa Baalbek Al Labwa secondary public school Beqaa Baalbek Aarsal Secondary Public School Beqaa Baalbek Al Ain secondary public school Beqaa Baalbek Al Fakiha secondary public school Beqaa Baalbek Ras Baalback Secondary Public School Beqaa Baalbek Deir Al Ahmar Secondary Public School Beqaa Baalbek Abdo Murtada Al Huseini mixed secondary public school Beqaa Baalbek Jawdat Rostom Haidar secondary public school Beqaa Baalbek Shmistar secondary public school Beqaa Baalbek Brital secondary public school Beqaa Baalbek Nabil Adib Sleiman mixed secondary Beqaa West Beqaa Joub Jannine secondary public school Beqaa West Beqaa Kamed Al Lawz Secondary Public School Beqaa West Beqaa Ghaza secondary public school University Scholarship Program Formative Evaluation – September 2015 85 Beqaa West Beqaa Al Manara secondary public school Beqaa West Beqaa Mashghara secondary public school Beqaa West Beqaa Al Qaraoun secondary public school Beqaa West Beqaa Sohmor Secondary Public School Beqaa Rachaiya Rashaya secondary public school South Lebanon Saida Dr. Hekmat Sabbagh Youmna El Eid Secondary for Girls (formerly Saida for Girls Secondary) South Lebanon Saida Saida Second Secondary Public School for Girls South Lebanon Saida Saida Mixed Secondary Public School- a.m. South Lebanon Saida Al Ghaziyeh Secondary Public School South Lebanon Saida Kfarhatta Secondary Public School South Lebanon Saida Aankoun Secondary Public School South Lebanon Saida Aadloun Secondary Public School South Lebanon Saida Al Babiliyeh Secondary Public School South Lebanon Jezzine Labaa Secondary Public School South Lebanon Jezzine Jezzine Secondary Public School Nabatieh Nabatieh Jbaa Secondary Public School Nabatieh Nabatieh Hasan Kamel Al Sabah Secondary Public School Nabatieh Nabatieh Al Nabatyieh for Girls Secondary Public School Nabatieh Nabatieh Jebshite Secondary Public School Nabatieh Nabatieh Rammal Rammal Secondary- Al Dowier Public School Nabatieh Hasbaiya Hasbaya Secondary Public School Nabatieh Hasbaiya Shebaa Secondary Public School Nabatieh Hasbaiya Kfeir Secondary Public School Nabatieh Marjayou n Marjaayoun Secondary Public School Nabatieh Marjayou n Khyam Secondary Public School Nabatieh Marjayou n Educator Mohamed Falha Secondary Public School Mais Al Jabal Nabatieh Marjayou n Majdel Selem Secondary Public School South Lebanon Sour Sour Mixed Secondary Public School South Lebanon Sour Sour Secondary Public School for Girls South Lebanon Sour Al Aabbasiyeh Secondary Public School South Lebanon Sour Deir Kifa Secondary Public School South Lebanon Sour Ali Jammal Model Orphanage Secondary Public School South Lebanon Sour Martyr Khalil Jaradi Secondary Public School/ Maaraka South Lebanon Sour Qana Secondary Public School University Scholarship Program Formative Evaluation – September 2015 86 South Lebanon Sour Kamal Salhab Secondary Public School- Ain Baal South Lebanon Sour Alma Al Shaab Secondary Public School Nabatieh Bent Jbeil Tebnine Secondary Public School Nabatieh Bent Jbeil Bint Jbeil Secondary Public School Nabatieh Bent Jbeil Rmeish Secondary Public School Nabatieh Bent Jbeil Aita Al Shaab Secondary Public School Mount Lebanon Aley Maroun Abboud Secondary - Kmatiye Branch Beqaa Rachaiya Al Rfaid Secondary public school Beqaa West Beqaa Al Marj Secondary public school Beqaa Baalbek Ras Baalbek Secondary public school - Al Kaa branch Beqaa Rachaiya Al Qaraoun Secondary Public School- Kawakaba branch Beqaa Zahleh Ali Al Nahri Secondary public school Nabatieh Bent Jbeil Al Sayyed Muhsen Al Amin Secondary Public School/ Shaqra North Lebanon Minieh￾Dannieh Al Sfireh Public School North Lebanon Akkar Hrar Secondary Public School Mount Lebanon Chouf Niha Secondary Public School South Lebanon Saida Al Sarafand secondary Public School South Lebanon Sour Al Bazouriyeh Secondary Public School Nabatieh Nabatieh Al Qsaibeh Secondary Public School North Lebanon Zgharta Rashiine Secondary Mixed Public School North Lebanon Minieh￾Dannieh Baddawi Secondary Public School Beqaa Baalbek Baalbek Second Secondary public school for Girls South Lebanon Sour Al Shahabiya Secondary Public School Nabatieh Marjayou n Al Tayba Secondary Public School Nabatieh Hasbaiya Shebaa Secondary Public School Kfarshouba branch Nabatieh Nabatieh Mayfadoun Secondary Public School Nabatieh Nabatieh Roumine Secondary Public School/ Ankoun branch Nabatieh Nabatieh Ansar Secondary Public School Nabatieh Bent Jbeil Kafra Secondary Public School Mount Lebanon Chouf Al Damour secondary public school Mount Lebanon Chouf Batloun Secondary Public School Mount Lebanon Chouf Barja Secondary Joun branch North Lebanon Akkar Bkarzla Secondary Public School North Lebanon Akkar Aydamoun Secondary Public School Beqaa Baalbek Al Labwa Secondary Public School- Shaat branch Beqaa Baalbek Tamnine Al Tahta Secondary Public School Beqaa Baalbek Tarayah Secondary public school Beqaa Baalbek Deir Al Ahmar Secondary Bouday branch University Scholarship Program Formative Evaluation – September 2015 87 Beqaa Rachaiya Haramoun Secondary public school - Rashaya English branch Beirut Beirut Al Allama Abdullah Al Alayli Secondary Mount Lebanon Chouf Chehim Secondary/ Al Zaarouriyeh branch North Lebanon Akkar Kfartoun secondary public School Beqaa Zahleh Saadnayel Secondary Public school Beqaa West Beqaa Al Swairi Secondary Public School South Lebanon Saida Doctor Nazih Al Bizri Secondary Public School South Lebanon Saida Haret Saida Secondary Public School South Lebanon Saida Al Marwaniya Secondary Public School South Lebanon Sour Shhour Mixed Secondary Public School Mount Lebanon Baabda Karnayel Secondary Public School - Ras Al Metn Branch North Lebanon Minieh￾Dannieh Beit Al Fakes Public School - Karsita Branch Beirut Beirut Al Amir Shakib Irslan Mixed Secondary Public school Beirut Beirut Al Alama Sobhi Al Mahmasani Mixed Secondary Public School Beirut Beirut Joubran Andraous Al Twaineh Secondary Public School Mount Lebanon Baabda Ghubairi Second Secondary Public School for Girls South Lebanon Saida Adloun Secondary Public School- Al Saksakieh branch South Lebanon Sour Sour Secondary Public School for Boys Marwahin branch North Lebanon Minieh￾Dannieh Mrah Al Siraj Public School North Lebanon Minieh￾Dannieh Markabta Secondary Public School North Lebanon Minieh￾Dannieh Al Sfireh Secondary Public School- Taran Branch South Lebanon Sour Rahman Younes Secondary Public Schoul- Qlaila Mount Lebanon Baabda Al Ghubeiri third secondary public school for girls Nabatieh Nabatieh Arabsalim Secondary Public School North Lebanon Akkar Masheha Mixed Secondary Public School Beqaa West Beqaa Lala Secondary Public School North Lebanon Akkar Martyr Walid Eido Secondary - Bokayaa public school Mount Lebanon Baabda Hussein Ali Nasser secondary public school-Hussein Maktabi elMur branch Mount Lebanon Aley Aramoun secondary public school- martyr Rafic Hariri branch Mount Lebanon Baabda Maroun Abboud Secondary Public School - Al Abbadiyeh Beirut Beirut Mufti Martyr Hasan Khaled Secondary Beirut Beirut Beirut Al Horj Mixed Secondary Public School Beirut Beirut Doctor Aman kabbara Chaarani Secondary public School Mount Lebanon Metn Jdaidet Al Matn Secondary Public School for Girls Mount Lebanon Metn Dekwaneh Secondary Public School Mount Lebanon Metn Dbayeh Secondary Public School University Scholarship Program Formative Evaluation – September 2015 88 Mount Lebanon Metn Mazraat Yashouh Secondary Public School Mount Lebanon Jbeil Kartaba Secondary Public School Mount Lebanon Kesrwan Ajaltoun secondary public school Mount Lebanon Kesrwan Ghosta Secondary Public School Mount Lebanon Chouf Munira Abou Merhi Al Bulbul Secondary Public School (Ketermaya) North Lebanon Tripoli George Sarraf Secondary public school North Lebanon Minieh￾Dannieh Sir Al Dinniyeh Secondary Public School North Lebanon Akkar Al Bireh Secondary Public School North Lebanon Akkar Rahbeh Secondary Public School North Lebanon Akkar Fnaydek Secondary Public School North Lebanon Koura Deddeh Public School North Lebanon Koura Amioun Secondary Public School North Lebanon Batroun Chekka Secondary Public School Beqaa Baalbek Al Nabi Sheet secondary Beqaa West Beqaa Saghbine Secondary Public School South Lebanon Saida Maghdousheh Secondary Public School South Lebanon Saida Martyr Nehmeh Hashem Secondary North Lebanon Tripoli Tripoli Al Kobeh Second Secondary Mixed Public School North Lebanon Koura Kfarakka Secondary Public School Mount Lebanon Kesrwan Al Moayssra Secondary Public School North Lebanon Akkar Al Hisa Mixed Secondary Public School North Lebanon Akkar Kobayat Secondary Public School for Girls Mount Lebanon Jbeil Jbeil Secondary Public School - Ehmej Branch Beqaa Hermel Al Kasr Secondary - Sahlat Al Maa Beqaa Hermel Al Hermel Al Namouzajieh Secondary public school Mount Lebanon Kesrwan George Efrem Al namouzajiya mixed secondary public school North Lebanon Akkar Wadi Khaled Secondary Mixed Public School Mount Lebanon Metn Bikfaya Secondary Public School- Beit Chebab branch North Lebanon Tripoli New Kobeh Secondary Mixed Public School North Lebanon Akkar Hisa Secondary Public School Sheikh Ayyash branch North Lebanon Akkar Kfartoun Secondary Public School Qunaya branch North Lebanon Akkar Fnaydek Secondary Public School-Meshmesh branch North Lebanon Akkar Al Kouachra Secondary Public School North Lebanon Minieh￾Dannieh Al Meniyeh Secondary Public School - Martyr Lieutenant Wissam Eid Branch Mount Lebanon Jbeil Jbeil Secondary Public School - Hsarat Branch Mount Lebanon Kesrwan Ghazir Secondary Public School - Yahchouch Branch Mount Lebanon Kesrwan Kfar zebiane secondary public School Mount Lebanon Aley Maroun Aboud Secondary public school - Sawfar Branch University Scholarship Program Formative Evaluation – September 2015 89 Mount Lebanon Aley Maroun Aboud secondary public school - Richmaya branch Beqaa West Beqaa Sohmor secondary public school North Lebanon Akkar Hrar secondary public school -Qabaait branch North Lebanon Akkar Wadi Khaled Secondary Bahiaa Hariri Branch for girls University Scholarship Program Formative Evaluation – September 2015 90 ANNEX 8: DISTRIBUTION OF STUDENTS BY MAJOR AND SEX Majors at Haigazian / USP I Sex Total Male Female Biology N 3 3 6 % 16.67% 6.38% 9.23% Business - Accounting N 1 0 1 % 5.56% 0.00% 1.54% Business - Finance N 1 3 4 % 5.56% 6.38% 6.15% Business Administration N 1 2 3 % 5.56% 4.26% 4.62% Business Advertising & Communication N 2 1 3 % 11.11% 2.13% 4.62% Business Human Resource Management N 0 3 3 % 0.00% 6.38% 4.62% Business Management Information Systems N 1 0 1 % 5.56% 0.00% 1.54% Computer Sciences N 2 3 5 % 11.11% 6.38% 7.69% Education N 0 1 1 % 0.00% 2.13% 1.54% English Literature N 0 2 2 % 0.00% 4.26% 3.08% Hospitality Management N 0 2 2 % 0.00% 4.26% 3.08% Mathematics N 2 5 7 % 11.11% 10.64% 10.77% Medical Lab Sciences N 5 21 26 % 27.78% 44.68% 40.00% Nutrition N 0 1 1 % 0.00% 2.13% 1.54% Total N 18 47 65 % 100.00% 100.00% 100.00% University Scholarship Program Formative Evaluation – September 2015 91 Majors at AUB/USP II Sex Total Male Female BADM N 0 3 3 % 0.00% 11.54% 6.00% Biology N 3 4 7 % 12.50% 15.38% 14.00% Business Administration N 0 1 1 % 0.00% 3.85% 2.00% Chemical Engineering N 4 0 4 % 16.67% 0.00% 8.00% Chemistry N 1 1 2 % 4.17% 3.85% 4.00% Civil Engineering N 0 3 3 % 0.00% 11.54% 6.00% Computer & Communications Engineering N 2 0 2 % 8.33% 0.00% 4.00% Computer Sciences N 7 1 8 % 29.17% 3.85% 16.00% Construction Engineering N 4 4 8 % 16.67% 15.38% 16.00% English Language N 0 1 1 % 0.00% 3.85% 2.00% Environmental Health N 0 1 1 % 0.00% 3.85% 2.00% Mathematics N 2 1 3 % 8.33% 3.85% 6.00% Mechanical Engineering N 0 1 1 % 0.00% 3.85% 2.00% Medical Lab Sciences N 0 2 2 % 0.00% 7.69% 4.00% Nutrition N 0 1 1 % 0.00% 3.85% 2.00% Petroleum Studies N 1 1 2 % 4.17% 3.85% 4.00% PTST N 0 1 1 % 0.00% 3.85% 2.00% Total N 24 26 50 % 100.00% 100.00% 100.00% University Scholarship Program Formative Evaluation – September 2015 92 Majors at AUB/USP IV Sex Total Male Female Architecture N 0 1 1 % 0.00% 3.70% 1.89% Biology N 2 2 4 % 7.69% 7.41% 7.55% Business Administration N 0 5 5 % 0.00% 18.52% 9.43% Chemical Engineering N 0 4 4 % 0.00% 14.81% 7.55% Chemistry N 2 0 2 % 7.69% 0.00% 3.77% Civil Engineering N 2 1 3 % 7.69% 3.70% 5.66% Computer Sciences N 11 2 13 % 42.31% 7.41% 24.53% Elementary Education N 0 2 2 % 0.00% 7.41% 3.77% Environmental Health N 0 1 1 % 0.00% 3.70% 1.89% Mechanical Engineering N 2 0 2 % 7.69% 0.00% 3.77% Medical Lab Sciences N 4 4 8 % 15.38% 14.81% 15.09% Nursing N 0 1 1 % 0.00% 3.70% 1.89% Nutrition N 2 3 5 % 7.69% 11.11% 9.43% Petroleum Studies N 1 1 2 % 3.85% 3.70% 3.77% Total N 26 27 53 % 100.00% 100.00% 100.00% University Scholarship Program Formative Evaluation – September 2015 93 Majors at AUB/USP V Sex Total Male Female Business Administration N 4 2 6 % 15.38% 7.41% 11.32% Civil Engineering N 0 1 1 % 0.00% 3.70% 1.89% Computer Sciences N 13 6 19 % 50.00% 22.22% 35.85% Electrical & Computer Engineering N 2 1 3 % 7.69% 3.70% 5.66% Mechanical Engineering N 2 0 2 % 7.69% 0.00% 3.77% Medical Lab Sciences N 1 9 10 % 3.85% 33.33% 18.87% Nursing N 1 4 5 % 3.85% 14.81% 9.43% Nutrition N 1 1 2 % 3.85% 3.70% 3.77% Petroleum Studies N 2 0 2 % 7.69% 0.00% 3.77% Political Sciences N 0 3 2 % 0.00% 11.11% 5.66% Total N 26 27 53 % 100.00% 100.00% 100.00% University Scholarship Program Formative Evaluation – September 2015 94 Majors at LAU /USP I Sex Total Male Female Biology N 2 0 2 % 9.52% 0.00% 3.39% Business Administration N 4 8 12 % 19.05% 21.05% 20.34% Chemistry N 0 1 1 % 0.00% 2.63% 1.69% Civil Engineering N 2 2 4 % 9.52% 5.26% 6.78% Communication Arts N 1 5 6 % 4.76% 13.16% 10.17% Computer Engineering N 3 2 5 % 14.29% 5.26% 8.47% Computer Sciences N 1 3 4 % 4.76% 7.89% 6.78% Economics N 0 1 1 % 0.00% 2.63% 1.69% Education N 0 5 5 % 0.00% 13.16% 8.47% Electrical Engineering N 1 1 2 % 4.76% 2.63% 3.39% Mathematics N 0 1 1 % 0.00% 2.63% 1.69% Mechanical Engineering N 4 0 4 % 19.05% 0.00% 6.78% Nursing N 2 2 4 % 9.52% 5.26% 6.78% Nutrition N 0 7 7 % 0.00% 18.42% 11.86% Psychology N 1 0 1 % 4.76% 0.00% 1.69% Total N 21 38 59 % 100.00% 100.00% 100.00% University Scholarship Program Formative Evaluation – September 2015 95 Majors at LAU/USP II Sex Total Male Female Business Administration N 4 9 13 % 17.39% 21.95% 20.31% Chemistry N 0 1 1 % 0.00% 2.44% 1.56% Civil Engineering N 2 2 4 % 8.70% 4.88% 6.25% Communication Arts N 1 4 5 % 4.35% 9.76% 7.81% Computer Sciences N 5 5 10 % 21.74% 12.20% 15.63% Education N 0 2 2 % 0.00% 4.88% 3.13% Electrical Engineering N 1 2 3 % 4.35% 4.88% 4.69% Hospitality Management N 1 1 2 % 4.35% 2.44% 3.13% Mechanical Engineering N 6 1 7 % 26.09% 2.44% 10.94% Nursing N 2 3 5 % 8.70% 7.32% 7.81% Nutrition N 1 11 12 % 4.35% 26.83% 18.75% Total N 23 41 64 % 100.00% 100.00% 100.00% University Scholarship Program Formative Evaluation – September 2015 96 Majors at LAU/USP III Sex Total Male Female Biology N 4 8 12 % 11.11% 14.04% 12.90% Business Administration N 3 11 14 % 8.33% 19.30% 15.05% Chemistry N 1 3 4 % 2.78% 5.26% 4.30% Civil Engineering N 8 4 12 % 22.22% 7.02% 12.90% Communication Arts N 0 4 4 % 0.00% 7.02% 4.30% Computer Engineering N 2 1 3 % 5.56% 1.75% 3.23% Computer Sciences N 4 3 7 % 11.11% 5.26% 7.53% Economics N 1 1 2 % 2.78% 1.75% 2.15% Education N 0 1 1 % 0.00% 1.75% 1.08% Electrical Engineering N 3 2 5 % 8.33% 3.51% 5.38% Mathematics N 1 3 4 % 2.78% 5.26% 4.30% Mechanical Engineering N 6 0 6 % 16.67% 0.00% 6.45% Nursing N 2 7 9 % 5.56% 12.28% 9.68% Nutrition N 0 5 5 % 0.00% 8.77% 5.38% Political Science N 0 2 2 % 0.00% 3.51% 2.15% Psychology N 1 1 2 % 2.78% 1.75% 2.15% Social Work N 0 1 1 % 0.00% 1.75% 1.08% Total N 36 57 93 % 100.00% 100.00% 100.00% University Scholarship Program Formative Evaluation – September 2015 97 Majors at LAU/USP V Sex Total Male Female Business Administration N 1 7 8 % 4.00% 20.59% 13.56% Chemistry N 2 5 7 % 8.00% 14.71% 11.86% Civil Engineering N 1 0 1 % 4.00% 0.00% 1.69% Communication Arts N 1 2 3 % 4.00% 5.88% 5.08% Computer Engineering N 1 0 1 % 4.00% 0.00% 1.69% Computer Sciences N 6 1 7 % 24.00% 2.94% 11.86% Economics N 1 0 1 % 4.00% 0.00% 1.69% Electrical Engineering N 2 0 2 % 8.00% 0.00% 3.39% Graphic Design N 0 1 1 % 0.00% 2.94% 1.69% Hospitality Management N 1 1 2 % 4.00% 2.94% 3.39% Mathematics N 7 3 10 % 28.00% 8.82% 16.95% Nursing N 1 5 6 % 4.00% 14.71% 10.17% Nutrition N 0 4 4 % 0.00% 11.76% 6.78% Political Science N 1 3 4 % 4.00% 8.82% 6.78% Social Work N 0 2 2 % 0.00% 5.88% 3.39% Total N 25 34 59 % 100.00% 100.00% 100.00% University Scholarship Program Formative Evaluation – September 2015 98 ANNEX 9: MEPI AND USP BUDGET BREAKDOWN Federal Cost University Cost Share Category % of total amount Federal Cost University Cost Share Category % of total amount Federal Cost University Cost Share Category % of total amount Federal Cost University Cost Share Category % of total amount Personnel and Fringe Benefit $ 151,520.00 $ 85,735.00 6.46% $ 510,736.00 $ 281,771.00 19.11% $ 450,914.00 $ 315,111.00 17.79% $398,400.00 $278,574.00 16.55% Travel $ 117,942.00 $ - 3.21% $ 91,306.00 $ - 2.20% $ 117,250.00 $ - 2.72% $106,660.00 $3,960.00 2.70% Equipment $ - $ - 0.00% $ - $ - 0.00% 0.00% 0.00% Supplies $ 74,000.00 $ 20,000.00 2.56% $ 54,600.00 $ 21,300.00 1.83% $ 74,558.00 $ - 1.73% $63,705.00 $0.00 1.56% Contractual $ 27,102.00 $ 6,000.00 0.90% $ 50,252.00 $ 7,000.00 1.38% $ 39,594.00 $ - 0.92% $34,128.00 $0.00 0.83% Other Direct Costs (including educational and student support costs) $ 2,508,220.00 $ 490,125.00 81.69% $ 2,084,517.00 $ 412,092.00 60.19% $ 2,317,684.00 $ 601,068.00 67.79% $2,597,107.00 $270,977.00 70.13% Indirect Costs $ 121,216.00 $ 68,588.00 5.17% $ 408,589.00 $ 225,417.00 15.29% $ 389,456.50 9.05% $0.00 $336,338.00 8.22% Total Costs $ 3,000,000.00 $ 670,448.00 100.00% $ 3,200,000.00 $ 947,580.00 100.00% $ 3,000,000.00 $ 1,305,635.50 100.00% $3,200,000.00 $889,849.00 100.00% % University total cost-share MEPI 2 LAU 18.27% 22.85% 30.32% 21.76% MEPI 1 LAU MEPI 1 AUB MEPI 2 AUB USAID Cost University Cost Share Category % of total amount USAID Cost University Cost Share Category % of total amount USAID Cost University Cost Share Category % of total amount USAID Cost University Cost Share Category % of total amount Personnel and Fringe Benefit $ 454,575.00 $ 654,272.00 7.74% $ - $ 591,363.00 8.06% $ 330,025.06 $ 618,199.73 11.66% $ 330,268.00 $ 576,293.00 10.75% Travel $ 13,000.00 $ - 0.09% $ - $ 13,200.00 0.18% 0.00% $ 194,254.00 2.30% Equipment $ 85,000.00 $ 17,500.00 0.72% $ - $ 72,100.00 0.98% 0.00% 0.00% Supplies $ 197,400.00 $ 36,242.00 1.63% $ 130,800.00 $ 36,220.00 2.28% $ 188,519.07 2.32% 0.00% Contractual $ 360,280.00 $ 2,028,500.00 16.68% 0.00% $ 94,435.83 1.16% $ 76,443.00 0.91% Other Direct Costs (including educational and student support costs) $ 8,093,460.00 $ 1,545,358.00 67.32% $ 5,000,997.00 $ 1,048,654.00 82.46% $ 5,108,370.46 $ 1,364,493.25 79.57% $ 5,335,179.00 $ 1,505,362.00 81.09% Indirect Costs $ 340,932.00 $ 490,704.00 5.81% $ - $ 443,522.00 6.05% $ 430,756.01 5.30% $ 418,258.00 4.96% Total Costs $ 9,544,647.00 $ 4,772,576.00 100.00% $ 5,131,797.00 $ 2,205,059.00 100.00% $ 5,721,350.42 $ 2,413,448.99 100.00% $ 5,936,144.00 $ 2,499,913.00 100.00% % University total cost-share USP3 LAU USP 5 LAU AUB 4 USP 5 AUB 33.33% 30.05% 29.67% 29.63% For more information, please visit http://www.socialimpact.com Social Impact, Inc. 2300 Clarendon Boulevard Arlington, VA 22201 Suite 1000 Tel: (703) 465-1884 Fax: (703) 465-1888 www.socialimpact.com