November 30, 2008 This publication was produced for review by the United States Agency for International Development. It was prepared under the Global Evaluation and Marketing II BPA Number EDH-E-00-08-00003-00. It was prepared by the Aguirre Division of JBS International, Inc. and its authors are Gerald Wein and Sarah Auten. Evaluation of the Clinton and Master’s Degree and Presidential Scholarship Programs and the Palestinian Faculty Development Program Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. i TABLE OF CONTENTS Acronym List ............................................................................................................... i Executive Summary ................................................................................................... ii I. Introduction ............................................................................................................1 II. Methodology..........................................................................................................1 A. Demographics .....................................................................................................3 III. Component 1: The Clinton and the Master’s Degree Scholarship Programs and the Presidential Scholarship Program .............................................................4 A. Pre-departure Activities.......................................................................................4 B. Participants’ Experiences in the United States...................................................7 C. Participants’ Experiences upon Returning to Palestine......................................9 D. Conclusions.......................................................................................................21 E. Recommendations ............................................................................................23 IV. Component 2: the Palestinian Faculty Development Program (PFDP) ........24 A. The Ph.D. Fellows Program..............................................................................25 1. Ph.D. Pre-program Activities......................................................................25 2. Ph.D. Participants’ Experience in the United States..................................29 3. Ph.D. Participants Impact on Higher Education.........................................36 B. The Short-term Fellows Program......................................................................41 C. Conclusions on the Short-term Fellows Program.............................................44 D. Recommendations on the Short-term Fellows Program ..................................44 APPENDICES Appendix A Instruments Appendix B Contacts List Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. i ACRONYM LIST AED Academy for Educational Development AMIDEAST America-Mideast Educational and Training Services, Inc. ATC Anti-terrorism Certification CSP Clinton Scholarship Program GEM II Global Evaluation and Monitoring II GTD Global Training for Development HESI Higher Education Support Initiative IQC Indefinite Quantity Contract JBS/A Aguirre Division of JBS International, Inc. MDSP Master’s Degree Scholarship Program NGO Non-governmental Organization OSI Open Society Institute PA Palestinian Authority PFDP Palestinian Faculty Development Program PSP Presidential Scholarship Program RFTOP Request for Task Order Proposal RFQ Request for Quotations SPSS Statistics Program for the Social Sciences START Strategic Technical Assistance for Results with Training (IQC) TOEFL Test of English as a Foreign Language USAID U.S. Agency for International Development USG United States Government WBG West Bank and Gaza Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. ii Executive Summary This study reviews several USAID/West Bank and Gaza (USAID/WBG) programs under which Palestinians were sent to the United States for graduate study. The first of these programs included the Clinton Scholarship Program (CSP) – renamed the Master’s Degree Scholarship Program (MDSP) – and the Presidential Scholarship Program (PSP). These two human resource development programs provided Master’s degree training in the United States to 158 Palestinians to relieve shortages of well-trained personnel in the economic and social sectors. These programs together cost USAID approximately $18.8 million. The focus of this evaluation is an examination of the impact of these programs on participants and, to the extent possible, on Palestinian society and to identify lessons learned that could be applied to future programming. The second part of this evaluation examines two elements of USAID/WBG’s on-going Palestinian Faculty Development Program (PFDP). PFDP aims to improve higher education in Palestine; two components being examined in this study are Ph.D. training in the United States for 34 Palestinians and short-term U.S. training for eight faculty and administrators from Palestinian universities. As these programs are on-going, the focus of this part of the evaluation is to assess progress to date and to identify any midterm program improvements. Component 1: Clinton/Master’s Degree and Presidential Scholarship Program Initiated in 2000, CSP sought to contribute to Palestinian economic and social development by providing Master’s degree training to promising Palestinians to address the shortage of qualified managers in the business, public administration, health, urban planning, law and engineering sectors. It was renamed the Master’s Degree Scholarship Program in 2003; however, the recruiting practices, goals and students supported under the activity remained the same through the duration of the contract. Altogether, the CSP/MDSP provided scholarships to 94 participants, of which 89 successfully completed their Master’s degrees. In 2003, USAID/WBG initiated PSP as a follow-on program, which continued until 2008. PSP redefined the fields of study somewhat, adding scholarships in the IT, environmental sciences and education sectors. In total, 69 Palestinians successfully completed their Master’s degrees through PSP, bringing the total output for the two programs to 158 Master’s degree recipients. Although the recruitment process placed an emphasis on recruiting women, the Master’s degree recipients included 36 percent women and 64 percent men. Recruitment of women posed a challenge given Palestinian societal reservations about women traveling and living alone. A number of women who had applied and been selected eventually withdrew due to family concerns. Impact on the Participants’ Knowledge and Skills All 73 Master’s degree participants who responded to this evaluation reported that their training greatly increased their job-related skills. Seventy-seven percent indicated that their technical and professional knowledge had greatly increased, and 69 percent indicated that they had developed new skills and habits that would allow them to keep abreast of changes in their fields. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. iii Participants also indicated development of specific professional and soft skills such as communications, self-reliance, problem solving, working with peoples of diverse cultures and time management in the workplace. Many of these are characteristics of successful leaders, and some are critical to developing a next generation of Palestinian leaders. Several respondents mentioned having had constructive dialogue with Jews in the United States, something that they had not experienced in Palestine. Generally speaking, Palestinian culture is relatively homogeneous; thus, for most participants, having an opportunity to interact with people of diverse cultures in the United States was a new and sometimes challenging experience. These experiences made them more tolerant of others’ differences and helped to develop their own interpersonal skills. Ninety￾six percent of participants held more favorable views towards the American people as a result of their experiences in the United States and 30 percent had a more favorable view of the U.S. government. The program offered four-month internship opportunities after participants completed their degree programs for practical training in their fields of study. Those who had done internships found them helpful in supporting their technical and professional knowledge – most especially in allowing them to experience U.S.-style work environments. Many of those who did not participate in internships expressed regret that they did not have that experience, especially those who had limited work experience prior to their participation in the program. Impact on Participants’ Careers Data from USAID suggest that 84 percent of participants returned to Palestine to complete their two years’ required service, and 80 percent of participants contacted by the team currently reside in Palestine. Although the data do not represent the entire universe of participants, they suggest quite a respectable percentage of participants are living and working in Palestine, contributing to the social and economic progress. The evaluation provides considerable evidence that program participants’ study in the United States contributed to their subsequent professional development. In response to an evaluation team survey of returned participants, 92 percent of respondents said that the opportunity to study in the United States enabled them to find better employment in their field or a related field. Ninety-five percent reported either substantially or somewhat increased levels of responsibility and 94 percent reported substantially or somewhat increased levels of pay compared to the position they held before the program. Respondents overwhelmingly felt that having earned their Master’s degree in the United States enhanced their professional competitiveness over those who studied elsewhere: 90 percent felt their U.S. degree was more advantageous than a Palestinian Master’s degree and 89 percent felt their U.S. degree was more advantageous than a degree from another country in the Middle East region. However, in the politically and economically troubled Palestinian environment, earning a Master’s degree from the United States has not been a guarantee of full employment. Many returning participants reported initial difficulty finding jobs, especially in particularly troubled political and economic years such as 2005. At the time of the study, ten survey respondents (14%) were unemployed, of whom seven were seeking employment and three were not currently looking for work. Analysis revealed no apparent correlation between Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. iv length of elapsed time since graduation and employment status. However, a higher percentage of women than men were unemployed. Of those who were unemployed, there was little reported difference regarding a desire to work, which would indicate that women’s higher unemployment rate is not due to lower motivation to work outside the home. Overall, unemployment rates among former participants appear to be lower than for Palestinian workers in general. Impact on Palestinian Development With the program only recently concluded, societal impact has not yet been fully realized. Even in longer-term analyses, measuring the societal impact of education based on a group of limited size is extremely difficult, as it is impossible to know with any degree of certainty what individuals might have achieved or not achieved in the absence of that education. However, in the case of the two USAID Master’s degree programs, the findings that a high percentage of participants are residing in Palestine and that participants are moving into positions of increased responsibility and volunteering in a variety of community activities suggest a positive impact on Palestinian economic and social development. There is also considerable evidence of a multiplier effect, as many participants expressed a sense of obligation to pass their expanded interpersonal and professional skills for the benefit of the broader Palestinian community. Respondents had shared the following skills and values with colleagues and friends: being independent, developing a positive attitude and trusting your abilities, being self-motivated and taking initiative, communication and public speaking skills, group work, household budgeting, professionalism, leadership, hard work, flexibility, honesty, adaptability, reconciliation and tolerance. Participants’ Leadership in their Local Communities Participants had undertaken leadership activities both in the workplace and through volunteering in their communities. One alumnus works for an NGO that develops monthly activities for 35 volunteers to benefit local communities. Another participant had volunteered to create the Clinton and Presidential Association with six other alumni. The main goal of this association was to initiate activities to serve their community and to maintain connections among all program participants in Gaza, as well as to exchange professional skills. Some 75 percent of respondents said they had learned about volunteer community service through their experiences in the United States and respondents offered a lengthy list of organizations with which they had volunteered. Participation in Alumni Activities About half of the survey respondents had participated in alumni activities. Of those who had not participated, lack of time and interest were the main impediments to participation. Several respondents commented that previous alumni events were primarily social in nature and thus did not offer sufficient incentive for attendance. Many respondents indicated that a more professionally-focused alumni network might be more appealing to them. As several participants expressed difficulty in readjusting to life in Palestine and a feeling of “being dropped” after the program ended, a professionally-focused alumni program that brought participants together for a speaker’s series or for job networking purposes might address issues of professional growth, unemployment and feelings of isolation simultaneously. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. v Component 1 Recommendations 1. Develop a follow-on program. This evaluation suggests that USAID’s Master’s degree scholarship programs developed participants’ professional capacity, increased contributions to Palestinian development through their employment and through volunteer activities, and enhanced their image of the United States. Although other opportunities for Master’s degree study exist in Palestine, those opportunities do not offer the same combination of skills development and changes in outlook and behavior that U.S. educational programs provide. In view of these factors, the evaluation team believes that a follow-on activity would be a good use of USAID resources. 2. Renew the U.S.-based orientation period before participants begin classes. In order to ensure that participants make the most of their educational experiences and come away with positive impressions of their time in the United States, participant training programs should allow participants a period of adjustment before they begin their classes. Similarly, assistance in finding housing would be desirable, as most students arrive at their U.S. locations with little knowledge of what areas are safe and convenient for public transportation. 3. Develop a professionally-based alumni program. The highly motivated and enthusiastic leaders who participate in this program appreciate opportunities for ongoing professional development. In addition, given the difficulty of finding employment in Palestine, this group could be developed to facilitate networking for job opportunities. Returned participants could be utilized as mentors for future participants and creative ways could be explored to engage groups such as the American Chamber of Commerce or American Business Association. 4. Develop more effective internship and entrepreneurship activities. Internship experiences were beneficial to those who participated, and many of those who had not participated expressed regret that they did not or could not take advantage of this opportunity. Any new program should attempt to assure such opportunities to all participants. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. vi Component 2: The Palestinian Faculty Development Program In contast to the Master’s degree programs that sought to upgrade human resources for roles across the economy, the Palestinian Faculty Development Program has an institutional development focus. It is designed to improve Palestinian universities by developing teaching methods in these universities and by increasing the pool of excellent professors and administrators. The program has ten components. This evaluation reviews only the two components that involve U.S. training, the Ph.D. Training Program and the Short-term Fellows Program. PFDP is a public-private partnership of USAID, AMIDEAST and the Open Society Institute (OSI). This cost-sharing arrangement has allowed each partner to leverage its resources, but has also limited the project’s scope and the involvement of Palestinian organizations working in higher education. The Ph.D. Program Overall, PFDP’s Ph.D. scholarship program is progressing well. Although the number of participants is a third less than the target, the 32 participants selected and sent to the United States have, with minor exceptions, been successful academically. This success attests to the care taken in selecting candidates. The Ph.D. studies are in public administration, education, political science, urban planning and social work. However, interviews with university administrators and professors revealed equally high interest in fields such as computer science, engineering, and nursing. The evaluation team believes that, from the perspective of maximizing Palestinian universities’ improvement, the impact of the program might have been greater if the technical focus of the Ph.D. element had incorporated some of these other disciplines. The limited number of eligible fields of study also had the effect of limiting the pool of qualified applicants and may thus have contributed to the shortfall in participants selected for the program. The project’s implementing partners managed the participant selection process very professionally and succeeded in identifying candidates capable of successfully completing Ph.D. programs in the United States. The participants represent a good geographic distribution (74% from the West Bank and 26% from Gaza), nine different Palestinian universities, and a reasonable gender distribution (35% women). However, the fact that some participants either had limited teaching experience or were not full-time professors when selected raises the prospect that those participants might not teach full time upon their return, which would reduce impact on their respective universities and on Palestinian development. Because of the USG’s “no contact” rule in place during the project’s early phase and the continuing “anti-terrorism certification” requirement, the roles that the Palestinian Ministry of Education and local universities would typically play in a USAID higher education development project have been much diminished. This fact has not been missed by Palestinian university administrators, who express frustration at the very limited role they have played. Similarly, both university administrators and the participants themselves are concerned about the limited contact that participants in the United States have with their Palestinian universities. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. vii Participants were placed in 16 U.S. universities, many of which enjoy widespread acclaim for their excellence. Nevertheless, the evaluation revealed that a number of participants felt that their Ph.D. programs do not match their needs as well as they expected. Those participants indicated that they had not been adequately involved in the placement process. Project implementers agree that the participants’ involvement in placement was minimal, despite repeated efforts to bring the participants into the process. Because the first year for any Ph.D. student is usually a “generic” one, the participants’ concern about this issue may diminish as they progress in their programs. Moving participants to another university at this time is probably neither desirable nor feasible. Implementing partners’ support for participants in the United States has been quite good. Participants report that OSI and AMIDEAST are responsive and do their best to find solutions to problems that arise. A few participants reported being very upset that they have not been able to bring their families, despite assurances that they would be able to do so. Not surprisingly, participants would like larger stipends, but the case for this was not argued too strongly. Most participants’ progress appears to be acceptable or better, although a small number are struggling with academic pressures. One participant has already successfully completed his Ph.D., returned home and is contributing to his university’s improvement. However, another participant failed his comprehensive exams, dropped out of the program, returned home and is currently unemployed. Almost all participants (96%) report having made significant or very large improvement in their knowledge and skills in their technical areas, improved their research ability (81%) and strengthened their study skills (78%). Participants also assessed their leadership skills as having improved substantially, with more than 70 percent judging their improvement to be substantial or very large. Seventy-three percent rated their problem-solving ability to have improved significantly or greatly; 72 percent had made significant or large improvements in problem-solving, and 84 percent made similar improvements in being self-reliant. There is also evidence that participants have become more favorably disposed toward Americans and toward the United States. Ninety-three percent indicated that their knowledge of U.S. democracy had increased moderately or substantially; 92 percent had similar increases in their knowledge of U.S. values and culture and 82 percent in their knowledge of U.S. religious and ethnic diversity. An increased understanding of the United States is an important “public diplomacy” benefit, particularly important in view of participants’ role in Palestine. The 32 participants still in the United States have ambitious plans for contributing to Palestinian development when they return home and can be expected, thanks to their training, to make a significant contribution to university development. The participants are aware of a number of obstacles they will confront upon returning home: the difficult and unstable political environment, universities’ severe financial constraints, and the impact of participants’ limited contact with their home universities at the present time. The latter constraint is one that project implementers could try to address now. As participants return to Palestine, they will hopefully become involved in PFDP’s Palestine￾based activities focused on improving teaching and learning. The evaluation team was impressed by AMIDEAST’s desire to “institutionalize” its various improved teaching and Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. viii learning activities within Palestinian universities so that efforts do not cease when the project ends. USAID and the implementing partners have made numerous mid-course corrections, adjusting to the political environment, changing program rules and reprogramming resources to increase program impact. Looking beyond the project, additional Ph.D. training clearly remains a priority for Palestinian universities. Recruitment of professors has become increasingly difficult, as older professors (many of whom were trained in earlier USAID programs) retire and others leave to earn better salaries either in international organizations, the private sector or other countries. Recommendations for the Ph.D. Program 1. Expand the role of Palestinian organizations within PFDP. The evaluation team recommends that the program creatively pursue Palestinian university involvement, as political constraints allow. Further, to the extent USG regulations permit, the team encourages USAID to support AMIDEAST’s efforts to “institutionalize” PFDP activities at Palestinian universities so that these efforts that not cease when the project ends. 2. Open communications regarding participants’ concerns about placement. The implementing partners should discuss placement problems with participants concerned about this issue. 3. Address the one or two remaining concerns over family travel. The team recommends that USAID inform any participants still waiting for news about their families’ travel of any problems that remain. 4. Utilize other PFDP activities to ensure returned participants’ continued engagement. As participants return to Palestine, AMIDEAST should make every effort to involve alumni in PFDP’s other activities focused on improving teaching and learning. 5. Consider reprogramming options that include additions to the current program. With under-spending in the scholarship components and four years remaining in the project, the team recommends that USAID and its partners consider additional reprogramming options to address the many unmet needs of Palestinian universities. The program might consider, for example, intensive state-of-the-art workshops run in the West Bank by U.S. universities to upgrade the technical/professional knowledge of professors in Palestinian universities. 6. Strengthen linkages among alumni and between alumni and the United States. USAID should explore whether an association of Palestinian graduates of U.S. universities might be an effective vehicle to help alumni keep abreast of developments in their fields while cementing relationships with the United States. 7. Consider providing additional Ph.D. scholarships as part of future university development programs. Providing additional Ph.D. training would clearly benefit Palestinian universities. However, USAID should consider such scholarships as part of a comprehensive university development program. If USAID determines that further Ph.D. training is a priority, the team encourages eligible disciplines be determined by the needs of the target universities. Any new university development program should, as the political situation permits, increase the involvement of Palestinian universities in planning and implementation. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. ix The Short-term Fellows Program The Short-term Fellows program provides experienced Palestinian academicians and administrators an opportunity to make two visits of up to four months each to U.S. university campuses where, as visiting faculty, they develop skills in teaching, research and administration. The program has thus far sent only eight participants to the United States; an additional ten scholars have been selected for early 2009. Interviews with five of the eight returned scholars indicated that their visits were highly productive. They developed courses and learned new teaching and research methods that they have put (or expect soon to put) into practice. Some have found that the benefits significantly exceeded expectations: one participant developed and signed an agreement establishing a faculty and student exchange with his U.S. host university. The most significant constraint on this program appears to be limited demand. This reflects initial limits on the technical disciplines of the program, the reluctance of university administrators to see important faculty members miss a semester of teaching, and the financial burden that participation placed on the participants and their families. With respect to the financial burden, all of the interviewed participants commented that the small PFDP stipend forced them to live like graduate students instead of like visiting faculty and in some cases to spend several thousand dollars of their own money. The implementing partners have attempted to address these issues. The technical areas have been greatly expanded and administrative staff has been added; AMIDEAST is attempting to coordinate more closely with Palestinian university administrators, and the financial support for participants has been increased by $500 per month. Recommendations for the Short-term Fellows Program 1. Continue efforts to implement this program. The team applauds the efforts of the implementing partners to step up the level of implementation of this program. The evaluation team believes that this program has demonstrated very positive benefits and encourages the partners to give it high priority. 2. Monitor performance to determine whether further adjustment in the financial stipend is advisable. The implementing partners and USAID should closely monitor the results of the Short-term Fellows traveling to the United States early in 2009 to see if the very positive results found by this evaluation are the norm. If a further increase to the participants’ financial stipend (e.g., an additional $500/month) would make the program more effective, that cost would be small relative to the potential gains. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 1 I. Introduction This document presents the results of work done by the Aguirre Division of JBS International (JBS/A) under the GEM II Indefinite Quantity Contract (IQC) to evaluate recent and on￾going USAID programs that provide graduate-level scholarships to Palestinians to study in the United States. This work was requested by the USAID Mission to the West Bank and Gaza (USAID/WBG). The evaluated programs are categorized into two components: • Component 1 includes two completed Master’s degree scholarship programs: the Clinton Scholarship Program (CSP), renamed the Master’s Degree Scholarship Program (MDSP), and the follow-on Presidential Scholarship Program (PSP). • Component 2 includes the U.S.-based Ph.D. and Short-term Fellow components of the on-going Palestinian Faculty Development Program (PFDP). The period of performance for this evaluation activity was between September 26 and November 30, 2008, with five working days spent in Washington D.C., and 28 working days in-country to collect and analyze data, submit a draft report and present key findings to USAID/WBG. The final report was submitted after the evaluation team returned to the United States. A. Goals of this Evaluation The goals of this evaluation are as follows: • To assess the impact of USAID West Bank and Gaza investment in U.S. long term training in meeting [a] shortage of professionals in [targeted] sectors; • To determine whether and to what extent U.S. academic training has directly contributed to the Palestinian society and economy; and • To provide recommendations for further programming in this area. II. Methodology This evaluation posed some methodological challenges because, while Component 1 was completed in early 2008, Component 2 is in progress and most scholars are still in the United States undertaking their programs of study. Therefore, the Component 1 evaluation measured impact and focused on participants’ experiences since returning to Palestine, and Component 2 was a mid-term evaluation concentrated more on implementation aspects and participants’ goals for the future. This evaluation utilized a mixed-method research design that gathered data with both quantitative and qualitative methods. Data were collected from a variety of stakeholder groups, including participants, program implementing staff and other stakeholders in both the West Bank and Gaza and in the United States. Data collection for the long-term participants in each component included an online, close-ended survey aimed at all long-term study participants and in-depth interviews of a randomly selected subset of participants. Two focus groups were also organized for a number of Component 1 survey respondents (see Appendix A for a copy of the evaluation instruments). Participants in the focus groups were Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 2 selected largely on the basis of their proximity to the sites chosen for those sessions. For the Short-term Scholars element of Component 2, the team relied on interviews with a sample (five of eight) of the returned scholars. Quantitative data were analyzed using SPSS and qualitative data were analyzed in MaxQDA. As is common in the evaluation of scholarship programs, the scholarship recipients are the principal source of information about the effectiveness of the program, its impact on their lives, and subsequent impact on society. For Component 1 of this study, JBS/A made two attempts to provide limited independent confirmation of participants’ assessments. First, the evaluation team attempted to identify and contact a group of Palestinian scholars who were initially accepted into the program but were unable to participate for a variety of reasons. This group was selected for their demographical similarity to program participants. The expectation was that these individuals could provide a point of comparison in regards to their career development to those who had studied in the United States. However, as discussed later in this report, the comparison group was limited in its usefulness in drawing conclusions about the impact of program participation. Second, the evaluation team sought to interview participants’ employers in an attempt to confirm reported performance changes. However, most participants did not offer the names of their supervisors, and of the six employers who were interviewed only one had worked with the participant prior to training. Thus, although this effort did provide limited confirmation of skills apparently attributable to U.S. training, the evaluation remains almost exclusively dependent on the self-reporting of participants. However, the evaluation team is convinced that the reporting of participants’ represented their best efforts to accurately portray the program and its impact upon them. Tables 2.1 and 2.2 itemize key stakeholder groups for each component, the survey methods implemented with each group, and the targeted and actual sample sizes for each stakeholder group (see Appendix B for a list of contacts). Table 2.1. Component 1 Sampling Strategy Stakeholder Method Target Actual Survey Universe (154) 75 started, 73 completed Interviews 25-30 25 (12 from West Bank, 13 from Gaza) Returned Scholars Focus Groups 2 groups of 7 participants 1 group of 9 (Gaza) 1 group of 3 (West Bank) Non-returned Scholars Interviews As available 1 Comparison Group Interviews 25 7 Employers Interview Selection 6 USAID Program Managers Interview 1 4 Implementing Partner (AED) Interview 1 1 Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 3 Table 2.2. Component 2 Sampling Strategy Stakeholder Method Target Actual Current Ph.D. Survey Universe (34) 27 Scholars Interviews 10-15 12 Returned Ph.D. Scholar Interviews 1 2 Short-term Fellows Interviews Not included in original proposal design 5 University officials Interviews Not included in original proposal design 3 Ministry of Education officials Interviews 1 0a USAID Program Managers Interviewees 1 4 Implementing organizations (OSI & AMIDEAST) Interview 2 3 a Although Ministry officials were listed as a stakeholder group in the original RFTOP, once in country, USAID advised the evaluation team that this group had limited involvement in the program due to political restrictions and would be unlikely to have much to contr bute to the evaluation effort. Given the condensed timeframe of the evaluation, the team elected to focus its efforts elsewhere. A. Demographics Of the 71 Component 1 respondents who identified their gender in the online survey, 45 are men (63%) and 26 are women (37%). Fifty-seven respondents (80%) were currently living in the West Bank and Gaza, and 68 respondents (96%) indicated that they had lived in Palestine since completing the program. Reasons for not living currently in Palestine included pursuing Ph.D. degrees abroad, the difficult political situation, and challenges in finding a job. All of those living abroad indicated intent to return to Palestine at some point in the future. In Component 2, 14 of 34 participants (34%) of the universe of Ph.D. participants are women. Of survey respondents, seven of 27 (27%) are female. Four of 12 interviewed participants (33%) are women. The two returned Ph.D. participants and five returned short￾term scholars that the team interviewed were men. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 4 III. Component 1: The Clinton and the Master’s Degree Scholarship Programs and the Presidential Scholarship Program The USAID scholarship program originally known as the Clinton Scholarship Fund Program (CSP) was developed in 2000 under the Global Training for Development (GTD) Indefinite Quantity Contract (IQC) and implemented by the Academy for Educational Development (AED). This program sought to contribute to Palestinian economic and social development by addressing the shortage of qualified managers in business, public administration, health, urban planning, law and engineering sectors. After the change in Administration in Washington, the CSP was renamed the Master’s Degree Scholarship Program (MDSP) in 2003. The recruiting practices, goals and students supported under the activity remained the same through the duration of the contract. When the GTD contract ended, a follow-on program was developed under the START IQC, and CSP became a component under the Higher Education Support Initiative (HESI) task order. The 80 students still studying under the CSP contract were administered under the HESI task order. In 2003, the Presidential Scholarship Program (PSP) (2003 – 2008) also began under START as a follow-on providing 70 additional scholarships; this program was also implemented by AED. The follow-on activity expanded the fields to include Master’s degree scholarships in the IT, environmental sciences and education sectors. For purposes of this report, the CSP/MDSP and PSP programs will be combined under the title Component 1, unless specific differences between the CSP/MDSP and PSP programs are being addressed, in which case the particular program will be identified. A. Pre-departure Activities 1. Participant Selection and Placement To recruit students to the program, AED distributed brochures and advertised through local media outlets to inform potential candidates of the opportunity. Many people also learned about the program through word-of-mouth. Applications were reviewed for completeness and for consistency with the program’s standards and goals. Those who met this standard underwent two rounds of interviews. The applications of those selected were passed to USAID/Washington to be vetted and approved. The recruitment process placed an emphasis on recruiting women; however, recruiting Palestinian women for an overseas exchange program posed a challenge given Palestinian societal reservations about women traveling and living alone. To respond to that concern, the implementing organization made an effort to place female students near U.S.-based family members whenever possible. Nevertheless, the program experienced difficulty recruiting women, and some who were selected to participate withdrew at the last minute due to family concerns. The female participation rate was 30 percent in the CSP/MDSP and rose to 43 percent in the PSP. The implementing partner’s Washington office placed participants in appropriate U.S. colleges and universities based on participants’ professional and educational backgrounds, GRE/GMAT and TOEFL test scores and other factors. Several students who had placement issues were transferred to programs better suited to their needs. The first group of CSP students arrived in Washington, D.C. for orientation in August 2000. They subsequently Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 5 proceeded to their respective assigned institutions and began classes in September of that year. 2. Financial Arrangements The total costs of the CSP/MDSP and PSP to USAID was approximately $18.8 million ($9.0 million for the CSP/MSP and $9.8 million for the PSP). As the programs supported 158 Master’s degree students (of which 154 completed the program), the average USAID cost was approximately $120,000 per completed graduate. Program funds paid for GRE/GMAT and TOEFL testing under CSP/MDSP. Under PSP, the participant was asked to pay 25 percent of the testing costs. The program covered almost all expenses in the United States, including a housing and maintenance stipend, a settling-in and book allowance, a computer, and an allowance for a return ticket home once during their two￾year stay. 3. Participants’ Motivation for Applying The online survey for returned CSP/PSP participants asked respondents to summarize their original motivations for applying to the program. Out of 69 responses, the most frequently￾mentioned goals were general professional development and general educational goals. Specific goals regarding Palestinian development activities were the third most-commonly stated objective, followed by specific interest in receiving their Master’s degree from a U.S. university (exclusive of those who mentioned a general interest in receiving a Master’s degree without regard to location or school). Some respondents indicated a desire to experience U.S. culture and life in the United States, and personal development goals. Most respondents were generally happy with their progress since returning to Palestine, but a few expressed frustrations regarding the extent to which they had been able to implement their goals with respect to community development and leadership. In most cases, this was attributed to ongoing political difficulties and their impact on living and working conditions in Palestine and most especially in Gaza. 4. Participant Orientation Activities Under CSP, the implementing partner organization had planned to conduct English language training and cultural information sessions at Birzeit University, near Ramallah. However, the Second Intifada and subsequent restrictions imposed by Israel at that time made access to the university next to impossible for most participants, and it was easier and more cost￾effective to gather students together for training in the United States. Thus, although some I thought this program is designed for me … I can make a change to myself, family, friends, community, country. I knew that by getting this training in the USA I will able to achieve many goals that will lead to peace and prosperity for Palestinians and Israelis alike. I know this may sound not very true at this stage, but I believe this will come [with the] right young leaders. My goal was to develop my career, develop my skills, and become a community leader … I did develop my career and skills, but I have not yet become a leader … there are not many opportunities here in Gaza. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 6 limited orientation was conducted in Palestine before participants departed for the United States, the program conducted English and math training, test preparation (GRE/GMAT and TOEFL) and graduate school study habits courses at the Economics Institute in Colorado. The pre-academic training in the United States was typically two to three months in duration but sometimes extended for as long as six months for those needing more English language training. By the time PSP was initiated, the political climate had improved to the point where pre￾academic training could be conducted in Palestine utilizing online and regular face-to-face formats. The length of the pre-academic training for PSP varied according to the needs of the participant in terms of language ability and in the length of time it took for the participant to obtain a visa for the United States. This training was not full-time, except for one intensive four- to six-week training session in the West Bank. Otherwise, participants kept their jobs and studied in their free time. Program alumni gave the orientation and preparatory activities somewhat mixed reviews. In the evaluation’s online survey, nearly 70 percent of respondents rated the information and preparatory activities as excellent or very good. When responses were disaggregated by program, more CSP scholars were highly satisfied with the training program than PSP scholars – although not overwhelmingly so (77% of CSP scholars felt the training was excellent or very good versus 64% of PSP scholars). Table 3.1 details participants’ responses by program. Table 3.1. Participants’ Assessment of Pre-program Orientation Activities Excellent Very Good Fairly Good Not Very Good Poor n Clinton/Master's Degree Scholars 29.4% 47.1% 17.6% 0.0% 5.9% 34 Presidential Scholars 15.4% 48.7% 28.2% 7.7% 0.0% 39 Total 21.9% 47.9% 23.3% 4.1% 2.7% 73 The CSP participants’ higher ratings of the orientation may well be attributable to their longer preparation period in the United States. For PSP students, the dual pressures of starting a new academic program while simultaneously adjusting to life in the United States were extremely challenging. PSP scholars frequently suggested that a period of U.S. orientation of at least one month would have been beneficial for acclimation and organization of life in the United States before classes began. Other changes recommended by respondents included: more involvement in and information regarding their placement into educational programs that suited their needs; a longer period of English-language training; more information about localities (climate, society, living conditions, perceived “prestige” of one U.S. location versus another); and increased social support (e.g., assistance finding housing, paying bills, finding a locksmith, getting a cell phone plan, taking care of parking tickets). The issue of housing assistance was especially concerning for female participants, most of whom had never lived alone before. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 7 B. Participants’ Experiences in the United States 1. Placement of Participants in Academic Programs The implementing partner was quite successful in placing participants into appropriate academic programs: 80 percent of the online survey respondents rated their university program’s match with their professional interests as excellent or very good. One exception involved a student who had been placed in a non-accredited program, which caused him a good deal of difficulty and frustration before he was moved to a more appropriate program. A number of the interview respondents offered enthusiastic comments about their U.S. alma maters. They mentioned, for example, top-20 national academic rankings of their institutions or programs, the breadth and depth of their professors’ knowledge, diverse student bodies and the wide array of supplementary activities and opportunities within their communities to complement their classroom experiences. 2. Internships and Entrepreneurship Activities In order to give participants an opportunity to apply the skills they had learned, the program offered four-month internships to participants prior to their return to Palestine. Almost half (46%) of the survey respondents participated in some form of internship during their time in the United States. Most of these were shorter than six months, but several respondents had engaged in internships lasting six to eleven months, and one participant had done a year-long internship. (Program guidelines dictated only four months of post-graduation internship time.) Participants who had completed internships found them helpful in supporting their technical and professional knowledge. Eighty-eight percent felt that the experience was extremely or somewhat effective in supporting their technical and professional knowledge, and 85 percent felt it was extremely or somewhat effective in helping them to acquire increased management and organizational skills. Several respondents mentioned specifically that participating in the internship had allowed them to observe U.S.-style work environments. Responses are detailed in Table 3.2. Table 3.2. Participants’ Assessment of Internships’ Professional Development Capacity How useful was your internship in the following: Extremely Effective Somewhat Effective Not Very Effective Totally Ineffective Don’t Know / Not Applicable Supporting technical and professional knowledge 39.4% 48.5% 12.1% 0% 0% Acquiring increased management and organizational skills 36.4% 48.5% 12.1% 3.0% 0% n = 33 When first arriving to the U.S., it was the first time for me and for many other girls to be there. I had to go to homes and places alone searching for a place in a totally new country. I think what I did was really not safe since I was not aware of the safe and the unsafe areas. Fortunately, after that, I got help from a relative of mine and found a good place. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 8 Many of those who did not participate in internships expressed regret that they did not have this experience, especially those who had limited work experience prior to their participation in the program. Respondents who did not participate in internships cited several reasons for not having done so: some were anxious to return to Palestine, while others were not aware of the internship opportunity or could not find an internship opportunity that would accept them for such a short timeframe. Many respondents commented that four months was too short for an effective internship or that employers would not agree to hire them for only four months. The program had planned to offer returned participants grants of up to $5,000 to launch promising new initiatives that capitalized on their educational experiences. However, changes in the Palestinian political environment resulted in most of the proposed projects being rejected: according to one estimate, out of 30 or 40 applications, only about five could be approved. Program alumni, particularly those who proposed (sometimes multiple) projects that were rejected were critical of the program’s failure to implement this program component. For example, one respondent first proposed a project working with municipalities on water issues, but USAID could not approve this project because of restricted ability to work with municipalities. He then proposed a project to produce clean environmental waste, but that project was rejected because it had the potential to violate the restriction against support for profit-making activities. He eventually developed a small council in his home village to provide management support for environmental issues. He found the entire process was frustrating and difficult. A less-motivated candidate might have given up without developing an acceptable project. 3. Personal and Professional Linkages Eighty-five percent of online survey respondents reported that they have stayed in contact with individuals they met in the United States. Most often, these connections were social, although many alumni had also maintained professional connections with professors and/or fellow students. Those connections were typically with a mix of Americans and non￾Americans. The nature and durability of these continuing connections varied considerably. Some alumni communicated with professors solely for letters of recommendation or on a social basis, while others continued to utilize their academic networks to obtain job-related advice, to request documents or updates on developments in their professional field, to seek advice on equipment or standards, or to obtain input on the development of projects. Several respondents commented that they are still active in their university’s alumni network and receive daily or monthly updates, listings of job opportunities in the U.S., and notifications of conferences and workshops. More often, interview respondents talked about the social connections they had made with faculty, fellow students and community members. Many of them said they exchange greetings during holidays or maintain regular communications through email or Facebook. Some of these connections were very strong: one respondent noted that she and her American friends make a special effort to meet in a mutually-accessible location like Cyprus or Europe whenever possible. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 9 C. Participants’ Experiences upon Returning to Palestine 1. Increased Job Opportunities with U.S. Master’s Degrees Data collected by the evaluation team suggest that returned participants are typically employed in positions of responsibility, serving as managers, policy and business advisors, project coordinators and managers, financial managers, etc. Often these are positions are with important companies or non-profit organizations. The following list exemplifies jobs that program alumni are filling: • Executive Manager, Palestinian Investment Company (Pal Invest); • Policy Manager, Ministry of Planning and Foreign Cooperation; • Chief of Electoral Affairs, Central Election Commission; • Instructor/Assistant Dean, Birzeit University; • Financial Manager, USAID; • Director, Local Development NGO; • General Manager, Watanieh Securities; • Project Coordinator, Water and Environmental Development Organization; • Financial Manager, Palestinian Investment Fund; and • Strategic Planner, Municipality Development and Lending Fund. Participants’ level of professional responsibility and direct contributions to Palestinian development are illustrated by this list, with alumni contributing to social and economic development through financial institutions, foreign assistance organizations, universities, and local NGOs. The team’s survey revealed considerable evidence that returned participants’ studies in the United States contributed to their subsequent professional development. Ninety-two percent of respondents said that the opportunity to study in the United States enabled them to find better employment in their field or a related field. Ninety-five percent reported either substantially or somewhat increased levels of responsibility and 94 percent reported substantially or somewhat increased levels of pay compared to the position they held before the program. Most participants who responded to the onlne survey (65%) have changed employers since undertaking their U.S. study. Of those who have returned to the same employer, most moved to positions of greater responsibility, salary and prestige. Respondents were also asked to indicate to what extent they felt that having earned their Master’s degree in the United States provided better professional opportunities than having earned their graduate degrees from a university in Palestine and the greater Middle East region. Overwhelmingly, the respondents felt that the U.S. degree had given them an advantage over local graduates: 90 percent felt their U.S. degree was more advantageous than a Palestinian Master’s degree and 89 percent felt their U.S. degree was more advantageous than a degree from another country in the Middle East region. Several respondents to the in-depth interviews had actually experienced this difference first￾hand, having earned a graduate degree from a local or regional university prior to their participation in this program. Those who had this expanded perspective were able to speak directly to the differences between the programs, and they all felt that having the additional After I participated in the program, I had a wider choice of work fields and I had more experience [to be able] to work with NGOs. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 10 cultural background and expanded life skills that their U.S. experience provided had made a significant difference in their professional development. On the other hand, returning participants, especially those returning during periods of more marked political instability, found considerable difficulty obtaining suitable jobs and some reported many months of unemployment. Ten of the online survey respondents (14%) indicated that they are currently unemployed. Of those, seven respondents are currently seeking work. Of those who were currently unemployed, all had been employed at some point since returning to Palestine and one of those is currently living in the United States. Of the six returned participants who are unemployed in Palestine, three are from Gaza, two are from the West Bank and one did not share identifying information. However, the 14 percent unemployed rate amongst survey respondents is better than the ILO reported 2007 unemployment rate of 21.3 percent for adults aged 10 or over.1 Although the figures are not directly comparable given the likelihood that the program alumni are more highly educated than the average Palestinian, it is a useful point of reference. In comparison, relatively few participants are unemployed, which suggests that the demand for program graduates is high. Unemployment for program alumni varies by sector. Table 3.3 below shows the gender, location and discipline studied by the unemployed participants. The largest number of unemployed former participants studied business. However, as more participants studied business than any other discipline, the rate of unemployed graduates (14 %) is only equal to the average of all participants. Anecdotally, some respondents expressed the opinion that MBAs are not in demand in Palestine. Table 3.3. Not Employed but Seeking Employment Gender Location Field of Study Female United States Geographic Information Sciences Female West Bank Business Administration Male Gaza Environmental Engineering Female Gaza Public Administration Female West Bank Business Administration Male Gaza Business Administration As is common in any modern economy, some former participants are working in sectors other than the one in which they studied. For example, the team heard that a few environmental engineers who were not able to find work in their sector moved into urban 1 ILO LABORSTA Online Database. Unemployment is defined as all persons above the age of 10 who during the reference period (2007) were: "without work," i.e. were not in paid employment or self-employment; "currently available for work", i.e. were available for paid employment or self-employment during the reference period; and "seeking work", i.e. had taken specific steps in a specified reference period to seek paid employment or self-employment. http://laborsta.ilo.org/cgi￾bin/brokerv8.exe, Accessed Nov. 3, 2008 Before the program, I was a government employee, now I work in the private sector. I was an employee, now I am a senior manager … it’s a 180 degree change. I already had an MBA from [a local] university, but this degree didn’t make any difference because it’s from here. They taught us the same basic curriculum that they teach in the U.S. but they don’t get the opportunity to see a different perspective, a different culture. The Master’s isn’t just the curriculum; it’s also the different people you meet, the different cultures, the different people. It’s a package. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 11 planning activities. However, as one respondent put it: “I wanted a job in [my field] but you need to go with the flow and see what the market needs. My education has still helped me a lot.” Table 3.4 shows that Health graduates seem to be more in demand and Public Administration graduates not as sought-after. However, as findings were not statistically significant, they may reflect these individuals’ experiences rather than an overall trend. Table 3.4. Participants’ Employment Status by Area of Study Are you presently employed? Yes No Total Percentage of Respondents who Studied this Field Business 19 3 22 30.1% Public Administration 2 2 4 5.5% Health 10 0 10 13.7% Urban Planning 2 0 2 2.7% Law 2 0 2 2.7% Engineering 7 1 8 11.0% Computers and Internet Technology 8 1 9 12.3% Environmental Sciences 4 2 6 8.2% Education 1 0 1 1.4% Field of Study Other 8 1 9 12.3% 63 10 Total 86.3% 13.7% 73 100% In an effort to identify other factors that may influence employment rates, participants’ employment status was compared to year of program completion, to determine if those who were unemployed were more recent graduates who had not yet had sufficient time to leverage their degrees into gainful employment. This analysis showed no apparent correlation between length of elapsed time since graduation and employment status. Finally, participants’ employment status and desired employment status was examined in relation to gender. Outcomes were not statistically significant, but overall a higher percentage of women were unemployed than men (19% of female respondents versus 7% of men). Of those who were unemployed, there was little reported difference regarding a desire to work, which would indicate that women’s higher unemployment rate is not due to lower motivation to work outside the home. Table 3.5 details these findings. Table 3.5. Unemployment Status by Gender Women Men Total % 25% 7% 14% Unemployed n 7 3 10 Currently Seeking % 72% 67% 70% Work n 5 2 7 Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 12 Anecdotally, several of the female interview respondents mentioned that they are undertaking dual roles as both mothers and wage earners. Although it was not clear to what extent economic interests factored into this decision, there was a clear interest on the part of these respondents to use their professional skills in the workplace, suggesting that their professional lives were equally important to their sense of identity as were their familial responsibilities. Female role models in the workforce are important to any future development in Palestine. From the perspective of the program’s impact on Palestinian society and the economy, a potentially greater concern than unemployment is that because of the region’s instability and lucrative opportunities in the Gulf and elsewhere, a significant number of former participants may have left Palestine. For any former participants motivated to seek employment elsewhere, having a Master’s degree from the United States enhances their prospects for securing a position. However, as far as the evaluation team can determine, this does not appear to have been a large problem. USAID data suggest that 81 of 89 CSP/MSP participants (91%) returned to the region, and 58 of 69 PSP participants (84%) returned and completed their obligation of working a minimum of two years in the region.2 This suggests an overall return rate of 88 percent. Impact, of course, is determined not only by whether participants returned but by whether they remained after the minimum required period following training. Comprehensive data on this question are not available because contact has been lost with a considerable number of participants. However, of the 73 former participants who responded to the team’s survey, 57 (80 %) were living in Palestine; although the team cannot isolate the societal impact of the returned participants, the fact that most participants are working in the region and that they have moved to positions of greater responsibility indicates that such an impact is reasonable to expect. This impact will be enhanced if the political and economic environment improves, allowing participants to realize their potential. 2. Changes in Participants’ Professional Development Apart from earning the actual Master’s diploma, the benefit of program participation that respondents mentioned most frequently was development of both specific professional skills and soft skills such as communications, self-reliance, problem solving, working with people of diverse cultures and time management in the workplace. Respondents to the online survey were asked to characterize their development in technical and professional knowledge, ability to conduct scholarly research, and ability to maintain professional skills over time. All of the respondents said their technical and professional knowledge had increased either greatly or somewhat; 95 percent said their research skills had increased greatly or somewhat, and 97 percent felt that they were better able to keep themselves up-to-date on innovations in their field. Overall, these responses point to the development of a more highly skilled and informed workforce on the part of the 154 participants trained under this activity. Table 3.6 details respondents’ answers. 2 These data were provided to the evaluation team in the scope of work that USAID/WBG prepared. I always had an ambition to assist my people to have better lives, especially the young … Having my Master’s degree [has] allowed me to grow career-wise and be able to be part of decision making and in implementing programs that I see needed for this country. I have also been an example for other women and in some way or another I have empowered them. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 13 Table 3.6. CSP/PSP Scholars’ Studies’ Affect on Knowledge and Abilities Greatly Increased Somewhat Increased Did Not Change Don’t Know/ Not Applicable Technical and professional knowledge 76.7% 23.3% 0% 0% Ability to do high quality, professional research 54.8% 39.7% 4.1% 1.4% Development of new skills and habits that keep scholars up-to-date on innovations in their field 68.5% 30.1% 1.4% 0% n = 73 Interview respondents offered insight into how the skills they learned in the United States have been applied in their professional lives, changing their work habits in Palestine and their expectations for their colleagues’ performance as well. One respondent observed that whereas it is unusual to meet deadlines in Palestine, his work ethic has changed – he said he “scared” his boss when he started at his current job because he approached things differently than his colleagues. His time management skills were better and he was not so focused on being perfect on the first try; he learned to do a first draft and then go back and revise later or even in collaboration with others. This sharing of innovative work styles represents the kind of change that will truly impact Palestinian development in the long run. In order to gain a broader perspective on the extent to which participants’ training has affected their performance in the workplace, the evaluation team sought to interview participants’ employers.3 As a result of this effort, the team conducted six telephone interviews with participants’ employers. These employers gave the participants very good reviews in terms of organization, efficiency, language skills and leadership qualities. These assessments of the former participants’ skills and abilities are consistent with the participants’ self-assessments. However, as only one of the employers interviewed had worked with the participant before their visit to the U.S., they had little perspective as to how these qualities might have been enhanced by the program. Regarding soft skills, most respondents were pleasantly surprised to find how much they had matured and developed over the course of the program. Studying in U.S. universities, completing internships in U.S. businesses and interacting with a wide range of cultures and backgrounds both socially and professionally had affected change in participants’ views, as well as their professional skills. Former participants judged many of their required workplace skills to have improved markedly. Table 3.7 details survey responses assessing improvement in nine skill areas: problem solving, career advancement, self-reliance and independence, summarizing 3 The team asked participants being interviewed if they would be comfortable with the team interviewing their supervisors. If participants answered in the affirmative, they were asked to provide their supervisor’s name and contact information. The most important is not just the difference in the salary - there is a difference in the performance. I learned how to do the work. I have learned how to respect my clients, how to respect my employees … the personal reflects the professional and vice versa. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 14 complicated ideas, listening to others, critical thinking, tolerance of diversity, negotiating with colleagues and flexibility. Table 3.7. Most-improved Skills Very Large Or Significant Improvement In Skills N Problem-solving skills 91.8% 73 Knowing how to advance their career 91.8% 73 Being self-reliant and independent 90.4% 73 Summarizing complicated ideas 90.4% 73 Listening to others’ suggestions or concerns 89.0% 73 Expressing their ideas and feelings (critical thinking) 89.0% 73 Being tolerant of others’ differences 89.0% 73 Negotiating with colleagues 87.7% 73 Being flexible 87.5% 72 Many of these are characteristics of successful leaders, and some are critical to developing a next generation of Palestinian leaders who might be called upon to deal directly in resolving the Palestinian-Israeli conflict. It is worth noting that several interview respondents mentioned specific interactions with both pro- and non-Zionist Jews in the United States, and talked about these relationships very respectfully. 3. Independent Confirmation of Benefits In an effort to determine to what extent changes in career development and participation in volunteer activities can be attributed to the scholarship program, the evaluation team attempted to identify a group of Palestinians with similar demographic makeup to the participants, but who had not participated in this program. The implementing organization provided a list of 25 people who had applied to and been selected as participants for the CSP/PSP program but who were unable to participate due to a variety of reasons. As no contact information was available for these individuals, the evaluation team was able to locate and interview only seven members of this group. Although the sample was smaller than hoped for, the data do not indicate any marked difference between the participants and the comparison group. Of the seven interviewed, four had earned their Master’s degree through other channels – two in Britain through other scholarship opportunities, one from a regional university, and one from a university in Hebron. All except one was employed (the exception was not looking for work as she was a stay-at-home mother) and most worked in relatively senior positions. Three had volunteered in their communities and filled leadership roles. Table 3.8 details their educational and employment status, their level of career satisfaction and any community leadership activities they shared with the evaluation team. It made me look at people as equal, all humans, with no bias on color or religion. Palestine is very homogeneous, but now I can be with Jews, Muslims, Hindus, Buddhists … whatever. I respect people for who they are. I came to realize the difference between Jews and Israelis. Although I live under occupation, I made tons of Jewish friends in the States, some of whom were supportive of Palestinian rights. This experience gave me exposure to people of many different backgrounds. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 15 Table 3.8. Comparison Group Responses in Key Categories Degree Degree Location Job Status Career Satisfaction Community Leadership R1 MA in Literature England University in Gaza – previously supervised 5 employees Satisfied with career status Volunteers teaching English and translating books R2 MBA Hebron Executive manager at Palestinian company, supervises 22 employees Very satisfied with current job - administrative position allows him to meet new people internationally and to visit new countries. Volunteered as a leader in a center for peace research; represented the Center in Jordan, Ramallah, and Hebron. Also volunteered with youth organization in Bethlehem R3 MS in Translation London Lecturer in a university in Gaza Managed to improve professional and economic status Volunteered translating books for charitable societies. Also filled a leadership position at a local library supervising book selection process. R4 MBA Cairo Vice president for an import/export firm, supervises 10 employees Satisfied with current job R5 Had not yet earned Master’s degree -- Surgical department in a Gaza hospital Unsatisfied because did not continue higher education R6 Had not yet earned Master’s degree -- Housewife R7 Had not yet earned Master’s degree -- Vice president at a Palestinian university. Supervises 34 employees, member of the test preparing and interviewing committees. Not very satisfied, would like to obtain a Masters Degree. Although the comparison group was not dramatically different from the participant group, this does not indicate that the CSP/PSP program lacks impact. Rather, it may be a product of the selection process for the comparison group: the comparison group did not randomly sample from the entire age cohort or across a variety of educational cohorts. The comparison group was comprised of those who have aspirations in line with the participating group and have similar leadership qualities and ambitions as program participants. They are the kind of people who will excel given any type of opportunity and who are already inclined to pursue international studies in order to advance their career objectives. In addition, the success of the comparison group speaks to the quality of the selection process, in that the program is clearly identifying the right kind of participant – those who will be the best leaders in their communities. It also demonstrates the perceived value of the program overall, as it clearly attracts ambitious and intelligent individuals who seek to increase their competitive advantage in their intended careers. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 16 The value of the CSP/PSP program lies in its ability to build the necessary technical knowledge and professional resource networks essential to a successful career. It provides scholars an avenue through which they can obtain the skills and experience necessary to become the next generation of leaders in Palestinian society. Alumni of this program indicated that having an opportunity to study in the United States increased their professional competitiveness in a way that studying at another university in Palestine or the greater Middle East region would not have done. It also vastly improved their understanding of the United States. While more than half of the comparison group members had received their Master’s degrees, none of them had done so in the United States. Therefore, while it may be valid to argue that those who do not have an opportunity to participate in this program would find opportunities elsewhere to obtain a Master’s degree, it does not appear as likely that they would have the same chance to develop the life skills and approach to cultural diversity that CSP/PSP alumni had attained. These benefits of U.S. study can be expected to impact Palestinian society over the longer term. USAID may wish to conduct a considerably larger study at some point in the future, perhaps in several years when the career paths of participants and non-participants are more fully developed, to undertake a more in-depth review of this issue. However, even with considerably more time and resources, any evaluation will need to confront similar methodological difficulties, particularly in retroactively identifying an appropriate comparison group, separating the many factors leading to individuals career development, and estimating impact on society. Another approach that USAID might consider for future impact studies would be to utilize two comparison groups, one of a similar demographic makeup and one of a more diverse sample, with respondents across age and educational cohorts. 4. Improved English Language Skills As might be expected after living in the United States for two years, most participants reported vastly improved English language skills. Those who did not feel their English had dramatically improved were already proficient in the language before going to the United States. Most participants appreciated the pre-program language training and said it had helped to raise their language skills to a level adequate to complete a Master’s degree program at a well-respected university. Very few respondents reported experiencing difficulty in completing their course work in English. However, participants’ exposure to a U.S. education did lead to increased opportunities in jobs involving international contacts where English language ability is an important asset. More than 80 percent of survey respondents reported that their current job requires them to work within an international context. Interviewed participants attributed this to their employers’ appreciation for the skills they had learned during their time in the United States, including their English language skills and experience working with peoples of diverse backgrounds. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 17 5. Understanding Cultural Differences Generally speaking, Palestinian culture is relatively homogeneous; thus, for most participants, having an opportunity to interact with people of diverse cultures in the United States was a new and sometimes challenging experience. Participants reported having encountered a wide variety of people in classes and in their personal lives. These experiences had not only made them more tolerant of others’ differences, it made them see themselves differently and helped to develop their own interpersonal skills. 6. Changed Opinions of Americans and of the United States Participants’ experience in the United States also changed their view of Americans and, to a lesser extent, of America. Respondents to the online survey were asked to describe how their understanding of various aspects of American culture changed after their participation in the exchange program. Not surprisingly, the greatest change had occurred in participants’ understanding of daily life in the United States, with almost 99 percent of respondents indicating a substantial or moderate change. However, this was closely followed by a moderate or substantial change in the religious and ethnic diversity in the U.S. (96%) and U.S. culture and values (93%). Table 3.9 details their responses. Table 3.9. Change in Participants’ Understanding of the United States Substantial or Moderate Change Minimal or No Change Don’t Know / Not Applicable Daily Life in the US 98.6% 1.4% 0% Religious and Ethnic Diversity in the US 95.9% 4.1% 0% US Culture and Values 93.2% 4.1% 0% Freedom of speech and press in the US 91.8% 8.2% 0% US Economy 89.0% 9.6% 1.4% US Democracy 78.1% 21.9% 0% Voluntary Community Service 75.0% 20.8% 4.2% N = 73 Some 77 percent of survey respondents indicated that they had shared with friends and colleagues the knowledge or skills they gained as a result of the program. Although many indicated sharing professional knowledge, there were many more who have shared cultural and social aspects of American life. One participant, who teaches English, said she shares It changed my outlook - I have a broader perspective. The world as I define it has a wider definition. I now see my culture, religion as Palestinian as part that interacts, influences, and is influenced by other components of world culture. Being in the U.S. was a big plus in gaining exposure to a variety of cultures, political viewpoints - it really widened my horizons. I feel very lucky and privileged to have studied in the U.S. I grew and became more mature. I have experienced much more independence than other Palestinian women. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 18 experiences celebrating holidays like Christmas and Thanksgiving in American homes. Another respondent commented that he has shared: “that Americans are great, and the American nation is so great because of freedom.” Questions regarding increased understanding of U.S. culture and values were complemented by questions regarding how much participants’ opinions of U.S. people and government had changed after their experiences in the United States. Nearly 96 percent of respondents indicated that their views towards the American people were more favorable or much more favorable as a result of the program. Although only 30 percent of respondents felt more favorable or much more favorable towards the U.S. government, this is still a fairly good indication of progress, considering the rather tempestuous political background. Table 3.10 details these responses. Table 3.10. Participants’ Changed Opinions of the United States More or Much More Favorable No Change Less or Much Less Favorable Don’t Know / Not Applicable n American People 95.8% 4.2% 0% 0% 71 U.S. Government 30.1% 52.1% 15.1% 2.7% 73 In-depth interview respondents elaborated on areas in which they felt their understanding of both the American people and with the U.S. government had increased. Most talked about how they felt the American people were kind, generous and caring about others, which they felt was a change from the way Americans are portrayed in popular media. One respondent said her opinion had “totally changed. I had many negative stereotypes about American people. But after visiting there, I learned how good Americans are, through the people I met in my school and in my neighborhood. They are friendly, generous and have big hearts.” 7. Leadership in their Local Communities Survey data suggest that U.S. training has led returned participants to become more involved in community activities and, in those activities, to serve in leadership capacities. Nearly half (49%) of the respondents to the online survey reported having volunteered with organized groups outside their workplace to improve social and economic conditions for Palestinians. Perhaps more noteworthy is the fact that of those who did undertake volunteer activities, most (78%) said they had served in a leadership capacity with these organizations, and all but one of those said their overseas experience had helped them to play this leadership role. Also, some 75 percent of respondents said they had learned about volunteer community service through their experiences in the United States. The respondent who made the above comment currently supervises and coordinates 35 student volunteers and develops monthly activities for them to benefit local communities. Another respondent had volunteered to create the Clinton and Presidential Association with six other alumni. The main goal of this association was to initiate activities that serve their community and to maintain connections among all program participants in Gaza, as well as to exchange professional skills. One of the things I got from the States was I did research about volunteer activities in the U.S. In the U.S., volunteerism is emphasized. Here, the culture of volunteerism in Palestine is not deep. My NGO works to improve this. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 19 Other organizations in which CSP/PSP alumni had volunteered included: • Planning Department at Al Bireh Municipality; • Health and Environment Committee at Ramallah Municipality; • Palestinian Disabled Society; • Educational Development Society; • National Al Marasad for Media; and the • Strategic Research Center. Examples of typical volunteer activities included: • Teaching community English classes on a volunteer basis; • Translating proposals for charitable organizations in Gaza pro bono; • Registering voters for the PLC elections; • Helping to prepare the Palestinian disabled team for the Beijing games; • Serving as a Board member for an NGO called the House of Water and Environment; • Serving as a member of the Palestinian Securities Exchange; and • Serving as a coordinator at Birzeit. 8. Multiplier Effect in the Community As the above examples demonstrate, returned participants are employing their new skills in community activities. Many participants expressed a sense of obligation to pass their expanded interpersonal and professional skills on to other members of their community through professional and personal connections. One interview respondent, who also taught at a local university, volunteered with Palestinian high school students to offer instruction on time management and personal development. Another participant commented: “It was part of my goal to share the knowledge I gained while in the United States with friends, family, and most importantly colleagues in the organization and the field at large.” In an open-ended question, respondents identified the following knowledge, skills and values that they had sought to pass on to colleagues and friends: being independent, developing a positive attitude and trusting your abilities, being self-motivated and taking initiative, communication and public speaking skills, group work, household budgeting, professionalism, leadership, hard work, flexibility, honesty, adaptability, reconciliation and tolerance. These soft skills were greatly valued by the participants but perhaps more importantly, were being shared to the benefit of the entire community, which is a key element in development efforts for all of Palestine. 9. Support from Implementing Partner Organization Participants were generally happy with the support they had received from the implementing partner while they were in the United States. However, after having spent an average of two years in the United States, many participants reported having difficulty returning to their home communities and resuming their former lives. This reverse culture shock came as a surprise to many, although older participants seem to have re-adapted to their former lives more easily than the younger participants. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 20 Generally speaking, alumni felt more support was needed upon returning to Palestine. One respondent commented: “They cared a lot about [us]; they took all our needs into consideration, however, the program ended too soon.” Many respondents felt that USAID had an obligation to provide more assistance with job placement upon returning to Palestine, as most participants reported a period of several months’ lag between completing their program and finding a job. While this is beyond USAID’s capacity, USAID could support other activities described below that might help returned participants to find appropriate employment. 10. Ongoing Learning Opportunities and Alumni Networks None of the participants who were interviewed had earned a Ph.D., though many expressed interest in doing so; one survey respondent indicated that he was pursuing a Ph.D. out of the country. A few respondents had participated in professional skills development courses specific to their areas of specialization. However, most had concentrated their efforts on their jobs and outside activities. About half (47%) of survey respondents had participated in alumni activities. Among those who had not participated, reasons mentioned included the difficulty of travel around Palestine, lack of awareness about the activities and lack of time or interest. With respect to their lack of interest, several respondents commented that the alumni events were primarily social in nature and did not offer sufficient incentive to motivate busy alumni to attend. Many respondents indicated that a more professionally-focused alumni network might be more appealing to them. As more than half of the survey respondents indicated that they maintain contacts with other program alumni on a social basis at least monthly, it appears that an organized alumni network that provides only social opportunities is unnecessary. Conversely, a professionally-focused alumni program that brought participants together for a speaker’s series or for job networking purposes would address both unemployment issues and feelings of isolation or “being dropped” by the program after completion of their degree program. Facilitating professional networks among alumni (many of whom have secured high-level employment and may have contacts that could be utilized to help other alumni) could be a way to help returnees find jobs on their own. Creative engagement of local organizations such as the local Chamber of Commerce may also help to address some of these issues and perceptions. We appreciate all of you have done so far, but can you fund us for Ph.D. programs? I am dreaming of that every day. • It’s hard when you feel that you’re a stranger in your own community. You change but the people here don’t change. • In the first three months, my mind was in the U.S., but my feet were in Palestine. • My family and I never recovered from the reverse cultural shock; I felt isolated from the support I [had] received. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 21 D. Conclusions The CSP/PSP programs have been successful in helping Palestinian participants to obtain Master’s degrees, and program alumni are generally very satisfied with the outcomes. Important conclusions about the program and its impact include the following: Return of participants to Palestine. Most of these Palestinians returned home, are employed in the region, and are contributing to Palestinian economic and social progress. Participants’ increased job opportunities. The CSP/PSP program resulted in participants gaining important new technical and leadership skills and attitudes that are important in the workplace. Most participants currently hold positions of prestige and influence, which will enable them to become leaders for future development efforts in Palestine. Participants credit their U.S. study as an important factor in their subsequent career advancement. Employers (although few in number) spoke of participants’ skills that may well have been developed in the United States, but they were unable to speak authoritatively on where those skills had been developed. Unemployment among returned participants is modest. Participants’ increased leadership roles. Of those program participants who had taken on supervisory roles in their jobs, many were making concerted efforts to pass on important soft skills such as: meeting deadlines; working collaboratively; workplace efficiency; conflict resolution; and communication skills. Many had also volunteered in their communities to share the knowledge and skills they had developed as a result of their time in the United States. Although many saw themselves as leaders before participating in this program, earning their Master’s degree and broadening their professional and personal horizons was an important element in their own leadership development. Program impact on Palestinian economy and society. Given the relatively short time that most participants have been back from their studies and the very difficult political and economic situation in the Palestinian territories, measuring societal impact at this time is extremely difficult. However, the impact on the participants involved in these programs is evident, and those participants now hold positions of increased responsibility where they can impact the organizations in which they work and their communities. Participants are well￾prepared, well-positioned and highly motivated to make a significant contribution to Palestinian development in the future. Public diplomacy benefits for the United States. The experience of living in the United States had a profound impact on participants’ overall outlook and on their professional development specifically. The cultural diversity of the United States, the experience of living independently, and interacting with American people helped participants to grow and mature and broke down stereotypes. Program alumni have returned with a sense of responsibility to become “cultural ambassadors” – to share their experiences with other Palestinians through volunteer activities or simply by telling family and friends informally. The creation of professional linkages. Participants have utilized relationships formed in the United States with professors and fellow students to assist them in professional development since returning to Palestine. More often, personal relationships with professors and students had endured – both with American and international students. Alumni networks have not been particularly successful thus far, but shifting the focus to ongoing professional development activities may fill a need articulated by many participants to remain connected Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 22 to USAID and to continue their career development after returning to Palestine. These professional connections are not occurring organically (in comparison to social networking among alumni), so any growth in this area will need to be purposefully nurtured by USAID. Quality of program implementation. Overall program implementation and management were successful. Participants were generally satisfied with the support they received from AED, although some participants who had not received a longer period of U.S.-based orientation felt that this area could be expanded. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 23 E. Recommendations 1. Develop a follow-on program. The evaluation recommends that USAID consider a follow-on program for Master’s degree training, particularly in disciplines that are weak or unavailable in Palestinian universities. The Palestinian economy would certainly benefit from additional people trained in the United States, not only because of the technical knowledge they bring back, but also because of the new ideas and new perspectives they gain from their time in the United States. The individuals that receive U.S. training are very likely going to become prominent Palestinian leaders in years to come. In addition to these benefits to Palestinian society, the U.S. gains through the positive image such programs create among the population and through the development of future leaders who have an understanding and appreciation of the United States. Palestinian universities do not offer equivalent programs or experiences to their students. 2. Renew the U.S.-based orientation period before participants begin classes. This study suggests that future participant training programs should allow participants a period of adjustment in the United States before they begin their classes. Similarly, it may be beneficial to offer more support in finding housing, even if this is facilitated through university student services, as most students arrive in their U.S. locations with little sense of what areas are safe and convenient for public transportation. Post-program support and follow-up should be expanded to help those who need it readjust to life in Palestine. 3. Develop a professionally-based alumni program. With only limited opportunities for continuing education in Palestine, it is difficult for returning participants to keep abreast of new developments in their fields. An effective alumni association could partially address this need by facilitating technology transfer through access to professional literature, Internet resources and special programs. With USAID support, such an organization could keep former participants in touch with the United States, e.g., by organizing a series of state-of￾the-art courses given by U.S. universities. An alumni association would also serve to keep former participants in touch with each other. For example, this group could be developed to facilitate networking for job opportunities. As the CSP and PSP students have returned and secured employment, they can be utilized as mentors for future participants. Such an organization might well become a recruiting ground for U.S. and international companies and organizations. USAID could also explore creative ways to engage groups such as the American Chamber of Commerce or American Business Association. 4. Develop more effective internship and entrepreneurship activities. These experiences were beneficial to those who participated, and many of those who had not participated expressed regret that they did not or could not take advantage of this opportunity. Institutionalizing some of these elements would encourage participation. In regards to the $5,000 post-training grants, although this element could be very beneficial to Palestinian development, many of the restrictions imposed on proposed projects limited its usefulness and accessibility. If these restrictions cannot be lessened, it may be better to direct program funds towards other elements such as the U.S.-based pre-program orientation period. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 24 IV. Component 2: the Palestinian Faculty Development Program (PFDP) Unlike the more general human resource projects assessed in Component 1, PFDP is an institutional development program to improve higher education in the West Bank and Gaza. PFDP has two major objectives: (1) to promote the expansion, retention and professional development of promising academics in the social sciences and humanities and (2) to revitalize and reform teaching in target disciplines at Palestinian higher education institutions, thereby promoting an institutional culture of teaching and learning. Launched in October 2005, the PFDP is an innovative collaborative effort of three organizations: USAID/WBG, OSI and AMIDEAST. The program is implemented under a Cooperative Agreement between USAID and AMIDEAST, which is considered the primary implementing organization. Funding is provided by USAID and by OSI. OSI, referred to in USAID documentation as “the Cost Sharing Partner,” also has major implementation responsibility. PFDP resulted from an unsolicited proposal submitted to USAID jointly by AMIDEAST and OSI. The proposal included two training elements at U.S. universities: Ph.D. scholarships for Palestinians teaching at the university level and short-term visits by more senior Palestinian faculty members seeking to improve the content and/or their methods of teaching. Other elements of the program, which are not covered by this evaluation, involve a variety of in￾country seminars, colloquia and other events intended to introduce Palestinian university faculty and administrators to improved teaching and learning methods. Those other program elements are financed by USAID and administered by AMIDEAST. In many ways PFDP resembles numerous USAID-funded higher education development projects in dozens of countries around the world for the past fifty years. It provides opportunities for professors to obtain doctoral degrees; it encourages teaching methods that involve active student participation; it introduces host country institutions to advances in educational technology; it encourages joint research and other linkages with U.S. universities, etc. However, there are also distinctive features of PFDP that are important to highlight, as they have influenced program design and implementation, and they will influence impact. First, whereas typical USAID higher education development programs are financed largely by USAID with some counterpart contribution from the host country government, USAID finances only $8.85 million (54 %) of the $16.3 million PFDP budget. The remaining $7.45 million, primarily the costs incurred in the United States of PFDP’s Ph.D. and Short-term Fellows programs, is contributed by OSI and the U.S. universities hosting PFDP scholars. OSI obtains 75 percent tuition waivers from participating U.S. universities and it finances the remaining 25 percent of the tuition costs and its own administrative expenses with its own funds. Although this cost-sharing arrangement is seen as a benefit by each of the partners, this partnership also conveys significant non-financial costs. Each of the key actors needs to make concessions to the priorities and constraints of the others. Early in the project, the USG’s “no contact” policy made it impossible for the partners to effectively involve and work collaboratively with the Palestinian Ministry of Education. Currently, the fact that USAID is subject to the USG prohibition on collaborating with organizations unwilling or unable to sign the Anti-terrorism Certification (ATC) keeps its partners from working closely with Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 25 some of the Palestinian universities -- the intended beneficiaries of the program. As a result, Palestinian education leaders have had little say in the determination of the program’s technical priorities or in the selection of candidates for training. OSI’s limited focus within higher education on social sciences and humanities prevents USAID and AMIDEAST from addressing needs in other disciplines that they and Palestinian authorities might wish to address. These constraints and their effects are discussed in more depth below. A. The Ph.D. Fellows Program The PFDP Ph.D. Fellows Program is intended to increase the number of professors with doctoral degrees in the social sciences and humanities. The program was designed to provide full scholarships to up to 50 scholars. However, due to selection issues discussed below, only 34 Palestinians began their Ph.D. programs at U.S. universities under the program: 21 in 2006 and 11 in 2007. No additional candidates will be sent to the United States in light of time limitations on the project and due to the fact that it normally takes four years or more to complete a doctoral program. At the time of this report, one scholar had already successfully finished his program, one had failed his comprehensive exams and returned home, and 32 scholars remained in the program. As noted above, the program operates with a cost-sharing arrangement that allows USAID, OSI and the universities to leverage their funds with the contributions of others. The total estimated cost of the Ph.D. program is $7.6 million, exclusive of OSI and AMIDEAST administrative costs. This suggests a total cost per student of a little under $224,000 (not including project administrative costs and the difference between the universities’ tuition charges and their real costs). USAID’s share of these costs is approximately $3.16 million (42%). The universities contribute approximately $2.68 million (35%) in the form of tuition waivers, and OSI contributes about $1.74 million (23%) to cover a variety of participant support costs. As in the previous section on the Master’s degree programs, the evaluation team’s findings presented below are divided into three sections: the first dealing with activities prior to participants leaving for training; the second, findings about the training and support activities related thereto, and the third about actual or potential results following training. 1. Ph.D. Pre-program Activities a. Technical Disciplines The Ph.D. scholarship program is focused on five technical disciplines: public administration and management, urban and regional planning, political science and government, social work and education. Table 4.1 below shows that there is a fairly even distribution of Ph.D. candidates across the technical disciplines with the exception of social work, which has fewer. Table 4.1. Number of PFDP Doctoral Degree Scholarships Awarded, by Sector Public Admin & Mgt. Urban & Regional Plan. Political Science & Govt. Social Work Education Totals Scholars 9 7 7 2 9 34 % of total 26% 21% 21% 6% 26% 100% Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 26 From USAID’s perspective, the decision to limit PFDP’s Ph.D. program to these particular disciplines within the social sciences reflected the desirability of having OSI as a cost-sharing partner. However, that design decision meant precluding other disciplines – such as engineering, business administration, medicine and IT – some of which were priorities of the earlier USAID programs, which many Palestinian universities regard as high priorities, and which are clearly important for Palestinian development. The small number of eligible technical disciplines limited the pool of qualified and interested candidates for Ph.D. study. This explains, in part, the reduction in the number of Ph.D. candidates selected and placed from 50 to 34. Many of the Palestinians with whom the evaluation team spoke, including current Ph.D. participants, also strongly urged a broadening of the technical disciplines in future investments. Additionally, the limited number of eligible candidates in the priority disciplines may have contributed to some discipline “switching” or “stretching.” The team interviewed one scholar who indicated that he had switched fields from Philosophy to Political Science to become eligible, although he had no real desire to switch fields. This may well have contributed to his poor performance in his Ph.D. program. Another participant worked in Public Health and was interested in getting a Ph.D. in that field. She accepted a scholarship to a Public Administration program on the hope that her public administration studies could focus on the health sector. Although any scholarship program can be “gamed” in this fashion, this may well be more prevalent in PFDP than in other programs because there were so few eligible disciplines and not a great many candidates to choose from in those fields. b. Selection of Scholarship Participants Identifying and selecting appropriate candidates for the Ph.D. program was not an easy task. AMIDEAST worked to encourage demand from eligible candidates through advertisements in leading newspapers, visits to university campuses and emails to university professors and administrators. Palestinian universities were not asked to suggest candidates, although later in the process they were asked to indicate their concurrence. To be eligible, Palestinian candidates needed to be: (1) teaching at least part-time in one of the program’s five technical disciplines in a Palestinian university; (2) qualified in terms of their technical preparation; (3) able to achieve a minimum score of 480 on the TOEFL (although the program provided limited in-country English training); (4) interested in spending four years or more in the United States pursuing a doctoral degree, which meant forgoing their teaching salary in favor of a graduate-level stipend; (5) able to get an endorsement from his/her home university; (6) pass successfully through the USG vetting process; and (7) able to get a U.S. visa. In the 2006 and 2007 selection rounds, there were 115 and 73 candidates, respectively, that submitted full applications. AMIDEAST and OSI reviewed those applications to determine which appeared to meet the program’s qualifications and developed a selection process that was rigorous, professional and transparent. The process began with an initial screening to eliminate applications that were incomplete or which revealed that minimal qualifications were not met. AMIDEAST and OSI then organized Selection Committees to review applicants and ultimately to interview finalists in the program’s five disciplines. Committee members included experienced personnel from OSI and AMIDEAST and U.S. university Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 27 professors in the appropriate fields from institutions that were potential sites for the Ph.D. study. A USAID representative also participated in some committee reviews. Ultimately, the number of candidates who made it through the process – who met these multiple requirement and convinced selection panels and university admissions directors that they would successfully complete Ph.D. programs – was quite limited. This lengthy and complex process eventually resulted in 34 Palestinians being selected and placed; the target had been 50. Although this shortfall was disappointing, the fact that the selected participants have generally performed well (as discussed below) testifies to the selection system’s effectiveness in identifying individuals capable of successfully completing doctoral studies in the United States. The process sought to encourage participation of women, with a target of 20 percent. This low target reflected the fact that only 13 percent of Palestinian university faculties are women. The fact that 11 of the 34 Ph.D. participants (32%) are women is another positive result of the recruitment process established by the implementing partners. The current average age of the program’s Ph.D. students is about 39 years,4 which suggests that participants’ average age when they complete their degrees will be 41 to 43. Other things being equal, the evaluation team believes that selecting younger candidates would have been preferable, as participants completing the program would have more time to impact Palestinian higher education. However, the difficulty in finding a sufficient number of suitable candidates suggests that this option was not available. USAID and program implementers sought to include participants from a broad range of Palestinian universities in both Gaza and the West Bank so that benefits would not be concentrated in a small number of institutions. This objective was realized. Table 4.2 below indicates that participants were drawn from nine Palestinian universities. Birzeit University was particularly well represented, with 11 faculty members included in the program. This reflects that university’s relative strength in the region and its ability to attract particularly well-qualified faculty members. Twenty-four participants (74 %) were selected from West Bank universities and nine participants came from universities in Gaza. Table 4.2. Numbers of PFDP Ph.D. Participants, by Location and University of Origin University Location # % West Bank Universities 25 74 An-Najah National University Nablus 4 12 Al Quds Open University Bethlehem (2), Jinin, Nablus, Hebron 5 15 Al Quds University Abu Deis 3 9 Bethlehem University Bethlehem, WB 1 3 Birzeit University Birzeit, WB 11 32 Hebron University Hebron, WB 1 3 Gaza Universities 9 26 Al Azhar University Gaza 2 6 Community College for Applied Sciences and Technology Gaza 1 3 Islamic University Gaza 5 15 Al Quds Open University Rafah, Gaza 1 3 Totals 34 100 Source: AMIDEAST website and Al Quds Open University. Rounding errors affect some percentage figures. 4 This estimate is based on the ages reported by 25 respondents who answered this question in the online survey Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 28 Candidates’ prior teaching experience was, in some cases, quite limited. One candidate mentioned that she had only been teaching for one year before leaving for this program and others had taught only part-time. Although the team considers that it would have been desirable to select candidates with more teaching experience (and thus more clearly devoted to academic careers), the program’s selection of some individuals with limited relevant teaching experience reflects the reality of the university situation in Palestine where part-time teaching is quite common5 as well as the limited candidate pool from which candidates could be drawn. The limited teaching experience of some participants has not detracted from their academic performance in the program. However, selecting part-time faculty is a concern if it is an indicator that participants will return to part-time teaching upon their return. Although all of the survey respondents indicated that they would return to university teaching, it is not clear whether this will be on a full-time or part-time basis. The answer to this question will be an important determinant of their impact (and this project’s impact) on higher education in Palestine. c. Participants’ Goals When asked about their reasons for undertaking Ph.D. studies in the U.S., participants’ responses tended heavily toward personal goals – advancing my career, creating new opportunities for myself, etc. This contrasted with the responses from returned participants of the Clinton and Presidential Scholarship Programs, who more frequently mentioned social goals, e.g., to increase their contribution to Palestinian development. Nevertheless, PFDP participants now appear to be enthusiastic about implementing a wide range of constructive changes within Palestine. (Participants hopes for the future are discussed below.) A number of participants reported that they had other opportunities to pursue doctoral degrees, in Europe or elsewhere, but had preferred to study in the United States. One participant also compared PFDP favorably to the Fulbright Program, to which he had considered applying. The availability of a significant number of grants to undertake Ph.D. studies in other countries was confirmed in discussions with university administrators. An administrator from Bethlehem University noted, for example, that his university had eight faculty members currently studying for Ph.D.s overseas; this includes one PFDP participant, three others studying in the United States, two obtaining Ph.D.s in Germany, one in Denmark and one elsewhere. d. Pre-departure Programs AMIDEAST and OSI collaborated in providing pre-departure training and orientation to the participants. The team utilized its survey and interviews to solicit feedback on the effectiveness of those activities. Survey responses are summarized in Table 4.3 below. 5 The use of part-time faculty is most prevalent at Al Quds Open University where, according to information provided by that university, part-time staff make up 84 percent of the teaching faculty and provide 63 percent of the teaching. A senior administrator from another Palestinian university told the team that the use of part-time staff at his university was not more than 25 percent. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 29 Table 4.3. How satisfied were you with the pre-program orientation and preparation? Response Percent Response Count Very satisfied 15.4% 4 Satisfied 50.0% 13 Neutral 23.1% 6 Dissatisfied 0.0% 0 Very dissatisfied 11.5% 3 n = 26 Sixty-five percent were “satisfied” or “very satisfied,” which is presumably lower than the implementing organizations would hope to achieve. Interviewees indicated problems in two areas. First, the English training was inadequate for some. As a result, in their doctoral programs they struggled to understand lecturers and found that it took them far longer than their American counterparts to read texts and the innumerable assigned journal articles. Second, a number of participants complained that they found themselves inadequately prepared to deal with the administration of life in America, particularly with respect to off￾campus activities that involved their families. One talked about difficulties experienced with housing, banking and public transportation. Another noted that they had received no information about enrolling children in schools, health insurance for family members, etc. 2. Ph.D. Participants’ Experience in the United States a. Participants’ Assessment of U.S. University Placements Matching international students appropriately to U.S. academic institutions is a difficult undertaking and this was certainly the case with PFDP’s Ph.D. candidates. A number of the PFDP candidates had substandard GRE or TOEFL scores, making it necessary to convince universities to waive normal requirements. Some candidates did not articulate their academic interests clearly (or adapted their interests to match qualified sectors), and almost none had a good understanding of how U.S. doctoral programs operate. According to OSI, none expressed a clear preference for a particular university or program, although they were encouraged at various stages to do so. Very likely, candidates were overwhelmed by the number of possibilities in the United States, with its 4,000+ institutions of higher learning, and would have needed considerable guidance if they were to attempt to sort through the possibilities. In addition, OSI was limited in selection to universities willing to waive 75 percent of their normal tuition charges. Ten of the 16 universities ultimately chosen for PFDP scholars were those institutions with which OSI had previously worked out such an arrangement. The numbers of PFDP candidates by technical area and university are shown in Table 4.4. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 30 Table 4.4. Number of PFDP Ph.D. Scholars, by Sector and University Universities Educa￾tion Public Admin. Poli. Sci. Urban Plan. Social Work Totals Univ. of Massachusetts 4 4 Columbia University 2 1 1 4 Boston College 2 1 3 Brandeis University 3 3 Univ. of Washington 1 3 4 Cleveland State 2 2 University of Akron 2 1 3 Univ. of Pennsylvania 1 1 University of Denver 2 2 Univ. of Texas - Austin 1 1 Univ. of Texas - Dallas 1 1 University of Illinois 1 1 Southern Illinois Univ. 1 1 University of Florida 2 2 Portland State 1 1 University of Michigan 1 1 Totals 9 9 7 7 2 34 Although these universities are well regarded and undoubtedly offer very high quality programs, the evaluation team’s online survey results indicate a significant number of participants are dissatisfied with their placements. When asked how they rated the selection of their university program in terms of how closely it matched with their professional interests, 15 of the 27 respondents (56 %) rate the selection as “very good” or “excellent;” three respondents (11 %) rated their placement as “adequate,” and nine respondents (33%) rated their placement as “not very good” or “poor.” Participants expressed similar concerns in interviews. Some reported feeling pushed academically in directions they thought inappropriate or irrelevant to their interests. Several participants expressed the view that they did not have sufficient opportunity to participate in the placement process and that it had not been sufficiently transparent. Two expressed fear that they would be dropped from the program if they expressed their dissatisfaction. This concern may explain why the evaluation team heard more about this issue than had OSI or AMIDEAST. The evaluation team brought participants’ concerns about placement to the attention of the implementing partners. In response, the OSI project director pointed out that OSI and AMIDEAST had given participants several opportunities to participate in the placement process, but that they had not done so. AMIDEAST pointed out that this behavior (not speaking up about placement preferences and concerns) is also common in the Fulbright Program and in other scholarship programs. OSI also noted that it is not unusual for Ph.D. students to express frustration in the first two years of their programs at having to take established course requirements that appear at the Nobody contacted me to ask me [discuss] what I want[ed] to study. They just looked at my proposal and then they made their conclusion about the field, and they decided to place [me] at a university which really does not reflect exactly what I want. …the placement process was not transparent. We were informed of the placement and we were asked to take it or leave it. A PFDP Ph.D. Participant Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 31 time to be inappropriate or onerous. However, core courses are needed to build the conceptual background and analytical tools doctoral candidates will need to conduct research, to teach and eventually to communicate effectively with other professionals in their field. It is during their dissertation research that Ph.D. candidates have the opportunity to specialize on the specific topics that most interest them. After completing their dissertations, many Ph.D. students come to a much higher appreciation of courses that they initially thought of little interest or importance. The evaluation team has found the explanations and assurances put forth by OSI and AMIDEAST quite convincing. The team suspects (but cannot determine with any degree of certainty) that the placements were as good as the candidates could have reasonably expected, given the degree to which their interests were articulated and their backgrounds and qualifications would allow. At the same time, there are quite clearly different perceptions between the implementers and some participants about the openness of the placement process and the results the process yielded. While OSI feels that it made extraordinary efforts to give participants the opportunity to be heard on placement, some participants did not hear this message. One respondent reported being told prior to departing for the United States that s/he should be grateful and not question the proposed placement. Further, the fact that only four of the 32 participants have indicated concern to OSI while nine (of 27) expressed concern to the evaluation team suggests a reticence to express concerns about placement to the project implementers. b. Participants’ Views of the Quality of Project Support Despite concerns about placement, participants generally have a very positive view about OSI and AMIDEAST and their representatives in Palestine and New York. Participants indicated that they turn to those organizations with problems and feel that the staff does its best to solve them promptly. (As noted above, this openness in bringing problems and concerns to the implementing partners did not seem to prevail on the question of placement. The evaluation team cannot explain this apparent inconsistency.) One participant explained that when she was in Gaza last summer, they emailed her every week. Also, during that time the university refused to register her because the tuition fees were not paid. She contacted OSI and the issue was resolved. “The OSI staff is more than great.” A participant speaking about placement: It’s a common issue for other scholarship recipients too – they find a different reality on the ground [than they expected]. That is why they try to adopt their studies to their needs and take courses from other departments… And if you express discontent, you risk losing your scholarship; therefore students try to find their own ways of coping and adjusting. I get in touch with them whenever I have a problem or a question. They were very responsive and very supportive. They are trying to do their best within the powers and the authorities they have, which is sometimes very limited, but they try. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 32 There are, of course, some problems that are difficult to resolve. Several participants believe that their stipends are not adequate and becoming less so as U.S. prices increase. The fact that participants’ average age is 39 (versus younger graduate students) means that they have more dependents and more financial responsibilities. PFDP has allowed participants to take their families with them to the U.S, and half of the participants have done so. Seventeen families (65 dependents) have joined participants in the U.S. Given the present restrictions on getting visas to the U.S. and the restrictions that the Israeli Government places on the travel of Palestinians, this has required an extraordinary effort on the part of the implementing organizations and USAID for which they merit thanks and praise. Despite numerous successes, there have been several problem cases. One long-standing case has apparently now been resolved, but one case of a family residing in Gaza has not been resolved, and the participant seems not to have been told what has been done and why there is a continuing problem. c. Annual Participant Conferences Arranging an interesting and useful annual conference for participants from diverse disciplines is a challenge for conference organizers. Still, 80 percent of participants rated them as satisfactory or better in the team’s survey. The conferences provide an opportunity to attend to common administrative matters and, perhaps most importantly, give participants an opportunity to share experiences and to build relationships within the group that will allow them to work together for change upon their return. d. Participants’ Progress in their Ph.D. Programs With few exceptions, the participants are doing well academically. This suggests that the implementing partners did a good job in selecting candidates who could adjust successfully to American society and deal effectively with the academic challenges of a U.S. doctoral program. OSI and AMIDEAST report that, with the possible exception of three or four scholars who are struggling academically, current participants are progressing satisfactorily. Several interview respondents reported that they have passed their comprehensive exams and are working on their thesis proposals. USAID should expect, however, that there may be other participants who do not complete their doctoral programs. Getting a Ph.D. in the United States is difficult, and the success rate, even for Americans and other native speakers of English, is certainly far from 100 percent. Several participants indicated that they have ongoing involvement or communication with their Palestinian universities, and they are concerned that they are losing touch with those institutions and their needs. To make their dissertation research as useful as possible, some The whole program is designed for single students. They copied the Fulbright programs that last for one or two years, and they wanted to apply to us. They did not take into consideration that we have families and that we are staying for five years, not for two years. Therefore, we have different needs. I’m not saying that they have to pay us more or they have to pay for our dependents. At least they should be sensitive to our situation. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 33 discussion with those home institutions would be desirable. Lack of communications may also lead to participants having increased difficulty reintegrating into their home universities when their studies are completed. e. Internships and Other Work Opportunities Ph.D. students often undertake teaching or research assistantships and internships to apply what they have learned and to supplement their scholarship support. Most commonly, Ph.D candidates take up such positions after course work is completed, while they are working on their dissertations, a stage just now being reached by a number of PFDP participants. To date, only five of 27 (19%) of survey respondents have benefited from this type of opportunity. Three of the five found that experience “extremely effective” in increasing their professional and technical knowledge; two found it “somewhat effective.” Interviewees expressed considerable interest in these types of work/study opportunities but felt that there were a variety of obstacles to be overcome. Most important of these was their course workload, which thus far had precluded their participation. Among other factors, one interviewee reported that her university had a policy of offering assistantships to students who had little financial assistance, making it unlikely that she would be able to participate. Three respondents indicated that there were few or no opportunities of this type at their universities. Others expressed concern that their stipend from OSI would be reduced if they earned extra money. f. Extra-curricular Professional and Volunteer Activities Most participants find some time, albeit limited, to participate in a variety of learning activities outside of their coursework and research. The most common outside activity is speaking publicly about the Palestinian perspective on the Middle East, something that appears to be very common. Additionally, many participants have attended professional conferences and workshops related to their professional disciplines. Aside from these types of events, participants engage in a wide diversity of activities. Examples include: • Attending a meeting of the local urban planning commission; • Participating in a two-week internship in the office of the local Congressperson, which included a trip to Washington, DC where he learned a good deal about the workings of the Congress and had the opportunity to meet Senators and other senior officials; • Attending international student meetings and get-togethers; • Continuing selected work assignments (from earlier involvement with the Carter Center and other organizations); • Participating in meetings of an environmental NGO, and • Working with the local school board where his children study. The major constraints on participation in such activities are the participants’ heavy course work and, for about half of the participants, family responsibilities. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 34 g. Changes in Professional Knowledge and Skills The Ph.D. participants report significant increases in knowledge and skills that will allow them to impact Palestinian higher education when they return home. Of the 27 participants who responded to the team’s survey, the percentages that assessed themselves as having achieved either “very large improvement” or “significant improvement” were as follows: Table 4.5. Ph.D. Participants Assessment of Increased Knowledge and Skills Category Significant or Very Large Improvement Increased technical knowledge 96% Improved study skills 89% Improved research skills 81% Improved teaching skills 78% Improved networking skills 70% Increased knowledge of university administration 48% Average 77% These ratings should, of course, increase further as participants complete more of their doctoral programs. h. Views about U.S. Education versus Palestinian Education Participants are generally overwhelmed with the resources at their U.S. universities, making it difficult for them to make meaningful comparisons with what they experienced at home. Several participants indicated that the library resources at their U.S. universities are simply unimaginable in Palestine. The extraordinary resource levels in U.S. universities make it possible for professors to teach as little as one or two courses a semester and to focus on research. In Palestine, professors get paid by the course, and a full-time professor needs to teach four or five courses in order to earn a living salary. Funds for research are rarely available, explaining why very little research is conducted. Participants also noted important differences between the United States and Palestine in job security of professors. Once professors are hired in Palestine, they are generally assured a job for the rest of their career, and there are few incentives for quality teaching or research. In the United States, tenure is difficult to obtain and professors who earn it have other incentives to strive for excellence. i. Changes in Leadership Skills The extent to which the PFDP’s Ph.D. participants can contribute to the improvement of Palestinian education in the future will depend not only on the academic, research and teaching skills that they acquire but also on their ability to influence others. For this reason, the team attempted to assess the extent to which participants’ experience in the U.S. enhanced a number of leadership characteristics. The results of these queries are provided in Table 4.6 below. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 35 Table 4.6. Self-assessment on Improved Leadership Skills Very large improve￾ment Significant improve￾ment Modest improve￾ment Little or no change Response Count Being self-reliant and independent. 39% 46% 4% 12% 26 Speaking in public. 11% 59% 22% 7% 27 Listening to others’ suggestions or concerns. 19% 56% 19% 7% 27 Expressing your ideas and feelings. 26% 44% 22% 7% 27 Being tolerant of others different than you. 33% 37% 22% 7% 27 Leading a team and motivating others. 8% 46% 35% 12% 26 Being flexible. 37% 44% 7% 11% 27 Solving problems. 23% 50% 23% 4% 26 Changing plans to adapt to opportunities 26% 41% 33% 0% 27 Working to make changes in your community. 22% 48% 22% 7% 27 Willingness to take risks. 11% 48% 33% 7% 27 Negotiating with colleagues. 22% 59% 15% 4% 27 Summarizing complicated ideas. 15% 67% 15% 4% 27 Working within a budget. 15% 41% 33% 11% 27 Knowing how to advance your career. 30% 44% 22% 4% 27 Totals 22% 49% 22% 7% 100% The reported changes in leadership attributes are quite dramatic. Overall, in the 15 leadership categories, 71 percent of participants rated their improvements as “very large” or “significant.” These data reflect the fact that the participants have successfully tested themselves in extremely challenging situations -- professionally, culturally, socially and in other ways. This experience has built skills and confidence that should aid them in playing leadership roles when they return home. j. Changes in Knowledge and Attitudes about the United States U.S. scholarship programs for international students often lead participants to a better understanding of the United States. The team included several questions in its survey to determine whether this is occurring under PFDP. Table 4.7 provides the responses. Table 4.7. Changes in Knowledge about the United States Substantial Increase Moderate Increase Minimal Increase No Change Don't Know U.S. democracy 63% 30% 0% 7% 0% U.S. economy 37% 48% 7% 7% 0% U.S. values and culture 59% 33% 0% 7% 0% Daily life in the U.S. 56% 33% 7% 4% 0% U.S. religious and ethnic diversity 52% 30% 15% 4% 0% U.S. freedom of speech and press 41% 44% 7% 7% 0% Voluntary community service 41% 41% 11% 7% 0% Averages 50% 37% 7% 5% 0% Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 36 The table shows that more than 80 percent of respondents indicated their knowledge of the United States had increased moderately or substantially in all categories. These scores can be expected to rise further as the participants’ stay in the U.S. continues. This increased understanding of the United States is an important “public diplomacy” result of this program, particularly as these participants will return to university careers where they will help to shape students’ thinking for many years to come. k. Building Professional Linkages Professional linkages are an important way through which academics keep abreast of new developments in their fields. This is particularly true for an area like Palestine, where the scholarly community and research outputs are small. Seventy percent of recipients indicated that they had developed significant new connections with professionals in their field. In a follow-up question that included students and faculty, 93 percent indicated that they had built such relationships. The majority of respondents indicated that their ties with other students included a roughly equal mix of American and foreign students, with only two respondents indicating that their relationships were primarily with foreign students. Although these new relationships are encouraging, evaluation team members have found in conducting other studies that most of these relationships do not last once students return home unless there are funds to do joint research or to take on other collaborative projects. For the two returned PFDP scholars, one has had only minor professional contact with people from his university since returning, and the other has had none. 3. Ph.D. Participants Impact on Higher Education a. Returned Participants’ Experiences Only two of the program’s 34 Ph.D. participants have returned to Palestine. One completed his Ph.D. in Urban Planning in 19 months – an almost impossible feat. The fact that he “missed his family and had nothing to do but study” encouraged him to finish as quickly as possible. He took extremely heavy course loads (as much as 20 units) and prepared for comprehensive exams in ten days. That participant returned to Palestine about six months ago and is currently teaching four courses (Urban Theory, Management of Human Capital, Principles of Urban Planning, and Housing Policies and Strategies) in the Department of Geography at Al Azhar University in Gaza. Although he taught these courses before going to the United States, he now employs “new material and better teaching methods.” This participant is clearly a PFDP success story, having completed his degree with incredible speed and having become a more effective professor. However, completing the program so quickly meant that he was less involved in extra-curricular educational opportunities, did not serve as a teaching or research assistant and had less opportunity to build networks. He reported to the evaluation team that had he been able to take his family to the United States he would have stayed longer and gotten more out of the program. Also, this participant’s re-entry into Palestine was not without problems. His Ph.D. program was not quite what he or his university expected, and this has resulted in some still unresolved issues with his university about what he is qualified to teach. Outside the Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 37 university, he holds a senior position under the Palestinian Authority (PA) but is unable to fulfill his duties because of the current divisions between the PA and the authorities in Gaza. One other grantee has returned to Palestine, in that case after failing his comprehensive exams and dropping out of the doctoral program. He is hoping that his university will award him a Master’s Degree, as is commonly done for Ph.D. candidates who drop out after having successfully completed the Ph.D. course work. That individual was not rehired by his university and is currently unemployed. b. Plans of Future Graduates The evaluation team sought information about the direction individuals’ career plans are taking and how these may have changed since they left Palestine. Most participants (82%) indicated their intention to return to work at the same university at which they were previously working. Although it is obviously impossible to know what returning participants will in fact achieve, it is clear that they have high expectations and will attempt a great variety of programs that portend significant impact on Palestinian higher education and on the broader society. Following is a sample of the projects that participants propose to undertake: • Design new curriculum; • Develop a program to teach quality research methods; • Help colleagues to improve their teaching skills; • Start a foundation for public administration; • Transfer learning to faculty and students; • Undertake and publish research; • Establish a research center; • Establish a think tank; • Conduct policy research; • Be engaged in public life through research, advocacy and publishing; • Start a professional association; • Introduce improved methods of university administration; • Start a special education department; • Integrate children with special needs into regular classes; • Work to increase integrity and creativity in the university; and • Establish the first urban planning department. c. Anticipated Obstacles to Impact upon Returning The principal obstacles that participants perceive include: • The uncertain political environment, which makes all economic activity uncertain; • University politics, which often means that favoritism outweighs performance; • The lack of financial resources in the universities, which make it difficult to launch new initiatives; • The lack of a tradition of quality research, which means that there are few resources to undertake it or rewards when it is conducted; and • The minimal contact that participants in the United States currently have with their home universities, which means that the universities do not keep these individuals in mind and may not make plans to utilize them effectively when they return. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 38 4. Conclusions for the Ph.D. Program Progress of the PFDP scholars and potential impact. Participants’ academic progress has been good. USAID can expect that most of the 32 participants still in the United States will complete their degrees and return to Palestine to resume teaching. Participants report having learned a great deal about their technical areas, improved their research and study skills, become more effective leaders and learned a good deal about the United States. One participant has already successfully completed his Ph.D., returned home and is contributing to his university’s improvement. Thanks to their U.S. training, participants can be expected to make an increased contribution to university development. Management of the PFDP program. The two elements of the program that this evaluation reviewed, while doing well in many regards, have underspent their planned budgets. The implementing partners and USAID have managed PFDP creatively, making changes to on￾going activities to solve unforeseen problems and to address the changing political and economic environment and also adding new activities that offer promise of contributing to project goals. The evaluation team views this active management of implementation as highly commendable. The limited role of Palestinian organizations and its effect. The USG’s “no contact” rule in effect when the project was getting underway coupled with the current ATC requirement have greatly reduced the role that would normally be played in a USAID higher education project by the targeted universities and the Ministry of Education. As a result, PFDP consists largely of a series of interventions planned and implemented by outside organizations. Although those interventions may be effective, they are very likely to end when the project ends. This would be most unfortunate. AMIDEAST has raised this concern and is attempting to address it by “institutionalizing” various program elements within Palestinian institutions. The limited involvement of Palestinian organizations in PFDP’s planning and implementation has also generated some resentment among university administrators. This undermines the public diplomacy benefits that this program has otherwise generated. USAID and AMIDEAST efforts to reach out to and involve Palestinians education leaders will, hopefully, counter that problem and will help to make the program’s activities and impact more sustainable. Concerns over placement. Finding suitable U.S. university placements for 34 Palestinian scholars was undoubtedly a difficult job. Although the majority of placements are working out well, some participants are quite displeased, and they did not always share their concerns with the implementing partners. The evaluation team did not have adequate time to review the individual participant’s stated objectives, the communications between the participants and the implementing partners and the applications that OSI submitted on their behalf. Thus, it could not determine how this problem developed or whether it might have been avoided. Support for participants in the United States. The implementing partners’ support for participants in the United States has been quite good. Participants are generally pleased with the efforts of the implementing agencies and have a high degree of confidence in the individuals working on their behalf. As in all scholarship programs, the participants wish they could be given larger stipends. However, there is little evidence that this is a serious problem. Implementing partners’ efforts to facilitate the travel of dependents has been extraordinary, with the possible exception of one or two cases. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 39 Future USAID support for Ph.D. training. Although the team is inclined to see programs focused on upgrading current faculty as more cost-effective than training additional Ph.D.s, additional Ph.D. training clearly remains a priority for Palestinian universities. Recruitment of professors has become increasing difficult, as older professors (many of whom were trained in earlier USAID programs) retire and others leave to earn better salaries in either international organizations, the private sector or other countries. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 40 5. Recommendations on the Ph.D. Program 1. Expand the role of Palestinian organizations within PFDP. The evaluation team applauds AMIDEAST and USAID overtures to increase the involvement of Palestinian organizations. These efforts should enhance the image of the program and its public diplomacy benefits. The evaluation team recommends that the program creatively pursue Palestinian university involvement, as political constraints allow. Additionally, to the extent USG regulations permit, the team also strongly supports AMIDEAST’s efforts to “institutionalize” improved teaching and learning efforts at Palestinian universities so that the various upgrading efforts that PFDP conducts not cease when the project ends. 2. Open communications regarding participants’ concerns about placement. As participants have been in their respective programs for some time, moving them to another university would not be appropriate. However, the team believes that it would be helpful if the implementing partners acknowledge participants’ concerns and offer to discuss problems, probably on a one-to-one basis, with participants. 3. Address the one or two remaining concerns over family travel. The team believes that USAID should do its best to inform any participants of problems that remain. 4. Utilize other PFDP activities to ensure returned participants’ continued engagement. As participants return to Palestine, AMIDEAST should make every effort to involve alumni in PFDP’s other activities focused on improving teaching and learning. 5. Consider reprogramming options that include additions to the current program. With four years remaining in the project, the partners may wish to consider programming options to address the many unmet needs of Palestinian universities. If resources permit, PFDP might consider programs to upgrade specific faculties. For example, PFDP might organize and invite Palestinian professors to attend a series of two to four week summer workshops offered at one of the Palestinian universities by a prominent U.S. institution. 6. Strengthening linkages among alumni and to the United States. Participants returning from the United States usually expect to continue their professional relationship with their host U.S. university and former classmates. However, those relationships frequently end within a fairly short period. This is unfortunate, as those relationships offer economic benefits through technology transfer and political benefits through improved mutual understanding. USAID should explore whether a functioning alumni association might be an effective vehicle to address this problem. Such organizations need to offer their members tangible benefits (e.g., technical programs in their fields) that require resources. The provision of seed capital (with a plan for financial viability after several years) might be a useful investment. 7. Consider providing additional Ph.D. scholarships as part of any future Palestinian university development program. Providing additional Ph.D. training would clearly benefit Palestinian universities. However, the team is not able to determine whether that is the best investment that USAID might make to strengthen universities. This determination should be made on the basis of a more in-depth study of the Palestinian universities. If USAID determines that it has resources for such training, the team would encourage broadening the eligible disciplines to include business, IT, engineering and other technical subjects that the universities see as priorities. Any new program should, as the political situation permits, increase the involvement of Palestinian universities in planning and implementation. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 41 B. The Short-term Fellows Program The Short-term Fellows (ST Fellows) Program provides experienced Palestinian academicians and administrators an opportunity to make two semester-long visits to the U.S. to work with senior American educators on issues of university teaching, research and administration. Program documentation indicates that these trips are intended to provide Palestinian scholars the following opportunities: (1) to learn alternative approaches to teaching; (2) to develop appropriate curricula for their home institutions; (3) to engage in research in consultation with U.S. academic peers; and (4) to experience other aspects of U.S. higher education, such as administrative structures and student related services.6 The program was initially planned to provide opportunities to 50 Palestinian university faculty members; however, implementation has been slower than anticipated. To date, only eight Palestinian scholars have participated: seven initiated the program in 2007 and one in 2008. A third group of ten scholars, five each from the West Bank and from Gaza, is expected to travel to the United States in early 2009. Given the program’s slow start, AMIDEAST believes that it will not be able to support 50 ST Fellows before the end of the program. To obtain information about the effectiveness of this program, the evaluation team interviewed five of the eight ST Fellows who visited the United States in 2007 and/or 2008. Three of those Fellows serve in senior administrative posts at their respective universities and were able to give a university perspective as well as speak to their individual experiences. The team also interviewed the Vice President for Academic Affairs at An-Najah University. 1. The Program’s Technical Disciplines The program was initially limited, like the Ph.D. program, to the five disciplines previously mentioned. However, implementers learned quickly from experience that Palestinian interest in participation was not as strong as expected. Frequently, faculty members found it difficult to arrange to be away from their universities for a semester. Local universities were often not willing to continue salary payments, creating hardships for the visiting scholar and his/her family. One program response to this problem was to expand the number of eligible disciplines, which now include public health, urban planning, public administration, public policy, political science, international relations, economics, environmental science, education, history, philosophy, sociology, anthropology, cultural studies, literature, and media studies. Most of the 2009 ST Fellows are from disciplines are been recent additions to this list. 2. Selection of Fellows As in the case of Ph.D. scholars, applications for this program come directly from interested individuals rather than from the Palestinian universities. Thus, the role of the Palestinian universities in selecting participants has been limited, although AMIDEAST meets with university leaders to discuss priorities, and there has been dialogue with the universities to encourage them to continue salaries during the period of training. There is evidence that Palestinian university administrators are clearly not pleased by their limited role in PFDP, 6 Source. AMIDEAST website, Programs & Services, Palestinian Faculty Development Program. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 42 although it was not clear to the evaluation team whether their concern pertains specifically to the ST Fellows Program.7 AMIDEAST has established an applicant review process that appears to be quite effective. Particular emphasis is placed on choosing professors and administrators who are strongly motivated and able to work effectively in an unstructured environment. This is particularly important because the participants largely determine their own programs once they arrive at their U.S. universities. Final selection is made by the same Selection Committees (including AMIDEAST and OSI representatives and U.S. university faculty) that are used for the Ph.D. program. OSI then selects the university, the faculty and the mentor based on the background and interests expressed by the respective Palestinian scholar. OSI and AMIDEAST appear to have had excellent success. They have selected candidates who are self-starters and highly motivated. All of the returned ST Fellows that the team interviewed appear to have worked very long days during their visit(s) to the United States. They described taking classes, giving classes, working with professors, conducting research and taking initiative to identify individuals who could be of assistance to their endeavors. They also seem to possess the necessary interpersonal skills to approach busy academics and establish effective relationships. 3. Participants’ goals The goals within this project component are, of course, specific to the interests of the selected Fellow. Some are quite specific, such as designing a specific course. Others are more general, such as becoming familiar with the U.S. approach to special education or bilingual education. Several mentioned their desire to build longer-term relationships with individuals or institutions in the United States. 4. Selection of U.S. Universities and Mentors The selection of universities and mentors has proceeded quite well. Four of the five fellows interviewed were satisfied with their universities. Each felt that the department offered a wide array of professional resources that could be tapped. The mentors in most cases were people who could be very helpful professionally. One Fellow had selected his own mentor and was extremely pleased with the results. Another found that the mentor that OSI had selected was not the right person, but he quickly found another professor within the department with whom he shared interests. One fellow that the team interviewed found both the department and the mentor inappropriate. However, professors in that department put him in touch with professors in other departments, some of whom had interests that overlapped with his own. He also took initiative to meet with professors at other universities in the area where there were shared interests. Although the initial visit did not achieve all of its objectives, it established relationships that set up a very successful second visit. 7 Palestinian university administrators expressed this concern at an AMIDEAST-organized meeting on November 5, 2008. One particularly outspoken critic described the program’s approach as “paternalistic.” Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 43 In sum, although there have been a few problems with respect to placements and mentors, these were successfully resolved because the individual Fellow’s programs are unstructured, and the Fellows themselves have taken the initiative to correct those problems. 5. Achievement of Specific Objectives Although the Short-term Fellows program got off to a slow start, the results achieved to date from the limited investment have been excellent. All of the Fellows interviewed found their experience in the United States extremely professionally useful. Each considered his goals to have been largely achieved, although several of those seeking to do research found that time was not always adequate to finish projects that they had begun. Most came back and found that they could apply what they had learned. One developed an agreement with his host university for a student and faculty exchange program. He also returned with new ideas about university administration, which resulted in his being asked to accept a promotion to the position of Vice President for Administration. Several developed new courses, which have been or will soon be introduced. Although one was demoted from his position as chairman of his department when he returned home, this seems to have been only tangentially related to his trip to the United States. Even in that case, the scholar expects that he will soon introduce the new course and materials that he developed during his visit. 6. Obstacles Encountered Finances. All of the Fellows interviewed complained of serious problems with the program’s financial arrangements. They found that the living allowance was sufficient only to allow them to live as graduate students, in run-down accommodations and/or sharing a room or small apartment with students. However, they are not graduate students but senior faculty and administrators. Their contacts were mostly with other U.S. faculty and senior administrators Returned Short-term Fellows reported that the program’s limited financial allowance sometimes put them in awkward and embarrassing situations with their U.S. counterparts. Rent required up to 70 percent of their stipend. The problem was compounded by the lack of financial support from their home universities, which at best provided partial salary payments while they were away. The implementing partners has taken action to address this issue, adding a $500/month payment mainly to ensure that participants’ family responsibilities at home could be met. Former participants indicated that this adjustment was very helpful, but they feel that the program continues to pose a financial hardship and puts them in an inappropriate position vis￾à-vis their U.S. counterparts. This issue also has deterred some scholars from participating in the program and others have declined to return for the second visit as a result of the financial burden it poses. Housing. Several of the Fellows complained that housing was both time-consuming and difficult to find and expensive. Arriving in cities they did not know, they were not aware of which neighborhoods might be dangerous or which offered access to public transportation. I learned more in four months at [his U.S. university] than I learned in ten years at [my home] university. Evaluation of CSP/PSP and PFDP Aguirre Division of JBS International, Inc. p. 44 C. Conclusions on the Short-term Fellows Program Program Effectiveness. In the limited time the Short-term Fellows Program has been implemented, it appears to be quite successful in providing learning and growth opportunities to Palestinian faculty and administrators. Fellows return with new ideas and enthusiasm and they appear to be making strides in introducing change into their respective universities. Since the cost to USAID is limited to travel, living expenses and program administration, the program appears to be a cost-effective method of introducing change into the Palestinian university system. Although the ST Fellows Program is considerably behind schedule, having thus far sent only eight participants to the United States, the implementing partners have taken action to make the program more attractive. Program Constraints. The most significant constraint on this program appears to be limited demand. This reflects initial limits on the technical disciplines of the program, the reluctance of university administrators to see important faculty members miss a semester of teaching, and the financial burden on the participants and their families. With respect to the financial burden, all of the interviewed participants commented that they were forced to live like graduate students instead of like visiting faculty, and several spent several thousand dollars of their own money. The implementing partners have attempted to address these issues. The technical areas have been greatly expanded and administrative staff has been added; AMIDEAST is attempting to coordinate more closely with Palestinian university administrators, and the financial support has been increased. The evidence suggests that this program yields considerable benefits at low cost. It thus appears to be very cost-effective, and the team suggests that efforts be made to utilize it fully. A further modest upward adjustment to the financial stipend for participants may be desirable to increase Palestinian interest in the program. In addition, OSI efforts to work with university officials prior to the visiting Palestinian fellow’s arrival to arrange short-term housing appropriate to a visiting professor would be very helpful. D. Recommendations on the Short-term Fellows Program 1. Continue efforts to implement this program. The team applauds the efforts of the implementing partners to step-up the level of implementation of this program. The evaluation team believes that this program has demonstrated very positive benefits and encourages the partners to give it high priority. 2. Monitor performance to determine whether further adjustment is advisable in the financial stipend. The implementing partners and USAID should closely monitor the results of the ST Scholars traveling to the United States early in 2009 to see if the very positive results found by this evaluation are the norm. Particular attention should be given to the adequacy of the financial stipend and to participants’ experience finding housing. If a further modest upward adjustment to the financial stipend for participants (e.g., an additional $500/month) would make the program more effective, that cost would be small relative to the potential gains. OSI might also review whether it could work with the host universities to provide housing assistance.