Evaluation of USAID Higher Education Portfolio* *TASK ORDER NO. REE-I-07-05-00053-00 ORDER NO. 09 WITH ACADEMY FOR EDUCATIONAL DEVELOPMENT Volume I Assessment Report Submitted August 29, 2008 Revised Submission November 21, 2008 This program evaluation is made possible by the support of the American People through the United States Agency for International Development (USAID.) The contents of this evaluation are the sole responsibility of the Academy for Educational Development and its authors, and do not necessarily reflect the views of USAID or the United States Government. Evaluation of USAID Higher Education Portfolio* *Task Order No. REE-I-07-05-00053-00 Order No. 09 with Academy for Educational Development Volume I Assessment Report Academy for Educational Development Washington, DC This program evaluation is made possible by the support of the American People through the United States Agency for International Development (USAID.) The contents of this evaluation are the sole responsibility of the Academy for Educational Development and its authors, and do not necessarily reflect the views of USAID or the United States Government. Assessment of USAID Higher Education Portfolio (Volume I) Table of Contents List of Appendices (Volume II)....................................................................................................... i List of Tables .................................................................................................................................. ii List of ACRONYMS .....................................................................................................................iii Chapter 1: Executive Summary .................................................................................................. 1 I. Introduction ................................................................................................................................. 1 II. The State of Higher Education in Pakistan: 2008 ...................................................................... 2 III. Merit and Need-based Scholarship Program ............................................................................ 3 IV. Science and Technology Cooperation ...................................................................................... 3 V. Fulbright Scholarship Program .................................................................................................. 4 VI. College Improvement Grant: Forman Christian College, a Chartered University (FCCU) ..... 4 VII. Endowments and Other USAID and USG Programs.............................................................. 5 VIII. Gender Equity ........................................................................................................................ 5 IX. The Higher Education Commission.......................................................................................... 5 X. Strategy for USAID and Recommendations for the Future....................................................... 6 Chapter 2: Evaluation Objectives and Methodology ................................................................ 8 I. Introduction ................................................................................................................................. 8 II. Overview of USAID Pakistan.................................................................................................... 8 III. Evaluation Objectives ............................................................................................................... 9 IV. Methodology........................................................................................................................... 10 Chapter 3: Goals, Objectives, Achievements, and Summary Findings and Recommendations for a USAID Higher Education Strategy for the Next Five Years......... 13 I. Goals, Objectives and Achievements ........................................................................................ 13 II. Monitoring and Evaluation....................................................................................................... 20 III. General Conclusions ............................................................................................................... 20 IV. Policy Level Recommendations: A Five-Year Strategy for USAID Higher Education Support ....................................................................................................................................................... 21 V. Program-Level Conclusions and Recommendations ............................................................... 23 VI. Higher Education Commission: Conclusions and Recommendations (Portfolio Level) ....... 30 Chapter 4: The Context and State of Higher Education in Pakistan..................................... 34 I. Introduction ............................................................................................................................... 34 II. The Higher Education Commission (HEC), Origins of the HEC ............................................ 35 III. Government of Pakistan and HEC Goals................................................................................ 38 Chapter 5: Merit and Need-based Scholarship Program ....................................................... 41 I. Participant Selection and Awards.............................................................................................. 41 II. Site Visits – Participant Observations...................................................................................... 43 III. Management Capacity and Institutional Strengthening.......................................................... 45 IV. Analysis and Review of the Effectiveness of Monitoring and Evaluation (M&E) Tools...... 45 V. Relevancy, Effectiveness and Efficiency................................................................................. 46 VI. Merit and Need-based Scholarships Conclusions................................................................... 48 VII. Recommendations ................................................................................................................. 49 Chapter 6: Science and Technology Cooperation Program ................................................... 50 I. Proposal Solicitation and Selection ........................................................................................... 50 II. Site Visitations and Relevancy................................................................................................. 51 III. Management Capacity and Institutional Strengthening.......................................................... 52 IV. Analysis and Review of the Effectiveness of Monitoring and Evaluation (M&E) Tools...... 53 V. Effectiveness: Achievement of Project Objectives Against Planned Targets and Intermediate Results........................................................................................................................................... 54 VI. Science and Technology Cooperation Conclusions ............................................................... 56 VII. Recommendations ................................................................................................................. 57 Chapter 7: Fulbright Scholarship Program............................................................................. 58 I. Participant Awards .................................................................................................................... 58 II. Site Visits – Participant Observations...................................................................................... 59 III. Access to Scholarships............................................................................................................ 59 IV. Relevance of Educational Experience .................................................................................... 62 V. Usefulness of the Program ....................................................................................................... 64 VI. Employment Impact................................................................................................................ 64 VII. Monitoring in the US and Follow-up in Pakistan.................................................................. 65 VIII. Benefits & Suggestions........................................................................................................ 65 IX. Management Issues – Capacity and Efficiency...................................................................... 66 X. Analysis and Review of the Effectiveness of Monitoring and Evaluation (M&E) Tools ....... 67 XI. Effectiveness: Achievement of Project Objectives Against Planned Targets and Intermediate Results........................................................................................................................................... 67 XII. Fulbright Program Conclusions............................................................................................ 69 XIII. Recommendations................................................................................................................ 69 Chapter 8: Forman Christian College (A Chartered University) .......................................... 71 I. Strategic Plan............................................................................................................................ 71 II. Innovative Programs ................................................................................................................ 71 III. The Academic Programs and Teaching .................................................................................. 72 IV. Infrastructure........................................................................................................................... 72 V. Administration ........................................................................................................................ 73 VI. Faculty Development and Scholarships.................................................................................. 74 VII. Interfaith Cooperation ........................................................................................................... 76 VIII. Gender Equity ...................................................................................................................... 76 IX. Need and Merit-based Student Scholarships .......................................................................... 76 X. Observations about FCCU Programs....................................................................................... 78 XI. Analysis and Review of the Effectiveness of Monitoring and Evaluation (M&E) Tools...... 79 XII. Effectiveness: Achievement of Project Objectives Against Planned Targets and Intermediate Results........................................................................................................................................... 79 XIII. Forman Christian College University Improvement Grant Conclusions............................. 81 XIV. Recommendations................................................................................................................ 82 Chapter 9: Other USAID and USG Financed Programs........................................................ 83 I. Partnership for Women in Science, Technology and Engineering (STE)................................. 83 II. University of Agriculture (UAF), Faisalabad .......................................................................... 86 III. Lincoln Corners ...................................................................................................................... 87 IV. Department of Gender Studies/University of Peshawar......................................................... 88 V. Human Rights Journal/University of Peshawar....................................................................... 88 VI. Other Endowments: Endowment for Agriculture, University of Peshawar and Endowment for Pakistan Forest Institute, Peshawar......................................................................................... 89 VII. Grants and Endowments Conclusions................................................................................... 89 Chapter 10: Gender Equity........................................................................................................ 91 I. Introduction .............................................................................................................................. 91 II. USAID Supported Institutions ................................................................................................. 93 III. Gender Equity Conclusions .................................................................................................... 96 IV. Recommendations................................................................................................................... 97 Chapter 11: Evaluation of the Budget of the Higher Education Commission ...................... 98 I. Introduction .............................................................................................................................. 98 II. Increasing the Budget for Higher Education............................................................................ 98 III. Monitoring and Budget Controls .......................................................................................... 104 IV. Conclusions........................................................................................................................... 105 V. Recommendations................................................................................................................. 106 Chapter 12: Evaluation of Higher Education Commission and its Implementation of the Medium Term Development Framework (MTDF) and other Government of Pakistan Higher Education Goals ........................................................................................................... 107 I. Introduction ............................................................................................................................ 107 II. Faculty Development ............................................................................................................. 107 III. Improving Access to Higher Education................................................................................ 112 IV. Promoting Excellence in Learning and Research................................................................. 116 V. Quality Improvement and Accreditation................................................................................ 118 VI. HEC Industry/Business Linkages: Relevance to the Economy............................................ 123 VII. Leadership, Governance, and Management ........................................................................ 126 VIII. Improving the Physical and Technical Infrastructure........................................................ 128 IX. Public-Private Higher Education Cooperation and Development...................................... 131 X. HEC Goals in the Context of International Standards and Expectations.............................. 134 XI. General Assessment of the Higher Education Commission at the Portfolio Level.............. 135 XII. Analysis and Review of the Effectiveness of Monitoring and Evaluation (M&E) Tools... 137 XIII. Effectiveness: Achievement of Project Objectives Against Planned Targets and Intermediate Results.................................................................................................................... 138 XIV. Conclusions........................................................................................................................ 140 XV. Recommendations............................................................................................................... 141 List of Appendices (Volume II) The following appendices are compiled in Volume II of this report: I Institutions and People Interviewed by Institution II Profiles of Universities and Institutions Visited III New Programs Proposed for the Higher Education Commission IV The Background and Context to Higher Education in Pakistan V Participating Higher Education Institutions in USAID Merit and Need-Based Scholarship Program VI HEC Merit & Need-based Scholarships: Focus Groups and Site Visits -Participant Observations VII US-Pakistan Science and Technology Participating Universities/Institutions and Research and Development Institutes VIII Science and Technology Cooperation Site Visit Examples (HEC & MoST) IX Fulbright Scholarship Program: Focus Groups and Site Visits -Participant Observations X Report of Thomas Armor- Additional field work in support of AED Evaluation XI Application for Financial Assistance: Forman Christian College (A Chartered University) XII Comparison of Annual Recurring Grant to Institutions of the Punjab Province with their Expenditure on Pay & Allowances 2003-04 to 2007-08 XIII Faculty Development Programs at the HEC XIV HEC University/Industry Research and Linkages Programs XV Private Universities/Institutions Declared Eligible for Public Funding XVI USAID Contribution to the Government of Pakistan Higher Education Policy and Goals: Effectiveness, Efficiency and Relevance of the Programs XVII USAID Contribution to GoP Higher Education Goals Evaluation of USAID Higher Education Portfolio i Academy for Educational Development August 29, 2008 List of Tables # Table Title Page 1 Programs Addressing USAID Higher Ed Indicators 21 2 Results of the Operational Plan for Pakistan through June 15, 2008 24 3 Results of Operational Plans for Pakistan to June 15, 2008 25 4 Number of Universities and DAIs in Pakistan: 1947-Present 42 5 Number of Public and Private Universities & DAIs in Pakistan (2008) 42 6 Summary Program Status of Merit and Need-base Scholarships 48 7 USAID Merit & Need-Based Program (1000 Scholarships) 50 8 Funding for Science and Technology Cooperation Project (million US $) 57 9 Proposals Received and Funded (Phases 1-3) 58 10 HEC/MoST Grants to Date 58 11 Budgets for HEC Project Grants (Rupees million) 62 12 Other USAID &USG Projects 91 13 Enrollments in HEIs: ;2001-2006 by Gender 99 14 Enrollment by Discipline, Level and Gender, 2004-05 99 15 HEC Recurring Budget Allocations 2001-02, 2007-08 106 16 HEC Development Budget 2001-02, 2007-08 106 17 Trends in Higher Education Per-Student Spending 2001-02, 2007-08 107 18 Public Higher Education Institution Revenues 2001-02, 2007-08 108 19 Spending by Public Higher Education Institution 109 20 Enrollment in HEIs (Public and Private), 2001-2006 120 21 Enrollments by Types of HEIs, 2003-08 121 22 Changes in Faculty and Proportion of Ph. D faculty and Teacher-Student Ratios, 2003- 2008 122 23 Enrollment in HEIs, 2001-2006 by Level and Gender 123 24 Enrollments in HEIs 2001-2006, illustrative 139 Evaluation of USAID Higher Education Portfolio ii Academy for Educational Development August 29, 2008 List of ACRONYMS AED Academy for Educational Development AEPAM Academy of Educational Planning and Management AIOU Allama Iqbal Open University AUP Agriculture University Peshawar BOT Build-Operate Transfer CEO Chief Executive Officer COE Centre of Excellence CDWP Central Development Working Party DAI Degree Awarding Institution DDWP Departmental Development Working Party ECNEC Executive Committee of National Economic Council EFA Education For All EMIS Education Management Information System ESR Education Sector Reform FCCU Forman Christian College (A Chartered University) GER Gross Enrollment Rate GIK Ghulam Ishaq Khan (Institute of Engineering) GNI Gross National Income GoP Government of Pakistan HE Higher Education HEC Higher Education Commission HEIs Higher Education Institutions HEMIS Higher Education Management Information System IMS Institute of Management Sciences LAN Local Area Network LEA Local Education Authority LID Learning Innovation Division LUMS Lahore University of Management Sciences M&E Monitoring and Evaluation MTDF Medium Term Development Framework NEP National Education Policy NER Net Enrollment Rate NWFP North West Frontier Province OECD Organization for Economic Co-operation and Development PPP Public-Private Partnership QA Quality Assurance QAA Quality Assurance Authority QAD Quality Assurance Division QAC Quality Assurance Committee QEC Quality Enhancement Cell R&D Research and Development Rs Pakistani Rupee SCHE Steering Committee of Higher Education SMC Scholarship Management Committee STE Women in Science, Technology, and Engineering Program UGC University Grants Commission VC Vice Chancellor Evaluation of USAID Higher Education Portfolio iii Academy for Educational Development August 29, 2008 Evaluation of USAID Higher Education Portfolio Academy for Educational Development Chapter 1: Executive Summary I. Introduction This Evaluation of the USAID Higher Education Portfolio was carried out by the Academy for Educational Development (AED) on behalf of USAID under Contract # REE-I-00-05-00053, Task Order # 09, from April through June 2008. The Evaluation Team began work in April and was completed in August 2008 with field work from April 17 to June 14 2008 and August 4–12. US development assistance to Pakistan began in 1951 as a modest effort to help this newly independent country respond to the turmoil that surrounded its creation and to help foster national development. In later years USAID included substantial assistance to education. Following almost a decade hiatus in US assistance to Pakistan for political reasons, in 2002 the US Government agreed to provide $100 million in assistance to Pakistan’s Education Sector Reform over several years. From 2005 through 2009, additional funds were allocated from the $300 million per year budget for development assistance. US Government support to the education sector has grown in importance in recent years expanding from about $20 million in 2002 to $60 million in 2007 and over $140 million in 2008 annual budget. USAID support to Pakistan’s higher education programs, begun in 2003 and 2004, coincided with a long overdue effort in Pakistan to transform higher education and bring its quality up to international standards. The Higher Education Commission (HEC) was created in 2002 to facilitate that effort and to prepare a master plan for higher education. This evaluation examines the impact of these efforts on higher education in Pakistan, quantifies the contribution of USAID higher education programs to the overall goals and makes recommendations about higher education strategies for the future. USAID’s renewed assistance specifically to higher education was an extremely timely intervention given the decades of neglect of the higher education sector, which at that point was in a disastrous state with only 2.6 percent of its college age students obtaining a higher education. The quality of its academic programs had fallen over the years, and most of its graduates had neither the skills that would make them competitive internationally, nor training that would prepare them for employment in business, industry, or the professions. The aims and objectives of the HEC’s Medium Term Development Framework (MTDF) can be summarized as: improved access to higher education; improved quality with a focus on faculty development and improvement of the infrastructure; and relevance to national needs.1 USAID goals, spelled out in the Interim Strategic Plan,2 focus on increased knowledge, training, and 1 Ibid, MDTF, pp. iii-v. 2 USAID/Pakistan, Interim Strategic Plan May 2003-September 2006, Islamabad, May 2003, pp. 6, 10-18. Evaluation of USAID Higher Education Portfolio 1 Academy for Educational Development August 29, 2008 infrastructure development in higher education as well as improved education policy, planning, delivery, and increased access. Thus, there is a high degree of congruence between the goals of the GoP and USAID. The evaluation team examined each of USAID’s higher education programs and presents it findings in the chapters which follow. The team also reviewed the state of higher education, the work of the HEC, the financial situation of higher education as represented by the HEC budget, and the current state of higher education in the context of the MTDF. The team drew conclusions about the effectiveness of these programs and made recommendations about them. At the policy level, the team made suggestions about strategies that might be employed by USAID over the next five years. II. The State of Higher Education in Pakistan: 2008 The state of higher education in Pakistan is improved from its condition in 2002 when the HEC was established. The percentage of students in higher education has increased from 2.6 percent of collage age students to 3.9 percent. About 3,850 faculty members have been supported for indigenous Master’s and PhD training, with the first of them now obtaining their degrees and beginning to make an impact on quality. In addition, more than 400 from 2005- 2008 Master’s and PhD students have been part of the Fulbright Program. While the percentage of PhDs, earlier at 23 percent of total faculty in 2002, has not increased due to the rapid increase in enrollments (faster than PhDs can be produced), absolute numbers of PhDs have grown from 2,579 in 2003-4 to 3,645 in 2007-8. Enrollments have increased an average of 14.7 percent a year since 2003 – far better than the HEC target of 10 percent a year. Progress has been made in infrastructure improvement in laboratories, IT, equipment, and buildings. Teacher training for universities has also been impressive with more than 8,300 faculty members participating in service training courses since the HEC began its work. Publications have increased over the last few years with a 30 percent increase in publications in impact factor journals between 2005 and 2006. There is evidence that research currently underway, and reviewed by this team, will have tremendous practical importance to development in Pakistan. Some of it is funded by USAID through support of the Pakistan-US Science and Technology Cooperative Program. The overall budget of the HEC, recurrent and development, has grown 350 percent since the HEC came into being. That has had a significant impact on the nation’s universities and research institutions. At the same time, it is important to emphasize, that many of the gains represent “catch-up” given the neglect of higher education for many years. Pakistan still remains far behind most of its neighbors in terms of quality of higher education, access, and capacity. While transformation has taken place, it will only continue if the government continues to provide the funding and support promised. The opportunity for higher education in Pakistan to move into the ranks of high quality providers is now a real possibility, but as the most recent ranking of higher education institutions world wide demonstrates, even in 2008, not a single Pakistan university ranks in the top 500 in the world. There remains a great deal more that must be done. Recent government cuts in the HEC Evaluation of USAID Higher Education Portfolio 2 Academy for Educational Development August 29, 2008 budget (see Chapters 3 & 11) do not bode well for the future. There are other serious challenges to success as we discuss in more detail in Chapter 12. The MTDF provides a vision and blueprint for higher education transformation. The progress made to date is substantial; but higher education in Pakistan has a long way to go before it will achieve its goals and produce the quality of higher education needed to make its graduates competitive in the global market, provide the basis for a knowledge economy essential for national development, and produce conditions that will assure the well-being of Pakistan’s citizens. We saw substantial improvement since the World Bank assessment of higher education in 2005-6. III. Merit and Need-based Scholarship Program The Merit and Need-based scholarship program, through the HEC and funded by USAID, is available to needy students admitted to universities in the fields of agriculture and business administration. To date, more than 900 poor and disadvantaged students have had access to higher education thanks to this program. In a context in which 70 percent of the population lives on less than $2 per day, this has been an important part of the effort to increase access to higher education. Students interviewed were very pleased with the program and most reported they could not have attended a higher education institution without it. Most of those receiving scholarships successfully complete their program. The program suffered from management and organizational difficulties at the outset which have largely been overcome. However, late payment continued to be a serious problem in 2008 – in some cases causing real hardships. We could find no good reason for the delays. The team recommends greater decentralization of the program by HEC, improved efforts to make payments on time, and streamlined reporting mechanisms. The HEC acknowledges these issues and new policies were put in place in August 2008 which we believe will improve the timeliness of payment performance. The number of female participants is low at only 17.5 percent. A major outreach effort is needed to assure that secondary school girls understand that funding is available if they are admitted to approved universities. Expansion of the subject areas eligible to include the sciences, health fields, and quantitative social sciences will also reach fields that attract more female students. The current choice of fields unintentionally discriminates against women. The Merit and Need-based Scholarships provide an effective program which meets the access goals of HEC and USAID, as well as goals designed to help people from disadvantaged and underprivileged areas. The program has leveraged additional donor funding from the Japanese International Cooperation Agency. We recommend that the program be continued with appropriate staff increases at the HEC. IV. Science and Technology Cooperation The US-Pakistan Science and Technology Cooperation project has provided significant capacity building for staff at universities and institutes in applied research, assisted with critical Evaluation of USAID Higher Education Portfolio 3 Academy for Educational Development August 29, 2008 technology transfer, and holds the promise of important contributions to national development in critical areas such as agriculture, health, and water resources. It has contributed to HEC goals of faculty development, quality improvement in science and technology, and expansion of the research sector. Links with US universities and research institutions were very effective, although exchange visits continue to be hampered by US delays in issuance of visas and security issues for US researchers. The project has suffered from its complex trifurcated structure in that no institution has responsibility for dissemination of information and research findings. The program is well run, funding tightly controlled, and its academic quality carefully monitored. The HEC and Ministry of Science and Technology need to insist that the results of the program are tracked. The team was not able to obtain data on seven USAID performance indicators since there were few records of them being tracked or reported. Where we did find data, progress seems to be substantial. The team recommends that an innovation unit be created at the HEC to help ensure that the findings of this applied research reach the application or commercialization stage. The evaluation team recommends support of the program. At the present time only one third of the highest ranked research projects were funded due to a lack of funds. There is a large pool of important research waiting for funding. The quality of the programs reviewed suggests that expansion of the project would increase the contributions to national development. V. Fulbright Scholarship Program The Fulbright Scholarship Program is making a major contribution to the MTDF in terms of faculty development, quality improvement, research capacity, linkages with US universities, and mutual understanding. Since the program was reestablished in 2005 with USAID funding, it has produced 412 Masters and PhDs. The return rate of scholars from the US is almost 100 percent. Training at high-quality institutions is critical to the success of the MTDF. The program should increase its outreach efforts to women and to disadvantaged students, especially from the border areas. It should expand its efforts to prepare disadvantaged students for graduate study in the US. Students should be encouraged to take a six to twelve month internship or field experience following their Fulbright scholarship. VI. College Improvement Grant: Forman Christian College, a Chartered University (FCCU) This grant has assisted FCCU to make significant progress in development of a US style liberal arts curriculum, in expansion of its science program, and in development of a four-year undergraduate curriculum focused on problem solving, active learning, and interaction between students and faculty members. FCCU is a model for the institutional transformation needed to bring higher education in Pakistan to international standards. Although there were concerns about administration of the USAID grant, those issues are resolvable and the required changes are underway. FCCU was criticized in one USAID report Evaluation of USAID Higher Education Portfolio 4 Academy for Educational Development August 29, 2008 for lack of transparency related to the selection processes, religious bias, and inattention to student demands. These findings in the USAID report were not supported by the information gathered in our three visits to the University. The institution serves as a model of inter￾religious harmony with the majority of students Muslim, the selection processes were well advertised and transparent. The student body is excited about the new curriculum and eager to be part of the change process underway. We recommend that the USAID programs at Forman be continued. The team was impressed by the progress made since the University was returned to private control in 2003. While much more needs to be done to achieve its goals, it is an excellent example of the kinds of transformation that should be taking place in all higher education institutions in Pakistan. VII. Endowments and Other USAID and USG Programs The three endowments administered through USAID are designed to encourage agricultural teaching and research, faculty development, technology transfer, applied research, and agricultural development. Only the endowment at the University of Agriculture, Faisalabad, has been funded to date, but it is off to an impressive start. We recommend that USAID review the arrangement made in 2005 with the University of Peshawar and the Pakistan Forest Institute. The institutions’ plans show great promise for making a contribution to national development. There are several smaller projects including in Woman’s Studies, Fatima Jinnah Women’s University, and Lincoln Corners. All were judged to be proceeding very well and are discussed in more detail in Chapter 9. VIII. Gender Equity Although there has been some improvement, the percentage of women receiving benefits from USAID funded programs is lower than expected. In some parts of the country a long history of religious and cultural factors remain powerful forces in discrimination against women. Nonetheless, a more concerted effort needs to be made to overcome these barriers, to encourage girls to attend secondary school and to follow courses of study that allow them to go to university. They also need to be more aware of the opportunities that are available for support. Programs such as that funded by USAID at Fatima Jinnah Women’s University are good models of the benefits that can result. The key to progress is increasing the number of women in all programs to the point where there is a critical mass of women in higher education who are able to encourage others to join them. We recommend that the HEC encourage universities to develop a comprehensive program to encourage women to enroll in higher education, especially women from disadvantaged and border areas. IX. The Higher Education Commission Evaluation of USAID Higher Education Portfolio 5 Academy for Educational Development August 29, 2008 We evaluated progress on major goals of the HEC as spelled out in the MTDF. That assessment can be found in Chapter 11. In brief, the team found that significant progress has been made on almost all of these goals and urge continued support for the HEC and the transformation of higher education. Faculty development, increased access for students (at almost 15 percent a year), science and technology improvement, infrastructure development, and quality improvement have been particularly successful areas. We applaud the leadership of the HEC for its vision, planning, and project implementation. It is essential to maintain the autonomy and integrity of the HEC as currently established if progress is to continue. Our assessment of the HEC since its inception is laid out in Chapter 12. We conclude that: the long overdue transformation of higher education in Pakistan under the leadership of the HEC has made solid progress over the last five years. What is most impressive is the breadth of reforms – faculty development, quality assurance, infrastructure rehabilitation, information technology staff training, research facilities, academic publications, increased access, improved accountability, expansion of campuses, and improved salaries and working conditions – and its insistence on quality improvement throughout the system. While the progress to date is notable, the situation is very fragile. Progress will continue only if government keeps its funding commitments and external donors such as USAID continue to support these efforts. X. Strategy for USAID and Recommendations for the Future Higher education transformation in Pakistan has been successful over the last five years – one of the most successful models in the developing world. However, higher education is at a critical stage in its development and needs strong continued support from the GoP and external donors, including USAID. The goals of the MTDF and the GoP are being realized broadly and while substantial progress has been made, continued investment and improvement are necessary if Pakistan is to catch up with its neighbors in the region and reach a stage of development that will allow it to be competitive globally. USAID should build on its very successful programs in higher education and possibly expand its efforts in higher education over the next five years. Its contribution to date has been particularly helpful in spurring high quality faculty development, increasing access to poor and disadvantaged students, increasing applied research in critical areas for national development, improving links with US institutions, and fostering a culture of quality and high standards. In addition, USAID programs have engendered a tremendous amount of good will among students, faculty, and higher education leaders in a difficult political climate. The Fulbright program is a vital contributor to preparing well-trained Master’s and PhD students so essential to the success of the plans to establish university and research centers of excellence. The Fulbright program has shown great progress in its efforts to foster gender equity. Those trained in the US have an almost 100% return rate and are eagerly sought by universities, government, and other employers. Evaluation of USAID Higher Education Portfolio 6 Academy for Educational Development August 29, 2008 Special attention should be given to improving gender equity, recognizing the long history of cultural and religious discrimination against women. Similarly, special efforts need to be continued to help disadvantaged students throughout Pakistan, but especially in the border areas which have suffered from neglect and economic deprivation for decades. Additional support to higher education is recommended for programs of teacher training, enhancing social science and policy analysis programs, and training for middle management and for technical personnel in science, technology, and IT. USAID should also work with the HEC to identify additional areas in which funding might be provided. The GoP should be encouraged to continue its strong support of higher education to bring it up to international standards and set the conditions to spur national development. No developing nation has moved into the ranks of developed nations without a high quality higher education system.3 While a great deal of progress has been made in higher education in Pakistan, it has a long way to go to become a high quality system. External assistance, including USAID grants and endowments, will be essential to make that happen. 3 For a discussion of these findings along with data see: World Bank, Construction Knowledge Societies: New Challenges for Tertiary Education, Washington DC, 2002. Evaluation of USAID Higher Education Portfolio 7 Academy for Educational Development August 29, 2008 Chapter 2: Evaluation Objectives and Methodology I. Introduction This study was undertaken in accordance with an agreement between the United States Agency for International Development (USAID) and the Academy for Educational Development (AED) for an independent evaluation of USAID’s higher education portfolio under the USAID FORECAST IQC number REE-1-00-05-00053-00, Task Order # 09, in response to RFTOP No 391-2008-009. The material that follows presents our evaluation of the higher education portfolio, the condition of higher education in general, and recommendations about a USAID strategy for higher education in the future and the kinds of programs it might consider funding over the next five years. II. Overview of USAID Pakistan The history of US bilateral assistance to Pakistan goes back to 1951 with a modest effort to help the newly independent country respond to partition and the tremendous population upheavals that ensued. It focused on infrastructure, food, and technical support. The programs grew to a total of almost $400 million per year by 1960. They totaled $5.1 billion over the years up to 1982.4 From 1982 to 1993, USAID provided another $2.28 billion for development projects. There was a seven year pause in USAID due to the US-imposed nuclear non-proliferation sanctions against Pakistan. The mission reopened in July 2002. In 2002, the US Government signed a five-year, $100 million agreement with the GoP to support its Education Sector Reform initiative. USAID supports both basic and higher education. The annual appropriation has grown from $20 million in 2002 to $60 million and now over $140 million annually and includes the largest Fulbright Program in the world. The higher education portfolio is under the foreign assistance objective of “Investing in People” and is designed to “Encourage the adoption of conducive social policies and deepen the capabilities of key social institutions, which includes establishing the relative roles of public and private sector service delivery.” The goals are spelled out further in the USAID/Pakistan Interim Strategic Plan, May 2003-September 2006 which is designed to “aggressively address the root causes of instability within Pakistan.”5 USAID’s education objectives for Pakistan focus on Strategic Objective 3 which includes to: “Increase knowledge, training, and infrastructure to develop high quality education programs for girls and boys throughout Pakistan.”6 That includes improvement and strengthening of education policy making and planning, improved delivery of educational services, and increased access to higher education. Educational growth and development also have important impacts on Strategic Objectives 4, 6, and 7 being: a more participatory, representative, accountable and democratic Pakistan; increased economic opportunities for the poor; and improved health for vulnerable populations.7 We note that in its study of knowledge societies and economic development, the World Bank concluded that no 4 USAID, History of USAID in Pakistan, http://www.usaid.gov/pk/mission/background/index.htm, dl. 5/25/2008. 5 USAID/Pakistan, Interim Strategic Plan May 2003-September 2006, Islamabad, May 2003, p. 6. 6 Ibid. p. 6. 7 Ibid. pp. 10-18. Evaluation of USAID Higher Education Portfolio 8 Academy for Educational Development August 29, 2008 society reaches the stage of a developing nation without a high-quality higher education system. The Bank notes that: “Indeed tertiary education is central to the creation of the intellectual capacity on which knowledge production and utilization is dependent….”8 Creation of a knowledge economy is central to national development. Current USAID goals are designed to do just that. In addition to specific support for Pakistan in the areas noted above, the US has provided almost $800 million in non-project assistance to the GoP and released a similar amount of Rupees to support the Federal Budget, which has an indirect affect on the national education budget. USAID is currently investing $86 million in higher education through a variety of programs including the Fulbright Program, need and merit-based scholarships, a college improvement program, science and technology cooperation, several endowments in agriculture, gender studies, and other capacity building projects. The evaluation of these and other projects follows. III. Evaluation Objectives The project team undertook an evaluation of USAID funded or administered projects at the program, portfolio, and policy levels. The HEC was reviewed at the program and portfolio level. The objectives of the evaluation are specified below: A. The Program level: 1. To assess the programs funded by USAID in terms of the following: i. What was their relevance in the context of Pakistan and its goals? ii. How efficient were each of them both in terms of goals and use of resources? iii. What was their effectiveness? 2. To review the achievements of the programs against planned targets including: i. Increased knowledge; ii. Increased training levels; and iii. Improved infrastructure. 3. To assess programs against intermediate results based on the PMP including: i. Strengthened educational sector policy making and planning, especially in relation to the Higher Education Commission (HEC); ii. Increased access to higher education. 4. To describe the successful achievements and challenges of the programs. 5. To document lessons learned. 6. To highlight innovative strategies and interventions. 7. To assess gender balance in the scholarship and other programs and make recommendations for improvement. 8. To assess the effectiveness of the monitoring and evaluation by USAID. 9. To assess any changes made by USAID to improve project management, project planning, and implementation. B. The Portfolio level: 8 World Bank, Constructing Knowledge Societies: New Challenges for Tertiary Education, Washington DC, 2002, p. 8. Evaluation of USAID Higher Education Portfolio 9 Academy for Educational Development August 29, 2008 10. To review the Higher Education Commission’s plans in higher education, including budget levels and provide insights as to how USAID’s programs can supplement and complement them. 11. To assess the relevance, efficiency and effectiveness of the projects overall in meeting the objectives of USAID, the MTDF, and other higher education goals of the GoP & the HEC. 12. To determine what recommendations can be made after reviewing USAID projects, for a higher education strategy relevant for Pakistan and consistent with USAID mission? What would such a strategy look like? What would be its budget implications? C. The Policy level: To undertake an assessment of USAID and GoP (through the HEC) policy as laid out in the USAID/Pakistan Interim Strategic Plan: May 2003-September 2006 and the GoP MTDF and subsequent policies of the HEC. 1. To assess higher education achievements and plans in terms of international expectations and Pakistan’s current level of development. 2. To review the Higher Education Commission’s plans in higher education, including budget level and provide insights as to how USAID’s program can supplement and complement them. 3. To review USAID agency-wide higher education goals and provide the Mission with recommendations for a five-year higher education strategy relevant for Pakistan and consistent with current political and strategic interests of the Mission. The evaluation of the higher education programs and HEC was conducted within the context of objectives described in the following documents: the USAID/Pakistan Interim Strategic Plan May 2003 - September 2006, the FY2007 Operational Plan for Pakistan, the Mission Performance Plan FY 2008 for Pakistan, USAID’s Strategic Plan: Fiscal Years 2007-2012, and the Project Monitoring Plan (PMP) Indicators. These objectives are listed in Chapter 3. IV. Methodology The project team was made up of Dr. Fred M. Hayward, team leader, an experienced international consultant and former university professor, department chair, and dean; Dr. Gerald Boardman, a higher education specialist with more than twenty years of experience in international education; Dr. R. A. Farooq, a university professor and educational practitioner with experience in the Ministry of Education, and Mr. Rafiq Jaffer, a participant training specialist. A follow up visit to Pakistan was undertaken between August 4-12 by Dr. Thomas H. Armor, a management and organization development specialist. The team worked closely with Dr. Randy Hatfield and Ms. Aabira Sher Afgan of USAID on the goals for the study, the Plan of Work, and the schedule for the evaluation. They provided invaluable advice and assistance in the course of this evaluation. The Mission provided the team with a large number of documents, correspondence, and other material about the higher education projects funded by USAID. Team members also collected a great deal of data and other material about higher education generally, specific material about higher education change, Evaluation of USAID Higher Education Portfolio 10 Academy for Educational Development August 29, 2008 The Higher Education Commission (HEC), various universities and institutions, and other relevant programs. We reviewed the budget for the HEC and budget data on many of the programs, assessed reports on USAID projects and carried out a wide range of interviews over the eight weeks the team was in Pakistan. Interviews were conducted with officials in the Ministry of Education, the Ministry of Science and Technology, the advisor to the Prime Minister, the Planning Commission and in other organizations. The team received generous assistance from everyone who was approached. They are especially appreciative of the assistance and information provided by the HEC leadership and staff. The team utilized the following methods in the course of this evaluation: • A review of available data on higher education in Pakistan, USAID reports and documents on the USAID higher education portfolio, HEC planning documents including the Medium Term Development Framework (MTDF), annual reports, protocols, budget data, education staffing and enrollment statistics, and other documents; Ministry of Education plans and other relevant documents including the draft National Education Policy 2008; USAID/Pakistan Interim Strategic Plan May 2003 – September 2006; USAID, History of USAID in Pakistan, USAID, Mission Performance Plan FY 2007, 2008; USAID, FY2007 Operational Plan Pakistan; US Department of State and USAID, USAID Strategic Plan: Fiscal Years 2007-2012; Government of Pakistan, Expenditures on Education 2005-06; Ministry of Education, Vision Discussion Document 1; Ministry of Education, Education Policy 1998-2010; Higher Education Commission, Annual Reports, 2002-2007; World Bank, Higher Education Policy Note Pakistan: An Assessments of the Medium Term Development Framework (2006); United States Educational Foundation in Pakistan (Fulbright Commission), Annual Report 2006 & 2007. • Discussions with USAID Islamabad staff including about the Scope of Work, another about Key Questions and the Plan of Work for the evaluation, as well as other advice and assistance provided by USAID. • Open-ended interviews with 117 people (See Appendix I for list of people interviewed) in Islamabad, Rawalpindi, Faisalabad, Lahore, Karachi, and Peshawar with: o Senior administrators and staff of the Higher Education Commission; o Vice Chancellors or Rectors and senior staff, heads of institutes, deans, faculty members, and program administrators to get an overview of the institutions and their assessment of higher education changes since 2002; o A sample of students receiving merit and need-based scholarships in various HEIs; o Principal Investigators and staff of science and technology projects to get an overview of their research programs and the progress made to date, and the relevance of the research to national development needs. Interviews with the Senior Program Officer at the National Academy of Science and US partners in several of these projects. • Interviews with officials at the Ministry of Education; Science and Technology, and Office of the Prime Minister (See Appendix I a list of people interviewed). • Site visits to eighteen Universities and Institutes in Islamabad, Peshawar, Rawalpindi, Lahore, and Karachi. Interviews with more than 150 students who received USAID Evaluation of USAID Higher Education Portfolio 11 Academy for Educational Development August 29, 2008 supported scholarships (See Appendix II for brief descriptions of the Universities and Institutions visited). • Site visits to the Higher Education Commission, the US Educational Foundation, and others related to the project goals. • Focus groups with Fulbright returnees during three sessions in Islamabad, Karachi, and Lahore. • A matrix developed to help the team identify measures of relevance, efficiency and effectiveness for each of the programs reviewed. • Interview of a random sample of USAID funded need-based scholarship students, faculty participants in the USAID-funded Master’s and PhD programs, and general faculty at Forman Christian College University. • Analysis of quantitative and qualitative data gathered during the course of the evaluation. • The follow up visit involved interviews with administrators, staff, faculty and students where appropriate, meetings with USAID staff, and review of pertinent documents. That visit focused on administrative, monitoring, and evaluation issues at the Higher Education Commission, Forman Christian College University, and the US Education Foundation Pakistan (USEFP). Evaluation of USAID Higher Education Portfolio 12 Academy for Educational Development August 29, 2008 Chapter 3: Goals, Objectives, Achievements, and Summary Findings and Recommendations for a USAID Higher Education Strategy for the Next Five Years I. Goals, Objectives and Achievements A. Goals In the chapters which follow are the team’s detailed evaluations of USAID funded and administered programs in Pakistan over the last five years. This chapter presents our major findings and overall recommendations for a future oriented strategy for USAID for the next five years. We present these policy level recommendations in the context of the main higher education USAID goal as applied to Pakistan: “The goal of USAID’s work in higher education is to improve the human and institutional capacity of host country higher education institutions (HEIs) to effectively contribute to economic, political and social development. USAID achieves this goal via a strategy of country-driven investments in programs, policies and institutional relationships that strengthen and engage host country higher education institutions as core development assets.” B. Summary Achievements - Indicators A summary analysis of program achievements in terms of the main USAID goal was conducted using the following USAID indicators of strengthened capacity: - short and long term training programs in technical sectors of critical importance to Pakistan’s development - degree programs in fields of critical importance - technical assistance and extension programs in a range of technical sectors (health, agriculture, infrastructure information communication technology, natural resource management, business development, science and engineering) - applied research - policy analysis The summary, findings and recommendations are based on a review of the following USAID￾funded and administered programs and the post-2002 implementation of the Medium Term Development Framework (MTDF) by the Higher Education Commission (HEC). The programs are: - Merit and Need-Based Scholarship Program Evaluation of USAID Higher Education Portfolio 13 Academy for Educational Development August 29, 2008 - Fulbright Program - Science and Technology Cooperation Program - Forman Christian College Improvement Grant - Agriculture University of Peshawar - Fatima Jinnah Women’s University - University of Peshawar Table 1 summarizes the findings for each of these programs in terms of USAID’s higher education indicators noted above. The ensuing chapters will describe the findings and recommendations in greater detail. Table 1: Programs Addressing USAID Higher Ed Indicators Short/Long￾Term Training Degree Programs T.A./Extension Tech Sectors Applied Research Policy Analysis Merit/Need ● ● Fulbright ● ● ● Science/Tech ● ● ● Forman Christian C ● ● ● Agric U-Peshawar ● ● ● Fatima Jinnah Women’s U. ● ● ● Univ Peshawar ● Higher Ed Commiss. ● ● ● ● ● C. Objectives In addition to the agency-wide goal and indicators described above, USAID-Pakistan’s achievements have been operationalized through objectives described in a number of documents, including the USAID/Pakistan Interim Strategic Plan May 2003 – September 2006, the 2008 Mission Performance Plan for Pakistan, and the FY2007 Operational Plan for Pakistan. We have focused on the Investing in People objective, Program Area 3.2 for Education, Program Element 3.2.2 for Higher Education, and the Sub-Elements and Indicators that fall under this Program Element. We have summarized the primary sub-elements below, along with a few additional objectives from key documents. This is followed by tables summarizing the key indicators under the Higher Education Program Element, along with indicators included in the 2007 Operational Plan. What is striking is the congruence of USAID achievement with the Pakistan Medium Term Development Framework as discussed in detail elsewhere in this report. Policy Development and Reform (Sub-Element 3.2.2.1). Over the last six years, overall USAID support for the HEC and the MTDF has added substantially to the HEC’s ability to improve education policy making. Today Pakistan is one of the world’s leaders in education innovation and has made more progress in improving the quality of higher education than any other developing nation during this period. Support for Forman Christian College University is also making a significant impact in allowing it to move to a four year, US style, liberal arts and Evaluation of USAID Higher Education Portfolio 14 Academy for Educational Development August 29, 2008 science curriculum that is building state-of-the-art capacity and serving as a model for other universities in Pakistan. A perhaps unintended consequence of the development of the Merit and Need-based Scholarship program is the role it has played in helping participating institutions develop their own policies for education policy focusing on students from less developed areas. They have launched outreach programs, developed systems to check eligibility and family income, and to advise students. They are, for the most part, way ahead of institutions not participating in this program. Institutional Capacity Development (Sub-Element 3.2.2.2). The Fulbright Program, the Merit and Need-based Scholarships, HEC support, the FCCU capacity building grant, and support for Fatima Jinnah University are all making major contributions to infrastructure development by providing training for students at all levels. Those at the graduate level in particular will return to Pakistan to enhance the capacity of university education, government, and business, which is suffering from a shortage of faculty members with Master’s and PhDs in critical areas. Forman Christian College University is a key institution with an academic focus that includes problem solving, increased interaction between faculty members and students, active learning and individual creativity. The HEC master plan for higher education also includes improved academic programs that emphasize teaching and learning following this model. FCCU is revising the academic curriculum of 18 programs with help from its USAID grant and it is already showing results. We see this as having benefits far beyond academic life and adding to the quality of higher education that is needed to create a more participatory, democratic Pakistan. The process is also improving accountability in higher education. Regional, National and Local Development (Sub-Element 3.2.2.3). Increasing the capacity and quality of higher education, is key to national, regional and local development. USAID programs in Science and Technical Cooperation, the Fulbright Program, support for FCCU, the Merit and Need-based Program and support for HEC programs make important contributions to that effort. The results of these programs, as spelled out in the chapters that follow, are clear testimony to their centrality to development in Pakistan. Professional Development (Sub-Element 3.2.2.4). The contributions of USAID programs to knowledge and training have been significant through the Fulbright program, with 412 fellowships to date since 2005 and 196 additional scholars selected for 2008. The Merit and Need-based Scholarship Program has assisted 906 scholars to date, allowing students who could otherwise not afford higher education the opportunity for such training. Support for Fatima Jinnah University has consisted of 26 students at the Bachelor’s and Master’s levels. Added to that are the contributions of the US-Pakistan Cooperative Science and Technology Program which is providing training in state of the art techniques to both students and faculty members. The endowment for the University of Agriculture (UAF), Faisalabad is at the early stages but is providing funds for agricultural ressearch in critical areas for Pakistan. We were impressed by the care being taken in selecting projects for research. The endowment will also fund research and training over the long term – a lasting benefit of this joint Agriculture/Education project. Host Country Strategic Information Capacity (Sub-Element 3.2.2.5). The Merit and Need￾based Scholarship program involved development of extensive selection, reporting and record keeping processes that involved expansion of information management of the 11 participating Evaluation of USAID Higher Education Portfolio 15 Academy for Educational Development August 29, 2008 institutions. In addition, FCCU received assistance from USAID to install a new management information system. Increased Access to Higher Education (USAID/Pakistan Interim Strategic Plan May 2003 – September 2006, IR 3.4: Expanded Public-Private Partnerships to Improve Access and Delivery of Education Services). There has been an 89% increase in access over the last five years, far ahead of the HEC target of 10% per year. The Merit and Need-based Scholarship program is especially important to this effort in that it targets students from poor families – those most in need of such support. The team was impressed by the care taken to verify family income, advertise the availability of these fellowships, and provide advice to the students. Economic Growth and Development (Mission Performance Plan FY 2008: Performance Goal 5). Higher education is the key to economic development as spelled out in detail in the 2002 World Bank Report on higher education and knowledge societies. Most directly linked to those efforts are the Fulbright scholarship programs, the Merit and Need-based Scholarship Program, the endowment at the University of Faisalabad, and scholarship support for Fatima Jinnah University. We note in particular the outstanding academic programs at Fatima Jinnah University which are producing highly trained women graduates in highly technical fields, preparing them for leadership positions in government, business, and universities, and helping to build a critical mass of women professionals who will help break down the biases against women that have had such a devastating effect on Pakistan (see Chapters 8 and 9). The following tables summarize data collected against the Higher Education Program Element 3.2.2 indicators, along with indicators included in the USAID Pakistan 2007 Operational Plan. Evaluation of USAID Higher Education Portfolio 16 Academy for Educational Development August 29, 2008 Table 2: Results of the Operational Plan for Pakistan through June 15, 2008 Science and Technology Cooperation Program, Fulbright Program, and Merit and Need￾based Scholarship Program Based on a Table (p. 168) from the USAID Pakistan 2007 Operational Plan Operation Plan Indicators (From 2007 Operational Plan) By 10/01/2006 (From 2007 Operational Plan) Actual by 6/15/2008 (from team assessment) Number of higher education partnerships between US and host country higher education institutions that address regional, national, and/or local development needs. 11 21 Number of research institutions participating NA 10 Number of US partners NA 46 Number of host-country individuals receiving USG funded scholarships to attend higher education institutions NA 1491 Percentage of women Merit and Need based Fulbright 33% 37% Ag. 14.8%, Bus. 20.7% 40% Number of host-country individuals trained as a result of USG investments involving higher education institutions 15 271 Number of people trained in operational research 99 NA* Number of higher education partnerships between US and host country higher education institutions that address regional, national, and/or local development needs 681 1491 Number of host country individuals completing USG￾funded exchange programs conducted through higher education institutions 0 67 Number of host-country institutions with increased management or institutional capacity as a result of USG investments involving higher education institutions NA 24 *This data was not available at the time of the review, see Chapter 6. Results Achieved with USG Assistance By 10/01/2006 Actual by 6/15/2008 Number of USG-funded scholarship and Exchange programs conducted through higher education institutions 1 1 Number of host-country individuals receiving USG￾assistance Number of Men Number of Women 104 69 35 412 114 82 Number of Masters and PhDs completed 0 67 Number of outreach sessions conducted 200 255 Number of students reached through outreach sessions 15,000 40,302 Evaluation of USAID Higher Education Portfolio 17 Academy for Educational Development August 29, 2008 Table 3: Results of Operational Plans for Pakistan to June 15, 2008 Merit and Need Based Scholarships Program, Fulbright Program & FCCU Numerical Results Achieved with USG Assistance Results Achieved with USG Assistance By 10/01/2006 Actual by 6/15/2008 Number of host-country individuals receiving USG￾Funded scholarships to attend higher education institutions (HEIs in Pakistan) 784 1,073 Number of participating institutions that benefited from capacity building to help needy students, develop effective techniques to identify meritorious but needed students, and provide them with guidance 0 11 Number of USG-funded scholarship and exchange programs conducted through higher education institutions 1 1 Number of institutions with improved Management Information Systems (MIS) as a result of USG Assistance 1 1 Number of institutions that have used USG- assisted MIS system information to inform administrative/management decisions 1 0 Number of library books purchased with USAID funding 0 2,833 D. Summary Achievements – Specific Outcomes The team found clear evidence of progress toward meeting USAID’s goal of capacity-building of HEIs, presented below with brief excerpts of outcomes on the Mission’s Operational Performance Indicators. More detail is provided in subsequent chapters: 1. Partnerships between US and Pakistani higher education institutions that address regional, national, and/or local development needs. Science and Technology Cooperation Program: “The partnerships involved in these 46 research grants are impressive and include 21 Pakistani HEIs, 10 Pakistani R&D Institutes, and 41 US universities and R&D institutes/agencies.” 2. Individuals participating in USG-funded exchange programs conducted through the Higher Education Commission (HEC). Fulbright Program: “Since September of 2005, 412 Pakistanis (334 Master’s, 65 PhDs, and 13 post-doctoral) have departed for the US for participation in the Fulbright program and 67 Master’s participants have completed programs and returned. There are 196 grantees (120 Master’s, 62 PhDs, and 14 post-doctoral) already selected for the 2008 Fulbright program.” 3. Individuals receiving USG-funded scholarships to attend Pakistani universities. Evaluation of USAID Higher Education Portfolio 18 Academy for Educational Development August 29, 2008 Merit and Need-Based Program: “Since 2004-05, 906 scholarships have been awarded and 178 students have completed programs; 101 new scholars are short-listed for the 2007-08 Masters Program.” 4. USG-supported organizational improvements that strengthen the institutional capacity of Pakistani institutions. Forman Christian College: “USAID has made a major contribution of $1 million to the construction of a new science building that will have 84 faculty offices, 19 research labs, 20 teaching labs (including 4 computer labs), 24 classrooms, an examination hall, and an auditorium plus other related facilities such as a library/computer room for student work.” Forman has also embarked on a major reorientation of the academic curriculum moving from three year program to a four year program following a US liberal arts and science model. In addition, in keeping with its strategic plan, FCCU is undertaking a reorganization and expansion of its management and administrative program (see chapter 6). 5. Pakistani individuals trained as a result of USG investments involving Pakistani universities. Forman Christian College: “The University has emphasized faculty development ….Part of its plan has been to encourage faculty members without Masters and PhDs to seek further studies. With USAID grant funds, the University has been able to support 60 faculty members (27 percent of the total) for that purpose.” 6. Pakistani institutions with increased management or institutional capacity as a result of USG investments involving HEIs. Higher Education Commission: Support for the tenure-track system has resulted in an increase in the number of high-quality faculty members returning to or remaining in Pakistan. In addition, development of the merit and need based scholarship program at each participating institution has resulted in substantial capacity building including techniques and procedures to identify and verify that the family incomes meet the standards for the scholarships, that methods of selection conform to the requirements of the institution and USAID, that funding is properly accounted for, and that the students are performing satisfactorily. 7. Number of people trained in operational research Fulbright Program: “At the request of the HEC, the sciences were identified as a priority field for the PhD…. The disciplines are … beginning to show a wider spread with the 2008 grantees showing 32 percent in Engineering, Computer Sciences and Sciences; 26 percent in Economics, Finance, Business Administration, and Public Policy; 13 percent in Telecommunications, Communications, and Journalism; 12 percent in the Social Sciences and English; 9 percent in Public Health, and 8 percent in the Arts and Humanities.” Evaluation of USAID Higher Education Portfolio 19 Academy for Educational Development August 29, 2008 In addition, the Science and Technology Cooperative Project involves training in the use of state-of-the-art equipment, learning new techniques from American colleagues, and training for researchers and graduate students. Although aggregate data were not available (partly as a consequence of the trifurcated nature of the administration of the program as noted in Chapter IV), all of the eleven programs reviewed by the team contained extensive operational research, training, and capacity building components. II. Monitoring and Evaluation While we used and value quantitative monitoring and evaluation tools, we also found qualitative methodology to be useful. In addition to measures such as throughput, repetition rates, graduation rates, return rates of those sent abroad, employment rates, and evidence that graduates who have left Pakistan are now returning (as they are – with at least 35 PhDs returning and assisting with placement by HEC last year), we also included some additional measures of our own which we hope will be helpful. For example, findings are presented from interviews, focus groups, and site visits. Each of the chapters discusses monitoring and evaluation on its own, both by USAID and HEC as well as by the institutions themselves. III. General Conclusions As a result of our evaluation, we conclude that: • Higher education transformation in Pakistan has been successful over the last five years – one of the most successful in developing countries. However, it is currently at a critical stage in its development. Its long-term success will be dependent on a number of factors, including funding increases by the GoP, and continued external support from USAID and other funders like Japan, Great Britain, and the World Bank. The progress to date is fragile and dependent on continued external support to a significant extent including: access to training, cooperation, and links to US higher education institutions. • The current situation in Pakistan is one of transition, uncertainty, and insecurity. In this environment, continued improvement of the higher education system is critical to security, stability, democratization, and national development. • The goals of the GoP for higher education transformation, as spelled out in the MTDF and in the Draft Education Policy 2008, are being realized and implemented effectively with solid results to date. While progress has been made, much more needs to be done to create the high-quality higher education system so essential to national development, stability, and improved opportunities for its citizens. Many of the major changes are just beginning to have an effect, others, such as the expected benefits of increased PhD training, will only have a marked impact in improving the quality of higher education when Master’s and PhD students return. • USAID support to date has helped the country realize some of the major goals of the Pakistan-United States Strategic Partnerships, signed by Presidents Bush and Musharraf on March 4, 2004. That assistance has helped increase access to education, expand US cooperation and linkages, promote exchanges of students and scholars, encourage research collaboration, and expand financial support for higher education and training. Its goals remain relevant today. Evaluation of USAID Higher Education Portfolio 20 Academy for Educational Development August 29, 2008 • USAID can take a great deal of credit for the successes in the higher education transformation to date, given its contributions to a number of critical programs. USAID support has made a significant contribution to the progress made to improve the quality, relevance, and access in Pakistan and should continue to play a critical role in the future. • USAID would benefit from adding additional staff with expertise in higher education to work with partners on USAID’s higher education portfolio. That would allow a more holistic and productive relationship with project administrators and improve monitoring and evaluation substantially. • USAID goals – as spelled out in foreign assistance objective “Investing in People,” the USAID/Pakistan Interim Strategic Plan, and the Pakistan-United States Strategic Partnership, 9 – and the higher education goals of the GoP are congruent. This has maximized the value-added contribution of USAID support. • USAID programs have engendered good will among university students, faculty, and administrators in spite of the animosity toward the US expressed in the press and in public forums. Our interviews with more than 200 students, 100 faculty members, a dozen university vice chancellors and senior administrators, Ministry officials, HEC staff, and others found universal goodwill toward the US and its support. • It will be important to build on that goodwill by continuing to support and strengthen the current programs and to consider a number of new initiatives. USAID has an opportunity to continue to help move Pakistan in the direction of high-quality higher education, national development, democratization, and improved citizen well-being. IV. Policy Level Recommendations: A Five-Year Strategy for USAID Higher Education Support The AED review team makes a number of policy level recommendations to USAID as it develops a higher education strategy for the next five years. • Build on the very successful programs in higher education and emphasize the importance of higher education to Pakistan’s future development. Models for sustainable development can be found in student support (Merit-Need-based Scholarship, Fulbright); curriculum development (Forman Christian College, Fatima Jinnah Women’s Univ); research (Science-Technology Cooperation); and faculty development (Science￾Tech, Forman Christian College, Fatima Jinna, and the University of Agriculture Faisalabad.). • Focus more attention on special programs that help attract more women students as well as disadvantaged students from the border regions, which have suffered for decades from neglect, economic decline and instability. The young people in those areas have few opportunities for higher education or employment and are ripe for recruitment by terrorists. Higher education provides an important alternative for young 9 Governments of the United States of America and Pakistan, Joint Statement: Pakistan-United States Strategic Partnership, Islamabad, 4 March 2006. Evaluation of USAID Higher Education Portfolio 21 Academy for Educational Development August 29, 2008 people and the chance to improve both their own well-being and that of the region. One possible strategy would be for universities to engage in careful analyses of economic needs, employment opportunities and community participation rates, after which practical programs can be developed and implemented which will result in meaningful jobs and an advancement of national goals. • Recognize the centrality of faculty development to the success of the MTDF. The Fulbright program in faculty development is producing high-quality Masters and PhD graduates for universities, government, and business. The kind of high quality training needed in Pakistan can best be obtained in the US through the Fulbright Program since the US provides the best graduate education in the world. Pakistan has set world￾class targets as its benchmarks for higher education. To meet these high level requirements and build the high quality higher education system needed will require the best training available. The administrators of the Fulbright program have made a major effort to increase participation of women and people from disadvantaged regions. That effort should be fully supported and expanded. In addition, USAID might consider supporting the development of pilot joint degree programs in which universities collaborate on curricula and students complete a portion in each country. Candidates for these programs might be selected through the Fulbright process. • Since quality improvement in higher education is associated with the quality of primary and secondary education, we recommend that the GoP undertake a major review of the troubled primary and secondary education sectors, both within the purview of USAID’s primary and secondary education programs and in the context of their long term importance for higher education. A Primary Education Task Force and a Secondary Education Task Force, based on the model of the Higher Education Task Force that led to the creation of the HEC, could be one means of accomplishing this. Of particular importance in this review would be an analysis of the vertical articulation between different education levels. Such an examination might focus on students who make successful transitions from primary through secondary and postsecondary education. The main goals of this effort would be to increase graduation rates and access to higher education. • Consider programs to support the HEC in three areas suggested by this evaluation: o Teacher Training (both in higher education institutions and by distance-education with face-to-face contacts). o Enhancement of Social Science and Policy Analysis programs including faculty development to improve quality to international levels. o Training for technical personnel in science, technology, IT and middle level administrators. • Work with the HEC to identify new areas of needed support. Suggestions of the evaluation team include: o Support for Centers of Excellence – proposals to come from institutions to the HEC. Evaluation of USAID Higher Education Portfolio 22 Academy for Educational Development August 29, 2008 o Creation of an Innovation Center – implementation and commercialization of applied research findings and basic research findings where relevant. o Development of programs that encourage applications from women and disadvantaged students from the border areas by supporting selected institutions to set up special programs focusing on recruitment, preparation, retention of these students. Expand outreach to these areas, including advice on academic requirements; special pre-university programs in math, science, and English; mentoring by other students and faculty; and opportunities for research experience. o Assistance to the HEC to help with its efforts to put an institutional accreditation process in place. V. Program-Level Conclusions and Recommendations Based on our evaluation of each of the projects funded by USAID, the team presents the following conclusions and recommendations at the program level. These include identification of important successes in meeting the aims of the MTDF, recommendations to improve the administration, evaluation, and monitoring of these programs, and suggestions regarding a USAID higher education strategy for the future. A1. Merit and Need-based Scholarships Conclusions • The Merit and Need-based Scholarship Program has helped more than 900 poor and disadvantaged students have access to higher education – most of whom would not be there otherwise. Higher education has increasingly become the province of the elite and wealthy. Even public higher education, with low fees, is out of reach for most Pakistanis, 70 percent of whom live on less than $2 per day. • In spite of initial start-up problems and delays, the program has matured to include uniform income reporting, a better understanding and utilization of the need concepts and measurement of need, improved provision of information about the scholarships to the public, and closer links with the participants. • The number of women admitted to the program, at 17.5 percent, is far too low. While we recognize the difficulty of recruiting women, especially in rural areas of the country, universities should be encouraged to undertake more outreach to encourage women in secondary schools to go to university. • The program has contributed to significant capacity building at the participating universities in their financial aid and scholarship offices. • The success of the Merit and Need-based Program has encouraged Japan to emulate the program. It has also stimulated participation in scholarship programs by several major corporate contributors to individual universities. • The administration of this program is too centralized at the HEC, with onerous reporting requirements which we believe are no longer necessary. The participating universities have now shown that they can administer these programs effectively and the various centralized reporting and selection mechanisms are time-consuming and create delays in allocation of scholarship. Evaluation of USAID Higher Education Portfolio 23 Academy for Educational Development August 29, 2008 • There are systematic and serious problems with late payments. In several cases the local institutions have had to arrange for food to be given to students while they waited for their funds to be processed or personally give them loans to tide them over. In a few cases students have received their grants after completing their studies. This totally negates the aims and goals of the programs and reflects negatively on both the HEC and USAID. The problem seems to be a combination of understaffing at the HEC (in part due to a government freeze on hiring), initial problems in getting funding released, and misunderstanding of the procedures by some universities. • Many students who received fellowships have become articulate ambassadors for the US – in one institution going so far as to suggest that a USAID/HEC Fellowship Alumni Association be formed nationally among those who received fellowship support. A2. Merit and Need-based Scholarships Recommendations As noted in Table 3D, the Merit and Need-based Scholarships program has addressed two areas mentioned in the USAID goals for higher education: short/long-term training programs in technical sectors and degree programs. Specific recommendations (also listed in Chapter 5) appear below. • Increase outreach and opportunities for more disadvantaged students to have a university education. • Expand the disciplines available for scholarships to include critical areas in the sciences and quantitative social sciences. This would also provide more opportunities for poor and disadvantaged women and to increase gender equity in critical areas. • The HEC should continue oversight of the program but with increased staffing to reflect the size of the program and to ensure prompt payment of stipends to the students. Electronic transfers should be used to speed funding to university bank accounts or directly to students wherever possible. We are pleased to report that the HEC has responded to our expression of concerns and revised its procedures to speed up the payments to students.10 • Decentralize the selection process to the participating higher education institutions. The HEC should continue to have one representative on each institutional selection committee. • Reduce reporting requirements to an annual basis with the report to be due toward the end of the academic year. Where errors have been made, excess funding should be deducted from the institution’s next year’s allocation. This is a case where monitoring and review are excessive and are hindering the efficient operation of the program. • Attempt to provide scholarship support commensurate with standard tuition rates. • Increase the number of disadvantaged students receiving scholarships. Currently, scholarships are available for less than a third of those students eligible for scholarships. The Federal and Provincial government and the institutions can provide more assistance in this area and also examine how to leverage additional funds from other sources for 10 Confirmed during a follow up visit to the HEC by Tom Armor in August 2008. Evaluation of USAID Higher Education Portfolio 24 Academy for Educational Development August 29, 2008 need-based scholarships. This is a promising area for the private sector to demonstrate corporate social responsibility. • Encourage women to seek higher education and to apply for these scholarships. More outreach activities are needed and more culturally appropriate opportunities are needed, such as a greater use of the women’s universities. To increase the number of women applying it is important to increase the range of disciplines to include those more likely to attract women. B1. Science and Technology Cooperation Conclusions The US-Pakistan Science and Technology Cooperation project has provided significant capacity building for staff at the universities and institutes in applied research, assisted in critical technology transfer, and promises to make important contributions to national development. We were impressed by the quality of the research programs reviewed, the staff involved, and the potential to significantly effect development in vital areas such as agriculture, the environment, and health. • The programs involve projects critical to Pakistan’s development. At the same time they are making contributions to US research in providing access to critical information and identification of diseases and other problems that are not yet affecting the US, but which will affect the livelihood of both countries in time. Among these are techniques for identifying new high risk TB viruses that are now beginning to show up in Pakistan; development of disease resistant strains of citrus; and work to develop wheat varieties that are resistant to a “virulent” wheat fungus which devastated crops in East Africa and is now moving through Iran11 toward Pakistan. • This project is an excellent value for USAID with $10.5 million of the funding coming from the HEC. The projects examined were well run, tightly budgeted, and showed promise of significant results. • Links with US institutions are continuing, and have been, very effective. We observed a high level of cooperation and shared knowledge between Pakistan and US grantees which is of great mutual benefit to both parties. US partners interviewed were very positive about the effectiveness of the projects and the mutual benefits. • One serious weakness of the program is its trifurcated nature in Pakistan, which makes scaling up of findings, commercialization, and dissemination difficult. We suggest mechanisms for improved coordination, information sharing and commercialization below. • Payments to grant holders were often made late. That has posed serious problems for the researchers involved and limited the success of the program. There has often been a lag of six months to more than a year in providing the funds to some Pakistan participants. We were not able to find a defensible reason for the delays, which we regard as unacceptable. 11 FAO, “Wheat killer detected in Iran Dangerous fungus on the move from East Africa to the Middle East,” FAO Newsroom, Rome, 5 March 2008. Evaluation of USAID Higher Education Portfolio 25 Academy for Educational Development August 29, 2008 • In a number of cases funding on the Pakistan side was cut below the levels needed to carry out effective research. Officials at the HEC acknowledge the problem. We make recommendations to overcome it below. B2. Recommendations for Science and Technology Cooperation As noted in Table 3D, the Science and Technology Cooperation program has addressed three areas mentioned in the USAID goals for higher education: short/long-term training programs in technical sectors, technical assistance and extension programs, and the improvement of applied research. Specific recommendations (also listed in Chapter 6) appear below. • Create an Innovation Unit at the HEC to publicize, develop, and commercialize promising projects for general public use and for commercialization. This could be linked to the HEC patent office. • Create a central unit in the HEC (which could be the Innovation Unit, above) to facilitate communication and information sharing between participants. The trifurcation of the program means that no one is aware of the total program other than the Academy in Washington. Thus Pakistan is not getting full advantage of the program and its results. • Streamline the administration of this program and ensure that payments are made in a timely fashion. • Improve communications during the Pakistan part of the selection process in terms of providing prompt feed-back to applicants regarding their proposals. • Ensure that adequate allocations are made to cover the cost of the projects, and cut the number of projects rather than underfund them. • Ensure that data on program achievements are collected and reported from grant holders. C1. Fulbright Program Conclusions The Fulbright Program in Pakistan is managed by the US Educational Foundation in Pakistan (USEFP), has made significant contributions to Pakistan in providing training for students, building university linkages, and in helping to create mutual understanding. It is making an important contribution to meeting the goals of the MTDF. • The USEFP has made an important contribution to quality improvement in higher education and is assisting the GoP in meeting the quality improvement goals of the MTDF – especially in terms of providing high quality training for Masters and PhDs in the US, and which help provide the essential environment for further study and certification of student learning. • Graduates of the Fulbright Program have excellent reputations and are eagerly sought by universities, government, and other employers. The Fulbright Program makes it easier for graduates to get a job or promotion. It helps with qualifications, learning new tools and gaining new knowledge and experience. • The return rate of Fulbright Scholars is close to 100 percent. Returnees were enthusiastic about their US experience and are building positive links with US institutions which seem to be continuing since their return. There were very positive responses about the Fulbright program at institutions we visited. Evaluation of USAID Higher Education Portfolio 26 Academy for Educational Development August 29, 2008 • Oversight of USEFP is complex due to its multi-agency structure and function – a combination of the Department of State, and USAID, with a Bi-national Pakistan-US Board and major contributions by the HEC of $10.5 million. All of these agencies feel they have a major stake in USEFP policy. It became clear to the team during our interviews that this has led to some confusion as to responsibilities that need to be resolved. • The Executive Director’s leadership of the program is outstanding. Nonetheless, administration decisions are too centralized in her office. Some of the day-to-day tasks need to be decentralized, subject only to review by the Executive Director. The addition of middle management staff will be needed for this decentralization, which will also provide an upward path for staff that will hopefully aid in staff retention. • The Fulbright Program is expensive but with the benefit of providing Pakistan with the best higher education in the world – a critical key to the long-term development of quality higher education in Pakistan. The Fulbright Program is one of the most worthwhile investments USAID has made in Pakistan higher education. • The selection process is reasonable, fair, and transparent. Senior staff are committed, lauded by the students, and have worked hard to improve gender balance and minority area representation. C2. Recommendations for Fulbright Program As noted in Table 1, the Fulbright Program has addressed three areas mentioned in the USAID goals for higher education: short/long-term training programs in technical sectors, degree programs in fields of critical importance, and the improvement of applied research. Specific recommendations appear below. • Continue efforts to increase the number of women by targeting women through advertising, field trips, and education fairs. • Target minorities and disadvantaged areas, focusing especially on encouraging applications from FATA, NWFP, Balochistan, and FANA areas. While Fulbright participation has been fairly broad geographically, too few candidates currently come from the border areas – areas that are key to peace and stability in Pakistan in the long run. • Support classes to prepare Fulbright grantees from disadvantaged areas in English language, math, computer skills, and other preparation prior to their departure for the US. • Focus primarily on PhD training except in areas where a Masters in regarded as a terminal degree. • Encourage the Executive Director to delegate more authority on day-to-day matters to an expanded middle management staff. Increase middle management and establish clear, documented participant selection decisions and financial management procedures. • Support PhD and Master’s graduates to have a 6 to 12 month internship or fieldwork experience in the US prior to their return given the good return rate and the benefits of such work in addition to their formal higher education training. • Send several students in selected critical areas to the same US university. Focus on academic areas of critical need to help develop teams which would return to the same university to create a “center of excellence.” Evaluation of USAID Higher Education Portfolio 27 Academy for Educational Development August 29, 2008 • Include the topic of racism in the US in the pre-departure orientation. Many participants had experiences with racism for which they were not prepared. D1. Forman Christian College (FCCU) Improvement Grant Conclusions • The College Improvement Grant to Forman Christian College (a Chartered University)12 was designed to assist FCCU in its development of a four-year bachelor’s degree program, upgrade its faculty, strengthen its programs in basic science and information technology, develop masters programs, and provide scholarships to help meritorious poor students attend FCCU. Its academic curriculum is modeled on a US broad liberal education, with high-quality academic programs, and problem solving participatory teaching and learning for students. • The University was criticized on its handling of some aspects of the grant in a USAID report on “Fund Accountability.”13 The concerns related to text book purchases, a charge for an HR course, equipment purchases, and criteria for a scholarship program. • FCCU was also criticized for what was seen as a lack of transparency in student and faculty scholarship programs, lack of gender equity, bias against non-Christians, difficulty locating library books purchased using USAID funds, concerns about equipment purchases, student unhappiness with the academic program, and lax administration. The team did examine the concerns and allegations carefully and found no basis for them. However, there are outstanding audit findings (the result of a third￾party audit) related to these issues which FCC still needs to correct. • There was no evidence of gender bias; students said the institution has made a major effort to recruit women since it regained control in March 2003. We found students and faculty to be very happy with the new program. There was no evidence of bias against Muslims – the majority of students are Muslims. • The administration of the University is somewhat understaffed, with 12 Executive Staff and a management team of about 29 positions, of which 7 are currently vacant. The situation should improve as key vacancies are filled. We were informed that recruitment for some of the vacancies is currently underway.14 • Forman has a thorough evaluation and monitoring system at every level. Faculty teaching is reviewed annually; a well-conceived academic assessment plan to review each unit on a 3 year cycle is being implemented. Administrative staff members are required to have an annual performance appraisal. Students must pass a comprehensive exam in English and a comprehensive examination in their major field graded by an external examiner. This level of review is unusual in Pakistan and a good model for other institutions. • FCCU is one of the most impressive and progressive institutions we visited during the evaluation in Pakistan. Their use of USAID funding has produced positive results. 12 Forman Christian College received University status in 2004. Hereafter we will abbreviate it as FCCU though in some documents FCC is used. 13 A. F. Ferguson & Co, Forman Christian College, Lahore, Fund Accountability Statement Audit and Review, (n.d, circa July 2007). 14 The follow up visit resulted in a report that sixteen searches are currently underway at FCC, several of which are for administrators including a Purchasing Officer. Evaluation of USAID Higher Education Portfolio 28 Academy for Educational Development August 29, 2008 • The institution is a model of the kinds of higher education change desired throughout Pakistan if the goals of the MTDF are to be realized. We found the USAID programs to be especially effective and efficient at FCCU, setting the stage for marked improvement in its programs and making substantial progress in returning FCCU to the status it once had as one of the top two universities in Pakistan. • FCCU can serve as a prototype for the new four-year curriculum advocated by the HEC. FCCU is a US style institution with a focus on active learning, problem solving, critical thinking, and individual creativity. It is an outstanding example of the kinds of projects USAID has supported in the past with excellent results. D2. Recommendations for Forman Christian College Improvement Grant As noted in Table 3D, the Forman Christian College Improvement Grant has addressed the following areas mentioned in the USAID goals for higher education: Short/Long Term Training, Degree Programs and Technical Assistance. Specific recommendations appear below. • Expand administrative staff involvement in USAID funded programs and fill vacant management positions. A plan for administrative expansion is spelled out in its Strategic Plan. • Complete revisions of its procurement policies in accordance with USAID requirements. FCCU should quickly respond to several outstanding USAID reporting requests to regain their confidence that FCCU understands it’s responsibilities in this area. • Continue the Merit and Need-based scholarships and faculty development program. • Upgrade library facilities. • Offer short courses to faculty members from other universities. This would require at least one teaching lab, video conferencing, and some additional staffing. • Develop procedures and be more responsive to USAID reporting requirements (most of them standard practices), E1. Gender Equity Conclusions • The percentage of women receiving benefits from USAID funded programs is lower than it should be. While some progress has been made in both the Fulbright and Merit and Need-based Scholarships in recent years, a continuing concerted effort is needed to improve gender equity. In the latter case, expanding the number of academic disciplines to include those in which there are larger numbers of women students – in the sciences and quantitative social sciences – would rapidly increase the overall percentage of women and make a major contribution to gender equity. • We recognize that in some areas of the country the cultural and historical biases against education for women remain. However, we are convinced that once a critical mass of young women are graduating from secondary school, and a sizeable number of them go to university, there will be a major change in the willingness, indeed the eagerness, of families to send their daughters to university. FCC has greater potential for rural women students when/if women’s housing facilities can be expanded. Evaluation of USAID Higher Education Portfolio 29 Academy for Educational Development August 29, 2008 • The National Academy of Higher Education at the HEC has the best record of gender equity with 50 percent women in its in-service faculty development programs. • The Fulbright Program deserves commendation for its emphasis on gender equity and its focus on recruitment of women. The percentage of women since the program was restarted in 2004 was 33 percent women in 2005, 37 percent in 2006, 30 percent in 2007, and 42 percent selected in 2008. • While the HEC had fewer women senior employees than expected a few years ago, it has recently made progress in this area. Two women were recently promoted to director general. E2. Gender Equity: Recommendations Because improvement in Gender Equity addresses all five USAID higher education indicators - Short/Long-Term Training, Degree Programs, Technical Assistance, Applied Research and Policy Analysis – the recommendations below are especially important. • Encourage the HEC and higher education institutions to work toward gender equity in all programs and increase the number of women members of staff and on the faculty. • Create, under the HEC, pilot outreach programs at higher education institutions, designed to work with secondary schools, local communities, and parents to encourage young women to go on to higher education, take courses necessary to qualify for admission, and make them aware of the opportunities for financial support. • Under Fulbright, continue to encourage more women to apply for fellowships with a goal of 50 percent women. • Set up mentoring programs within higher education institutions for first year female students to assist them in adapting to the institution and their studies. • Recommend to the HEC that it reconsiders the age limit of 35 years for women pursuing advanced degrees with HEC support. Those who take time off to have children after completing their undergraduate program are likely to be 35 or a little older. • Broaden the areas of study under the HEC and the programs to include areas which are more attractive to women students and which provide them with employable skills and talents in what appear to be socially acceptable areas of work. These include the social sciences and sciences in addition to agriculture. VI. Higher Education Commission: Conclusions and Recommendations (Portfolio Level) In addition to the evaluation of USAID programs at the project level, we were asked to look at the overall activities, the portfolio level, of the HEC and to make an assessment of its current funding and prospects for the future. A. Higher Education Commission Conclusions • For the first time since 1947, higher education in Pakistan has undergone significant transformation and improvement that was long overdue. Higher education in Pakistan remains well behind international standards and is behind most of its neighbors in the region. On the other hand, the progress to date is substantial. Evaluation of USAID Higher Education Portfolio 30 Academy for Educational Development August 29, 2008 • The evaluation team was very impressed with what has been achieved to date. In the last four years the higher education system, once among the weakest in the developing word, has made major improvements across a wide range of areas including: faculty development, upgrading laboratories and infrastructure, increased access, quality improvement, accountability for faculty and administrators, access to library material through the digital library, accreditation and quality assessment. • The ongoing commitment to quality in all the programs is especially noteworthy. The HEC has managed to maintain those standards in the face of strong resistance in some areas and the massiveness of the transformations they are undertaking in higher education. The HEC commitment to international standards as benchmarks for quality improvement is vital to success. It will be important to maintain those high standards if the HEC is to meet its own quality expectations. • The HEC has proven to be receptive to suggestions from Pakistani higher education institutions and from external experts. This has resulted in several significant modifications of plans when they seemed not to be having the desired effects or needed revision. This openness and flexibility has helped overcome some of the early opposition to these reforms and improved the outcomes. • Budget controls and monitoring were noted by USAID as problems as a result of a Control Environment & Risk Assessment. 15 Senior HEC management has made a concerted effort to respond to all recommendations from the risk assessment and a USAID report on July 22, 2008, indicates they have succeeded.16 Overall, we are impressed by the level of review and control over both the recurring and the development budgets at senior management levels and think the new budget control system will resolve the problems noted. • The MTDF, which elicited strong opposition from a number of major universities and higher education institutions a few years ago, is now receiving approbation. We heard almost unanimous praise for the HEC and the progress made in upgrading and reforming higher education from almost every institution visited. What has changed is that the overall attitude toward the HEC is positive. • It is clear that the goals of the MTDF, and subsequent HEC and national higher education goals, have been broadly accepted by universities and other higher education institutions as the basis for moving forward. The positive results in terms of improved infrastructure, faculty quality, teaching, research, and service are clear. While acceptance of these goals is an important step forward, Pakistan still has a long way to go to build the quality programs needed to be among the very best higher education systems in the region. • What is striking in the international context is that the HEC, in four short years, has managed to achieve such broad transformation and quality improvement in the direction of international standards. We can think of no other system that has done as well in such a short period of time. The HEC has managed to bring one of the weakest higher education systems in the world up to reasonable standards with the promise of creating a 15 Controller, USAID, “Control Environment & Risk Assessment – Higher Education Commission,” February 4, 2008. 16 USAID, “Status on Control Environment & Risk Assessment (CERA) – Higher Education Commission,” July 22, 2008. A report that notes that previously identified weaknesses have been corrected or resolved and the HEC meets required standards or is in the process of meeting them. Evaluation of USAID Higher Education Portfolio 31 Academy for Educational Development August 29, 2008 number of centers of excellence in teaching, research, and service over the next few years. • The budget situation of the HEC at the beginning of our evaluation (April 2008) looked good and commitments of government funding, while not as high as initially expected, were promising. Funding levels and promises from a number of donors, including the World Bank17 and a number of governments, had been reasonable. However, the situation deteriorated just before our departure with the failure of government to make its usual quarterly payment to the HEC. Some of those funds did eventually arrive – but only one third of the expected total. This does not bode well for the future. Furthermore, the macro finance situation has deteriorated and the World Bank has put its funding on hold until it improves. We note our strong concern that government decisions and financial failures may badly damage the progress made to date in higher education. • The prospects for creating a quality higher education system in Pakistan look very good, assuming that the government continues to fund higher education at promised levels, and external support continues or is resumed. At the present time, the per capita expenditure remains low by international standards, although the overall higher education budget has increased 350 percent over the last five years. The budget allocations this year did not increase as much as expected and that is a worrisome sign for the future. Added to that is the failure of government to make full payments to the HEC in the last quarter. Without continued funding at promised levels, the progress made to date is likely stall. • We want to emphasize the importance of high-quality higher education to the economic, social, and political development of Pakistan. As the World Bank has demonstrated,18 no nation has moved into the ranks of developed nations without high quality higher education. Pakistan is not yet at that stage, but has made significant progress in that direction. • In addition, we want to emphasize the importance of high-quality higher education to democratization and political development. A successful focus on quality improvement will result in significant improvements in the lives of the people of Pakistan. The emphasis on creativity, active learning, critical thinking, student outcomes, high quality research, and a higher education system relevant to the needs of the nation, is already paying dividends in the growing level of confidence of students, faculty members, alumni, and citizens in the higher education system. B. Higher Education Commission Recommendations In the light of the conclusions drawn from the team’s evaluation of the Higher Education Commission, its portfolio, its budget, and funding prospects, we make the following recommendations: • Continue joint HEC/USAID programs, and expand the Fulbright, the Merit and Need￾based Scholarship, and the Pakistan-United States Science and Technology programs. 17 World Bank funding is to be a Development Policy Operation (DPO) of $100 million. This means that there is no money going to the HEC or any Ministry, but directly to the Treasury. Nonetheless, that is designed to enable the government to fund the HEC and other projects reviewed by the Bank. However, the DPO can not be released until the macro situation is fine – which is not the case at the present time. Funding is on hold until the Bank has confidence that the macro situation is back on track. 18 Ibid., Constructing Knowledge Societies (2002). Evaluation of USAID Higher Education Portfolio 32 Academy for Educational Development August 29, 2008 • Consider three new programs (see Chapter 12 and Appendix III for details on all three programs): 1. A major new teacher training program for primary and secondary teachers to be inaugurated at existing teacher training institutions and through a new distance education program, 2. A program designed to improve the quantitative social sciences in Pakistan so that they can make a badly needed contribution to policy-analysis, policy-making, higher education teaching, service, and research, 3. A program to provide training for technicians and other middle level administrators to overcome shortages in these areas and bring this sector up to international standards. • Provide higher education scholarship support for people from poor, underdeveloped, and impoverished areas of Pakistan. In addition to scholarships for tuition, room and board, the HEC might consider other strategies for supporting disadvantaged students. For example, it might encourage universities to partner with secondary schools to “early identify” potential achievers and provide supplementary academic and motivational services such as tutoring, mentoring and field trips. • That HEC focus on institutional needs and program requirements including: o To insure that funds for programs such as the Merit and Need-based Scholarships and the Science and Technology Cooperation Project are released in a timely fashion. o To insure that USAID reporting requirements for the Science and Technology Project are carried out not only as a grant requirement but also as a means of documenting program success. o To improving gender equity in all programs including management in the universities and the HEC. o To build stronger links with business and industry, establish incubator programs, and improve relevance of university programs in science, business, engineering, and technology areas. o To identify, hire, and train more high quality mid-level staff at the HEC, especially women. There is need to plan for succession at the HEC with a more active internal training program as well as an effort to identify star talent to groom for leadership positions. • Maintain the autonomy of the HEC, regardless of the governance arrangement it chooses. Much of the success of the HEC is attributable to its autonomy, its control over funding once allocated, its senior administration and staff, and its success in working with higher education institutions, both public and private, throughout Pakistan. We believe a major change in the structure or autonomy of the HEC would be detrimental to the successes in higher education to date and to the long term prospects of higher education in Pakistan. • Continue USAID collaboration and assistance to the HEC, the Ministry of Education, Ministry of Science and Technology, and Ministry of Agriculture. Continue to focus on programs that help foster improvement in the quality of higher education in Pakistan and thus national development, productive employment, and democratization. For example, consider creating a coordinating body among the three organizations that could periodically meet to compare activities and share ideas. Evaluation of USAID Higher Education Portfolio 33 Academy for Educational Development August 29, 2008 Chapter 4: The Context and State of Higher Education in Pakistan I. Introduction Pakistan became independent on August 14, 1947, following a protracted struggle ending British rule and resulting in partition of colonial India into two sovereign states. At the time of independence in 1947, Pakistan had only two universities, the University of the Punjab, Lahore, (founded in 1882) and Sindh University, Jamshoro (founded in 1947). Higher education has been held in high esteem in Pakistan and is looked to as an important contributor to socio-economic development and as a producer of trained manpower. Universities are seen as centers of higher learning and creators of new knowledge. However, since independence they have made only limited contributions to both. By constitutional mandate, education is a provincial responsibility (See Appendix IV for an overview of the development of education in Pakistan). Most higher education institutions were funded by the provincial governments throughout most of the post-independence period. In spite of its reputed importance, however, higher education remained a neglected sub-sector of education until the establishment of the University Grants Commission in 1974. At that time, funding to the universities began to be made through the University Grants Commission. Up to 1987 there were only 26 universities and degree awarding institutions (DAIs) in the country. With the promulgation of the National Education Policy (1998-2010), there was a significant increase in the number of institutions of higher education, with the total growing from 18 to 78 between 1988 and 2008 (Table 1 below). Private education was now encouraged (Table 2) and is currently 45 percent of the total. The private sector grew in particular in Sindh, Punjab and NWFP. The first major post-independence changes in higher education came after the creation of the Higher Education Commission (HEC) in September 2002. It was established to guide higher education policy and improve the quality of public and private universities and degree awarding institutions. The establishment of the HEC and its subsequent efforts to transform higher education resulted in substantial increases in funding for higher education. The HEC’s efforts played an important role in improving the quality of higher education in Pakistan over the last five years. The Higher Education Commission has been successful in attracting substantial amounts of additional funding from a variety of donors and overseas agencies. Evaluation of USAID Higher Education Portfolio 34 Academy for Educational Development August 29, 2008 Table 4: Number of Universities & DAIs in Pakistan from 1947 - Present Region 1947 1948-57 1958-67 1968-77 1978-87 1988-97 1998-2008 Federal - - 1 1 1 - 14 Punjab 1 - 2 2 2 5 24 Sindh 1 1 1 3 2 9 21 NWFP - 1 2 1 2 2 13 Balochistan - - - 1 - 1 4 AJK - - - - 1 1 1 NA - - - - - - 1 Total 2 2 6 8 8 18 78 Source: Higher Education Commission, Statistics, May 5, 2008, http.//www hec.gov.pk Table 5: Number of Public and Private Universities & DAIs in Pakistan (2008) Region Public Private Total Federal 14 3 17 Punjab 19 17 36 Sindh 13 25 38 NWFP 12 9 21 Balochistan 5 1 6 AJK 1 2 3 NA 1 - 1 Total 65 57 125 Source: Higher Education Commission, Statistics, May 5, 2008, http.//www.hec.gov.pk In 2001, Pakistan was ranked among the lowest in the world in higher education enrollment rates. Other Asian developing countries, such as India and Korea, stand at 10 percent and 68 percent respectively. According to a report of the steering committee for higher education in 2001, only 2.6 percent of the students between the ages 17-23 were enrolled in universities. The total increased to 2.9 percent in 200519 and by 2008 was 3.9 percent. The target for the Government of Pakistan was to almost double the percentage of college age students by 2010 to 5 percent of the 17-23 age group.20 II. The Higher Education Commission (HEC), Origins of the HEC The higher education sector in Pakistan has a long history of problems going back for decades. The higher education system was marked by low quality in general, little research output, and a faculty in which only 23 percent had PhDs. The total enrollment of students of college age was among the lowest in the world with only 2.6 percent of college and university age students enrolled in 2001. This put it among the lowest in the world along with Ethiopia at less than 1 percent, Madagascar 3 percent, and Afghanistan 2 percent.21 This is in sharp contrast to India which was about 8 percent. Pakistan’s overall expenditure on education was also among the 19 Government of Pakistan, Report of the Steering Committee for Higher Education, 2001. Government of Pakistan, Economic Adviser’s Wing, Finance Division. Pakistan Economic Survey 2005-06. Islamabad: 2006, p. 168. 20 Ministry of Education Policy 1998-2010, httq://www moe.gov.pk/edupolicy4htm. 21 See UNICEF/WB study (2003). Evaluation of USAID Higher Education Portfolio 35 Academy for Educational Development August 29, 2008 lowest in the region with only 2.1 percent of its budget going to education.22 This was compounded by a low literacy rate of 56 percent (44 percent among women) and a weak economy with a per capita GDI of $600 and PPP of $2,170.23 The higher education sector had remained largely stagnant since the establishment of Pakistan in 1947, not because its problems were unrecognized – indeed there had been at least ten reviews of higher education since independence – but due to failure to agree on policies to improve the system or, if policies for change were promulgated, failure to implement them. Since independence, there had been reports on higher education in 1947, 1951, 1959, 1966, 1969, 1970, 1972, 1979, 1992 and 199824 and six major education policy documents.25 Nonetheless, higher education had been stagnant throughout most of this period and was generally of poor quality. The universities lacked a culture of faculty research. Very little research was taking place and there were few publications. Not a single university in Pakistan ranked in the top 500 institutions in the world.26 The system as a whole neither met the needs of the nation nor prepared students to compete in an increasingly demanding world economy. The reform of higher education and the creation of the Higher Education Commission grew out of the appointment of a Presidential Task Force on Higher Education in 2001. That was followed in 2002 by the appointment of the Task Force on Improvement of Higher Education in Pakistan which produced a report that gained broad acceptance in government and the higher education community. The report was followed by the appointment of a Steering Committee on Higher Education which recommended the creation of the Higher Education Commission in September 2002. The Higher Education Commission replaced the University Grants Commission which had been responsible for making decisions on finance for public universities and had limited policy authority over federal universities. In its later years it had become ineffective and dysfunctional. During that period, authority over the provincial universities and colleges rested with the regional governments. The governor in each province was chancellor of the university and appointed the vice chancellor. The individual universities had a high degree of autonomy over their own governance and expenditures as well as over the colleges and other institutions that were affiliated with them. They were chronically under funded and of low quality. The powers of the Commission were laid out in the Higher Education Commission Ordinance LIII of 2005. In that document, the HEC was given extensive powers over higher education. Authority was centralized as never before and its powers extended to private tertiary education. The HEC’s authority covers “…any university established or incorporated under any law….”27 The HEC was given Ministerial status with the Chairperson of the Commission being a member of Cabinet and reporting to the Prime Minister – in many respects similar to a Ministry of Higher Education, with the Executive Director being the equivalent of a Deputy Minister. 22 Higher Education Commission, HEC Annual Report 2002-2003, p. 8. 23 World Bank; World Development Indicators, 2006, p. 21. 24 Javed Hasan Aly and the National Education Policy Review Team, Education Policy in Pakistan: A White Paper, February 2007, p. 1. 25 World Bank, Higher Education Policy Note: An Assessment of the Medium-Term Development Framework, June 28, 2006, p. 6. 26 HEC, Medium Term Development Framework, Islamabad, 2005, p. iii. 27 Government of Pakistan, Higher Education Commission Ordinance, 2002, chapter 1, p. 2. Evaluation of USAID Higher Education Portfolio 36 Academy for Educational Development August 29, 2008 The powers of the HEC include to: “…formulate policies, guiding principles, and priorities for higher education institutions;” to promote “…socio economic development of the country;” to plan for national development of higher education, and to establish budget recommendations for government for the operation and development of higher education.28 The HEC also was given the power to regulate the opening of new institutions, both public and private, to oversee the curriculum, to set requirements for new faculty members and conditions for the promotion of faculty members. The HEC was given extensive powers to improve quality, evaluate, and accredit institutions. It had authority to establish national or regional evaluation councils or a national accreditation body.29 It was given the power to set budgets for the general operation of higher education institutions, for outreach, research, support for linkages between industry and higher education, and to promote “…educational programmes that meet the needs of the employment market…” – that is, to foster relevance to national needs. It was authorized to “…promote the study of basic and applied sciences in every field of national and international importance….”30 The HEC was also charged with collecting information and statistics on higher education. From its inception, the HEC began to put in place a number of programs designed to improve the quality of higher education. The process of transforming higher education was a top down approach under the HEC. This program included: increased funding for university base budgets; a major effort to upgrade science laboratories and university facilities; an indigenous PhD program; support for faculty participation in international conferences; support for professional conferences in Pakistan; and many others. An innovative program to encourage faculty research, “Promotion of Research in Universities,” was begun in 2002-3, funded through a special allocation. It grew from Rs. 47,000 million in 2002-2003 to Rs. 270,000 million in 2003-2004, an increase of 474 percent in one year.31 Worried about the quality of PhD training in Pakistan, the HEC initiated a major foreign PhD program to give students state-of-the-art training and raise quality to international standards. To insure that these young scholars returned to Pakistan, the HEC introduced several innovative programs to guarantee positions on their return, provide research support, travel awards, and improved salaries. In 2004, the HEC began work on the Medium Term Development Framework (MTDF), designed to spell out a strategic plan for higher education from 2005-2010. That effort was lead by Dr. Sohail Naqvi, Executive Director of the HEC. In part this was a response to a call from the Planning Commission for a strategic plan from the HEC. Overall it marked a recognition that the transformation of higher education required a clear vision and goals which would foster fundamental change and a detailed plan for implementation. The Planning Commission responded positively to the HEC’s four page outline for change, promised substantial funding to the HEC if appropriate plans could be developed along the lines proposed, and encouraged them to develop a strategic plan. The result was the Medium Term Development Framework: 2005-10 (MTDF). That document spelled out the major challenges for the HEC, a vision, and a series of goals and objectives for change. A rough budget for the MTDF was prepared by the HEC – later 28 Ordinance, ibid. chapter 2. 29 Ordinance, ibid, chapter 2, section 10, (3), p. 7. 30 Ibid. p. 8. 31 Higher Education Commission, Annual Report 2002-2003, p. 11. Evaluation of USAID Higher Education Portfolio 37 Academy for Educational Development August 29, 2008 spelled out in much more detail and modified with input from a team of World Bank higher education specialists.32 The MTDF focused on building a knowledge-based economy and identified three major themes for higher education policy: access, quality (focusing at the outset on faculty and infrastructure development), and relevance to national needs. Within those themes the MTDF spelled out programs for: faculty development; improved access; promoting excellence in learning and research; ensuring relevance to the economy and industrial linkages; leadership development and improved governance and management, quality enhancement including quality assessment and accreditation, and development of the physical and technical infrastructure. Initially, there was substantial resistance to the transformation proposed in the MTDF. In spite of opposition, by 2005-2006 the HEC was in control of higher education policy nationally and had made major strides in improving access, relevance, and the infrastructure of higher education. Quality improvement and accreditation were being put in place, new institutions were planned, and a new group of PhDs (trained at home and abroad), were beginning to return to the Universities, bringing new knowledge and state-of-the art training to Pakistan’s universities. Some 2000 scholarships have been awarded to Pakistani students, supported by the government and other donors, for study in doctoral programs in engineering, sciences, and other programs in countries like Austria, Britain, China, France, Germany South Korea, and the US. The commission plans to open nine new engineering universities across the country in collaboration with Austria, France, Germany, Italy, Sweden and other countries, although there is currently some pressure to reduce the number given the effects of current economic difficulties on the budget. The recurring budget also grew impressively during this period from approximately Rs 3 billion ($45 million) in 2001-2002 to almost Rs 15.77 billion ($243 million) in 2007-08. Funding for the Development Budget grew at an astonishing rate from a mere Rs 420 million ($6.5 million) in 2001-02 to about Rs 18 billion ($279 million) in 2007-08. In spite of fears to the contrary, universities and other degree granting institutions continued to maintain a great deal of autonomy. Indeed, by 2008 the HEC began to decentralize authority and decision-making and to concentrate on higher education policy for the future. Much of the ground-work for quality improvement of higher education was in place. The key now was to take advantage of this foundation to encourage faculty excellence, build strong programs, and foster knowledge creation for national development and the well being of the people of Pakistan. III. Government of Pakistan and HEC Goals The major goals of the Government of Pakistan and the HEC since it was established have focused on quality improvement, access, and relevance to national needs.33 The goals and objectives of the HEC were specified in detail in its Medium Term Development Framework 32 For a detailed examination of that budget in the context of several budget scenarios see: World Bank, Higher Education Policy Note, 2006. 33 Higher Education Commission, Medium Term Development Framework 2005-10, 2004, pp. iii & iv. Evaluation of USAID Higher Education Portfolio 38 Academy for Educational Development August 29, 2008 (MTDF) and in the Government of Pakistan’s Draft National Education Policy 2008.34 The goals and objectives of the GoP and the HEC are summarized below. A. Goals and objectives of the Higher Education Commission: 2005-10 The aims of the Medium Term Development Framework 2005-10 (MTDF) are spelled out below: • Faculty development including Master’s and PhD programs and short term courses. • Improving access with a target of 5 percent of the age cohort by 2010. • Promoting excellence in learning and research. • Increased relevance of higher education to the economy including effective industry linkages between higher education, business and industry. • Leadership, governance, and management. Improvement in higher education leadership, governance, transparency, efficiency, and accountability. • Quality assessment and accreditation. Increase quality throughout the system (including teaching, research, and service) and establishment of a national system of accreditation for public and private higher education institutions. • Improving the physical and technical infrastructure. Improve laboratories, expand and enhance access to information technology, upgrade institutional facilities, and expand campuses to meet student demand and national research and development needs. • To increase higher education funding to levels adequate to support improved quality (with the goal of achieving international standards), increased access, and relevance to national development. B. Government of Pakistan Goals The goals of the government for education are spelled out in the National Education Policy 2008, which contains a section on higher education. The goals include: • To “… produce responsible, enlightened citizens to integrate Pakistan in the global framework of human centered development.”35 • A major focus on access, equity (gender; rural-urban; provincial), and quality. • To encourage private provision at all levels.36 • Higher education-specific goals, among which are to: o Improve quality o Merit-oriented admissions o Equitable opportunities o Efficient administration, teaching, and research o Develop civic responsibility, social cohesions, and more tolerant society o Enhance research, innovation, economic growth, relevance and development37 o Promote and assist faculty development o Provide service to the nation o Improve teaching and learning 34 Government of Pakistan, Draft National Education Policy 2008, Ministry of Education, Islamabad, April 14, 2008. 35 Ibid., Draft National Education Policy, p. 2. 36 Ibid. Chapter 1. 37 Ibid. pp. 51 & 52. Evaluation of USAID Higher Education Portfolio 39 Academy for Educational Development August 29, 2008 o Increase access38 The goals and objectives of the Government and the HEC have been spelled out in detail making evaluation much easier than is often the case. The team also had the advantage of access to the very detailed 2006 analysis39 of the HEC by a World Bank team of which one of the members was a part. That gave the team an excellent basis for comparison of the changes that have taken place between 2005 and 2008. 38 Ibid. chapter 9 39 World Bank, Higher Education Policy Note Pakistan: An Assessment of the Medium Term Development Framework, 2006. Evaluation of USAID Higher Education Portfolio 40 Academy for Educational Development August 29, 2008 Chapter 5: Merit and Need-based Scholarship Program “I would have gone to Kuwait as a laborer if not for the scholarship.” (A student from Peshawar) “No one will be denied admission because of financial need.” (IBA – Karachi flyer) The USAID Merit and Need-Based Scholarship program (July 2004 - December 2010, USAID $7,333,350) is implemented by the HEC. The major aims of the program are: 1) to enable the academically qualified, yet financially needy Pakistani students, to continue university studies in the fields of agriculture and business administration in selected public and private universities/institutions of Pakistan; and 2) to enhance the institutional capacity of the participating institutions in implementing need and merit based scholarship programs. The program is designed to award 1,000 indigenous scholarships to needy and financially disadvantaged students for study of agriculture and business administration at the undergraduate level (four years) and the graduate level (two years), along with a few special scholarships for students in the earthquake region. The scholarships are awarded at 11 Higher Education Institutions (HEIs), 9 public and 2 private institutions (see Appendix V). The assessment team visited 8 of the 11 HEIs and conducted administration and student interviews at each site. Table 6: Summary Program Status of Merit and Need-based Scholarships Area of studies: Public/Private: Gender distribution: Special scholarships: Geographic distribution: Pending scholarships: 480 (14.8 percent female) from agriculture & 426 (20.7 percent female) from business administration 779 from public sector institutions & 127 from private sector institutions. 159 (17.5 percent) females & 747 (82.5 percent) males 32 special scholarships have been awarded to students from earthquake effected areas. Punjab (302), Sindh (292), NWFP (251), Federal Capital (34), and Balochistan (27). 101 candidates are short-listed for the 2007-08 Master’s Program I. Participant Selection and Awards The program is available to qualified “needy” students enrolled in their first semester/term at the participating HEIs. The universities/institutions make scholarship information available to the students through their regular admissions campaign process, information displays at notice boards, web sites, department notices, newspaper advertisements, and student and faculty contact. A. Selection The “academically qualified” applicant first obtains admission to one of the 11 participating institutions. Once admitted, the following steps are involved in determining “financially needy” status: • Standardized Application Form: the applicant completes a 12 page form requesting personal information along with supporting family financial status information and documentation on total monthly income, monthly expenditure, disposable monthly income, fixed assets, current assets, and general remarks. Each institution has a focal person (financial aid or scholarship office staff member), who is available to assist the applicant in clarifying the requested information. • Data Evaluation, Interviews and Short-Listing: the data are compiled in a standardized format and presented to an eight-member Institutional Scholarship Award Committee (ISAC) with university management and faculty representatives, notable scholar (external), HEC representative, and two community representatives. This committee verifies the data, conducts an interview, and short-lists the potential candidates. In many of the institutions, there are also department-level committees which review the application prior to submission to the institution–level committee. • The file of short-listed candidates is then forwarded from the institution to the HEC Scholarship Management Unit (SMU) in Islamabad. This unit reviews financial data only and makes a recommendation back to the submitting institution. • Final awards are then made by the institution in accordance with the number allocated. About 20-25 per cent of the initial applications were awarded scholarships. Waiting candidates (alternatives) are also listed. • The continuation of awards is based on a semester review of academic performance by the SMU in accordance with the HEC and institutional academic performance policies. In site visits to eight of the eleven participating institutions, the team obtained comments from the institutional staff about the process. Their consensus was that it is providing transparency, fairness, and an unbiased selection of “financially needy” students. First year problems with defining merit versus need and in developing and standardizing a set of objective and transparent procedures were overcome. Currently, the process seems to be implemented in a fairly uniform and unbiased manner. All candidates selected are clearly “financially needy” while some rejected are also needy. Partial scholarships would be helpful in such cases. More institutional autonomy in monitoring and re-allocating any remaining funds (for example from the small number of drop-outs) would be helpful. Transfer of payments from the HEC has been especially slow, which has caused high stress and a financial burden for the students in trying to survive for up to a year or more on borrowed funds, and caused a great deal of anxiety for those who administer the programs and often have to try to find funds to help the students survive. B. Awards Since 2004-05, 906 scholarships have been awarded and 178 students have completed programs; 101 new scholars are short-listed for the 2007-08 Masters Program. Since there are only two years remaining in the program, no new undergraduate scholarships are currently being awarded Evaluation of USAID Higher Education Portfolio 42 Academy for Educational Development August 29, 2008 as the undergraduate scholarship is for four years. The only new scholars will be in the two-year Master’s Program. As Table 4 indicates, the program has been implemented on a reasonable schedule since inception. Table 7: USAID Merit & Need-Based Program (1000 Scholarships Planned) Program 2004-05 2005-06 2006-07 Awarded/ Active Completed Undergraduate (4 yrs) 144 350 0 494 0 Masters (2 yrs) 148 142 122 412 178 Totals 292 492 122 906* 178 Gerald Boardman: Sources HEC, USAID, and other reports 2008 *Note: 101 new scholars are short-listed for 2007-08 Master’s Program. II. Site Visits – Participant Observations The assessment team interviewed a sample of the Merit/Need-based scholars from each of the universities/institutions visited. Selected findings and comments were as follows. Further details are available in Appendix VI. A special outreach effort is needed to identify and assist the talented but “financially needy” student. Most of the HEIs are handling the advertisement of the scholarships in conjunction with their admission process. Only a few are making a special effort to reach out through television, radio and on-site information visits to school and other locations. One example of an excellent outreach effort is the Institute of Business Administration (IBA) in Karachi. In addition, IBA brings highly talented and needy students on-campus and provides 6-8 weeks of preparation (in English, computer skills, math and communication skills) as part of the admission process at IBA’s expense (Karachi). The student scholarship recipients are needy and several mentioned that they would have dropped out if not for the scholarship. Several of the male students indicated that they had lost their father and without the scholarship they would have had to drop out of the university. Seventy five percent of the students attending the institution are from rural areas throughout Pakistan, with the majority of them financially needy (Faisalabad). The scholars found the application process took time but was easy and fair. Several students commented on how simple and easy the application process was and how appreciative they were of the US in providing the scholarship (Faisalabad). Others had more difficulty. One student in Peshawar did complain that the application process takes a lot of time. A majority of the scholars are completing their program and plan to either go on to further education or to obtain employment. All 30 of the undergraduate agriculture students awarded scholarships in 2004-05 will complete their programs, with 26 entering the Master’s program and the other 4 accepting employment. All 25 of the Master’s degree scholars from 2004-2005 completed their program and all obtained employment in the agricultural sector within six months of their graduation, most were employed Evaluation of USAID Higher Education Portfolio 43 Academy for Educational Development August 29, 2008 with the Punjab government and the others in the private sector. Of the 170 scholarships awarded to date, only 3 have dropped out of the program for academic reasons (Faisalabad). The number of female scholars participating was low. At the Agricultural University in Faisalabad, 90 undergraduate scholarships were awarded (12.2 percent female) and 90 graduate scholarships (5.5 percent females). Traditionally, fewer females go into the agricultural discipline, especially the plant science field, which is one scholarship focus areas, as the plant science field tends to be more field-oriented. More women tend toward the veterinary science field, which was not a part of the program. Further discussion appears in Chapter 10 Gender Equity. There were consistent delays in the transfer of funds Comments from students included many references to the fact that there had been a delay in the transfer of the scholarship funds which causes undue hardship because of the high need levels of the students (Faisalabad). As one administrator noted: “There was a time-lag on transfer of payments and some procedural problems early on; students were frustrated and some may have dropped out.” (Karachi). One student reported: “Delayed payments were a big problem for me. I had to take loans from my friends for up to one and a half years.” (Karachi). People complained that the HEC committee approval was very slow. It was reported that 8 applicants initially selected turned down their fellowships because of the delay. (Islamabad). Several students told us that it was taking about 10 months to receive their payments. Expand the number and disciplines covered by the scholarships. Several administrators requested that the institution receive more scholarships because of the high percent of “financial needy” students and there were many others who were equally needy who could not be supported. It should be noted that this institution added an additional 45 need￾based scholarships at its own expense. The team also heard requests that the scholarship be extended through PhD studies. Several administrators and students suggested that the number of scholarships be expanded to include the fields of animal science and veterinary science as well as agricultural economics and agricultural engineering (Faisalabad). The scholarship is sufficient to cover all costs and the recipients were very thankful for the scholarship. The students indicated that the scholarship is sufficient to cover all costs. The students were most thankful for the scholarship and all expressed their gratitude. (Faisalabad). As to the amount of money involved, most said: “Yes – the scholarship amount was sufficient.” (Students in Karachi). However, tuitions varied between and within institutions and the institution had to assist in some disciplines. Tuition had increased since the scholarship amounts were set in many cases (Karachi). Many positive benefits have been derived from the scholarships and the internships. One student said: “The scholarship changed me – I am more motivated, am working harder, know that I can achieve success, have broader sense of power, learned to think on my toes, am more punctual.” (Karachi). Another reported: “Initially I had to get a job to make it – now I can study full-time.” (Peshawar). One student noted: “Leadership, finance and marketing courses Evaluation of USAID Higher Education Portfolio 44 Academy for Educational Development August 29, 2008 were all relevant and really helped me in my internship.” (Karachi). An administrator reported that nearly 70 percent of the internships lead to a job (Karachi). A student in Peshawar noted: “I learned to compete and now have more self-confidence.” Suggestions for the future included a student organization for scholarship recipients, opportunities for additional needy students, regular payment of scholarships, and special emphasis for FATA. A number of suggestions were made during the interviews. In Peshawar, the student group as a whole told us they would like to form: “a student society for scholarships – one tied in some way to the USAID program.” They were very grateful for the scholarships and eager to be identified with USAID. Several students reported that they were worried about scholarship opportunities for the next group of students if the program was not renewed. They also suggested that it was important that payments be made regularly (Peshawar). Students suggested that there was a need for a quota for FATA to insure that enough people from that area were supported (Peshawar). It was also suggested that there should be a way to re-allocate the unused funds (Peshawar). III. Management Capacity and Institutional Strengthening One of the aims of the program was to enhance the institutional capacity of the participating institution in implementing need-based scholarships. Based on the review of the universities/institutions visited, including interviews with the institutional personnel involved in the financial aid process and a review of the forms and procedures being used, it was determined that each financial/scholarship office had a good understanding of the financial aid processes and procedures. Each had established a functioning Student Financial/Scholarship Aids Office and each was using a similar application process and system for determining financial need. Information exchanges between the institutions and the HEC about the process and procedures to be used have been excellent. The main formats for the initial forms and procedures were drafted by one of the institutions and then adapted, adopted and refined over time based on the experiences and input from the other institutions. The overall task of the application and review process is very time-consuming with multiple data points, supporting documents, and re-checks along with an interview of each applicant. The result has been a fairly standardized and transparent data collection and scoring system for making application and determining “financial need.” The institutions are bearing the costs of the entire advertisement, application, and screening process. The concept of “financial need” is being accepted by the parents, applicants, and institutions and the institutions are developing capacity to manage the task. The process and procedures have been standardized with standard evaluation criteria and follow up mechanisms to ensure continuity, along with proper operating procedures for stipend payments. IV. Analysis and Review of the Effectiveness of Monitoring and Evaluation (M&E) Tools Quarterly progress reports are being submitted by HEC to USAID according to a standard format. Included in the reports is information on activities undertaken in the quarter, accomplishments during the quarter, challenges encountered, planned activities of the next Evaluation of USAID Higher Education Portfolio 45 Academy for Educational Development August 29, 2008 quarter, women’s activities, human interest stories, and project level results to date. The institutions are keeping the participant records and can produce the requested information. The HEC is making field visits to the training sites and monitoring the selection and financial aid activities. The evaluation team reviewed a sample of the progress reports submitted. Most of the numbers are consistent with those at the institutional level. The information is mostly output data concerning the numbers of applications, scholarships awarded, candidates selected, active participants, and those who have completed their fellowships. The team found it to be generally reliable. As more of the scholars complete their programs, there is a need to compile follow-up information on employment and impact. Some of the institutions are beginning to add this information to their data base and will be able to provide such data in the future. The M&E information is supportive of the USAID project objectives and intermediate results (IRs) and indicates that objectives are being achieved. The follow-up data will be essential to show the ultimate relevancy and effectiveness of the program. V. Relevancy, Effectiveness and Efficiency Included in this section are comments related to the relevancy, effectiveness and efficiency of the Merit and Need-Based Scholarship Program. A. Relevancy The two fields receiving emphasis in the Merit and Need-Based Scholarship Program are Agriculture and Business Administration. Both are fields of high relevance and need in Pakistan. The information from the institutions visited indicated that the graduates are employable, especially those with a masters degree. An Agriculture Major with a Master’s Degree is virtually assured of employment and a Business Administration major with a Master’s Degree is equally employable. The undergraduate scholars from 2004-05 are currently completing their bachelor’s program. Many are planning on going on to higher education, especially those in agriculture, while others will be seeking employment. Several have received employment offers based on their internships. Comments from the institutional representatives indicated that a Business Administration bachelor’s degree graduate has a 70-80 percent chance of finding employment within six months of graduating while the Agriculture bachelor’s graduate really needs a Master’s Degree to ensure employment. The program directly supports both the Improving Access and Ensuring Relevance, core aims of the MTDF. All of the current active scholars (906, including 159 females) from the Merit and Need-Based Scholarship program were marginal students financially, many were rural poor, and the scholarship provided an opportunity for training and employment directly supporting Improving Access. All of the graduates will be in relevant areas and available to support the further development of Pakistan. B. Effectiveness: Achievement of Project Objectives Against Planned Targets and Intermediate Results Evaluation of USAID Higher Education Portfolio 46 Academy for Educational Development August 29, 2008 The aim of USAID/Pakistan Strategic Objective 3 is to increase knowledge and training and to improve infrastructure. The Merit and Need-Based Scholarship has two objectives that support this USAID objective. Objective 1: To enable the academically qualified, yet financially needy Pakistani students, to continue university studies in the fields of agriculture and business administration in selected private and public universities of Pakistan. Of the HEIs, 11 (9 public and 2 private) are participating. The plan is that 1,000 Merit and Need￾Based scholars (merit/financially needy students) will receive increased knowledge and training under this program. To date, 906 of the 1,000 scholarships have been awarded (2003 - 2007); 480 (14.8 percent female) in Agriculture and 426 (20.7 percent female) in Business Administration. Of the 906, 178 have completed. The active scholarships will result in 494 undergraduate degrees and 412 Master’s Degrees. There are an additional 101 candidates short￾listed for the 2007-08 Master’s Program. All of the completing students have obtained employment or are going on to further education. The selection process has resulted in about 25 percent of the applications being awarded scholarships. All of the scholars are “financially needy” and many would not have attended or would have dropped out of the university without this scholarship opportunity. Our projection is that objective one will be met. Objective 2: Enhance the institutional capacity of the participating institutions in implementing need-based scholarships. There is no physical infrastructure associated with the program, only institutional strengthening. Part of the institution-building has been to provide mechanisms for standardized data gathering and evaluation procedures and forms, common methods of data analysis, and protocols for information and data analysis. Based on an on-site review of 8 of the 11 universities/institutions, it was determined that each had a good understanding of the financial aid processes and procedures. Each had established a functioning Student Financial/Scholarship Aids Office and each was using a standardized application process and system for determining financial need. There are few drop outs and completers are finding employment. The main issue has been the slowness in approvals and fund transfer. The processes being utilized are effective except for the fund transfers, which could be corrected by providing more institutional monitoring autonomy (see recommendations). The process for determining “financially needy” is time-consuming but effective. Objective 2 is being met. Comments from the participants indicate that the scholarships have been beneficial to them in meeting both their personal goals and financial needs. Follow-up information on program completers and employment rates need to be continued over the life of the program. Sustainability of the Student Financial Aid/Scholarship Offices will be determined over time, though we see no problems. The program is effective in meeting participant, USAID and HEC goals and objectives, although there is a critical need for improved transfer of funds and additional tracking of effectiveness, which might include some anecdotal stories of the longer￾term benefit derived from the scholarships from completers after graduation and upon employment, and of longer-term institutional strengthening that takes place. Evaluation of USAID Higher Education Portfolio 47 Academy for Educational Development August 29, 2008 C. Efficiency How efficient is the program? The implementing partner/HEC administrative and management costs are minimal and there is no institutional overhead being charged. The costs associated with the institutional Financial Aid/Scholarship Offices and selection committees are being borne by the institutions. The selection process could be more efficient but the costs are all institutional. Selection is running at 20-25 percent of the applicants. Completion rate of the scholars is running above 95 percent with most, if not all, receiving employment in a relevant field (Agriculture and Business Administration) after they finish their education, resulting in a very efficient program. Some are remaining in school through a Master’s degree prior to entering the workplace. These are “financially needy” participants who might otherwise have dropped out of school for personal reasons as per the comments in previous sections. Based on the information available, the Merit and Need-Based Scholarship Program is rated as an efficient program. Our field visits showed that those programs are efficiently run with committed and well-trained staff. VI. Merit and Need-based Scholarships Conclusions • The Merit and Need-based Scholarship Program has helped more than 900 poor and disadvantaged students have access to higher education – most of whom would not be there otherwise. Higher education has increasingly become the province of the elite and wealthy. Even public higher education, with low fees, is out of reach for most Pakistanis, 70 percent of whom live on less than $2 per day. In that regard it meets strategic objective 3 to increase knowledge and training as well as infrastructure to develop high quality programs for boys and girls. It also meets intermediate goals 3.4, increased access in that it has opened access for almost 1000 poor and underprivileged student who might otherwise not have been able to gain a higher education. • In spite of initial start-up problems and delays, the program has matured to include uniform income reporting, a better understanding and utilization of the need concepts and measurement of need, improved provision of information about the scholarships to the public, and closer links with the participants. • The number of women admitted to the program, at 17.5 percent, is far too low. While we recognize the difficulty of recruiting women, especially in rural areas of the country, universities should be encouraged to undertake more outreach to encourage women in secondary schools to go to university. • The program has contributed to significant capacity building at the participating universities in their financial aid and scholarship offices. • The success of the Merit and Need-based Program has encouraged Japan to emulate the program. It has also stimulated participation in scholarship programs by several major corporate contributors to individual universities. • The administration of this program is too centralized at the HEC, with onerous reporting requirements which we believe are no longer necessary. The participating universities have now shown that they can administer these programs effectively and the various centralized reporting and selection mechanisms are time-consuming and create delays in allocation of scholarship. Evaluation of USAID Higher Education Portfolio 48 Academy for Educational Development August 29, 2008 • There are systematic and serious problems with late payments. In several cases the local institutions have had to arrange for food to be given to students while they waited for their funds to be processed or personally give them loans to tide them over. In a few cases students have received their grants after completing their studies. This totally negates the aims and goals of the programs and reflects negatively on both the HEC and USAID. The problem seems to be a combination of understaffing at the HEC (in part due to a government freeze on hiring), initial problems in getting funding released, and misunderstanding of the procedures by some universities. • Many students who received fellowships have become articulate ambassadors for the US – in one institution going so far as to suggest that a USAID/HEC Fellowship Alumni Association be formed nationally among those who received fellowship support. VII. Recommendations • Increase outreach and opportunities for more disadvantaged students to have a university education. • Expand the disciplines available for scholarships to include critical areas in the sciences and quantitative social sciences. This would also provide more opportunities for poor and disadvantaged women and to increase gender equity in critical areas. • The HEC should continue oversight of the program but with increased staffing to reflect the size of the program and to insure prompt payment of stipends to the students. Electronic transfers should be used to speed funding to university bank accounts or directly to students wherever possible. We are pleased to report that the HEC has responded to our expression of concerns and revised its procedures to speed up the payments to students.40 • Decentralize the selection process to the participating higher education institutions. The HEC should continue to have one representative on each institutional selection committee. • Reduce reporting requirements to an annual basis with the report to be due toward the end of the academic year. Where errors have been made, excess funding should be deducted from the institution’s next year’s allocation. This is a case where monitoring and review are excessive and are hindering the efficient operation of the program. • Attempt to provide scholarship support commensurate with standard tuition rates. • Increase the number of disadvantaged students receiving scholarships. Currently, scholarships are available for less than a third of those students eligible for scholarships. The Federal and Provincial government and the institutions can provide more assistance in this area and also examine how to leverage additional funds from other sources for need-based scholarships. This is a promising area for the private sector to demonstrate corporate social responsibility. • Encourage women to seek higher education and to apply for these scholarships. More outreach activities are needed and more culturally appropriate opportunities are needed, such as a greater use of the women’s universities. To increase the number of women applying it is important to increase the range of disciplines to include those more likely to attract women. 40 Confirmed during a follow up visit to the HEC by Tom Armor in August 2008. Evaluation of USAID Higher Education Portfolio 49 Academy for Educational Development August 29, 2008 Chapter 6: Science and Technology Cooperation Program “The project has helped in creating a buzz in microbiology, and raised the status of microbiologists in the country. Many MBBS students are willing to work voluntarily on the project.” (S & T co-operative research grant scholar at AKU) The Pakistan-United States Science and Technology Partnership Program (USAID $6,902,034; March 2005 - September 2008) aims to improve the capacity of Pakistani public and private science and technology institutions, predominately institutes of higher education, to better contribute to human and economic development in Pakistan. The activity is intended to increase the strength and breadth of cooperation and linkages between Pakistan’s scientists and institutions with counterparts in the US. The activity supports the USAID objectives to: 1) improve the quality, relevance, or capacity of education and research at Pakistani institutions of higher education in science and technical areas; 2) improve the capacity of Pakistani public and private science institutions to support industry competitiveness; and, 3) increase the capacity of science and technology to improve the well-being of ordinary Pakistani people. It also meets a number of the aims of the GoP and the MTDF including: 1) faculty development, 2) promoting excellence in research, 3) increase relevance of higher education to the economy, and 4) quality improvement. The Pakistan counterparts are the Pakistan HEC and the Ministry of Science and Technology (MoST). The Government of Pakistan is providing a total of $9 million; USAID is contributing about $7.0 million over a three-year period, and the US State Department about $1.2 million. The National Academy of Sciences in the US is the implementing partner. The projects include a combination of USAID, HEC, MoST, and other US funding in addition to in-kind contributions from each of the institutions involved. Unspent funds are carried forward. Table 8: Funding for Science and Technology Cooperation Project (million US $) Source Phase 1: 2005-08 Phase 2: 2006-09 Phase 3: 2007-10 Total HEC 3.0 2.5 2.0 7.5 MoST 0.5 0.5 0.5 1.5 USAID 2.0 2.0 2.0 7.0 US State Dept. 1.2 1.2 Total 5.5 5.0 5.7 17.2 Gerald Boardman: Sources HEC, USAID, and other reports 2008 I. Proposal Solicitation and Selection The joint Science and Technology program operates on an open-merit, peer review basis which emphasizes quality, relevance and potential for impact. The proposals are classified as highly relevant to the program objectives, marginally relevant, and not relevant. Only the highly relevant proposals are considered. Four annual phases of solicitation and selection of proposals were conducted in 2005 through 2008. Some of the Phase I and II projects have been completed with Phase III projects mostly just beginning and Phase IV at the award stage. Evaluation of USAID Higher Education Portfolio 50 Academy for Educational Development August 29, 2008 In-country proposals are solicited through national newspaper advertisements, through MoST, and HEC announcements and websites and in the US through National Academy of Sciences announcements. Some partnerships have been built on existing networks while other partnerships have resulted where the Pakistani researcher has sought out a top resource person or research laboratory in the US with which to link. Table 9: Proposals Received and Funded (Phases 1-3) Proposals received Proposals funded Percentage funded HEC 354 31 8.7% MoST 115 15 13.0% Total 469 46 9.8% Gerald Boardman: June 2008, from HEC & MoST data. The program has received in excess of 100 proposals at each phase with about a fourth being rated as highly relevant. The program has been able to fund about a third of the highly relevant proposals for about a 9.8 percent success rate for submitted proposals. The proposals are jointly prepared by the US and Pakistani scientists working together and independently reviewed by a minimum of three experts in Pakistan and three experts in the US. The results are then rated and compared and, if necessary, additional experts are asked to review the proposals. The final selection is made by a joint committee of scientists from the US and Pakistan. The proposals have included a wide range of fields important to Pakistan and the US, including agriculture, infectious diseases, water treatment, earthquake engineering, environmental and forensic sciences. The principle officers/investigators have been primarily males although there are three female principal investigators and several women have participated in the exchanges. Table 10: HEC/MoST Grants to date Phase 1 (2005) Phase 2 (2006) Phase 3 (2007) Sub-total (Phase 4 in Process) HEC 8 9 14 31 14 MoST 8 3 4 15 5 Sub-total 16 12 18 46 19 Gerald Boardman: June 2008, from HEC & MoST data. As indicated in Table 7 above, through Phase 3, 46 applied research grants have been awarded. An additional 19 applied research projects have been selected for implementation during the next year. The partnerships involved in these 46 research grants are impressive and include 21 Pakistani HEIs, 10 Pakistani R&D Institutes, and 41 US universities and R&D institutes/agencies (see Appendix VII). II. Site Visitations and Relevancy The assessment team interviewed the HEC and MoST Project Managers, had a conference phone call with the National Academy of Science (NAS) senior program officer and US partner institutions, made on-site field visits to 10 Pakistani S & T partnership institutions and R & D institutes, conducted interviews with principal investigators and their associates, and viewed the following research projects and facilities: Evaluation of USAID Higher Education Portfolio 51 Academy for Educational Development August 29, 2008 • Pakistan Council of Research in Water Resources (PCRWR)/Islamabad • Quaid-e-Azam University (QAU)/Islamabad • University of Arid Agriculture (UAA)/Rawalpindi • University of Agriculture (UAF)/Faisalabad • National Institute for Biotechnology and Genetic Engineering (NIBGE)/Faisalabad • Lahore University of Management Sciences (LUMS)/Lahore, • University of Punjab (UP)/Lahore • Centre for Excellence in Molecular Biology (CEMB) & Centre for Applied Molecular (CAMB)/Lahore • NED University of Engineering and Technology/Karachi • The Aga Khan University (AKU)/Karachi Overall, the assessment team was very impressed with the quality and relevance of the research being conducted and related laboratory facilities. All researchers were highly complimentary of the international exchanges and networking process, expertise and cooperation of the US resource persons, facilities visited, laboratory techniques observed, computer simulation models, and workshop training received while in the US and/or when the resource person visited Pakistan. Grantees in both the US and in Pakistan like the program. The completed projects have been very successful with high quality, relevancy and impact. Two illustrations of projects visited by the evaluation team follow. Additional examples appear in Appendix VIII. • Three water resource projects with the Pakistani Council of Research in Water Resources (PCRWR) successfully adapted a Water-Shed Management simulation model, used grass hedges in water-shed management, and won quality assurance awards for their desalinization and assessment of water quality lab work. The work of these projects is affecting water quality throughout Pakistan. • The Intensification of Forensic Services and Research at the Centre for Applied Molecular Biology project through the Center for Applied Molecular Biology (CAMB) has been helping law enforcement agencies do DNA forensics work – a first in Pakistan with unlimited potential. Each project involves a small amount of funds and when utilized in partnership with experts in the US, can leverage access to resources and development resulting in major national impact for Pakistan as well as benefits to the US. As the projects are completed, what is important is that there exists a mechanism for marketing and commercialization of the results of these applied research projects to improve the daily lives of the people and further the social and economic growth and development of Pakistan. III. Management Capacity and Institutional Strengthening Proposal development, screening and implementation are highly participatory between the US and Pakistani researchers, ensuring high quality and relevancy in the research and results generated. These processes are working well. Laboratories are being enhanced and techniques modernized, although there is need for some improvement in project administration and management at both the HEC oversight level and at principal investigator and institutional Evaluation of USAID Higher Education Portfolio 52 Academy for Educational Development August 29, 2008 support level. Included are some illustrative comments based on the external monitoring reports and the on-site interviews. • There were delays in purchase of software/equipment. This was due to a delay in HEC release of funds and in university procedures; e.g., project was to start in January and HEC release of funds occurred in July. • Equipment, travel and contingency funds were cut by HEC to the level that will make it very difficult to complete the project. • A project that was scheduled to start in September of 2007 is still waiting for release of funds. The Master’s students that were to work on this research are soon to graduate and no longer available. • Collaborative activity involved the training of Pakistani personnel in the US training went extremely well. • The audit report needs to be submitted. • US visa issues was a common issue. Many researchers have been waiting 8-10 months and there is still no word on their visa, even though they had traveled abroad and to the US previously. • The researchers are committed and proud to be doing applied research in spite of the delays and inconveniences. Some issues related to the Science and Technology grants were administrative and should be resolved with improved oversight. Delays in HEC project approvals and fund transfers are clearly an issue. The US partner side is authorized sooner and able to start their work while there may be up to a six-month delay on the Pakistani side. US visa issues and travel of US researchers to Pakistan continues to be a major issue. Male and female Master’s and PhD students are being involved as research fellows and gain valuable experience. Research is of high quality and appropriate research paper presentations and articles are resulting from the work. The assessment team observed that the R&D Institutes set up by MoST had very extensive facilities and equipment. More infrastructure support in terms of laboratory equipment and research capacity building is needed at the Pakistani university level. A closer working relationship is needed between some of these Institutes and the universities. IV. Analysis and Review of the Effectiveness of Monitoring and Evaluation (M&E) Tools Progress reports from the researchers are received on time and there is external monitoring of the progress and financial reports, which is important in ensuring quality research. The external evaluation reports are shared with the researchers. The evaluation team reviewed a sample of these reports along with a sample of the proposals submitted and researcher progress and annual reports. There is a need for improved clarity in performance indicators and in tracking these indicators. The project began the implementation of the TraiNet system about a year ago and is trying to be responsive to participants. This is a new process and, as with all new reporting requirements, takes time to implement fully. There is also need for a workshop for the researchers on how to prepare progress and annual reports as the reports vary widely in format and quality. There is substantial improvement in laboratory equipment resulting from the grants. Evaluation of USAID Higher Education Portfolio 53 Academy for Educational Development August 29, 2008 There has been a clear increase in knowledge based on the process being utilized in proposal development and review, conduct of the research, and international exchanges and workshops taking place, although specific indicators of increased knowledge are not being documented sufficiently. This is especially true of the training conducted in Pakistan. The US side is documented better. US researchers have been requested to work with their counterparts to record numbers of workshops, training activities, and participation of Pakistani women. USAID branding is also an issue as some are doing this while others are not as well. Project objectives are being met but not documented as well as they should be. M&E indicators need to be better documented and more in accordance with USAID performance tracking procedures. There is a need to clarify project managers’ and implementing agency roles and responsibilities. The MoST project manager makes some site visits to projects in conjunction with his regular work. Project amounts for in-country oversight are small. Most of the oversight is financial. The project mangers could play an important role in the monitoring area. The results of the projects are impressive, but a better job of M&E documentation and reporting is needed. Even something as simple as a comprehensive list of the grants, PIs and partner institutions is difficult to acquire. Every list obtained by the assessment team was different and each had inconsistencies. V. Effectiveness: Achievement of Project Objectives Against Planned Targets and Intermediate Results The program has funded 46 applied research grants in the first 3 of 4 phases of the program. All of the proposed research was judged by the review committees to be “highly relevant” to the development needs of Pakistan. An additional 19 applied research grants are being awarded during Phase 4. Some of the Phase 1 and Phase 2 research projects are completed although a few are still in-progress, most of the Phase 3 projects are in-progress, and the Phase 4 projects are yet to begin. A. Indicators of effectiveness To achieve the goal of increased knowledge and training, the USAID Mission has specified the following seven indicators against which performance under the Science and Technology Program is meant to be tracked and reported: o Number of faculty participating on the research grants o Number of Master’s and PhD students doing research in conjunction with their degrees o Number of training workshops conducted and number of participants o Number of papers presented at conference proceedings o Number of papers published in conference proceedings o Number of articles published in journals o Number of international development visits taken Each of the indicators should be regularly reported as part of the progress reports for each of the grants and compiled annually by the implementing agency. The numbers will be impressive. The team found that the quality of reporting varied among the researchers who were far more focused on research papers presented and articles published than on the required progress reports. It was Evaluation of USAID Higher Education Portfolio 54 Academy for Educational Development August 29, 2008 not possible for the team to compile any meaningful quantitative information for the indicators. However, the assessment team identified numerous examples in their on-site visits and review of the project reports. Included are three examples of resulting publications: • Submission and acceptance of a research paper to the Transportation Research Board Journal – this is a highly reputed and well-distributed journal in the area of Transportation Engineering. • Three papers published in international conference proceedings related to research conducted as the result of the establishment of the Extrusion Center at the Institute of Food Science and Technology/University of Agriculture, Faisalabad. • Six journal publications produced at the Centre of Excellence in Water Resource Engineering at University of Engineering and Technology, Lahore. Overall, the program is highly effective in meeting USAID, HEC and GoP indicators and goals although there is a need for improved documentation. This documentation should also include supportive anecdotal stories of interest. B. Indicators of efficiency How efficient is the program? Site visits indicated that the administration of the programs at the institutional level is very efficient and well done. In-country administrative and management costs are minimal. The HEC management costs are running less than one percent and the MoST costs are even less as the program manager is handling the S&T project on top of his normal work. The implementing partner general and administrative costs are contractual and set by the administrative contracting office. At the institutional level in Pakistan, overhead costs were nominal and typically covered by the institution. Principal investigators are receiving their salaries from the institutions as well. Typically, the research grant budgets contain the following line items: consumables, services/travel and subsistence, support personnel (research fellows), laboratory equipment, contingency, and overhead, where it exists. The average full-time cost of a research fellow varied from about $75/month to $150/month depending on qualifications. HEC administration was efficient with the exception of slow payment of funds in some cases. HEC administrative costs were running about 2.4 percent over Phases 1-3. We found the project administration to be efficient and very low cost. Table 11: Budgets for HEC Project Grants (Rupees million) Phase 1 Phase 2 Phase 3 Faculty development/research grants 200.00 160.0 200.11 HEC admin./support personnel 1.68 2.69 HEC admin./contingency 1.20 5.0 2.40 HEC admin./equipment 0.62 Total 203.50 165.0 205.20 Gerald Boardman: June 2008, from HEC data. Note: funding in Phase II was reduced to utilize carry over funds from Phase I. Relative to impact, the projects are selected based on relevance and only a third of the proposals rated as “highly relevant” are being funded so only the top priority projects are funded; thus, the selection process is set up to support only the most relevant (i.e., highest impact) projects. Evaluation of USAID Higher Education Portfolio 55 Academy for Educational Development August 29, 2008 Graduate students are participating as research fellows and are receiving their Master’s and PhDs as the result of the research generated from participation on the projects. The training provided is at the highest level; very cost-effective considering the experience and quality of resource persons available. The research fellows are exposed to the top researchers and research techniques in their fields as the result of their participation. No data exist on employment of the students following their graduation as it is still too early in the program but, overall, based on interviews at the institutional level, the graduates will be highly sought after as the result of the experiences gained from their participation on such high quality research projects. VI. Science and Technology Cooperation Conclusions • The US-Pakistan Science and Technology Cooperation project has provided significant capacity building for staff at the universities and institutes in applied research, assisted in critical technology transfer, and promises to make important contributions to national development. • Most of the programs involve projects critical to Pakistan’s development. At the same time they involve contributions to US research in providing access to critical information and identification of diseases and other problems that can affect the livelihood of both countries. Among these are techniques for identifying new high risk TB viruses that are now beginning to show up in Pakistan; development of disease resistant strains of citrus (cooperative research that is helping protect against citrus diseases in Pakistan but also beneficial to the citrus industries in California and Florida); and work to develop wheat varieties that are resistant to a “virulent” wheat fungus which devastated crops in East Africa and is now moving through Iran41 toward Pakistan. • This project is an excellent value for USAID with about half the funding coming from the HEC. • The projects examined were well run, tightly budgeted, and showed promise of significant results. • Links with US institutions are continuing and have been very effective. We observed a high level of cooperation and shared knowledge between Pakistan and US grantees which is of great mutual benefit to both parties. US partners interviewed were very positive about the effectiveness of the projects and the mutual benefits. • One serious weakness of the program is its trifurcated nature which makes scaling up of findings, commercialization, and dissemination difficult. We suggest mechanisms for improved coordination, information sharing and commercialization below. • Payments to grant holders were often made late. That has posed serious problems for the researchers involved and limited the success of the program. There has often been a lag of six months to more than a year in providing the funds to some Pakistan participants. We were not able to find a defensible reason for the delays. They are unacceptable and should be eliminated as part of the project renewal. • A problem arises from the selection process in which, in a number of cases, funding on the Pakistan side was cut below the levels needed to carry out effective research. 41 FAO, “Wheat killer detected in Iran Dangerous fungus on the move from East Africa to the Middle East,” FAO Newsroom, Rome, 5 March 2008. Evaluation of USAID Higher Education Portfolio 56 Academy for Educational Development August 29, 2008 Officials at the HEC acknowledge the problem. We make recommendations to overcome it below. • A major challenge is how to include more Pakistani female scientists in the exchanges and in project management positions. Women were observed as research fellows, working in the laboratories, and as head of laboratory facilities. This was especially true at the Pakistan Council of Research of Water Resources, CEMB, CAMB, and NIBGE. • The USAID Mission has specified seven indicators against which performance under the Science and Technology Program is meant to be tracked and reported. Each of the indicators should be regularly reported as part of the progress reports for each of the grants and compiled annually by the implementing agency. Currently the reporting system is not in place and operating. The team could not compile any meaningful quantitative information against the indicators. • The selection process is very rigorous with many worthy programs remaining unfunded because of limited resources. At the present time only about a third of the proposals ranked “highly relevant” are funded for a success rate of only 9.8 percent of applicants. There is significant potential to increase the impact of this program if additional funding can be found. VII. Recommendations • Create an Innovation Unit at the HEC to publicize, develop, and commercialize promising projects for general public use and for commercialization. This could be linked to the HEC patent office. • Create a central unit in the HEC (which could be the Innovation Unit, above) to facilitate communication and information sharing between participants. The trifurcation of the program means that no one is aware of the total program other than the Academy in Washington. Thus Pakistan is not getting full advantage of the program and its results. • Streamline the administration of this program and ensure that payments are made in a timely fashion. • Improve communications during the Pakistan part of the selection process in terms of providing prompt feed-back to applicants regarding their proposals. • Ensure that adequate allocations are made to cover the cost of the projects, and cut the number of projects rather than underfund them. • Ensure that data on program achievements are collected and reported from grant holders. Evaluation of USAID Higher Education Portfolio 57 Academy for Educational Development August 29, 2008 Chapter 7: Fulbright Scholarship Program “I was an employee. When I came back, I was a manager. I was told to start thinking as a manager, to think about issues and how to resolve them. I learned that NGOs can learn many ways to make profits and achieve sustainability.” (Male participant). “My friends told me that I would not get a scholarship because I was a woman – this was not true, it was an equal opportunity.” (Female participant) In accordance with a Memorandum of Agreement signed in April of 2005, USAID transferred to the US Department of State $15 million to establish a US Education Foundation in Pakistan (USEFP, Fulbright – Commission) to provide two-year Master’s degree training to Pakistanis in disciplines critical to the social and economic development of the country. The program was expanded to include PhD training in December of 2005, when USAID and HEC signed a Memorandum of Understanding, with USAID contributing an additional 4.5 million per year for PhD scholarships and HEC contributing $10.5 million.42 USEFP also receives $800,000 a year from the Department of State, Bureau of Educational and Cultural Affairs. The Pakistan Fulbright program is the largest in the world. The program targets early to mid￾career professionals and works to foster both an academic pursuit and a mutual understanding of the people of Pakistan and the US. I. Participant Awards Announcements about competition for Fulbright Scholarships include advertisements in Pakistan’s leading English newspapers; Dawn, The News, The Daily Times, The Frontier Post, and Balochistan Times. Also, ads were run in the Dawn Review and Spider, The Monthly Herald, and the Friday Times Weekly. The Foundation holds on-site informational sessions at the Fulbright offices, selected higher education institutions, at other interested agencies, and at various educational fairs throughout Pakistan. Since September of 2005, 412 Pakistanis (334 Master’s, 65 PhDs, and 13 post-doctoral) have departed for the US for participation in the Fulbright program and 67 Master’s participants have completed programs and returned. There are 196 grantees (120 Master’s, 62 PhDs, and 14 post￾doctoral) already selected for the 2008 Fulbright program and are in the process of applying for their US University admissions and visas. At the request of the HEC, the sciences were identified as a priority field for the PhD. The US Educational Foundation was responsive to the HEC request as well as making a concerted effort to encourage more females and other disadvantaged participants to apply. The Fulbright program has made important strides in this area over the past several years. The percent of females selected has increased from about 30 percent in 2005 to 42 percent selected as 2008 grantees. The geographic spread of the participants has become more representative of the country. The 42 See: The United States Educational Foundation in Pakistan, Annual Report 2006 and Annual Report 2007. Evaluation of USAID Higher Education Portfolio 58 Academy for Educational Development August 29, 2008 spread of the 2008 Master’s and PhD grantee cohort is as follows; Punjab – 88, Sindh – 42, ICT – 23, NWFP – 14, Balochistan – 14, Northern Areas – 0, FATA – 2, and AJK – 0. The disciplines are also beginning to show a wider spread with the 2008 grantees showing 32 percent in Engineering, Computer Sciences and Sciences; 26 percent in Economics, Finance, Business Administration, and Public Policy; 13 percent in Telecommunications, Communications, and Journalism; 12 percent in the Social Sciences and English; 9 percent in Public Health, and 8 percent in the Arts and Humanities. The Fulbright management should be commended for their efforts in this area as significant improvement has occurred since the early years in 2005 and 2006. II. Site Visits – Participant Observations In order to evaluate the experiences of the Fulbright recipients, the team interviewed a sample of Fulbright scholars who had returned to Pakistan at three of the Foundation Offices: Islamabad, Lahore and Karachi. The interviews were carried out as focus groups using a list of questions but adopting an open-ended discussion format facilitated by one member of the evaluation team. Fourteen of the 67 returned Fulbright scholars (7 males and 7 females) participated in the focus group sessions. The participants attended Master’s programs during 2005-2007 at universities such as John Hopkins (public health), Harvard (economics), Florida State (IT), University of Pennsylvania (non-profit management), University of Kansas (physics), upstate New York (environmental policy), New York University (computer science), SUNY Albany (public health), University of Pittsburg (public health), and New York University (MBA). The Fulbright scholars are employed in areas including Public Health, IT, the NGO sector, a teaching assistant at LUMS, a Physics College lecturer, a Section Officer in Environmental Protection Department, a Program Manager of a software company, and one working in the ADB-funded Decentralization Support Program. Two other women returnees are doctors, one is working at AKU in the neural-surgery department and the second is awaiting a posting in the government health department. The focus groups were informal and semi-structured to obtain the information needed while at the same time to provide participants with an opportunity to talk about any other aspect of the program they wished. The facilitator assured the participants that their comments would be confidential and encouraged them to express their views frankly. A sample of responses is included below and full text appears in Appendix IX. III. Access to Scholarships Participants learned about the scholarship through a wide range of different sources including friends, relatives, newspaper advertisements, education fairs, and the HEC web￾site. Many participants said that they were influenced by their friends or relatives: “I studied with a friend who was applying so I also applied.” Another reported: “I found out about the program from my relatives who had been Fulbrighters.” Or another: “My brother had applied.” (female and male participants). Others saw the advertisement in the newspaper or the HEC website. Another participant was inspired by US scholars whom he met as part of his work. Some Evaluation of USAID Higher Education Portfolio 59 Academy for Educational Development August 29, 2008 obtained information at the USEFP education fairs. Most respondents felt that the program was well advertised and explained. We discussed the question of advertising with Foundation staff and learned that in the last few years they made a major effort to target women and men in rural and underserved areas such as the NWFP, conducting workshops, advertising in local language newspapers and meeting potential students in places like Quetta and Peshawar. Most participants felt communication about the Fulbright program was good and the information provided by USEFP useful. They were especially complimentary about the ease of the application process although a few expressed dissatisfaction. Participants especially liked the application form because of its fixed format and word boundaries, unlike other scholarships which had lengthy forms with repetitive questions: “The application process and supporting materials were relevant and complete.” “The process was facilitating.” “The process was quite different (not as restrictive in terms of age and other requirements) from the government scholarship process, World bank process, or faculty process.” The dissatisfaction expressed was mostly about some delays and/or communication gaps in the admission process: “My admission was delayed and it reached a point where I told them I would apply myself. I finally got the admission in April.” Another reported: “I didn’t get the college of my choice.” Several students reported that they were admitted to universities that were not on their list of priorities. One problem which is endemic to Fulbright Programs all over the world is the choice of universities for the participants’ programs. Even the HEC was somewhat naïve about the possibilities open to students, wanting them all to go to the top few US universities. The HEC and participants were not aware that there are 4,500 higher education institutions in the US, many as good as or better than these. On the whole, participants were satisfied with the process and complimented the Foundation staff on the care with which they handled applications, student questions, and preparation. In the information packet sent with application material, the Foundation should provide an explanation of the US system, the level of competition at certain of the institutions, and the institutions that have best programs in different subject areas. For example, there are more than 4,000 universities in the US. The pressure on some of the better known institutions is tremendous (e.g., Harvard gets about 10,000 applications for 200 places). Even students with straight “As” and perfect GRE scores do not necessarily get in. The pre-application information might also note the point about the best programs being in a variety of different institutions, often not those well known abroad. That might reduce some of the problems related to choice of institution, or at least make students aware of the highly competitive nature of the system. A few participants were dissatisfied with the HEC requirement that they sign a bond to guarantee their return, USEFP TOEFL requirements, and handling of the application. Evaluation of USAID Higher Education Portfolio 60 Academy for Educational Development August 29, 2008 A few participants expressed displeasure regarding the HEC paper work and the requirement of pledging family property as part of the bonding requirement. On the other hand, most respondents found this a reasonable expectation and when asked about it said they had no difficulty in doing that or getting the required information. One participant was unhappy about having to take the TOEFL after having completed a degree in the US before applying. However, another participant was of the view that TOEFL and GRE scores provided a good basis for decision making. There was no evidence of gender discrimination in any aspect of the application, selection, or operation of the program. In general, participants said that the process provided an equal opportunity to all. Nonetheless, it would be useful to discuss differences in male-female relations in the US and Pakistan. This is especially important for women coming from a conservative religious background. Speaking to the question of gender equity, one student noted: “Forget gender, it was fair for all.” Another noted: “The application process was very fair, without any discrimination, and a diverse group was selected.” (Female participant). One woman Fulbrighter felt she was advantaged saying: “I was favored being a woman.” Another women said: “My friends told me that I would not get a scholarship because I was a woman – this was not true, it was an equal opportunity.” Some Fulbrighters suggested that people from rural and disadvantaged areas had special difficulties that needed to be considered. We applaud the Foundation for it current efforts to increase the number of students from rural and disadvantaged areas. These students may need special help in preparing for their studies in the US. The existing pre-departure programs and English and computer science should be expanded to meet the need. As one respondent noted: “Often rural students were brighter than urban, but could not compete because of lower English competency.” (Male participant). Another remarked: “Rural students could not compete with English medium students.” (Female participant). Participants felt they were generally well briefed in the pre-departure orientation and did not experience any significant cultural shock in the US. A few exceptions are noted, especially by the women. The following instances of “cultural shock” were mentioned by the men: “I attended the orientation. It included legal stuff, teaching style, history, etc. It was very helpful.”; “We were well briefed.”; “The alumni conference in LUMS included an excellent preparatory session, including a dramatic role play.” And by the women: “The teaching style based on the discussion method was very different. We should have been introduced to it before.” One reported: “ My first semester was terrible as I had to take care of my son alone, there was a lot of statistics and epidemiology, and I got a ‘D’ grade. Things improved when my husband joined me.” Another said “I had no problems. I had great American friends. They allow you to do anything, unlike here.” Another reported: “It is easier to be a single parent in the US due to good day care facilities, bus timings, etc.” Participants had mixed views regarding their orientation in the US. Evaluation of USAID Higher Education Portfolio 61 Academy for Educational Development August 29, 2008 Some of the comments were: “During the orientation they gave us knowledge about how the university worked. They marketed themselves.” (Male participant). “In the initial orientation they told us about American culture, schooling system, studies. (Female participant): “Sometimes IIE made things too complicated; I got more help from the person at the international studies office at the school.” (Male participant). “Most of the problems were during the first four months.” (Female participant.) Participants found the Fulbright Program to be better than other scholarships in terms of appropriate funding, but some did experience shortages in funds associated with their field of study for the most part and, in a few cases, unexpected fees. Some participant comments follow: “I ended up saving money.” (Female participant). Another said: “I was better off than most other Pakistani students that were studying.”(Male participant). Some participants were impressed that Fulbright funded art supplies, attendance in workshops and seminars; others mentioned problems like having to pay an International Student Fee, shortage of funds for buying books, laptop, etc. Participants found the scholarship adequate for one person, but not for a family. Participants mentioned several instances of discrimination. We recommend that the issue of racism in the US be included in the orientation. Racism in the US needs to be discussed up front. The orientation might usefully include role￾playing exercises. Although conditions are improving in some parts of the US, racism and discrimination are real facts of life and the students should be made aware of it and helped to be prepared to deal with these situations when or if they occur. This part of the focus groups produced a very thoughtful and emotional discussion, which is a credit to the participants. The issue of racial and ethnic discrimination was a major one for most of the participants. We note some of their comments here: One Fulbright returnee reported: “In Pittsburg there was bias against people with brown skin, particularly among conservative elderly people.” (Female participant). Another reported: “In New York they considered Pakistanis terrorists – not so in Arizona. It was also true of some professors.” (Female participant). Another had a very bad encounter with a bus driver. She noted: “My student card had expired. The driver confiscated my card, harassed me, and sent me to the police station.” (Female participant). The issue was resolved and the police helped her obtain a new identify card. University faculty members did not escape as subjects of concern. One Fulbrighter reported that: “Some faculty had misconceptions about people from Asia or brown skins. They did not accept our ideas so easily, perhaps because they did not want to hear them. The attitude was that people coming from underdeveloped countries do not know anything. Faculty from big cities had a greater level of acceptance.” (Female participant). Others reported no problems of this sort, but they were in the minority. For example one said: “My faculty members were open to ideas.” (Male participant). IV. Relevance of Educational Experience Participants had a variety of goals in applying for the scholarship, with some being clearer than others. These goals were modified in some cases during their studies. Participants experienced varying degrees of success in achieving their goals. Evaluation of USAID Higher Education Portfolio 62 Academy for Educational Development August 29, 2008 Participants mentioned a variety of goals for applying, including establishing a non-profit organization (NGO), moving from administration to research, becoming a manager, etc. Some participants were unclear about their goals before they applied, but became clearer during or after their studies abroad. Some of the experiences were: “You may not change your goals, but they get refined. My goals and aims were refined.” (Female participant); “I was trying to leave my job and do research and consultancy. After my return I set up a consultancy with two others. The idea of going formal probably came during my US study.” (Female participant). “I was not very clear when I went abroad. After seeing all the art scenes I realized that I could make a future by being a print maker.” (Female participant). Other experiences were similar – many of the participants changed their thinking or their goals. Several indicated a need for some type of internship or work experience to complement their formal training or example, in the software industry: “The software industry prefers industry experience over education. I could not get experience in the US.” (Male participant). Another thought about higher education and student affairs: “I had an M.Sc. in Physics and went for the degree of Master’s in Higher Education. It was a slight challenge, but I managed. I would have liked some field training in the Student Affairs area; like an internship, but I was not allowed to work.” (Female participant). In general, the participants felt it was a great opportunity to be in the US, but wished they could have experienced the work world more. Participants were generally happy with the institution where they studied abroad and found courses relevant and challenging in most but not all cases. There was considerable variety in the institutions attended; for example; “John Hopkins gives you a great experience. You get a much bigger picture, learn about developing countries. It offers you a lot of breadth, but not enough depth. Ten months was not enough. We needed more time.” (Female participant). Another said: “John Hopkins was more community, Harvard was more policy. I chose JH because I wanted a community background.” (Male participant). Courses were challenging and exceeded expectations: “About 60-70 percent of the course was challenging. I studied because I wanted to, but did not have to stay up nights. The human rights course was an eye opener. Networking was a big thing. It was not entirely academic.” (Female participant). Another said: “They put me in CSE which required an electronic engineering background.” (Female participant). Another commented: “Not much memorization was required in the examinations. One had to get the gist of the book. We had the freedom even to challenge the author of the book.” (Female participant). There were a few disappointments: “The courses were interesting, not challenging, except finance and economics. So I was a little bit disappointed. I said to myself: have I come all this way to study this.” (Female participant). And another: “I wanted to do policy research. I selected the topic of identifying reasons for non-functional policies and gave the outline to the advisor. I worked for 6 months on the topic. But my advisor panicked, saying it was a topic for a PhD rather than a Master’s thesis. I had to change my topic very late. I did my Master’s project in 3 weeks on storm water discharge, which was not a policy issue.” (Male participant). Evaluation of USAID Higher Education Portfolio 63 Academy for Educational Development August 29, 2008 V. Usefulness of the Program Most participants experienced a significant increase in their knowledge and skills, and a broadening of their perspective as a result of studying abroad. There was an increase in knowledge and skills: “The econometrics course helped me a lot.” (female participant); “The program trained us to go and find out the facts. But the required books and journals were also available, unlike here. We became aware of the need for current knowledge. We always got a new perspective to explore and discuss in class through inquiry￾based methods.” (Male participant). There is greater independence in learning: “There were huge jumps in my work in two years. Previously, I never wrote about my work. If you went through a block, they held your hand. They taught me how to research my own work and analyze it.” (Female participant); “I do my work better, have increased analytical thinking, and developed a broader perspective. I do not face the problems that I faced earlier.” (Male participant). A few participants were able to use their newly acquired skills in designing and conducting research, and in teaching. All of the Fulbright returnees were Master’s degree participants, the PhD Fulbright returnees will not start to return for another year. It is the PhD returnees who will be moving more into the teaching and research professions, although some of the Masters participants took research and design courses as a part of their programs and have found these skills to be useful in their work. Included are a few examples; “I learned qualitative research methods. Now I am more organized as a teacher, give strict deadlines.” (Female participant). Another said: “I learned qualitative research methods in public health and am using the methods in designing a telemedicine program involving mobile phones.” (Female participant). One reported: “I took a course in bio￾statistics, and learned that I was not a good researcher, but a manager. I learned how to interpret research findings, and how to use them to develop a program for change, and evaluate the change. We gave cell phones to Lady Health Workers, and are studying their impact on health services.” (Male participant). Regarding teaching one reported: “I got a broader perspective. My teaching became more informal. Students talk more easily with me and give more respect to me because of my US degree compared to my colleagues without foreign degrees. They have this mind set.” (Female participant). VI. Employment Impact Some participants improved their employment position as a result of participating in the program, while others did not. Gender did not play any significant role. The employment situation differed among the participants as some went from jobs they planned to return to while others did not. Experiences varied – some illustrations in finding a new job follow: “Fulbright made it easy to get a job. I got what I asked for.” (Female participant). Another said: “I got a job ten days after my return, due to Fulbright.” (Male participant). One noted: “The software industry prefers industry experience over education. I could not get experience in the US. If I had been here I could have taken up a position which was taken up by others. Things are moving slowly.” (Male participant). Evaluation of USAID Higher Education Portfolio 64 Academy for Educational Development August 29, 2008 And for those returning to the same job or organization: “I am working at the same position. I had a Master’s and did a Master’s in the US” (Male participant); “I moved up to a better position in the software company.” (Male participant). Another said: “In the public sector, promotion is not based on education.” (Male participant). One reported: “I lost as my colleagues in the ADB are now consultants, but the US education was worth it.” (Male participant). A former government employee who is still waiting: “I am still waiting for my posting - since January 2008.” (Female participant) The Foundation has offered to help these participants. The experiences of returnees varied, although, most who applied for new jobs were successful. A few who were returning to their previous jobs or organizations struggled a bit though that was not universal. All the participants were thrilled about the experience and optimistic about the future. They know it is up to them to find a way to use their new skills and knowledge in the employment sector and are open to broader options. The Foundation is assisting them as they request assistance. Gender was not an issue. VII. Monitoring in the US and Follow-up in Pakistan We were interested to learn about any assistance Fulbright recipients received in the US from IIE and their Universities and as well as about any follow-up upon return. Assistance and support from the Institute of International Education (IIE) was mixed – some had more favorable experiences than others; several said IIE staff assistance improved after a few months with the arrival of a new focal person at IIE. Several participants complained about the first IIE official regarding delays in payments, responding to phone calls and emails, and lengthy documentation requirements. However, they praised the IIE focal person who joined after a few months. A male participant said that Indians received more help than Pakistanis. Participants filled a feedback questionnaire from IIE. Most participants found the university authorities much more helpful than IIE, but that is to be expected since they had face to face contact and could seek help more easily. Upon their return to Pakistan, several participants said that they were in touch with the USEFP Pakistan Director, who invited them to events, including alumni conferences, and offered to assist with seeking a job. VIII. Benefits & Suggestions Participants described a variety of benefits of attending the program, including development of tolerance, freedom to do what one wanted, and a prestigious degree. Personal and family benefits resulting from the program experience were significant as participants described everything from a changed family, to development of tolerance, increased self-confidence and career aspirations, independence, career changes, and a prestigious degree. Comments include: “My experience changed my family, who joined me in the US.” (Male participant). Another noted: “I developed tolerance and learned to give more space to others.” (Female participant). Another reported: “I have started to write more, am thinking of further studies, setting up a business.” (Female participant). One noted:” My professor motivated me to Evaluation of USAID Higher Education Portfolio 65 Academy for Educational Development August 29, 2008 study for my PhD.” (Male participant); “I changed from epidemiology to social behavior and community health. I got an extension for one semester through Fulbright.” (Female participant). And another said she benefited tremendously from: “Getting a degree from a prestigious institution, international exposure, and having my daughter with me.” (Female participant). Participants wanted to be allowed to apply for a Fulbright PhD scholarship following participation in a Master’s program; some desired permission to work after completing their degrees. One unanimous recommendation was emphasized by the participants – the need to change the one degree rule and allow eligibility to apply for a PhD Fulbright at a future date. As one student noted: “Those who have availed a Master’s program should also be allowed to apply for the Fulbright PhD program.” (Female and male participants). Several suggested that Fulbright allow them to have a work experience or internship opportunity: “We need to be allowed a work experience either after completing our degree or as part of the program.” (Male and female participants). Participants recommended increased opportunities for people from rural areas and more networking among Fulbrighters. There were two recommendations: - 1) more outreach to rural areas: “It is good that they have an office in Karachi. They should also open offices in Sukkur and Larkana.” (Male participant). Another suggested:” They should include more people from rural areas, help them in English.” (Female participant). 2) Still another commented that it would be useful if there could be more networking among Fulbrighters: “One person returning to an organization cannot produce change. They should select three American universities in the same field (e.g. public health) and send people in groups, form networks there, and back in Pakistan.” (Female participant). Another said: “They should form alumni networks in cities. USEFP could play a bridging role.” (Female participant). Participants found living on-campus more culturally enriching. They had mixed views of interacting with American and international students. Relative to living on-campus versus off-campus, a comment by one of the women was: “I lived on campus, had good opportunities for discussion, sharing.” (Female participant). One of the men noted that: “By living off-campus, with people I already knew, I missed the cultural part. A few thousand dollars more would have enabled me to live on campus.” (Male participant). Comments related to the interaction with Americans versus international students was mixed; “My experience with international students was more encouraging than with American students. American students were less responsive.” (Male participant). Another said: “I lived with American students. They were very responsive.” (Male participant). One noted: “Many organizations promoted interaction between international and American students. They were very helpful, invited me on Christmas and Good Friday, etc.” (Male participant). IX. Management Issues – Capacity and Efficiency The USEFP in Pakistan has experienced some growing pains since 2000 as it has gone from a small program to become the largest Fulbright program in the world. This rapid expansion was Evaluation of USAID Higher Education Portfolio 66 Academy for Educational Development August 29, 2008 not without certain organization restructuring and management adjustment issues. Additionally, during this period the application process was changed to an on-line application process. The administrative burdens increased dramatically including: handling over 1,000 applications a year, tracking and follow-up on returnees, setting up two new testing centers in Lahore and Karachi, and from 2007 taking charge of IT testing conducted by Prometric for all of Pakistan. The unique arrangement of governance, oversight and accountability for the USEFP would benefit from a cooperative effort to facilitate smooth operation of the administrative processes of the foundation. We suggest that an interagency committee be set up to informally coordinate efforts between the various agencies involved in the process. In addition, the management structure of the USEFP should be decentralized with the addition of middle management staff to provide wider program and financial control functions. The Executive Director should assume an oversight role of the selection process and participate in individual selection decisions only by exception. While US Fulbright training in the US is more expensive than training in Pakistan or in the region, it is an outstanding program without equal in terms of quality. It is an excellent long￾term investment for the US. It is a key to the realization of the quality goals of the MTDF and the goals for higher education of the GoP. It is providing major benefits to Pakistan and the US in terms of mutual understanding and linkages in higher education. The USEFP administrative service costs are less than 20 percent. The program is also funding most of the post-doctoral programs from the service fee – a significant contribution to Pakistan too. The low￾administrative costs coupled with a record of success by the awardees, and excellent return and job employment rates, creates an efficient program. The team heard that some people think the Fulbright Program in Pakistan should be run using private contractors. The team feels strongly that this would be disastrous to all aspects of the Fulbright program, would detract from effective capacity building efforts in Pakistan generally, reduce the number of scholarships significantly, undermine the efforts of the USEFP to build capacity among its Pakistani staff, and be counter to the goals of USAID to help build capacity, expertise, and management skills in Pakistan. X. Analysis and Review of the Effectiveness of Monitoring and Evaluation (M&E) Tools In terms of monitoring and evaluation reporting, in November of 2007 it was concluded that since the Fulbright funds are transferred from USAID to the Department of State, the reporting requirement for the funding rests with the Department of State and can be handled as a part of the ECA annual statistical report to Congress. This is true for all of the ECA, USAID, and HEC grantees. As noted earlier, added monitoring of completion of MA and PhD programs, as well as job placement of returnees (university, government, business, self-employed) would be useful. XI. Effectiveness: Achievement of Project Objectives Against Planned Targets and Intermediate Results Increased knowledge and training in higher education (Strategic Objective 3). The Fulbright Program is designed to “contribute to institutional capacity” and to assist in the development of Evaluation of USAID Higher Education Portfolio 67 Academy for Educational Development August 29, 2008 top researchers and professionals. It has had substantial success in that effort sending more than 400 scholars to the US to study with nearly 200 more selected for 2008. The team carried out interviews with returned Fulbright scholars, as noted above, and were impressed by their quality, training, and the attitudes about the US they brought back to Pakistan. They are already providing important contributions to universities, government, business, and the communities in which they life. The training aspect of the program is critical to the achievement of both USAID goals and those of the MTDF for Pakistan. We believe the program has more than met its interim goals and should be expanded substantially. Increase infrastructure development of higher education through scholarships, research, and linkages (Strategic Objective 3, and Operational Plan). The Fulbright Program has been one of the most important factors in increasing infrastructure development in Pakistan providing training for over 400 scholars since its inception with nearly 200 more scheduled for participation in 2008. Equally important is the fact that this training is in the United States – which provides the highest quality training in the world. That is exactly what is needed if Pakistan is to meet its very high targets for state-of-the-art training for university faculty and professionals. Interviews with returned Fulbrighters verify both the quality of their educational experience in the US and the kind of training, creativity, and enthusiasm they have brought back to Pakistan. Deepen the capacity of key social institutions (Investing in People). The team was struck by the rapport and community developed among the Fulbright participants during their participation. This “alumni association” has its own role to play as a social institution in Pakistan with its members active in their communities and as a group. In addition, they are adding to raising the standards of higher education, government, and the businesses in which they are not active participants. Establish roles of key public and private delivery (Investing in People). The Fulbright program is a key contributor to the development of private delivery of higher education by providing training to outstanding young scholars who return to teach in public and private higher education institutions or to work in government or the private sector. Without it such training, the pool of potential faculty members and research staff would be far smaller than it is and well below the critical needs for development in Pakistan. Increase access to higher education (IR 3.4 & PMP). As noted above, the Fulbright Program is one of the most important contributors to increased access to higher education in Pakistan. It is also critical in providing the high quality professionals need. Strengthen and improve education policy making (IR. 3.1). The Fulbright program is providing a pool of well trained faculty members and professional who will be the backbone of the next generation of policy makers in Pakistan. Some earlier graduates of the program are already in high positions, and those recently trained will make their way into such positions in short order. They will be important additions to policy planning, analysis, and policy-making in Pakistan in the future. Evaluation of USAID Higher Education Portfolio 68 Academy for Educational Development August 29, 2008 XII. Fulbright Program Conclusions • The Fulbright Program in Pakistan has made significant contributions to Pakistan in providing training for students, building university linkages, and in helping to create mutual understanding. • The selection process is reasonable, fair, and transparent. Senior staff are committed, lauded by the students, and have worked hard to improve gender balance and minority area representation. • The Program has made an important contribution to quality improvements of higher education and is assisting the GoP in meeting the quality improvement goals of the MTDF – especially in terms of providing high quality training for Master’s and PhDs in the US. • Graduates of the Fulbright have excellent reputations and are eagerly sought by universities, government, and other employers. The Fulbright Program makes it easier for graduates to get a job or promotion. It helps with qualifications, learning new tools and gaining new knowledge and experience. • The return rate of Fulbright Scholars is close to 100 percent. Returnees were enthusiastic about their US experience and are building positive links with US institutions which seem to be continuing since their return. There were very positive responses about the Fulbright program at institutions we visited. • The United States Educational Foundation in Pakistan also runs a number of additional important programs such as testing for GRE, TOEFL, and IT. The IT program recently received an award as the best IT program in South Asia. • The Fulbright Program is expensive but with a payoff, provision of quality higher education – a critical key to the long-term development of quality higher education in Pakistan. The Fulbright Program is one of the most worthwhile investments USAID has made in Pakistan. XIII. Recommendations • Continue efforts to increase the number of women by targeting women through advertising, field trips, and education fairs. • Target minorities and disadvantaged areas, focusing especially on encouraging applications from FATA, NWFP, Balochistan, and FANA areas. While Fulbright participation has been fairly broad geographically, too few candidates currently come from the border areas – areas that are key to peace and stability in Pakistan in the long run. • Support classes to prepare Fulbright grantees from disadvantaged areas in English language, math, computer skills, and other preparation prior to their departure for the US. • Focus primarily on PhD training except in areas where a Masters in regarded as a terminal degree. • Encourage the Executive Director to delegate more authority on day-to-day matters to an expanded middle management staff. Increase middle management and establish clear, documented participant selection decisions and financial management procedures. Evaluation of USAID Higher Education Portfolio 69 Academy for Educational Development August 29, 2008 • Support PhD and Master’s graduates to have a 6 to 12 month internship or fieldwork experience in the US prior to their return given the good return rate and the benefits of such work in addition to their formal higher education training. • Send several students in selected critical areas to the same US university. Focus on academic areas of critical need to help develop teams which would return to the same university to create a “center of excellence.” • Include the topic of racism in the US in the pre-departure orientation. Many participants had experiences with racism for which they were not prepared. Evaluation of USAID Higher Education Portfolio 70 Academy for Educational Development August 29, 2008 Chapter 8: Forman Christian College (A Chartered University) Improvement Grant “A general education like this is compulsory for success today. You need to be broadly based. To be competitive in society today you need to know something about everything, not just the field you are working in.” (FCCU Male Student) One of the oldest private colleges in Pakistan, Forman Christian College was founded in 1864. Forman was taken over by the government in 1972, along with a number of other private education institutions. Its quality and infrastructure deteriorated significantly during that period. Under government control it became an all men’s institution, except for the Master’s programs, in spite of having been the first college to admit women in the Punjab in 1902. The College was returned to the Presbyterian Church in March 2003, and received University status in 2004.43 It began to admit women once again to its undergraduate programs in 2004. The total student population in 2008 was 4,330 with 2,114 being at university level. The University has delayed plans to start a Master’s program for a few years, choosing to focus on its undergraduate programs and curriculum first. We believe that is the right decision though it is not popular with some faculty who are eager to move ahead with graduate studies. FCCU requested permission to shift USAID funds intended for the Master’s program to other programs in September 2007. They had not received a response at the time of this review in May-June 2008. The evaluation team made two trips to FCCU to evaluate the programs funded by a $5 million USAID grant for the period September 2004 – September 2009. During those visits the team talked with the Rector and other senior administrators, met with students on two different occasions, and talked with faculty and staff. A follow-up trip was made in August by Tom Armor. I. Strategic Plan The University has been developing a strategic plan which we read in draft form. It has a well￾developed statement of mission and goals as well as an assessment of the institution’s market position. The overall focus is on developing a high quality academic program, outstanding teaching and learning, and high quality graduates. The Strategic Plan includes continued emphasis on faculty development, academic support including the library, internships and international educational experiences for students, expanded central administration, improved research support and encouragement by the university, improved student life with an emphasis on teaching core values including mutual respect, tolerance, and understanding. II. Innovative Programs The team was impressed by the number of innovative programs initiated at Forman including faculty development workshops on campus, revision of the curricula to emulate the US model, its liberal arts program, a strong commitment to teaching and learning, and its efforts to develop 43 In recognition of its university status we refer to it hereafter when abbreviated as FCCU. Evaluation of USAID Higher Education Portfolio 71 Academy for Educational Development August 29, 2008 a center of excellence in science in conjunction with its new science building. In 2005, FCCU began a four-year Bachelor’s (Honors) program which is now in place using the semester system. It is designed to meet international standards. FCCU hopes to be accredited by the New England Association of Schools and Colleges (one of the best regional accreditors in the US) and has had a preliminary site visit by its director to explore that possibility. The Bachelor’s program has been mentioned by some educators as a model for other institutions. III. The Academic Programs and Teaching The University has made a major commitment to high quality teaching, and to a four-year liberal arts program that emphasizes problem solving, active learning, individual creativity, and discussion. • USAID funds have assisted the curriculum development efforts for 18 academic departments and allowed 9 programs to bring in US consultants to assist with this curriculum development. Workshops have been conducted on pedagogy for faculty focusing on both teaching and learning. • FCCU takes the evaluation of teaching very seriously and has developed one of the best evaluation systems we have seen. Teachers are evaluated in three ways – by their students, faculty members, and their deans. They receive regular feed back about their performance. Teachers are mentored if problems are found or if they request assistance. The team is not aware of any other university in Pakistan that takes teaching so seriously. • In interviews with students we were struck by the fact that most students spontaneously talked about the quality of the teaching, the creative relationship between faculty and students, and the quality of their programs. For many, that was the main reason they attended FCCU. • The faculty too, showed a strong commitment to their students, to teaching, and to rebuilding the university into one of the best in Pakistan. IV. Infrastructure The College is still recovering from the deterioration of its infrastructure during the years of government control. Nonetheless, it has made excellent progress in refurbishing and rebuilding much of that infrastructure, opening a new building for business and social sciences this year, beginning construction of a new science block, and is in the process of upgrading its Internet capacity and IT systems. The library remains a problem and needs to be upgraded and expanded. USAID has made a major contribution of $1 million to the construction of a new science building that will have 84 faculty offices, 19 research labs, 20 teaching labs (including 4 computer labs), 24 classrooms, an examination hall, and an auditorium plus other related facilities such as a library/computer room for student work. This will allow Forman to greatly expand its science teaching and research and attract the best scientists in the country. Such a facility is essential if FCCU is to attract first rate young scholars, provide the high quality training necessary to achieve international standards, provide the space necessary for first rate science teaching and research, and upgrade its laboratories to adequate levels. From interviews with faculty members in the sciences, it is clear that this facility is sorely needed and greatly Evaluation of USAID Higher Education Portfolio 72 Academy for Educational Development August 29, 2008 anticipated. It will be a major contribution to science training in Pakistan and is a critical part of the strategic plan. V. Administration The University suffered from an understaffed central administration in its early years. As a consequence, record keeping in some areas was lax and there was inattention to some administrative matters. The University’s basic Executive Management Staff is made up of 12 people.44 Its management team consists of 28 positions, of which 7 are currently vacant. The management team has been expanded and a number of positions were in the process of being filled in August 2008. The Rector told the team about an error in billing USAID for $33,351. This mistake was self￾reported to USAID by the Rector. The funds were refunded. We want to emphasize that this was an error about what was eligible for purchase under the USAID grant. Most of the purchases for which a refund was given were text books FCCU has a policy of procuring course textbooks and placing them in the central library for students to check out as reference books during the semester, reducing the book costs to students by bulk purchases. Most of the $33,351 went for the purchase of text books. Our review indicates that this expenditure, according to the grant agreement, was probably an allowable expense. We also note that in recognition of the need for more personnel for administration at FCCU, the Strategic Plan calls for increase in administrative staff, including an expanded financial aid office, which is now in process. FCCU should demonstrate that it is serious about responding to ongoing USAID requests for documentation and reporting requirements by initiating contact with USAID soon rather than waiting for further USAID requests. Similarly, USAID should strive to respond to FCCU submissions in a timely fashion. We believe the expansion of the FCCU administrative staff will improve the effectiveness of the administrative operations. The University was criticized about its handling of some aspects of the grant in a report on “Fund Accountability.”45 The concerns related to text book purchases, a charge for an HR course, equipment purchases, and criteria for a scholarship program. The program is now in compliance. Several other issues were outside our charge. FCCU was also criticized for what was seen as a lack of transparency in student and faculty scholarship programs, lack of gender equity, bias against non-Christians, difficulty locating library books purchased using USAID funds, concerns about equipment purchases, student unhappiness with the academic program, and lax administration. A member of our team, on a supplementary visit, did undertake a sample check of USAID funded books in the library in which the randomly made requested item was found and its purchase details documented. We did not audit equipment and were not requested to do so. We did check out the other concerns and allegations carefully and found no basis for them. There was no evidence of gender bias; 44 We note that USAID staff seemed to be of the impression that it was only two people. See Trip Report, by Aabira Sher Afgan, Education Management Specialist/CTO, 1-04-08. p. 5. 45 A. F. Ferguson & Co, Forman Christian College, Lahore, Fund Accountability Statement Audit and Review, (n.d, circa July 2007). Evaluation of USAID Higher Education Portfolio 73 Academy for Educational Development August 29, 2008 students said the institution has made a major effort to recruit women since it regained control in March 2003. During the prior period of government control it had been turned from a coeducational institution to an all male institution. We found students and faculty to be very happy with the new program. There was no evidence of bias against Muslims – the majority of students are Muslims. Both students and faculty commented on the success of the university’s programs on religious tolerance and mutual understanding. VI. Faculty Development and Scholarships The University has emphasized faculty development to improve the quality of the institution – to regain the status it once had as one of Pakistan’s outstanding universities. Part of its plan has been to encourage faculty members without Masters and PhDs to seek further studies. With USAID grant funds, the University has been able to support 60 faculty members (27 percent of the total) for that purpose. A. Faculty Scholarships The faculty development program was originally to have two tracks, one to be a master’s program to be presented by an American university during summers in Lahore. That part of the program fell through due to national security threats to the safety of foreign scholars which made it impossible for the university to provide insurance for visiting US scholars. The second track was for faculty members to undertake an approved Masters or PhD program at an institution locally – in most cases in Lahore. That became the only option. FCCU covers the cost of tuition for these programs, which has ranged from a low of $150 to a high of $2,442 with the average being about $1,200.46 B. Interviews with Faculty Members. We selected a sample of 32 faculty members randomly and were able to interview 29 of these faculty members, 16 percent of the faculty of the university. One sample was drawn from the group of those receiving faculty development support for their MA and PhD programs; the other drawn from faculty members who had not received support. We focused our questions on their views about faculty development, the process of selection, its transparency (including the way in which they found out about it), their perception of its fairness, gender equity, determination of need, religious bias, satisfaction with faculty development program, and their views about the state of FCCU. We asked faculty members if they were briefed about our visit before hand. They reported that they were not. The results were as follows: • All faculty members interviewed were aware of the faculty development program and very pleased with it. They had learned about it at faculty meetings, from colleagues, and from their departments. • The faculty members involved in the program, with one or two exceptions, were happy with it, felt it gave them new creative opportunities to grow intellectually, and had gained a great deal of confidence in the process. Several had completed the program and felt it had helped them tremendously. One noted that the program 46 Forman College, Faculty Support for M.Phil. or PhD Study, unpublished internal document provided June, 6, 2008. Evaluation of USAID Higher Education Portfolio 74 Academy for Educational Development August 29, 2008 already had the effect of producing changes. He felt there was: “… a new wonderful environment now at the University. It had also helped senior faculty, many of whom were now very guiding and supportive – not controlling.” • Any faculty member who was interested in further studies could apply, seek admission to an appropriate university, and if admitted, be awarded a grant. None of the faculty we interviewed knew of anyone who had been turned down for a grant. We later learned that one or two were turned down initially because the choice of institution was not appropriate for their field of study. They were awarded grants once admitted to an appropriate institution. • All the faculty members interviewed felt the program was an excellent one, important to the improvement of institutional quality, and critical to the development of FCCU. • About one third of the faculty members we interviewed had not taken part in the Masters or PhD program. However, 7 of these 10 had PhDs and thus were not eligible. • We were also struck by the commitment of the faculty to FCCU. All of those interviewed liked the new curriculum, though some noted that change always poses problems and will take some time to complete. Those interviewed felt it essential to improve the quality of education and were proud that FCCU was at the forefront of those efforts. We reviewed the requirements for the faculty development fellowships in the light of criticism that it was not advertised and that the fellowships were not widely available. A USAID report had criticized the selection process as unfair and largely unknown to the faculty. The findings are as follows: • The team found nothing in the Grant about a process for selecting faculty members for these awards for Masters and PhD study. The operative wording seems to us to be the statement: “We will assist faculty members whose needs can be met through PhD and M.Phil programs in Pakistan. We estimate that twenty-one (21) of our faculty members will be able to prepare themselves well by completing an M.Phil program here in Lahore at either the University of the Punjab or Government College University.”47 When the US summer master’s program fell through, the local program, which was originally going to focus on several programs such as Urdu and Islamic Education, was opened to all disciplines. We note that to date, 49 faculty members (26 percent of the faculty) are now in, or have been in, the Masters or PhD program. When taken together with those who already had PhDs, that constitutes 59 percent of the faculty, a much higher percentage than most universities in Pakistan (where the average is 23 per cent). We found that no one who applied for a faculty development grant, and had been admitted to an appropriate program, was turned down for funding. • The team found no mention of “need” in the document regarding faculty development though there may have been some oral discussion of need-based funding at some point in time. Since virtually everyone who applied received a grant, need does not seem to be an issue. 47 Ibid, p. 8. Evaluation of USAID Higher Education Portfolio 75 Academy for Educational Development August 29, 2008 VII. Interfaith Cooperation The College prides itself on its success as a model of interfaith cooperation with a faculty that is one-third Christian and two-thirds Muslim, as well as a mix of religious faiths in the student community. A number of students volunteered that this was an important attribute of FCCU and several noted their best friends were of other religious beliefs. A number of students saw this as one of the unique and vital features of the college. Of the total student population about 709 are Christian (about 16 percent), there are a few Sikhs and Hindus, but most of the rest of the roughly 3,600 students are Muslim (about 83 percent). VIII. Gender Equity The number of women students at the university is low at 23 percent of the total with only 19 percent in the need-based scholarships going to women. This is due primarily to the fact that women were excluded from the University during the period of government control in spite of the fact that FCCU was among the first higher education institutions in Pakistan to admit women. Since regaining autonomy in 2004, the College has worked to increase the number of women students, as discussed in more detail in Section X below. IX. Need and Merit-based Student Scholarships Concerns were expressed by USAID about the administration of the need and merit-based scholarship program. We heard no complaints about the program during our first visit, though we asked students, faculty, and administrators about both the scholarship program and their satisfaction with it. Given some difference between a USAID report and the team’s evaluation during our first site visit, the team returned to campus on June 10th to follow up on our first evaluation trip to FCCU. A. Interviews with Students We drew a random sample of students from 2006-07 and 2007-08 classes for the interviews, selecting 54 students randomly for possible interviews. That week was exam week and, thus, we were only able to interview 23 students, or 6 percent of the students who received fellowships. We asked the students if they had been briefed prior to the interviews. All of them said they had not. A summary of responses is presented below: • 11 were from the 2006 class, 12 from the 2007 class. All were content with the program and grateful to USAID for the support. • All of the 2007 students reported that they would not have been able to attend FCC without the fellowship. Eight out of 11 of the 2006 group reported they could not have attended without it – 87 percent of the total. • We reviewed the procedures for 2007 with the chair of the selection committee and note that the requirements included: o Full-time status. o A score of 65 percent or more on the BISE Intermediate Exam or O-level examination. Evaluation of USAID Higher Education Portfolio 76 Academy for Educational Development August 29, 2008 o Established financial need with family income of Rs. 20,000 per month or less. o A completed financial aid application. • We reviewed a sample of financial aid applications for 2007-08 and found them complete. We reviewed the financial records of students and found no exceptions to the family income maximum and saw several applications that had been rejected because the monthly income was too high. • All 12 of the students interviewed in a random sample of the 2007 group reported that they were from poor families. We reviewed the record of the Committee’s financial evaluation of students for qualification and saw that none of them exceeded the Rs 20,000 monthly salary maximum; most of their family incomes were much lower than that. • We asked the students if they felt religion was a factor in allocations of fellowships and all 23 said that was not the case. They noted that more Muslims had scholarships than Christians. • We asked about gender discrimination; 22 of 23 students said it did not make a difference. The one who felt it did (a woman) reported that she felt there was a slight preference for women. • When we asked students how they became aware of the scholarship, 22 percent said they saw an ad in the newspaper, 13 percent reported other sources of information, 17 percent said from friends prior to their applications. Thus, 52 percent the students knew about it prior to applying for admission. The other half learned about the program from flyers on campus, faculty members, or administrators. We obtained copies of the ads from 2006 and 2007 as well as campus flyers and handouts. • All 23 of the students thought that the selection process was fair. • When asked about their programs and satisfaction with FCCU, all of them were extremely positive, lauded the quality of the faculty, the liberal arts approach, and felt the new four year curriculum gave them greater opportunities to compete in globally by providing the information they needed, helping them learn to problem solve, and access the tools they needed. B. Understanding the Need and Merit Based Program at FCCU As part of the review, we examined the documentation for the grant, the Award of Grant 391-G-00-04-01036-00. In the document, the purpose of the grant is stated as follows: “The purpose of this grant is to provide support to Forman Christian College … in its development of a four-year bachelor degree program and strengthening of programs in basic science and information technology, as described in Attachment 2 of the Agreement entitled “Program Description.”48 That attachment is the FCC proposal. The section of the proposal on scholarships reads in part: “Our commitment is to educate young people in Pakistan regardless of their ability to pay. FCC does not wish to become a college that educates only students from wealthy families…” The proposal then discusses the need to charge tuition to cover costs, notes that the funds will cover tuition only, and thus the need for students to have some of their own resources. The text continues referring to the fellowship selection process and reads: “The scholarships will 48 USAID, Award of Grant, September 24, 2004, p. A1. Evaluation of USAID Higher Education Portfolio 77 Academy for Educational Development August 29, 2008 be awarded on the basis of a combination of need and merit as determined by a faculty committee. No student will receive funds beyond demonstrated need. Of those students with need, the top 200 students will be selected by the faculty committee based upon merit results plus results of an interview to determine their character and motivation.”49 • The team understands this to mean that what constitutes merit for these purposes will be determined by the faculty committee, and that need will be considered as a crucial part of the process, but that student interviews will also have an effect on selection as noted above. • The team thus enquired about what standards had been set by the Faculty Committee. They are reported above. • The team found that in the early years of the program, publicity did not mention that the funding came from USAID, nor was the need-based aspect of the scholarships operationalized. Both problems were corrected in 2007. In interviews with the administration, the team discussed the importance of the need aspect of the scholarships as a way to recruit more poor and disadvantaged students and that its availability be featured prominently in advertisements about the program. It was clear from our interviews with students that availability of scholarship funds was an important fact in their choice of FCC. The Rector noted that FCC would highlight need-based scholarships in future advertising, which already includes the newspaper and posting of flyers on campus. X. Observations about FCCU Programs A. Results of random interviews with student and faculty As noted above, a number of concerns were expressed by USAID about student and faculty unhappiness with the administration of FCC, the lack of transparency, and the small size of the grants. Some of the comments reported did not relate to the fellowship but to other concerns such as complaints about the high cost of living in Lahore, and worry that the program did not cover the cost of books and accommodation. These concerns do not relate to the administration of the USAID program but to economic conditions or the terms of the grant itself. The team asked respondents if they had any other concerns they would like to express. No additional concerns about the program were mentioned. The concerns we heard were a desire for faster Internet connections, a gym for girls, and improved facilities for men. In sum, the students and faculty we interviewed (a random sample in both cases) were very content with the USAID programs, had strong praise for the administration, were very pleased with the quality of the faculty, appreciated the new four-year program, and felt they were fortunate to be attending FCCU. We conclude that USAID funds are being wisely spent, that the institution has improved its monitoring processes, self-reported its errors, and moved to improve its administration. B. Perceptions 49 Ibid, Attachment 2, pp. 2-3. Evaluation of USAID Higher Education Portfolio 78 Academy for Educational Development August 29, 2008 It appears to the evaluation team that most of the problems and criticisms stem primarily from misperceptions. As opposed to the USAID funded Merit and Need-based Scholarships managed by the HEC, clearly designed as a need-based program, the FCCU Grant document has no such single criteria. It notes that merit criteria, to be determined by a university committee, and the results of interviews, will be part of the considerations for selection. In looking at the response of FCCU administrators to the concerns articulated by USAID, it seems to the evaluation team that the administration of FCCU was fully responsive to the suggestions made by USAID. • The evaluation team concludes that the administration at FCCU moved to emphasize the need aspects of the scholarship in 2007 and used need as the primary criteria for those new students meeting admission’s requirements, and who were awarded scholarships. • The administration was responsive to other concerns of USAID such as publication of information about scholarship and transparency of the process. For example, attached is a copy of the scholarship application form including questions about family income (Appendix XI). • The team found no implication or suggestion that anyone was misusing USAID funds in any way. • There were program weaknesses at FCCU which have been acknowledged and other problems as we have noted. Overall, the team finds the USAID Institutional Development Program at FCCU to be an excellent program. XI. Analysis and Review of the Effectiveness of Monitoring and Evaluation (M&E) Tools We have reviewed monitoring and evaluation issues in some detail in earlier material in the chapter and made a number of recommendations in that regard in what follows. Forman College needs to improve its monitoring of USAID program funds, as noted earlier, and is in the process of making changes to make that possible. The administration of the College was understaffed at the time of this review with a number of vacancies which are in the process of being filled. We recommend that the administration focus more extensively on USAID requirements and work closely with USAID staff to resolve problems. FCCU needs to be more responsive to USAID reporting requirements (most of them standard practices) and develop the procedures required, perhaps with assistance from USAID staff. XII. Effectiveness: Achievement of Project Objectives Against Planned Targets and Intermediate Results Increase knowledge and training in higher education (Strategic Objective 3). The key focus of the improvement grant for Forman Christian College University is increase knowledge and training by providing a four year bachelors based on the high quality US model of liberal education. As noted in the Operational Plan, it is to be a “world class education at Pakistan prices.” In addition, it is to provide assistance to “deserving and needed students, a MIS system for FCCU, strengthen science teaching, and help with professional development for faculty. As noted above, Forman is on the way to realizing all these goals. The four year program is in place, Evaluation of USAID Higher Education Portfolio 79 Academy for Educational Development August 29, 2008 the college has provided scholarships to 600 needy students who were adequately meritorious – most of whom would not have been able to obtain a university education without it. In addition, 60 faculty members received USAID assistance for professional development. We were impressed by the quality of the students, the new high quality curriculum (begun in 2005) and the work done to up grade 18 programs at FCCU. We believe that FCCU has more than met the interim goals of USAID and is well on its way to regaining it status as one of the outstanding universities in Pakistan. Increase infrastructure development of higher education through scholarships, research, and linkages (Strategic Object 3). FCCU infrastructure development has benefited from USAID support in the development of a new MIS system which is currently in place. It has also been able to utilize USAID funds to bring in experts to help assist in upgrading it academic programs setting state-of-the-art as its targets. We believe FCCU is well on its way to achieving these goals. It is already seeing an increase in application as a result of the new four year program. Students recognize that study at Forman improves their chances to gain entrance to PhD studies abroad and enhances their chances of employment. Infrastructure development is also improving as a result of the purchase of 2833 new books for the library, computer and science equipment which is being purchased, and a new science building which we saw was well underway. Deepen the capacity of key social institutions (Investing in People) and increase participation, representation, and democratic values (Strategic Objectives 4, 6, & 7). FCCU is building a strong higher education institution that also focuses on critical values of citizenship, high moral standing, integrity, and high quality. It is a model of inter-faith harmony and takes pride in the fact that though its founders were Christian educators the majority of its students are Muslims. In interviews with students and faculty we were struck by FCCU’s focus on interfaith cooperation and the close relationships between students of different faiths. We see FCCU as creating important social capital for a future democratic, participatory, Pakistan. Establish roles of key public and private delivery (Investing in People). The improvement grant to FCCU has been an important contribution to private delivery of higher education services. FCCU is a private institution and this support demonstrates a US commitment to assist private as well as public education. Without a strong private public education sector, Pakistan will not be able to meet the access goals of the MTDF nor the needs of business, education, government, and other employers. Critical to success is the fact that FCCU has set its standards high and is already well on its way to becoming one of the outstanding universities (once again) in Pakistan. Increase access to higher education (I.R 3.4, and PMP). FCCU now has 2,114 undergraduate students of which 21 percent are women. It is thus making a considerable contribution to access in Pakistan. It low enrollment of women is a consequence of the fact that when it was put under government control it was changed from a coeducation institution (it was one of the first in 1902) to an all male institution. It returned to coeducation status when returned to church control in 2003 and is in the process of expanded the number of women admitted with a target of 50 percent women. Evaluation of USAID Higher Education Portfolio 80 Academy for Educational Development August 29, 2008 Strengthen and improve education policy making (I.R 3.1). FCCU is at the forefront of innovative educational policy and is regarded as a model for higher education by senior officials in the HEC. The review team was impressed by the new curriculum and the progress that has been made to date to raise standards. Both students and faculty expressed enthusiasm for the changes in spite of the disruptions and increased cost caused by moving from a three year to a four year program. Forman is also in the process of revising its administrative structure to improve accountability, student counseling, and effectiveness of dealing with donor funding. XIII. Forman Christian College University Improvement Grant Conclusions • The College Improvement Grant to Forman Christian College was designed to assist the University in its development of a four year bachelor’s degree program, upgrade its faculty, strengthen its programs in basic science and information technology, and provide scholarships to help meritorious poor students attend FCCU. It’s academic curriculum is modeled on a US broad liberal education, with high quality world-class academic programs, and problem solving participatory teaching and learning for student, • The University was criticized about its handling of some aspects of the grant in a report on “Fund Accountability.”50 The concerns related to text book purchases, a charge for an HR course, equipment purchases, and criteria for a scholarship program. An error in charges for textbooks was self-reported by the institution. Another was a mistake in charging a $512 HR course to a faculty development part of the grant. Another concern related to the failure of the fellowship selection committee to consider need as well as merit and interview results in its scholarship decisions during the early years of the program. That error was admitted by the Rector at an earlier point and had been corrected by the time of our visit. We verified the new process of selection, determination of need, transparency of the process, and reviewed both the applications and financial material about a random selection of students. The program is now in compliance. Several other issues were outside our charge. • FCCU was also criticized for what was seen as a lack of transparency in student and faculty scholarship programs, lack of gender equity, bias against non-Christians, difficulty locating library books purchased using USAID funds, concerns about equipment purchases, student unhappiness with the academic program, and lax administration. We followed up on the other concerns and allegations carefully. In the process we carried out interviews of randomly selected students and faculty, reviewed student family financial statements, examined advertisements and flyers about scholarships, reviewed policies and procedures for scholarships, and interviewed administrators. However, by way of summary conclusion, we want to emphasize that we found the scholarship selection process for 2007-8 based on need as well as merit needed for admission and an interview. All this is in accord with the grant agreement. We found no evidence of religious bias in student fellowships or faculty grants. Students and faculty reported that scholarship programs were well publicized, faculty members denied any religious discrimination in faculty grants, noting that anyone who applied was eligible. • There was no evidence of gender bias; students said the institution has made a major effort to recruit women since it regained control in March 2003. During the prior period 50 A. F. Ferguson & Co, Forman Christian College, Lahore, Fund Accountability Statement Audit and Review, (n.d, circa July 2007). Evaluation of USAID Higher Education Portfolio 81 Academy for Educational Development August 29, 2008 • The administration of FCCU was somewhat understaffed. With 12 Executive Staff and a Management team of about 29 positions (7 are currently vacant), the situation should be resolved when key vacancies are filled in the near future. We were informed that recruitment for some of these vacancies is currently underway.51 (See Appendix X for the text of the follow-up report). • Both the evaluation team and the follow up visit noted administrative, monitoring, and evaluation issues between USAID and FCCU, and that there are procedural and reporting issues that need to be resolved. There is need for a problem solving approach and better communication between USAID and FCCU. • FCCU is one of the most impressive and progressive institutions we visited during the evaluation in Pakistan. Their use of USAID funding, in spite of some lapses in bookkeeping and communication at the beginning of the program – which they have acknowledged and largely rectified – has produced positive results. • The institution is a model of the kinds of higher education change desired throughout Pakistan if the goals of the MTDF are to be realized. We found the USAID programs to be especially effective and efficient at FCCU, setting the stage for marked improvement in its programs and making substantial progress in returning FCCU to the status it once had as one of the top two universities in Pakistan. • FCCU can serve as a prototype for the new four-year curriculum advocated by the HEC. FCCU is a US style institution with a focus on active learning, problem solving, critical thinking, and individual creativity. XIV. Recommendations • Expand administrative staff involvement in USAID funded programs and fill vacant management positions. A plan for administrative expansion is spelled out in its Strategic Plan. • Complete revisions of its procurement policies in accordance with USAID requirements. FCCU should quickly respond to several outstanding USAID reporting requests to regain their confidence that FCCU understands it’s responsibilities in this area. • Continue the Merit and Need-based scholarships and faculty development program. • Upgrade library facilities. • Offer short courses to faculty members from other universities. This would require at least one teaching lab, video conferencing, and some additional staffing. • Develop procedures and be more responsive to USAID reporting requirements (most of them standard practices). 51 The follow up visit resulted in a report that sixteen searches are currently underway at FCC, several of which are for administrators including a Purchasing Officer. Evaluation of USAID Higher Education Portfolio 82 Academy for Educational Development August 29, 2008 Chapter 9: Other USAID and USG Financed Programs “The scholarship changed me – I am more motivated, am working harder, know that I can achieve success, have broader sense of power, learned to think on my toes, am more punctual.” (Scholarship student) The USAID and USG programs to be discussed in this Chapter represent additional support to Higher Education in Pakistan from a variety of USG financial sources with different degrees of management oversight from the US agencies. The programs have demonstrated varying degrees of success in obtaining USG funding and PL480 sourced endowment funds and in specifics of program performance. Unifying themes of the seven grants and endowments include institution￾building, scholarship support, scientific research support and professional exchanges between Pakistani and American academic institutions. Table 12: Other USAID & USG Projects Program Dates USG Sponsoring Agency Amount 1. Fatima Jinnah Women’s U. Dec. 2007 USAID/Washington/EGAT Bureau $200,000 Grant 2. Univ. of Agriculture, Faisalabad Oct. 2003 ongoing US Department of Agriculture $12.4 mil Endowment 3. Lincoln Corners, various U’s Feb. 2006 ongoing US Department of State $180,000 Grant 4. U. Peshawar Gender Studies November 2006 to November 2007 US Department of State $200,000 Grant 5. U. Peshawar, Human Rights J. November 2006 to November 2007 US Department of State $200,000 Grant 6. Agriculture U. Peshawar October 2005 ongoing US Department of Agriculture $2 mil Endowment 7. Forest Institute Peshawar October 2005 ongoing US Department of Agriculture $1.3 mil Endowment I. Partnership for Women in Science, Technology and Engineering (STE) The Women in Science, Technology, and Engineering (STE) program is a partnership that links two provincial government institutions of higher education for women in Pakistan; Fatima Jinnah Women University, Rawalpindi, Punjab province, and Sardar Bahadur, Khan Women’s University, Quetta, Balochistan province, with a US partner institution, California State University – Fullerton (CSUF). The aim is to increase woman’s access to academic opportunities at the undergraduate and graduate level in the science and technology fields in order to participate in scientific research in Pakistan and to enter related professions. The USAID Washington Bureau for Economic Growth, Agriculture and Trade granted $200,000 in December 2007 to support this partnership endeavor. Evaluation of USAID Higher Education Portfolio 83 Academy for Educational Development August 29, 2008 For the current program, 11 top candidates were identified by department chairpersons for application for the Master’s degree program at CSUF with 1 candidate selected; 7 candidates were recommended by department chairpersons for application for a $2500 Master’s merit scholarship and 5 were selected; and 24 candidates were recommended by the department chairpersons for a $300 undergraduate scholarship and 20 were selected. All scholars are in the science and technology area. The foreign Master’s candidate is in the field of software engineering. The funds for the scholarships are paid directly to the scholar. Fatima Jinnah administrative time related to the scholarship selection, advisement and management is in-kind. No institutional overhead fee is charged. A Site Visitation and Institutional Relevancy An on-site visit was made to Fatima Jinnah Women University and a consultation was held with the Vice-Chancellor and Registrar along with interviews with three groups of the Merit Scholars sponsored under the program. Fatima Jinnah is the first and largest women’s university in Pakistan. It was established in 1998 and began as a Master’s degree granting institution with 25 students in the science and technology area. The institution became Fatima Jinnah Women University in 2002. The institution has continued its strong emphasis in science and technology fields. Today it has 4,065 students; 1,585 undergraduates in 15 areas, 2,455 graduate students in 13 areas, and 25 PhD students in 7 areas. The university has 850 students staying in residence halls. Fatimah Jinnah has 35 faculty members with PhDs and of these, 21 are full-time and 14 are visiting faculty. The institution has accepted the HEC tenure-track system and the review cases of several faculty members have been sent to HEC. If accepted, these faculty would be the first at the institution on the tenure-track path. About half of the students are able to find employment upon graduation and the remaining graduates either go on to further education or start a family. All students are required to take three semesters of English (one additional course is optional) and a computer course, as well as complete a six-week internship with a local business or industry in addition to their science and technology courses. All classes are taught in English. The institution has 948 computers. Many of the women are offered positions with the agency with which they do their internship. Admission is through an open-merit system based on prior academic record, an entrance examination, and an interview. About one in ten applicants are admitted. About 10 percent of those admitted are classified as need-based. Based on place of birth addresses, about 60-70 percent of the women are classified as rural. On-site visits to secondary schools are made as part of the recruitment process to encourage women to apply. B Participant Interviews Twenty two of the 25 women selected for the merit scholarships were available for a structured group interview, conducted in three sub-groups according to discipline. The questions were related to future goals, internship experience, quality and relevance of course work, research interest and perceived benefit of the scholarship. Of the 22 scholars interviewed, all Evaluation of USAID Higher Education Portfolio 84 Academy for Educational Development August 29, 2008 were either juniors or first year graduate students; 6 were in software engineering, 9 in the environmental and biological sciences, and seven in the behavioral sciences. Finding: The participants had high goals for further education and were science and technology oriented in their career choices. All of the participants plan to pursue a Master’s degree and most a PhD. All were highly motivated. Several indicated an interest in applied bio-research and some an interest in their own technology/software-related business. Careers mentioned were environmental biological scientist, agriculture/organic farming, clinical psychology, child education, graphic designer/web sites, and software engineering. Finding: The participants demonstrated an excellent command of English and strong communication skills. The women demonstrated self-confidence, self-esteem, and a firm command of English. They were relaxed and communicative with strong inter-personal skills. Finding: The participants indicated that their internship experience helped them to gain confidence about their skills and to compete in the workplace. They saw their course work as very relevant to their goals. Several of the women mentioned specific courses that were relevant to their internships, especially English and computer skills. They felt quite comfortable in the workplace, and they were accepted by their colleagues. Some worked in internships outside their fields, such as a bank position, but still felt they had learned a lot, were pleased by the experience, and were able to use some of their course skills. Finding: Benefit to be derived from the scholarship process is worth the effort – successful or not. All of the women were pleased to receive the scholarship and suggested that the application process helped them to grow in and of itself, whether they received the scholarship or not. They know they can compete academically and felt it was a learning experience that will prepare them for future opportunities. There were pleased with the amount of funding even though the $300 undergraduate scholarship was small by comparison to the larger $2,500 Master’s scholarship and the Japanese scholarships. Finding: News media and the influence it can have on cultural perceptions. At the end of each interview session the team asks students if they have any questions they would like to ask the team members. Several good questions, reflecting a larger view of the world, were asked during the open question period at the end. The women were aware that fewer Americans were visiting the education fairs and that many Americans were afraid to come to Pakistan. We discussed some of the perceptions formed based on the news media and how these perceptions are influenced by what we hear and read and how the US culture was probably different from what they might expect to find based on their news reports. We also discussed some of the cultural differences between Peshawar, Islamabad and Lahore and how important it is to have an understanding of each other’s culture at the grassroots and people-to-people level. C. Effectiveness and Efficiency Evaluation of USAID Higher Education Portfolio 85 Academy for Educational Development August 29, 2008 At Fatima Jinnah Women University, 20 women will receive an indigenous undergraduate degree, 5 women will receive an indigenous Master’s degree, and 1 woman will receive a foreign Master’s degree. All will receive increased knowledge and training as the result of the program. It is expected that most, if not all, will complete their work and then take scientific-career employment or go on to further education. These are all highly competent and motivated women. The selection process was open-merit and effectively managed by the administration. All of the women felt their internship experience was excellent; each has high aspirations for the future, and all are thankful to the American people for the scholarship opportunity. The institution receives no financial reimbursement for handling the program; consequently, the program is quite cost-effective. The completion rate is expected to be at, or near, 100 percent, resulting in an efficient program. History shows that the drop-out rate at Fatima Jinnah Women University is low, about 5-7 percent. D Management Capacity and Institutional Strengthening The Fatima Jinnah senior management works closely with the Punjab provincial Education Ministry. It also accepts the HEC goals, aims and policies and is working to support the HEC programs. The management indicated that the Government of Pakistan Education Sector Reform Plans as related to the HEC helped them to move on in the direction of the HEC objectives and policies from the beginning. Since Fatima Jinnah is a relatively new institution, they did not have to re-educate the faculty and were able to move in a positive and supportive way to help transform higher education from the beginning. The management is highly motivated. It is working to set up a career placement office and a comprehensive management information system to generate timely reports in response to information requests about their students and graduates. The management is future oriented and supportive of working with donors to further improve the quality and opportunities for their graduates. II. University of Agriculture (UAF), Faisalabad The endowment agreement for $12.4 million in PL480 resources through the US Department of Agriculture was signed in September 2003.52 Rs. 650 million (about $10 million) was received by the University in 2006. An amount of Rs. 70 million is still to be received from the Ministry of Finance, Islamabad. The objectives of the endowment are to promote and support agricultural research and faculty generally, to encourage the support of the exchange of agriculture scientists and agriculture experts, support technical training leading to a Master’s or PhD for faculty members at UAF, technology transfer, and product commercialization. The endowment is managed by a Board of Directors, has a Technical Advisory Committee, and an investment Committee. The program is in its early stages. Faculty members had been asked to identify what they saw as major problem areas for agricultural research and development in Pakistan. They also held a meeting with stakeholders on December 12, 2006 to discuss these needs. In March 2007 they invited applications for faculty development from staff members (and advertised nationally). 52 USDA, “Recipient Agency Agreement between USDA and UAF on Faculty Development, Technology Transfer and Product Commercialization (FETTPC) Programme.” September 15, 2003. Evaluation of USAID Higher Education Portfolio 86 Academy for Educational Development August 29, 2008 The Committee approved 8 applications and people are in the process of finalizing arrangements for them. The second round of faculty development selection was announced in January 2008 and 21 faculty members were nominated by their faculties. The technology transfer program is also in its early stages. After national advertisement of the competition, the selection committee received 51 concept papers, invited 38 to submit proposals, and of those, selected 14 worth Rs. 35,173 million. The projects are in the faculty of agriculture, agricultural economics, veterinary sciences, animal husbandry, and water management. The endowment committee is in an early stage of competition for the product commercialization component of the project. Endowment funds were invested in three different accounts (later four) in national banks with initial income being reinvested in the endowment. The return on investments was slightly in access of 10 percent during 2006-7. The Endowment has also funded two post-doctoral activities, and a number of workshops with agricultural professionals, stakeholders (including farmers), and agro-business. While it is too early to make judgments about the results of this investment, our discussion with the Vice Chancellor, our review of projects underway, and our discussions with faculty members and administrators at UAF was very positive. One of the projects we saw at UAF was focused on citrus diseases and developing disease resistant seedling – a critical need in this citrus growing area of Pakistan. The links that existed with the University of California, Riverside and the USDA, Riverside, were substantial and it was clear that a great deal of technology transfer was taking place. An interview with the counterpart scientist in Riverside emphasized the mutual value of the project to both the US and Pakistan.53 The potential for commercialization of these and other products resulting from research and development sponsored by the endowment is tremendous. The University of Agriculture, Faisalabad, has the opportunity to make the kind of contribution we have seen from Land-grant universities in the US. The team came away very impressed with the quality of the institution and the faculty and have high expectations for success in the future from the endowment. III. Lincoln Corners The Lincoln Corner project is a joint partnership between a host institution and the US Mission in Pakistan. The aim is to provide general information about the US and to serve as a reference library with access to US books, journals, DVDs and other related materials as well as a programming and public information resource center. In February 2006, the US Department of State granted $180,000 for this purpose. There are four Lincoln Corners in Pakistan; Islamabad, Karachi, Peshawar, and Muzzaffarabad. The assessment team visited the Lincoln Corner at the Islamic International University, main campus library, in Islamabad, and followed-up on a proposed center at the University of Punjab in Lahore. The Lincoln Corner in Islamabad has over 1,600 books, 24 periodicals, 120 CDs and DVDs, and 3 public computer work stations. The Corner is managed by a senior librarian, is well-organized, and has over 1,000 visitors each month as recorded in a daily usage log. The 53 Interview with Dr. Richard F. Lee, Research Plant Pathologist, UDDA ARS national Clonal Gerplasm Repository for Citrus and Dates, Riverside California, July 17, 2008. Evaluation of USAID Higher Education Portfolio 87 Academy for Educational Development August 29, 2008 website address is: www.iiu.edu.pk/lc/. The team was impressed by the program, its scope, and its usage. IV. Department of Gender Studies/University of Peshawar The Department of Gender Studies at the University of Peshawar became functional in 2005. It was initially established in 1989 at the request of the Ministry of Women Development as one of five Gender Centers in the public universities of Pakistan, and was to become a Center of Excellence, but this failed to develop. In 2003, Peshawar University converted the Center into a Gender Studies Department. In late 2006 the US Department of State granted $200,000 for one year support to the program. In 2005, at the request of, and with the assistance of, the HEC, it became fully functional as a department with 4 permanent faculty, 7-8 part-time faculty from other departments, and 4 others assisting from outside the university. The HEC is working to establish similar gender study programs at several other universities in Pakistan; e.g., the University of Punjab in Lahore and in Quetta. The Department of Gender Studies offers three programs: Masters, Post-Graduate Diploma and Post-Graduate Certificate Courses. Three teaching modules have been developed; gender and human rights, gender and social policy, and gender and media. Because of career opportunities in the government and non-government sector, including the tribal areas, much interest (male and female) is being created, although some of the male interest is because it offers a backup program if the male cannot get admitted to their program of choice. Initially, there were 113 applicants for 25 seats in the Master’s program. The department graduated its first class of 18 Master’s students in October 2007, 13 males and 5 females. All of the students did internships as a part of their course work, and most of the graduates are employed. The department has approached the HEC and the Federal Public Service Commission to recognize the degree, which would lead to even greater job opportunities. The Department also offers a 60-hour evening certificate course on Gender and Development. One permanent staff member is on leave at the University of Washington in the US working on a PhD in General Studies sponsored by the US Government. Due to the shortage of full-time faculty, the department chair expressed an interest in being able to have faculty participate in short-term capacity building courses abroad rather than the long-term program. Nonetheless, this kind of staff capacity building is crucial to improving the quality of academic programs on women. V. Human Rights Journal/University of Peshawar While at the University of Peshawar, the evaluation team sought to assess the Human Rights Journal. In late 2006 the US Department of State granted $200,000 in support of the journal. Members inquired about the International Human Rights Perspective Journal and the Human Rights Studies Center. The Center had been established in the Law Faculty of the University in 2000. The Center offices were found to be locked with padlocks and the Journal is no longer being published. The Journal had been an annual publication and was published with USAID Evaluation of USAID Higher Education Portfolio 88 Academy for Educational Development August 29, 2008 support. We understand that the journal was closed by the University at the request of the Law Faculty. VI. Other Endowments: Endowment for Agriculture, University of Peshawar and Endowment for Pakistan Forest Institute, Peshawar Neither of the endowments for the Agriculture University of Peshawar, in an amount of $2 million, nor that for the Pakistan Forest Institute, in the amount of $1.3 million, had been allocated to the institutions as of the evaluation team’s visit on May 26, 2008. The funds to be provided to the institutions were sourced in the PL480 local currency generated through the US Department of Agriculture. Although the grants were awarded in 2005, the funds have not yet been transferred to the institutions. We visited both institutions and talked with Vice Chancellor Prof. Said Khan Khalil of the Agricultural University and the Director General Raja Muhammad Zarif of the Pakistan Forest Institute. Both have conceived projects prepared for the funds when they are dispersed. VII. Grants and Endowments Conclusions • Overall the portfolio is strong and strongly supportive of higher education goals of GoP including the HEC’s MTDF and general aims of the 2008 National Education Policy for higher education of the Ministry of Education. There is a high degree of congruence between the goals of the GoP and those of USAID nationally and internationally. • Taken collectively, the projects are well conceived, meet their intended purpose, and have had a very positive impact on improving higher education, building capacity, improving the quality of the academic programs, helping to establish links with the US, and contributing to a positive image of the US in Pakistan. • Based on the small grant of $200,000 invested by EGAT in the STE program, this program is both efficient and relevant and will have a direct impact on the future development of Pakistan. The Fatima Jinnah Women University has an environmentally attractive campus, which is supportive of women and provides an opportunity for the students to increase their knowledge while working with strong female role models. Any programming that can provide further support to this University and its programs is encouraged. It is an excellent environment in which career-oriented women can study and learn. The programs are clearly relevant, effective and efficient. • The three endowments reviewed are designed to serve a wide range of purposes, including: to encourage agricultural teaching, research and service; capacity building; technical incubators; faculty development; technology transfer; product commercialization; and cooperation between Pakistan and US scientists. One of the endowments is active; the other two institutions have not yet received their endowment funding. • Our review of the one endowment program that is in place, the University of Agriculture, Faisalabad, found an active program which is well organized, administered, and dynamically led by the Vice Chancellor. The Endowment Board has invested the funds conservatively, put some of the early earnings back into the Evaluation of USAID Higher Education Portfolio 89 Academy for Educational Development August 29, 2008 endowment and worked out a careful selection process for grants and allocation of funds. An initial selection of projects has been carried out and several projects are in their early stages at the present time with others yet to begin. The team was impressed by the scope of planning and the potential. • Neither of the endowments at the Agriculture University of Peshawar nor that for the Pakistan Forest Institute has received its endowment funding. Although the grants were awarded in 2005, the funds have not yet been transferred to the institutions. We visited both institutions and talked with Vice Chancellor Prof. Said Khan Khalil of the Agricultural University and the Director General Raja Muhammad Zarif of the Pakistan Forest Institute. Both have projects prepared for the funds when they are dispersed. Evaluation of USAID Higher Education Portfolio 90 Academy for Educational Development August 29, 2008 Chapter 10: Gender Equity “The high-need female is not applying for admission to the institution; thus, the institution needs to place a higher priority on attracting and recruiting women – all disciplines.” (Student affairs: Faisalabad). “It is necessary to have female teachers to attract the female student. They had no trust in the male teachers.” (Karachi administrator) “My friends told me that I would not get a scholarship because I was a woman – this was not true, it was an equal opportunity.” (Woman Fulbright recipient) I. Introduction In accord with the MTDP goals, a major objective of USAID is gender equity – to ensure that girls and women have equal opportunities with those of boys and men, that cultural, educational, social, political, economic and other hurdles for women be reduced and that women have the opportunity to utilize their talents to the fullest. One of the tasks set out in the Scope of Work was to: “Assess the gender balance with regard to the scholarship programs and provide recommendations for integrating more women into higher education programs/opportunities.”54 While significant progress has been made in increasing gender equity in Pakistan, in some parts of the country, there is strong cultural opposition to equality for women, some of it attributable to very active conservative Islamic communities. Even the issue of schooling for girls is a contentious one in some parts of Pakistan with four girl’s schools destroyed during the last week alone of our mission.55 The cultural context has resulted in a high rate of gender inequality in access to education and a serious disparity between opportunities for males and those available to females at all levels of the educational pyramid. The historically high gender disparity in HEIs seems to have been reduced somewhat in recent years, with female enrollment rates growing by 108 percent between 2001 and 2006 compared to 78 percent in men. However, the rates of increase of women enrollment slowed down during 2004 and 2006 as compared to the men (Table 10) and this is a trend that needs to be watched carefully. 54 Ibid. p. 5. 55 See for example the newspaper article from June 14, 2008, “One more girls’ school blown up in Dir.” Delawar Jan, The News, June 14, 2008, p. 3. An unidentified spokes person said their action was designed to “eliminate obscenity” and that anyone challenging them would be “god’s enemy….” Evaluation of USAID Higher Education Portfolio 91 Academy for Educational Development August 29, 2008 Table 13: Enrollments in HEIs, 2001-2006 by Gender Male-Female Ratio Annual Increase (percent) Male Percent Female Percent Male Female 2001-02 173,460 62.9 102,243 37.1 2002-03 202,643 61.2 128,443 38.8 16.8 25.6 2003-04 243,605 57.6 179,030 42.4 20.2 39.4 2004-05 276,409 58.6 195,555 41.4 13.5 9.2 2005-06 308,476 59.2 212,997 40.8 11.6 8.9 Total 77.8 108.3 Source: Higher Education Commission: http.//www.hec.gov/pk. June 2008. There is also considerable difference in male-female ratios by discipline. Women outnumber men in a number of disciplines, including arts and design, chemistry, dental surgery, medical and health sciences, nursing, BA/BSc, and social sciences, but are lagging significantly behind men in disciplines such as agriculture, business administration, commerce, computer science and IT, law, public administration, and science (Table 11). Special attention should be given to disciplines with low numbers of women to ensure that differences are not due to discrimination. The data indicate that, despite the gains in the last few years, women are behind in most of the professionals. Table 14: Enrollment by Discipline, Level and Gender, 2004-05 Level Gender Total Discipline Bachelor Masters M.Phil PhD PGD Male Female Total Agriculture 6,080 1,343 10 518 25 6,202 1,774 7,976 Arts & Design 2,153 475 4 22 35 875 1,814 2,689 Biology 5,442 3,689 662 456 0 3,304 6,945 10,249 Business Administration 24,265 31,710 221 273 665 44,316 12,818 57,134 Chemistry 2,341 3,433 460 261 0 3,105 3,390 6,495 Commerce 8,268 3,183 16 7 182 9,125 2,531 11,656 Computer Science & I.T 32,949 8,497 697 72 211 34,906 7,520 42,426 Dental Surgery 1,031 17 0 0 0 429 619 1,048 Engineering 36,094 4,718 133 276 166 36,644 4,743 41,387 Law 5,375 701 313 19 15 4,893 1,530 6,423 Medical & Health Science 12,076 402 118 50 682 6,089 7,239 13,328 Nursing 235 12 0 0 0 62 185 247 Pharmacy 7,815 110 181 37 0 4,517 3,626 8,143 Public Administration 952 1,599 5 12 145 1,776 937 2,713 Science 8,044 7,398 804 332 96 11,076 5,598 16,674 Social Science 77,272 52,622 2,697 1,283 2,074 52,930 83,018 135,948 Veterinary & Husbandry 1,932 333 144 185 0 2,355 239 2,594 BA/BSc Pass 89,924 0 0 0 0 44,841 45,083 89,924 Discipline wise Total 322,248 120,242 6,465 3,803 4,296 267,445 189,609 457,054 Consolidated Enrollment * 7,685 4,654 1,891 640 40 8,216 6,694 14,910 Grand Total 329,933 124,896 8,356 4,443 4,336 275,661 196,303 471,964 Source: Higher Education Commission, January 1, 2008 * Discipline breakup not available Evaluation of USAID Higher Education Portfolio 92 Academy for Educational Development August 29, 2008 During our visits to various HEIs across the country, we met many articulate and determined women who were actively pursuing their studies with a strong career perspective. This suggests that where women have equal opportunities to gain admission to higher education, they will do as well, if not better, than their male colleagues. In some universities (e.g., University of the Punjab), gender parity has been achieved, and women outnumber men in a number of departments, including those traditionally regarded as a male domain. Even traditionally “male” departments like engineering are seeing increasing numbers of women on their rolls in many institutions. In Pakistan higher education overall, the number of women enrolled are significantly lower at Master’s and PhD programs. However, between 2001 and 2006, the actual increase in numbers of woman enrolled in M.Phil and PhD programs grew faster than that of men at a rate of 153 percent for men versus 201 percent for women at the M.Phil level and 103 percent for women at the PhD level versus 54 per cent for women. Nonetheless, in absolute numbers the gap remains far too great at every level. A number of specific initiatives have been taken to encourage and support women’s access to higher education. Women’s universities are being established in Peshawar and Multan, on the lines of women universities in Rawalpindi and Quetta. The various scholarship programs offered by the HEC have also enabled many women (and men) to enroll in and/or continue their higher education though fewer women are receiving grants than men. Distance learning has also contributed to increasing access of women to higher education. In 2004-05, distance learning constituted 40 percent of the total higher education enrollments, with 187,556 students, of whom 52 percent were women. In comparison, only 35 percent of students on university campuses are women. Since Pakistani women generally face problems of mobility in a highly patriarchal society, the flexible programs offered by the Allama Iqbal Open University (AIOU), and now the Virtual University, are contributing to addressing this critical need. Nonetheless, there is concern about the high drop out rates, the relevance of courses given, and the employability of graduates. Since questions have been raised regarding the quality of the graduates and problems in getting jobs, these issues are beginning to be addressed in several ways. AIOU has developed quality assurance policies and regulations, conducts feedback surveys in order to learn about its programs and make improvements, and offers course development workshops for faculty. II. USAID Supported Institutions In summary, USAID has selected to support institutions that were already sensitized to the need for gender equity or that were progressive enough to work with USAID to increase enrollments and scholarships for women. However, the percentage of women receiving benefits from USAID funded programs is lower than it should be, and we will discuss how program design may have unintentionally induced a lower number of female scholarship recipients. While progress has been made in the Merit and Need-based scholarships, as well as in the Fulbright scholarships, a continuing effort is needed. We recognize that in some areas of the country the cultural and historical biases against education for women remain. We are also Evaluation of USAID Higher Education Portfolio 93 Academy for Educational Development August 29, 2008 convinced that once a critical mass of girls are graduating from secondary school, and a sizeable number of them go to university, there will be a major change in the willingness, indeed the eagerness, of families to send their daughters to higher education institutions. The following sections present our findings with regard to each of the major institutions that received USAID funding. A. HEC Of the 906 Merit and Need-Based scholarships awarded through the HEC program the gender break down was: • 159 (17.5 percent) females and 747 (82.5 percent) males overall • 480 (14.8 percent female) from agriculture • 426 (20.7 percent female) from business administration The design of the HEC program included requirements that may have mitigated against a healthy increase in scholarships granted to women: • Prevailing norms restrict many parents and guardians of females from allowing or encouraging their daughters to apply for admission to higher education institutions and then to seek financial assistance. At times parents/guardians do not want to share financial information with a female child; similarly the female student is reluctant to request such information. More information needs to be available to people in rural and underdeveloped areas about higher education opportunities. An administrator in Islamabad suggested that: “Females are culturally more closed about financial status – males are about 80-85 percent of the applicants.” • The scholarship program focus was on agriculture and business administration, the former which is not traditionally an area for female studies. The information gathered and the interviews conducted at the Agriculture University in Faisalabad tend to support this theory. At the Agricultural University, 90 undergraduate scholarships were awarded (12.2 percent female) and 90 graduate scholarships (5.5 percent females). Traditionally, fewer females go into the agricultural discipline, especially the plant science field, which was mentioned as a field of the scholarships, as the plant science field tends to be more field-oriented. More women tend toward the veterinary science field which was not a part of the program. About 20-25 percent of the men applying for the scholarship were being accepted while only 15-20 percent of the females applying were accepted. The reason provided was that there was a higher percent of the males attending the institution that were from poor families than females. To emphasize the point, Faisalabad institutional admissions over the same time period showed 16.2 percent females for the undergraduate program and 36.4 percent females for the graduate programs, substantially higher than the scholarship program figures. To increase the number of women applying it appears necessary to increase the range of disciplines to include those more likely to attract women. Evaluation of USAID Higher Education Portfolio 94 Academy for Educational Development August 29, 2008 It should be noted that in addition to its efforts to increase scholarships to women, the National Academy of Higher Education at the HEC has the best record of gender equity with 50 percent women in its in-service faculty development programs. While the HEC had fewer women senior employees than expected a few years ago, it has recently made progress in this area. Two women were recently promoted to director general. We encourage the HEC to increase its recruitment of women especially to senior management levels. B. HEC/MoST Science and Technology Cooperation Data on numbers of Pakistani female scientists involved in the research and exchanges was not available at the time of the team site visits. However, women were observed as research fellows, working in the laboratories, and as head of laboratory facilities. This was especially true at the Pakistan Council of Research of Water Resources, CEMB, CAMB, and NIBGE. The challenge is how to include more Pakistani female scientists in the exchanges and in project management positions. C. Fulbright Program We single out for special mention the Fulbright Program for its emphasis on gender equity and its focus on recruitment of women. At the request of the HEC, the sciences were identified as a priority field for the PhD. The US Educational Foundation was responsive to the HEC request as well as making a concerted effort to encourage more females and other disadvantaged participants to apply. The Fulbright program has made important strides. The percentage of women Fulbright scholars since the program was restarted in 2004 are indicative of what other programs can achieve: • 33 percent women in 2005 • 37 percent in 2006 • 30 percent in 2007 • 42 percent selected in 2008 The US Fulbright program has consistently improved in achieving gender equity. The HEC indigenous in-service program is also very good, but more work is still needed in recruiting and selecting females for the HEC degree programs. The quality and relevance of the training and its impact will be measurable in the long-term, especially for the PhD programs since they are longer-term, typically three to five years. The move to tenure track faculty positions with performance criteria should help with both attractive salaries and improved quality. The 2003-2007 US Fulbright program is showing 21.8 percent PhD female scholars and 34.0 percent Master’s female scholars, while the 2008 US Fulbright grantees selected is showing 38.7 percent PhD females and 43.3 percent Master’s females. The HEC foreign scholarship degree programs (PhD and Master’s combined) is showing 21.0 percent females while the indigenous degree programs are showing a combined average of 19.0 percent females. The percentage of females participating in the HEC indigenous in-service programs is 50 percent. Evaluation of USAID Higher Education Portfolio 95 Academy for Educational Development August 29, 2008 There was no evidence of gender discrimination in any aspect of the application, selection, or operation of the program. In general, participants said that the process provided an equal opportunity to all. Speaking to the question of gender equity, one student noted: “Forget gender, it was fair for all.” Another noted: “The application process was very fair, without any discrimination, and a diverse group was selected.” (Female participant). One woman Fulbrighter felt she was advantaged saying: “I was favored being a woman.” D. Forman Christian College University (FCCU) The number of women students at the university is low at 23 percent of the total with only 19 percent in the need-based scholarships going to women. This is due primarily to the fact that women were excluded from FCCU during the period of government control in spite of the fact that FCCU was among the first higher education institutions in Pakistan to admit women. Since regaining autonomy in 2004, the College has worked to increase the number of women students. The Strategic Plan for FCCU calls for a goal of “aggressive” recruitment of women students to achieve a total of 33 percent women students in the short run and 50 percent in the long run. There are 42 women on the faculty, about 20 percent of the total. A goal for the faculty is to be 33 percent women by 2010-11.56 The university is planning to build more hostels for women students. After talking with students, the team suggests that the University also needs to provide sports facilities for women who are very eager to be involved in sports but for cultural reasons cannot do so in the open. In response to out inquiry about gender discrimination, 22 of 23 students said it did not make a difference. The one who felt it did (a woman) reported that she felt there was a slight preference for women. III. Gender Equity Conclusions • The percentage of women receiving benefits from USAID funded programs is lower than it should be. While some progress has been made in both the Fulbright and Merit and Need-based Scholarships in recent years, a continuing concerted effort is needed to improve gender equity. • We recognize that in some areas of the country the cultural and historical biases against education for women remain. However, we are convinced that once a critical mass of girls are graduating from secondary school, and a sizeable number of them go to university, there will be a major change in the willingness, indeed the eagerness, of families to send their daughters to university. The current economic conditions in Pakistan make it very difficult for a mother to stay at home rather than work. Since she must work, it is essential to give her the training and skills to find good employment in a place where she too can make a contribution to development and earn a reasonable income. 56 Forman Christian College University, Strategic Planning Document: Forman Christian College (A chartered University), 2008, pp. 7-8. Evaluation of USAID Higher Education Portfolio 96 Academy for Educational Development August 29, 2008 • The National Academy of Higher Education at the HEC has the best record of gender equity with 50 percent women in its in-service faculty development programs. • The Fulbright Program deserves commendation for its emphasis on gender equity and its focus on recruitment of women. The percentage of women since the program was restarted in 2004 was 33 percent women in 2005, 37 percent in 2006, 30 percent in 2007, and 42 percent selected in 2008. • While the HEC had fewer women senior employees than expected a few years ago, it has recently made progress in this area. Two women were recently promoted to director general. IV. Recommendations • Encourage the HEC and higher education institutions to work toward gender equity in all programs and increase the number of women members of staff and on the faculty. • Create, under the HEC, pilot outreach programs at higher education institutions, designed to work with secondary schools, local communities, and parents to encourage young women to go on to higher education, take courses necessary to qualify for admission, and make them aware of the opportunities for financial support. • Under Fulbright, continue to encourage more women to apply for fellowships with a goal of 50 percent women. • Set up mentoring programs within higher education institutions for first year female students to assist them in adapting to the institution and their studies. • Recommend to the HEC that it reconsiders the age limit of 35 years for women pursuing advanced degrees with HEC support. Those who take time off to have children after completing their undergraduate program are likely to be 35 or a little older. • Broaden the areas of study under the HEC and the programs to include areas which are more attractive to women students and which provide them with employable skills and talents in what appear to be socially acceptable areas of work. These include the social sciences and sciences in addition to agriculture. Evaluation of USAID Higher Education Portfolio 97 Academy for Educational Development August 29, 2008 Chapter 11: Evaluation of the Budget of the Higher Education Commission “The coming of the HEC was a boon for the university. This university was starved for funds and totally devoid of infrastructure prior to that.” (University Vice Chancellor May 2008) I. Introduction The Scope of Work called for a review of the “…Higher Education Commission’s plans in higher education including budget levels.” AED was also asked to: “…provide insights as to how USAID’s programs can supplement/complement them.”57 The review of the higher education master plan is discussed in detail in chapter twelve (below). In this section we examine the budget of the HEC from 2002 to 2008. Not only did the Presidential Ordinance No LIII of 2002 grant the HEC a great deal of authority, it also paved the way for an unprecedented level of autonomy over its budget that allowed a level of flexibility that resulted in important efficiencies that had not existed earlier. The HEC had the freedom to move funds between line items and thus respond both to crises and deal with delays in the implementation process. Thus the HEC was able to spend almost all its recurring and development budget each year by moving down its priority list because funds from a project behind schedule were available to be used in other areas toward the end of the year. It thus spent nearly 100 percent of the funds allocated. II. Increasing the Budget for Higher Education The HEC is responsible for submitting a budget to the Federal Government each year for the recurring and development sectors of the budget (See tables 12 & 13 below). The HEC allocates funds to the public universities and to development projects (including to some qualified private universities) as well as administering funds it receives from donors such as USAID and the World Bank. Part of the success of the HEC has been its ability to obtain substantial increases in its budget each year through 200758 which has allowed it to undertake the massive transformation now underway. After years of neglect, the budget is beginning to approach a reasonable level by comparative standards. The recurring budget grew tremendously during this period from approximately Rs 3 billion ($45 million) in 2001-2002 to almost Rs 15.77 billion ($243 million) in 2007-08 (See Table 12). This budget growth was in keeping with plans launched as part of the MTDF and represented a government commitment to creating a quality system of higher education. The fact that the HEC was able to sustain these increases in the face of opposition from some quarters is a tribute to its 57 USAID, “Section C – Scope of Work Evaluation of Pakistan’s Higher Education Portfolio.” January 30 2008, p. 5. 58 It is not clear whether the projected increases for 2008 will be forthcoming. Quarterly payments to the HEC to date (July 2008) have been less than promised. Evaluation of USAID Higher Education Portfolio 98 Academy for Educational Development August 29, 2008 stewardship and the ability to focus on a wide range of critical projects ranging from salary increases for faculty members to repair of an aging infrastructure. In short order, almost everyone could see the fundamental improvements in infrastructure and in the environment for teaching, learning, research, and public service. Table 15: HEC Recurring Budget Allocations 2001-02 to 2007-08 Universities Recurring Budget Allocation (Rs. Millions) 2001-02 2002-03 2003-04 2004-05 2005-06 2006-07 2007-08* 2,946.914 3,443.393 5,304.647 6,995.608 10,493.412 14,332.521 15,766.425 Source: Higher Education Commission, 14 June 2008. *Figures for 2007-08 are estimated. Funding for the Development Budget grew from Rs 420 million ($6.5 million) in 2001-02 to about Rs 18 billion ($279 million) in 2007-08 (see Table 13). This provided critically needed improvements to the infrastructure, laboratory improvements, and advances in IT. It was to spark the revival of higher education. There were other early development initiatives which helped improve quality quickly including development of a digital library which, by 2006, provided universities access to over 25,000 journals, 40,000 e-books, and a wide range of other material.59 Table 16: HEC Development Budget 2001-2 to 2007-08 Universities Development Budget Allocation (Rs. millions) 2001-02 2002-03 2003-04 2004-05 2005-06 2006-07 2007-08* 424.000 4,265.000 4,968.000 8,940.000 10,890.000 14,406.000 18,000.000 Source: HEC, Annual Report 2005-6, p. 131; update April 18, 2007. *Figures for 2007-08 are estimated. 59 Higher Education Commission, Annual Report: 2006-2007, pp. 61-62. Evaluation of USAID Higher Education Portfolio 99 Academy for Educational Development August 29, 2008 While the recurring and development budgets grew by about 350 percent from 2001/02 to the present time, this growth came after a long period of under-funding. Pakistan’s expenditures on education still remain low by international standards with much of the increase taken up by the costs associated with higher student enrollments. As noted in the World Bank study in 2006, “…what we are witnessing is more a rehabilitation campaign to restore the capacity of universities to function normally.”60 Table 17: Trends in Higher Education Per- Student Spending, 2001/02 - 2007/08 Fiscal year Spending Indicator 2001/02 2002/03 2003/04 2004/05 2005/06 2006/07 2007/08* Enrollments Enrollments (#) 135,123 159,103 183,708 208,756 231,601 263,414 316,278 Total Higher Education Spending (Recurrent + Development) Real Total Higher Education Spending (Rs Millions) 3,904 8,132 10,021 14,236 17,388 20,182 20,696 Real Per-student Total Higher Education Spending (Rs) 29,524 52,132 55,544 69,766 76,682 76,616 65,438 Higher Education Recurrent Spending Real Higher Education Recurrent Spending (Rs Millions) 3,480 3,996 5,413 6,647 8,431 10,102 9,664 Real Per-student Higher Education Recurrent Spending (Rs) 26,318 25,614 30,002 32,577 37,183 38,351 30,555 Higher Education Development Grants Real Higher Education Development Grants (Rs Millions) 424 4,137 4,608 7,589 8,957 10,080 11,033 Real Per-student Higher Education Development Grants (Rs) 3,206 26,519 25,542 37,189 39,499 38,265 34,883 Source: World Bank, Table 4, p.15 updated by the HEC, June 13, 2008. 60 Ibid. Policy Note, p. 11. Evaluation of USAID Higher Education Portfolio 100 Academy for Educational Development August 29, 2008 Table 18 : Public Higher Education Institution Revenues, 2001/02 - 2007/08 Fiscal year Spending Indicator 2001/02 2002/03 2003/04 2004/05 2005/06 2006/07 2007/08* Total Revenues 6,652 7,731 10,800 13,394 18,644 21,261 23,430 • HEI Government Revenues (Rs Millions) 3,513 4,104 6,284 7,948 11,716 12,378 13,681 • HEI Non-government Revenues (Rs Millions) 3,140 3,627 4,515 5,446 6,928 8,883 9,749 Nominal Revenue per Student (Rs) 50,309 49,557 59,857 65,641 82,219 80,712 74,080 Real Revenue per Student (Rs 2001/02) 50,309 48,066 55,521 55,718 65,749 56,892 45,406 Government Revenues/Total Revenues (%) 53% 53% 58% 59% 63% 58% 58% Non-government/Total Revenues (%) 47% 47% 42% 41% 37% 42% 42% Source: World Bank Table 13, p. 23. Updated by the HEC, June 13, 2008. Evaluation of USAID Higher Education Portfolio 101 Academy for Educational Development August 29, 2008 Table 19 : Spending by Public Higher Education Institutions, 2001/02-2007/08 Fiscal Year Spending indicator 2001/02 2002/03 2003/04 2004/05 2005/06 2006/07 2007/08* Spending by public HEIs Total public HEI spending (Rs Millions) 6,456 7,431 10,233 12,652 18,365 20,531 25,563 Nominal public HEI spending per student (Rs) 48,823 47,637 56,716 62,004 81,802 77,941 80,824 Public HEI spending vs. Non-salary spending Salary spending (Rs Millions) ** 3,531 3,930 5,525 6,561 8,984 10,153 12,718 Salary spending/Spending% 55% 53% 54% 52% 49% 49% 50% Source: World Bank Table 14, p. 25 updated by the HEC, June 13, 2008. * Figures for 2007-08 are estimated. ** Figures of Salary Spending for the years 2004-05 and 2005-06 have been updated. For conversion of Nominal into Real number, the CPI index of State Bank of Pakistan as published in its Annual Report for FY 07 has been used. For current financial year 2007-08 (FY 08), the inflation rate of 15% has been assumed. Evaluation of USAID Higher Education Portfolio 102 Academy for Educational Development August 29, 2008 The effects of the increases in student numbers is demonstrated by figures for the real increase in per student spending on the higher education recurrent budget, which, while the overall budget increased almost 350 percent, grew by only 45 percent from 2001-2 to 2006-7 (Table 14). As for the development budget, as can be seen in Table 14, while the overall budget increased tremendously during this period, most of the increase was in the first year (2001-2 to 2002-3) where it grew 7.3 times its earlier total. Thereafter, the per student increase over the next 5 years was a more modest 44 percent. As can be seen from Table 15, Public Higher Education institutional revenues also grew substantially, but government revenues as a percent of the total actually declined in 2006-7 (from 63 percent of the total to 42 percent) as did real revenue per student. Thus, while the budget for higher education has grown significantly over the last few years, much of this increase has been needed to cover the cost of increasing access. The increases in enrollments, while significant in a context in which Pakistan is well behind the rest of the world, at about 3.9 percent of university age students (compared to at least 8 percent for India), demonstrates that much more funding is required if Pakistan is to achieve both the increased access and the international standards it has set for itself. Its spending as a ratio of public spending on tertiary education to GDP is considerably less than many of its Asian neighbors such as Thailand, India, or Malaysia, though on a par with the Philippines and slightly higher than Indonesia.61 An indicator of the positive effects of budget increases on the ability of universities and DAIs to improve their infrastructure, provide better support for teaching, and increase funding for research can be seen in the budget data on salary expenditures for public education. Bearing in mind that faculty salaries have increased substantially in the last few years, we would expect the percentage of the budget devoted to salaries to have increased substantially. It has not. Salaries as a percent of total spending have dropped from 55 percent in 2001-2 to 49 percent in 2006-7 (Table 16). This is because the total funding made available by government has grown faster than the cost of the salary increases. That should help institutions improve the quality of their programs and is a very positive result of increased government funding for higher education. The contribution of non-government revenue to the total cost of higher education is substantial at 42 percent of the total cost in 2006-2007, up from 37 percent the year before and 41 percent in 2005-6. The contributions of the institutions themselves to the total cost is illustrated by the fact that in the Punjab, for example, the cost to public universities and DAIs of salaries alone, is more than the total government recurrent grants to these institutions. The government recurring grants total Rs. 3,097,985 million versus an expenditure on salaries of Rs. 5,049,134 (See Appendix XII). Looking at the overall government expenditures on education, we see a decline over the last few years as a percentage of GDP. In 1995, prior to the creation of the HEC, it was 2.5 percent of GDP, by 2000 it was 2.06 percent of GDP, in 2002 it was only 1.51 percent, and in 2007 it had recovered somewhat to was 1.93 percent. The percentage of the education budget going to higher education has varied over the years being 18.5 percent in 1985, 10.4 percent in 2002; and 15.6 percent in 2008. The critical point to be made here is that although expenditures for tertiary 61 The rates respectively are: Pakistan .4; Malaysia 2.7; India .7; Indonesia .3; and Philippines .4. See Ibid. Policy Note, p. 14. Evaluation of USAID Higher Education Portfolio 103 Academy for Educational Development August 29, 2008 education have increased substantially, they are still below those of countries that are making progress in national development, and far below what is needed to bring higher education in Pakistan up to the quality standards required for major national development. The evaluation team heard complaints about excessive funding going to higher education. We did not see any evidence of that. The HEC itself is understaffed and lives on a modest budget. Funds are well managed overall, with some slippage when the HEC changed accounting systems – problems which have since been corrected.62 The major development expenditures have gone to meeting long overdue infrastructure and other equipment expenses needed just to keep up. Major expenditures have also been made for critical salary increases, research funding, faculty development, and other essential areas. For the first time in many decades, there is progress to report in higher education. In this context, continued growth in government funding is essential to sustain the development of higher education. III. Monitoring and Budget Controls Budget controls and monitoring were identified as a problem area in early 2008 as noted in the Control Environment & Risk Assessment. 63 The report identified weakness in accounting system controls, excessive staff turnovers, unfilled vacancies, missing documentation, low salaries, and shared responsibilities with other non-USAID programs. We did not review the accounting control system but we did discuss the issues with the Chairman and Executive Director in detail as well as with staff during the course of our evaluation. These issues were also discussed in the follow up visit by a staff member. A new accounting system is in place and should help with budget control and monitoring. The staff salary problems are a function of the civil service structure and out of control of the HEC, although they use the appointment of consultants as a way to pay appropriate salaries. The HEC is also working on changes in the salary structure. The vacancies are a product of a hiring freeze imposed on the HEC by government. We hope that will be short-lived. Senior HEC management has made a concerted effort to respond to all recommendations from the risk assessment and a USAID report on July 22, 2008 indicates they have succeeded.64 Overall, we are impressed by the level of review and control over both the recurring and the development budgets at senior management levels and think the new budget control system will resolve the problems noted. Senior management exercises tight on-going control of the HEC budgets, is aware of its financial status on a day to day basis, and reviews income and expenditures regularly. 62 USAID, Status on Control Environment & Risk Assessment (CERA) – Higher Education Commission, July 22, 2008. A report that notes that previously identified weaknesses have been corrected or resolved and the HEC meets required standards or is in the process of meeting them. 63 Controller, USAID, “Control Environment & Risk Assessment – Higher Education Commission,” February 4, 2008. 64 USAID, “Status on Control Environment & Risk Assessment (CERA) – Higher Education Commission,” July 22, 2008. A report that notes that previously identified weaknesses have been corrected or resolved and the HEC meets required standards or is in the process of meeting them. Evaluation of USAID Higher Education Portfolio 104 Academy for Educational Development August 29, 2008 IV. Conclusions One of the major achievements of the HEC during the period 2002 to 2008 has been to increase the budget for higher education to adequate levels for the first time in decades. This has required extensive planning, careful budget preparation, and most importantly, an effective job of making the case for higher education funding. In a context in which there are many claimants on funds – primary and secondary education, roads, healthcare, poverty, water, agricultural development, defense, and many others – all with good cases for increased funding, it is a tribute to the senior management of the HEC that it was so successful. HEC management had strong support during this period from the Ministry of Finance and the Prime Minister’s office. That would not have happened if the leadership of the HEC had not done an outstanding job of making the case for assigning a high priority to expansion of both the recurring and the development budgets for higher education. We want to emphasize what was noted in the World Bank Policy Note of 2006 that much of the budget increase merely brought the budget up to where it should have been all along; that it was more “a rehabilitation campaign” which brought the universities up to the level needed for normal operations.65 Since that time the budget has continued to increase and this is beginning to move higher education in the direction of international standards to which the HEC aspires. Nonetheless, there is a great deal more to be done to reach those standards. We note that no university in Pakistan is in the 2008 ranking of the top 500 universities in the world. Transforming higher education must be an ongoing process. There is no such thing as a one-time fix. In addition, as we have shown above, much of the growth in the budget is absorbed in the costs of increased enrollments – an increase of 72 percent since 2002. The important achievement of higher education at this time is that these enrollment increases have come in tandem with an increase in quality. That includes better trained faculty members (with many more to come from the Fulbright and other programs), improved laboratories, access to a digital library, improved research, enhanced teaching tools, and better teaching and learning. All these improvements (the basic equipment of higher education) will need to be kept in working condition and up-to-date. And as enrollment grows, more will be needed. In addition, the return of large numbers of PhDs from both universities in Pakistan and abroad, will mean a larger percentage of faculty with higher salaries (since they will be on tenure track), a need for more teaching equipment, laboratories, and other research support for them, and thus increased costs. It is important to recognize that all this is a normal expectation for any higher education system that has aspirations of contributing to national development. The demands on the budget of the government of Pakistan are enormous and the needs of many other programs also critical. Nonetheless, if Pakistan is to meet its development goals, the place to begin the effort is with higher education. High-quality higher education has other essential broad benefits for the nation including its ability to make major contributions to teacher training (see our recommendation in on that in chapter 12), in agriculture, health care, business, industry, clean water, construction, and a host of other areas. As a mater of priorities, the GoP has been wise to place higher education toward the top of its budget priority list. It will be important to 65 Ibid., Policy Note, p. 11. Evaluation of USAID Higher Education Portfolio 105 Academy for Educational Development August 29, 2008 continue to do so to achieve success in developing a high quality higher education system essential to national development. There are major clouds on the horizon for the HEC and its budget. Funding levels have started to decline below anticipated increases, though the total allocated was increasing at the time of our first visits to the HEC in April-June 2008. However, as the team left Pakistan in June 2008, there was a delay in payment of the quarterly funding to the HEC, and soon thereafter the recurring budget was reduced by Rs. 3.23 billion while the development grant for this year was reduced by Rs. 2.44 billion. This could have a negative effect on the progress being made if it is not reversed. V. Recommendations Adequate funding is the critical base on which successful transformation of higher education takes place. To date, the HEC has been impressive in both managing the budget processes for higher education and in overseeing and monitoring the implementation of the MTDF. The changes that have taken place cover all areas of higher education from increasing the quality of faculty members to software for monitoring expenditures. This task has not been without its detractors. Nonetheless, to date, the HEC has managed these efforts well and succeeded in getting the funding needed to implement the long term plans for higher education. In that context we make the following recommendations. • That USAID continue to provide support for the HEC. At the present time, USAID program are complementary to the goals of the MTDF and the goals of the GoP. • That HEC consider supporting a number of new programs recommended in chapter 12 (below). Evaluation of USAID Higher Education Portfolio 106 Academy for Educational Development August 29, 2008 Chapter 12: Evaluation of Higher Education Commission and its Implementation of the Medium Term Development Framework (MTDF) and other Government of Pakistan Higher Education Goals “This is the first time in the history of the country that we have seen a picture of the way a university should be. We are getting everything a 21st century university should have.” University Vice Chancellor, April 2008 I. Introduction The Scope of Work called for the team to “Review the Higher Education Commission’s plans in higher education including budget levels and provide insights as to how USAID’s programs can supplement/complement them.” In this chapter we review the Higher Education Commission’s plans for higher education and make an overall assessment of its achievement and challenges. This includes its oversight of higher education, implementation of the MTDF and other goals of the GoP. How has the HEC done since its inception? What can we say about its stewardship of higher education transformation? How does the transformation of higher education in Pakistan today compare with international expectations about high quality tertiary education systems? What is our overall assessment of the HEC? The goals for higher education are spelled out in the Medium Term Development Framework 2005-10 published in 2004 and reiterated and amplified in the Draft National Education Policy 2008, which adds goals for the future including: developing civic responsibility, tolerance, and human centered development.66 The team reviewed the work of the HEC in terms of each of the major HEC goals including: faculty development; improving access; promoting excellence in learning and research; quality improvement and accreditation; relevance; leadership, governance, and management; industry/business linkages, and encouraging quality private tertiary education. How successful has the HEC been in realizing the major goals of the MTDF? What are the major challenges? What have been the most striking successes? II. Faculty Development The HEC put faculty development at higher education institutions (HEIs) at the center of its national agenda and the MTDF with the goal being to improve the quality of teaching and research through targeted programs to enhance the pedagogical skills and qualifications of faculty members. This was aimed at increasing the number of PhD faculty from the distressingly low level of 23 percent. The key initiatives have focused on Master’s and PhD training in Pakistan and abroad for faculty and support for post-doctoral fellowships and research. Supportive programs have included a foreign-faculty hiring program, efforts to attract Pakistani academics working abroad to return to Pakistan, in-service pedagogical training, support for a tenure track system coupled with salary increases, a twinning program for collaborative research with partner institutions inside and outside of the country, one of the largest digital libraries in 66 Ibid, National Education Policy 2008, pp. 2; 21-52; chapter 9. Evaluation of USAID Higher Education Portfolio 107 Academy for Educational Development August 29, 2008 Asia, and the Pakistan Education and Research Network. Our evaluation of these programs over the past five years (2003-2008) is presented below. The data presented here are based on the HEC 2006-2007 Annual Report and updated with 2007-2008 data provided by the Finance Office, the Office of the Executive Director, and the HEC/HRD Office in April-June 2008, as well as interviews with staff. We include gender data when available. A. Indigenous Faculty Development Programs The indigenous PhD and Master’s scholarship programs are designed for individuals working in the public and private sector, research and development organizations, and in less developed or newly established universities. This has been a major initiative. During the past five years, 3,820 scholars have been active in the indigenous PhD programs and 82 have completed; 412 scholars have been active in the indigenous Master’s programs and 178 have completed their studies (See Appendix XIII for detail on each of the programs). During the past four years, 8,348 HEI faculty members participated in the HEC sponsored National Academy of Higher Education (NAHE) faculty development in-service training program to improve pedagogical, administrative/planning, communication, research/measurement, curriculum/material development, and English language skills. The in￾service program included both short and long-term courses with the percentage of female participants averaging 50 percent; the percentage of females participating in the long-term courses has actually exceeded 50 percent during the past three years. B. Foreign Faculty Development Programs The HEC envisions a PhD foreign scholarship program of 2,000 scholars in all disciplines weighted toward the science and technology areas; 608 scholars have been active in the foreign PhD programs and 118 have completed their work; and 964 scholars have been active in the foreign Master’s programs and 79 have completed. Currently, some 14 countries participate in this program. A survey of second year PhD scholars showed 220 research papers published in conference proceedings and journals and 179 conferences attended. During 2004 and 2005, 172 teacher educators (36 percent females) participated in one semester US training programs at George Mason University, University of Montana, and Oregon State University. The focus was on teacher training methodologies in administration, mathematics, science, and English as a Second language (ESL). C. Foreign Faculty Hiring and Visitors Program The objective of these programs was to hire expatriate Pakistani faculty members for a specified duration to ease the critical shortage of qualified faculty in public sector institutions of higher learning; 408 foreign faculty members have availed themselves of this program. Some 80 percent were in the science and technology area with 20 percent in the social science and business education area. Evaluation of USAID Higher Education Portfolio 108 Academy for Educational Development August 29, 2008 The visiting scholar applicant may be a foreign national or an expatriate Pakistani who is permanently settled abroad and maintains a position in a parent institution abroad. Since it began in 2003, 68 scholars have participated in this program. Remuneration and benefits include a basic salary, housing rent, and utility allowance components. Incentives for bringing indigenous Pakistanis, who have PhDs and are working abroad, back to Pakistan to teach and carry out research is important to the development of the country. In the past five years, 28 such Pakistani PhDs have returned full-time to Pakistan. D. Tenure Track System The introduction of a tenure track system by the HEC was a major effort to improve quality, insure accountability of faculty members, and increase salaries to competitive levels. The tenure track system was a center piece of the quality improvement process of the HEC. Unlike its US model, it was not driven by concerns about academic freedom (though they existed), but by a desire to hold faculty members accountable in a context in which there was little that could be done to punish those who missed classes, failed to keep up in their fields, or did no research. The new system required regular formal review of faculty achievements in teaching, research, and service, and held them accountable for their productivity and its quality. The benefits of participating in the tenure track system included salaries two to three times higher than existing university (civil service) salaries. Initial resistance to the tenure track system by current faculty members was substantial, in part because accepting the conditions came with the risk of losing one’s job if unsuccessful in a tenure review even if they were currently associate or full professors. There would also be a loss of benefits to older faculty members (retirement and health insurance) though the high salaries would more than make up for the latter. Resistance by older faculty members was understandable. For most of them, the possibility of substantially higher salaries was not worth the risk. That obstacle was soon eliminated when the rules were changed so that failure to obtain tenure following a review, allowed those faculty members to keep their jobs, civil service status, and salaries. The disincentive for existing faculty members about the loss of pensions and medical insurance was also reversed. By 2007 these benefits were made available to people who opted for the new tenure track system as well as the old – thus removing the last major obstacle for participation by high quality faculty members who had been employed for many years. These changes reflected the willingness of the HEC to listen to its critics and resulted in acceptance of the system by almost all the public universities in Pakistan. E. Quality, Relevance and Impact The impact of the tenure track system has been substantial, now that the conditions have been modified to remove the disincentives for current faculty members, and its contribution to quality is already being felt. The new conditions are attracting many Pakistanis to return home who would otherwise have sought jobs in the US, Europe, the Middle East, or in high quality institutions in Asia. It is now possible to live comfortably on an academic salary without a second job and working conditions (including equipment and research support) have also improved. That is helping attract people who are improving both the quality of teaching and Evaluation of USAID Higher Education Portfolio 109 Academy for Educational Development August 29, 2008 research and in doing so, the relevance of academic offerings. More than 90 per cent of new hires are opting for the tenure track system – and in many cases 100 percent (as at the University of Agriculture, Faisalabad) are doing so. The high quality and productive senior faculty members at most universities are now be nominated for consideration for tenure and many are achieving that status. There is still a great deal of pressure from some quarters to “grandfather” older faculty members into the system even when they are not qualified. As one faculty member put it: “Aren’t fifteen years of teaching the equivalent of a dozen publications by some younger man?” So far, the answer has been a very resounding “No!” We applaud that firmness and commitment to standards. The HEC indigenous in-service program is very good, though faculty members reported that some of the presenters were not up to expectations. More work is still needed in recruiting and selecting females for the HEC degree programs. We judge the progress to be successful to date, though the quality and relevance of the training and its impact will be measurable only in the long-term, especially for the PhD programs since they are longer-term, typically three to five years. The move to tenure track faculty positions with performance criteria should help with both recruitment and retention with competitive attractive salaries now in place and an improved working environment. The most immediate result of the faculty development effort will be an improvement in the quality of teaching and research. We are already seeing evidence of that in our interviews with faculty and students. Faculty participation in research and professional meetings has also increased with the availability of increased numbers of travel grants for conference participation. Publications have increased significantly in the past few years and are up 30% from 2006-2007 in impact factor journals.67 There has also been an increase in university research related patents. The data indicate an increase in each of these latter areas - travel grants, conferences funded, articles, and patents. The HEC is still in the early stages of its efforts, but indications are that the quality of the HEC’s investment in human resource development is showing results. The illustrative indicators of quality are all showing increasing trends. Socio-economic relevancy and impact should result from the increased research along with new products and services, although this will take time. In addition, placement has been excellent for those completing their programs. The HEC is assisting in this effort. To date 38 foreign PhD scholars have returned and been placed. This is in addition to 82 indigenous PhDs. To further ensure relevancy and impact, the indigenous PhD programs should include an internship/exchange with the industrial sector, research park, business incubator institute and/or some entrepreneurial course work, or capped with 3-6 months of foreign post-doctoral work. Development of faculty who are able to contribute to the development of Pakistan is the important key. International institutional collaborations will also play an important role in furthering faculty development and needs to be strengthened and expanded. The indigenous and foreign PhD programs have been predominately in the Science and Technology areas. More PhD and Master’s scholars are needed in the Social Sciences, Arts and Humanities, and Business Education areas. 67 HEC, Annual Report, 2006-07, p. ix. Evaluation of USAID Higher Education Portfolio 110 Academy for Educational Development August 29, 2008 F. Conclusions • The tenure track system is increasingly successful and the HEC has succeeded in resisting efforts to lower standards. There are indications that it is already having a positive effect on teaching, research, and public service. There is no question that it is proving a successful mechanism to hold faculty members accountable. • There has been a strong focus on human resource development – faculty development and technology training (especially, PhDs); now adding a focus on leadership/management, linkages (international and national – R & D and industry) and relevancy to economy will be important. • Higher education enrollments are increasing: thus, continued emphasis on high quality faculty development, quality assurance (e.g., tenure track system), and accreditation, is needed. • There has been excellent participation and support of HEC programs by foreign governments. • The percent of females participating in the HEC in-service programs is excellent. • Both the indigenous and foreign graduate programs have been highly successful in recruiting scholars. • Effective measures of impact are in place as well as support to encourage scholarly activity including travel grants, conferences funded, articles published, and placement of returning PhD scholars. • Indigenous Pakistanis are starting to return to Pakistan, via the reverse brain drain program, visiting scholar’s program, and foreign faculty hiring programs; this is a sign of success. Incentives for participation are good. • An effective career placement program for returning graduates is in place. Graduates are ensured a job upon returning and a number of them availed themselves of this service successfully. We are not aware of any returned scholars (sponsored or not) who were unemployed if they sought assistance. • There is a need for more emphasis on the humanities and social sciences in PhD programs. The early emphasis was in the science and technology areas and was understandable given the needs in those areas. • The effort to establish a series of Centers of Excellence at various high quality universities should be encouraged. Creating communities of scholars in priority disciplines and individual high quality universities is important to major improvement in both teaching and research. • Problems associated with obtaining US student visas are increasing and hindering the effective delivery of many programs. Some students obtained visas after classes began in spite of timely requests. This is a major deterrent to academic cooperation and is damaging to the US image in Pakistan. • It is essential to foster greater collaboration between foreign and Pakistani Universities if Pakistan is to create institutions meeting international standards. Solutions need to be found for a variety of issues complicating such exchanges and linkages including: difficulty with insurance (health, liability and life) and security for foreign resource people who come to Pakistan. Many excellent institutional links currently exist between Pakistan and US universities. It is important to build on that Evaluation of USAID Higher Education Portfolio 111 Academy for Educational Development August 29, 2008 base to expand the number of linkages and networks, requiring rethinking of these programs, facilitation by the US government, a commitment by the programs, and close cooperation with the Pakistan government. G. Recommendations • Continue to insist on high standards for tenure and regular review of faculty member’s performance. • Encourage more international exchanges and collaboration, including training programs. Provision of more travel grants would be useful. • Indigenous PhD programs should also include an opportunity to study or carry out research for six months abroad. Programs split between US and Pakistani institutions are one possibility. Continue to emphasize human resource development • Some indigenous PhD programs should include an internship or exchange with the industrial sector, technology research park, business incubator institute or require some entrepreneurial course work. Faculty need more awareness of, and experience with the commercial and industrial world. • The experience with short-term foreign training for students prior to graduate study has been very successful, both in preparing students for graduate studies at a later date and for improving relationships between the US and Pakistan. Expanding those opportunities would be especially useful at this juncture when the ties need to be strengthened. • While the in-service faculty development programs have worked well in many areas, there is a need for more content-specific training and a focus on pedagogy and research techniques related to teaching specific content rather than programs on generic teaching issues. III. Improving Access to Higher Education Increased access to higher education is a critical issue in Pakistan and a major goal of the MTDF. More and more of Pakistan’s young people seek a university education while increasing numbers are unable to obtain places and thus fail to get the training they need for employment while the country finds itself short of the skilled graduates it needs. At the same time employers are complaining about the lack of properly trained candidates for employment and about the relevance of their education. The lack of trained specialists has been a major factor in the failure to develop the economy at the desired rate. The literature on access to education, including tertiary education, identifies a number of problems arising from unequal access for poor youth, including further increases in poverty, the dangers of growing numbers of unemployed or underemployed, possible recruitment of these people as terrorists, or the creation of conditions that lead to chronic instability, conflict, and internal war.68 Pakistan is a classic example of such a phenomenon. Years of neglect of the 68 See, for example, World Bank. Constructing Knowledge Societies: New Challenges for Tertiary Education. 2002, p.19-20 Evaluation of USAID Higher Education Portfolio 112 Academy for Educational Development August 29, 2008 education sector have contributed to increasing unemployment and underemployment, rapid growth in crime, sectarian violence, and terrorism. Some progress has been made in the recent past, with the percentage of college-age students in higher education increasing from 2.9 percent in 2002 to 3.8 percent in 2005-06.69 Despite these gains, the percentage remains among the lowest in the world, and considerably below some of its main competitors such as India with 11 percent, 20 percent in China, 24 percent in Iran, and 32 percent in Malaysia.70 The percentage drops even further for women and for those from rural and disadvantaged groups. Of the students enrolled in Pakistani HEIs in 2005-06, only 41 percent were women.71 The efforts of the HEC to increase enrollment rates are essential for social justice and to tap the talents of all members of society. Data on access show that enrolment rates in public and private HEIs have increased by 89 percent between 2001 and 2006 (See Table 17). The HEC is more than meeting its target of 10 percent growth rate per year of 14.7 percent. Table 20: Enrollment in HEIs (Public and Private), 2001-2006 Enrollments in HEIs, 2001-06 0 100,000 200,000 300,000 400,000 500,000 600,000 2001-02 2002-03 2003-04 2004-05 2005-06 Source: Higher Education Commission, June 2008 The general universities72 have the largest intake with 70 percent of the total enrollment. However, the percentage increase in enrollments has been highest in the medical universities and degree awarding institutes (Table 18). This high growth in enrollments has been achieved through a mix of strategies, including establishment of new HEIs (increased from 44 in 1997 to 122 in 2008), expansion of existing facilities, and setting up new campuses (including classrooms, hostels, labs, libraries, sports facilities, etc.), recruitment of more and better qualified 69 World Bank, Higher Education Policy Note, 2006, p.67 70 Global Education Digest, 2007. Comparing Education Statistics across the World. UNESCO Institute for Statistics, 2007. 71 Higher Education Commission 72 General universities include those that offer general and pure science programs, as opposed to universities offering specialized professional programs such as engineering, medicine and agriculture. Evaluation of USAID Higher Education Portfolio 113 Academy for Educational Development August 29, 2008 faculty, and a variety of scholarship programs. These are discussed in greater detail in the other sections of this report. Table 21: Enrollments by Types of HEIs, 2003-08 2003-04 2004-05 2005-06 2006-07 2007-08 Percent increase General Universities 134,564 149,968 164,150 188,904 220,997 64% Agriculture Universities 14,682 16,793 21,760 24,682 27,881 90% Engineering Universities 20,028 24,762 24,056 26,497 30,766 54% Medical Universities 4,337 4,214 6,049 5,956 11,924 175% Degree Awarding Institutes 6,811 8,770 10,502 11,805 18,588 173% Other Institutes 2,080 2,671 3,377 3,633 4,107 97% Centers 1,206 1,578 1,707 1,937 2,015 67% Total 183,708 208,756 231,601 263,414 316,278 72% Percent increase in enrollments 13.6% 10.9% 13.7% 20.1% Source: Higher Education Commission, June 2008 A. Access and Quality One of the challenges for higher education in an earlier period was that increased access was not accompanied by an increase in institutional capacity. Thus, for example a single laboratory space was often shared by several students simultaneously. Although class size increased, there was not a proportionate increase in infrastructure or materials with a growing concern among educators in Pakistan that increased access was contributing to the lowering of standards. To quote from the Report of the Task Force On Improvement of Education in Pakistan: “The issue of raising the quality of education has been highlighted in each of the Five-Year Plans. The authors of the Fourth Plan (1970-1975) noted that it was necessary not only to spend more money on education, training and research but also to spend it effectively. Although this need was expressed in each plan, the lack of adherence to the stated discipline was not questioned. Sadly, whatever funds were available to the Education Sector were used for quantitative expansion and not for qualitative improvement. Consequently, standards of education deteriorated.”73 As a result, students did not receive the kind of training needed to compete in a global market, and the job market in Pakistan was flooded with unemployable 73 Report of the Task Force on the Improvement of Higher Education in Pakistan. March, 2002 Evaluation of USAID Higher Education Portfolio 114 Academy for Educational Development August 29, 2008 graduates. Well aware of this problem, the HEC has linked increased access to quality improvements. An overall assessment of success in that regard would be premature at this stage, but it is clear that the initial impact has been positive both in terms of increasing access and improving quality. Some proxy indicators of success are encouraging. For example, despite massive increase in enrollments, the teacher-student ratio has actually dropped from 1:21 in 2003-04 to 1:19 in 2008. Table 22: Changes in Faculty, Proportion of PhD Faculty, and Teacher-Student Ratios, 2003-2008 2003-04 2004-05 2005-06 2006-07 2007-08 Enrollment 183,708 208,756 23,1601 263,414 316,278 Percent increase in enrollments 13.6 10.9 13.7 20.1 Total Faculty 8814 9804 11,034 13,744 16,727 Percent increase in faculty 11.2 12.5 24.6 21.7 PhD Faculty 2579 2482 2241 3260 3645 Percent increase in PhD faculty (3.8%) (9.7%) 45.5% 11.8% Percent of PhD Faculty 29.3 25.3 20.3 23.7 21.8 Teacher-Student Ratio 1:21 1:21 1:20 1:19 1:19 Source: Higher Education Commission, May 2008. The actual number of PhDs declined somewhat from 2003-2004 to 2005-2006, though it began to increase again in 2006-07. The reduction in the percent of faculty members with PhDs partly reflects this decline and in part the fact that it is easier to increase the number of students each year than the number of faculty with PhDs given the fact that it takes at least four years to complete at PhD. Another proxy indicator of educational quality is the number of students completing degrees beyond the Master’s. The data indicates significant improvement in this area, with a 16 percent increase in M.Phil outputs and a 65 percent increase in PhDs between 2001 and 2006. The number of women is significantly lower than men in Master’s and PhD programs, and thus a major effort should be made to increase the percentage of women enrolled. The actual increase in numbers of woman enrolled in M.Phil and PhD programs grew faster than that of men between 2001 and 2006 at a rate of 153 percent for men versus 201 percent for women at the M.Phil level and 103 percent for women at the PhD level versus 54 per cent for women. Nonetheless, in absolute numbers the gap remains far too great at every level (Table 20). Evaluation of USAID Higher Education Portfolio 115 Academy for Educational Development August 29, 2008 Table 23: Enrollments in HEIs, 2001-2006 by Level and Gender Year Bachelor Masters M.Phil PhD PGD M F M F M F M F M F 2001-02 116,723 69,112 48,315 30,289 2,556 1,315 2,435 689 3,431 838 2002-03 136,547 86,923 56,826 37,859 3,150 1,692 3,122 1,016 2,998 953 2003-04 166,022 128,880 65,021 43,855 4,708 2,746 4,662 1,810 3,192 1,739 2004-05 190,996 138,937 73,167 51,729 6,016 2,340 3,596 847 2,634 1,702 2005-06 210,293 149,390 84,445 56,472 6,476 3,954 3,741 1,398 3,521 1,783 Source: Higher Education Commission, June 2008 B. Access and Disadvantaged Groups Access to higher education in Pakistan has been especially difficult for the most vulnerable groups, including women, the poor, rural areas, and minority groups. Most public sector universities provide some scholarship support to ensure that particularly meritorious students are not deprived of higher education due to financial constraints. Nonetheless, the funding barrier remains especially acute for students in disadvantaged groups and those in rural and disadvantaged parts of the country. In some HEIs, quotas for students from rural areas are in place. The HEC has also encouraged universities to open campuses in less developed areas. This strategy has started to have results with the establishment of 13 new sub-campuses of 9 universities in less developed areas. Nonetheless, the percent of university students from disadvantaged backgrounds remains far below their percentage of the population and these inequalities remain sources of opposition, hostility and instability. C. Conclusions The HEC to date has been successful in increasing access to higher education. Its initial steps to improve quality at the same time are promising ranging from improvements in the quality of the infrastructure to increasing the percentage of PhD faculty. These changes must be ongoing and in the end will depend on the willingness of government to continue to increase funding levels as it has over the last five years so that standards are not compromised in order to increase the numbers. The investments made by the HEC during the last five years are contributing to meeting the MTDF goal of increasing access for students, including women and disadvantaged students, to higher education institutions. The challenge ahead is to continue to upgrade institutions and maintain standards, particularly in newer campuses and those set up in less developed areas, develop links between HEIs and industry to increase the relevance and marketability of their programs, and to provide significant subsidies so that no deserving student is deprived of the opportunity for a higher education. IV. Promoting Excellence in Learning and Research A major aim of the MTDF is: “to ensure that all higher education students benefit from a high￾quality learning experience” and to “…increase the capacity of higher education institutions to carry out cutting-edge research….”74 One of the key elements in the HEC strategy for quality 74 Ibid. MTDF, p. 31. Evaluation of USAID Higher Education Portfolio 116 Academy for Educational Development August 29, 2008 improvement and relevance is to improve the quality of teaching through faculty training, teaching awards and by providing a variety of incentives to carry out research, attend professional meetings, and keep up with the changing nature of one’s academic field. A. Improvement of teaching and learning The HEC has established the Learning Innovation Division to provide short-term and long-term training programs for the faculty of the universities to enhance the pedagogical skills and promote research activities. Courses include: • Curriculum-based short-term courses/subject-based orientation – 6 to 12 days. More than 600 faculty members have taken content based courses and 238 have been part of the faculty professional development program. More than 450 faculty members participated in workshops and lectures that were part of the learning innovation series, and many other programs have been offered by eminent expatriate and Pakistani scholars. • Courses on Research Methodology for university faculty. • A program designed to introduce the semester system to faculty members in Pakistan. The semester system is intended to give the students a better understanding of their progress in mastering the subject, improve course quality, and allow students greater flexibility in terms of choices and breadth of study. • Assessment and Testing – awareness seminars for Heads of Departments, Controllers of Examinations, Paper Setters and stakeholders. A national committee on examinations was set up in October 2003 to try to improve the current examination system. • Teaching awards for outstanding teaching. • Transformational Change – a workshop for senior managers. This workshop was held by former US university president, Dr. Mary Linda Armacost, on leading transformational change, based on US experience with transformation. B. Higher Standards. The HEC set higher standards for M.Phil and PhD programs to raise standards of graduates; 36 universities and DAIs were reviewed. C. Research Improved research performance is expected from faculty in the new tenure track system. Higher quality research has a spillover effect for teaching. The HEC has put in place inducements for research including: • Increased funding for research • Peer review of submissions for approved journals • A plagiarism policy • Merit-based promotion system as part of tenure track. • Requirements of faculty members of measurable progress over the years. D. Improvement of instructional equipment Evaluation of USAID Higher Education Portfolio 117 Academy for Educational Development August 29, 2008 The HEC has been providing and enhancing equipment for teaching and learning, including: • Library enhancement as part of infrastructure improvement (discussed earlier) • Video conferencing and equipment for video presentations by PowerPoint and other mechanisms. • IT development (discussed earlier) • Digital Library (discussed earlier) E. English instruction The HEC will undertake English Language Teaching Reform in an effort to improve English language instruction and raise the level of English among graduates. This is a major problem identified by employers. F. Conclusions It is too soon to judge the full effects of the HEC’s focus on teaching and learning, but the efforts being made seem to be in the right direction. The key to quality improvement rests in the motivation of faculty members, and this will vary by institution and take time to change is some cases. The faculty members themselves have the primary responsibility to make the changes necessary to bring their courses up to expectations. That is happening a now at some universities. We saw that kind of change taking place at Aga Khan University; the University of Agriculture, Faisalabad; LUMS; and FCCU – all providing excellent examples of this kind of curriculum change. However, until larger numbers of new PhDs, trained in universities abroad, return and are present in larger numbers, the changes will be limited to those universities with major commitments to international standards. At the same time, the evaluation team was impressed by the efforts being taken at several universities to upgrade the curriculum. At FCCU, for example, changes in every major program are underway and there is an air of excitement and intellectual creativity on campus that has students excited and faculty members committed. We think these models will have an effect on other institutions and urge their support by the HEC and USAID. V. Quality Improvement and Accreditation Of the three major issues addressed by the Medium Term Development Framework (MTDF), the second priority was quality.75 One of the motivating factors behind the new higher education plan was the very poor quality of higher education in Pakistan. The HEC chairman noted in the Foreword to the MTDF that: “Not a single university of Pakistan is ranked among the top 500 of the world.”76 The percentage of students of college age attending institutions of higher education was among the lowest in Asia, only 23% of faculty had PhDs, there was no culture of research, teaching was not emphasized, and faculty members for the most part, had not kept up with progress in their fields. 75 The other two were access and relevance to national needs. HEC, MTDF, 2005, p. iii. 76 Ibid. Evaluation of USAID Higher Education Portfolio 118 Academy for Educational Development August 29, 2008 The HEC embarked on a number of immediate steps to improve quality, focusing on faculty development and upgrading university infrastructures. The HEC set an ambitious goal of sending 15,000 to 20,000 faculty members abroad for PhD training in fields of national priority over the next five years – that is by the end of 2010. That was supplemented by indigenous PhD training and foreign hiring of PhDs. For infrastructure improvement, an initial program to improve laboratories was started along with efforts to provide access to books and journals, acquire updated equipment, improve computer and Internet access, and enhance teaching aids.77 All this was designed to help Pakistan create a “knowledge based economy” in line with the recommendations of the World Bank 2002 study: Constructing Knowledge Societies: New Challenges for Tertiary Education. 78 From the outset, the HEC set international standards as the benchmark79 for quality improvement for institutions, programs and individuals at every level – the faculty, curriculum, student admissions, PhD requirements, promotions, equipment and laboratories, ICT, research, teaching, and service. All were to be judged comparatively based on international standards, and to the extent possible, subject not only to internal evaluation but to external review as well. As one looks back at the focus on quality improvement three years later, it was a wise decision that has paid dividends. Quality improvement and international standards were evident in a wide range of programs implemented by the HEC including: • Faculty development through Master’s and PhD training abroad. • Faculty development in Master’s and PhD programs in Pakistan. • In service training for faculty members run through the HEC Quality Assurance and Learning Innovation Division. • The policy on plagiarism. • Increased standards for faculty appointments. • The tenure track system. Quality improvement continues to be among the major foci of the HEC in all areas. At the program level, four accreditation councils have been established to date to accredit programs. The HEC has decided to limit program accreditation to the professions given the magnitude of the challenge. Universities will be reviewed under the institutional accreditation program with 15 standards used in the course of evaluation. That process will involve self-assessment by the institution in terms of the standards, a site visit by peer reviewers, and their recommendation for accreditation. In addition, the Quality Assurance Division has undertaken a number of programs designed to improve quality, working with the universities and encouraging them to be active on their own through the quality enhancement cells and their own institutional governance structure. Among the initiatives taken to improve quality are the requirement of a PhD for assistant professorship, increased criteria for M.Phil and PhD which went into effect on 31 May 2005, and the new eligibility criteria for faculty on tenure track80 which was implemented in 2006 (revised 77 Ibid, p. iv. 78 World Bank, Constructing Knowledge Societies: New Challenges for Tertiary Education, Washington DC (2002). 79 MTDF, p. 51. 80 Higher Education Commission, Eligibility Conditions for the Appointment of Faculty Members in Universities and DAIs, Islamabad, January 2007. Evaluation of USAID Higher Education Portfolio 119 Academy for Educational Development August 29, 2008 in 2007) but requires approval by each university for institutional implementation. Many universities have done that already. A. Quality Enhancement Cells Early in the quality improvement process, the HEC worked with several universities to establish Quality Enhancement Cells at 10 universities on a pilot basis; 9 of which worked well. Each is headed by a faculty member at the professor level. The staff of each Cell worked with the HEC and the institution’s faculty members on quality improvement. They will be the contact points for program self-assessment and eventual accreditation. By 2007, Quality Enhancement Cells were in place at 30 institutions. A spot check indicated that they are working effectively and laying the groundwork for self-assessment and institutional accreditation as well as assisting with existing program accreditation in areas in which Councils have been established. B. Program Accreditation The HEC has established a Quality Assurance Division to organize the accreditation process. Accreditation will by carried out through both program accreditation and institutional accreditation. 1. Program accreditation for professions (up to twelve Councils in the professions). To date the QAD has established four Councils. They are: • National Computing Education Accreditation Council • National Agriculture Education Accreditation Council • Teacher Education Accreditation Council • Business Education Accreditation Council 2. Linkages are being established with existing accreditation/certification bodies in Pakistan. The HEC is working on such linkages and MOUs with existing independent councils such as the Council for Architects and Town Planners, Pakistan Engineering Council, and several others. 3. Material is being prepared to assist in the program accreditation process.81 C. Institutional Accreditation Standards are being prepared for institutional accreditation by the Quality Assurance Division (QAD) of the HEC. After the QAD has completed a draft version of the Standards, they will be reviewed by the Commission, and when approved, shared with the universities for their comments and input. This is planned for late 2008. Rapid completion of this process, including discussion of the Draft Standards with institutions, is important to build on the momentum begun in quality assurance and to help get the institutional accreditation process underway so that institutions can begin to prepare their self-assessments. 81 See for example, Zia Batool and Riaz Hussain Qureshi, Good Practices for Quality Assurance for Accreditation Councils in Pakistan, Higher Education Commission, n.d. Evaluation of USAID Higher Education Portfolio 120 Academy for Educational Development August 29, 2008 A thorough institutional self-assessment takes 6-12 months. While that is going on, the QAD should begin the process of identifying a pool of peer reviewers. They should be distinguished academics and professionals with a record of fair judgments and no ethical blemishes. They might be nominated by higher education institutions, professional associations, and members of the Commission. Given the diversity of institutions, we would suggest that they start with a pool of at least 100 peer reviewers. Candidates will need to be screened carefully by a committee set up for that purpose, and those who meet the tests, provided with training (1-2 days) focused on the process, procedures, Standards, expectations, conflict of interest, ethical requirements, confidentiality, and related topics. This is a critical part of the process since it is the peer reviewers who make the judgments about whether or not institutions meet national standards, they are the face of the process, their interaction at institutions sets the tone for accreditation, and their own standards are key to its success or failure. D. Successes: The quality assurance process is off to a good start. • Standards for Accreditation. The HEC has begun the process of developing the Standards for Institutional Accreditation. They are currently being prepared by the Quality Assurance Division (QAD) and should be ready for review during 2008. In the mean time, the QAD should prepare material about the site visit process, training for peer reviewers, and other material related to accreditation. The QAD should organize pilot accreditation reviews (focusing on at least one public and one private university) as a test of the procedures and to insure that the Standards are clearly understood and that assessment procedures are effective. • Quality Assurance Committee. The QAD is in regular consultation with the Quality Assurance Committee of the HEC. This committee is composed of distinguished faculty and administrators. • Quality Assurance Agency. The HEC has recently established a Quality Assurance Agency (QAA) under the QAD to oversee the institutional accreditation process in the nation’s public and private universities. It will need to work with the QAD on the accreditation process, its timing, and organization. • Quality Enhancement Cells. Following up on the pilot project with 10 Quality Enhancement Cells, the QAA has expanded the number to 30 cells in universities. Their success should be reviewed at some point in the next year. • Quality Enhancement Programs. The HEC has undertaken a number of quality enhancement programs including: short-term training for faculty, teaching awards for outstanding teachers, research support, infrastructure improvement, video￾conferencing, workshops, and other training. E. Conclusions and Recommendations The HEC has overcome one of the biggest challenges faced by the QAD since its inception, which is a shortage of staff. It should now be able to move forward rapidly in developing the Standards for Institutional Accreditation, one of the most critical aspects of the Institutional Accreditation process, and move on to piloting the accreditation process at several universities. The HEC is making good progress in achieving the goals of quality assurance and accreditation. Evaluation of USAID Higher Education Portfolio 121 Academy for Educational Development August 29, 2008 • Standards: Our review of the Draft Standards suggests that they are in keeping with international standards for accreditation, drawing on some of the best material on accreditation to be found, namely the standards of the New England Association of Schools and Colleges, Commission of Higher Education in the US, and the Quality Assurance Agency of Great Britain,82 adapting this material to Pakistan’s culture, experience and needs. It will be important to complete the initial development of these standards in the next few months, have them approved by the HEC and then discussed with the public and private universities. The QAD has been wise to follow the general themes and language of existing accreditors since these Standards have both passed the test of time and been tested in courts of law. • Piloting the Standards. Part of preparation for accreditation should involve pilot testing the Standards, once the final draft is approved, on several institutions that volunteer for that task. This would include completion of a self-assessment document on their part in which they discuss their own level of compliance with each of the standards, a site visit by a team of 5 to 8 peer reviewers (depending on size and complexity of the institution), a final report by the team, and a recommendation to accredit or not to accredit by the appropriate HEC body. Experience in the US, South Africa, and elsewhere suggests that this is an important stage in the process to help the QAD and the QAA work out any deficiencies in the process, and put the Standards to the test. • Candidacy for Accreditation. The QAD may want to consider a stage often called Candidacy for Accreditation as a pre-accreditation stage in the process. The institution must meet certain conditions to qualify for Candidacy. An institution usually must be a candidate for accreditation for at least one year and have graduated its first class, before being considered for accreditation. • Peer Reviewers. The most critical part of the accreditation process is the selection and training of peer reviewers. They must be outstanding scholars and researchers who are highly respected in their fields, thoughtful in deliberation, noted for their fairness, and beyond reproach. The selection of peer reviewers and their training must be done with great care. They are central to maintaining high quality standards and to the legitimacy of the whole process. • Self Studies, Site Visits, and Evaluation. The QAD should begin now to develop procedures for Self Studies, Site Visits and Evaluation. The self-study should be built around the Standards for Accreditation. Procedures should be carefully laid out for the final decision for accreditation. Normally the recommendations of the site visit team and their report go to a commission or board made up of distinguished academics and professionals who review the recommendations of the peer review team, its report, and the self-assessment, and decide whether to grant accreditation, deny accreditation, or (if that is an option) put the institution on probation for a period of time (usually 1-2 years). • Education of the Public. Critical to successful accreditation is public education (including the universities) about the meaning and importance of accreditation. The 82 See for example the: Standards for Accreditation Commission on Institutions of Higher Education New England Association of Schools and Colleges, adopted in October 2005. Evaluation of USAID Higher Education Portfolio 122 Academy for Educational Development August 29, 2008 public should understand that this is a measure of institutional quality. A major problem in the developing world today is the large number of fraudulent and low quality institutions that claim to be giving tertiary education but are providing either very low quality or bogus degrees, fleecing the public, and leaving students ill￾prepared for the highly competitive job market faced by today’s university graduates. One of the best protections against that is a well-informed public. VI. HEC Industry/Business Linkages: Relevance to the Economy A major aim of the MTDF is to improve the relevance of higher education for the economy – “to ensure alignment of the generation of human capital with economic activity and national development objectives.”83 This goal responds to widespread industry and business concerns about the lack of relevance of much of higher education to national needs. The World Bank Higher Education Policy Note, Pakistan, emphasizes states that ensuring the relevance of university education to employment and economic development, including industrial linkages, is key to a country’s socio-economic development. A. HEC Vision, Strategy and Aims In accordance with the MTDF, one of the core strategic aims of the HEC is to promote interaction between the industrial and higher education sectors and to promote its alignment with economic activity and development. The four objectives of this effort are: 1) encourage university-industry collaboration, 2) establish industrial internship program, 3) set up technology parks, and 4) involve industry experts in university bodies. In support of these objectives, the HEC has prepared a document entitled, “Technology – Based Industrial Vision and Strategy for Pakistan’s Socio-economic Development.” The document contains 23 policy guidelines and objectives to help guide Pakistan’s scientific and technological capability along with supporting generic strategies and implementation requirements. B. Linkage Strategies The cornerstone of the HEC’s research and development strategy is to enable and nurture research at Pakistan institutions and to ensure that the research enterprise is sufficiently integrated with industry. The intent is to provide linkages between governments, higher education, research and development institutions, and industry through support for collaborative research and cost sharing that leads to further national development. The HEC has two main program strategies for ensuring relevance; 1) a university-research linkage program, and 2) a university-industry linkage program (See Appendix XIV for details). In addition to these linkage strategies, other options for support are included in the HEC Technology-Based Vision and Strategy document and as a part of the MTDF programs (e.g., corporate social responsibility, career centers, internships, technology parks, technology incubators, research and development centers, involvement of industry experts, and matching contributions). 83 Ibid, MTDF, pp. 42-43. Evaluation of USAID Higher Education Portfolio 123 Academy for Educational Development August 29, 2008 C. US Linkage Support Included within the linkage program strategies were two USAID supported programs; 1) a Pakistan-US joint Academic Research Program (NAS), and 2) a Competitiveness Support Program (CSP). In 2003, the GOP Ministry of Science and Technology and the US Department of State signed a Science and Technology agreement. In 2005, the USAID Mission in Pakistan agreed to assume funding and management responsibilities for activities resulting from this agreement, which resulted in the Pakistan-US Science and Technology Cooperation Program. Today there are 46 applied research projects being implemented in support of this program – 31 with the HEC and 15 with the Ministry of Science and Technology (MoST) and an additional 19 applied research projects selected for implementation during this next year. The projects include a combination of USAID, HEC, MoST, and other US funding. The completed projects have been very successful with high quality, relevancy and impact (See Chapter 6 for descriptions of some of the projects). These projects show how a small amount of funding can be utilized in partnership with experts in the US to leverage access to equipment and training resulting in national impact for Pakistan. The USAID/HEC supported Competitiveness Support Fund (CSF) was launched in April 2007 with an aim of promoting knowledge-based economic development. The purpose is to support industrial-based research proposals and develop a business incubator project. To date the project has conducted four linkage awareness workshops with industry on promoting private sector involvement in commercialization of the viable research projects. The projects are peer￾reviewed and must be recommended by at least two peer reviewers. D. Supporting Data of Successful Linkages Examples of HEC successful linkage partnerships have included development of a bio￾diesel fuel for running cars; a national hygienic baby foods project at Karachi University, several herbal products development projects at Kohat University, development of an integrated solution for a solar hybrid water-heating system for industrial processes, renewable/alternate energy solar home electrification of villages, and enzyme and microbial technology for industrial application. HEC has four domains for conducting industrial-sector relevant research at the university; improve new processes, improve new products, develop new processes, and develop new products. Included is an update on the MTDF industry-relevance indicators. Indicators # of joint university-industry projects – 10 # of technology incubators established – 5 approved (0 established) # of patents issued – 9 filed in US patent office in the past two years # of industry-sponsored research projects - 0 # of students undergoing one-year internships with industry – started in 2007; 25,000/yr. under the Prime Minister’s National Internship Program (NIP) – 2008 program is stopped Total yearly income of university from commercialization of research – not available Evaluation of USAID Higher Education Portfolio 124 Academy for Educational Development August 29, 2008 The university-industry thrust is new in the past two years. The data show some success as there are now 10 university-industry linkage projects, 5 approved technology incubators, and 9 patents filed. Since 2007, 25,000 Master’s program graduates have participated in a one year National Internship Program with industry. This program is now stopped with the caretaker government. A start has been made but there is a long way to go. E. Conclusions The research culture has been renewed in Pakistan through the HEC, especially applied research. University-to-university partnerships and university-to-R&D institute partnerships are feasible and being accepted, but linking relevant research to the industrial sector is a new concept. There are several challenges that will need to be overcome in order to establish a successful industry linkage applied research partnership: • There is a lack of confidence by the industrial sector and the research institutions in each other in terms of the benefits of such partnerships. More awareness strategies are needed to market the potential partners on the benefits. The owners of industry do not want the academics to make changes. They do not understand the need for new services, processes and products or, minimally, accept the idea that applied research can help the industrial sector. • There is a need for an increased financial contribution from industry to become viable. Currently, the contribution is about 20 percent while in Europe the contribution is closer to 70 percent. • There is need for an enterprise employment training study to help assess relevance of training needs and begin to establish better communication between industry and the training providers; likewise, there is a need for a graduate tracer study. • There is a need to model some of the “best practice” applied research strategies implemented in other developing countries so all will better understand the relevance and potential impact of applied research in development. • There is a need for more industrial sector internships. In summary, the HEC industrial sector linkage strategies have enhanced the HEIs ability to do relevant research. These programs have been quite successful. However, those who have tried to transfer technology or establish joint partnerships with the industrial sector have struggled. The confidence gap is immense and there is need for more work to close the gap before productive linkages are possible. F. Recommendations Included in this section are illustrative relevancy linkage themes to help close the industry and business relevancy gap in terms of quality and responsiveness of the HEI provider to the employer and economic development. Each HEI ought to review carefully its linkage strategy with industry and business to be sure it is doing all that is possible. The general themes include linkage strategies in the areas of empowerment, accountability, partnerships, quality assurance, responsiveness, and resource mobilization, all of which are essential to building Evaluation of USAID Higher Education Portfolio 125 Academy for Educational Development August 29, 2008 strong university-industry partnerships and are possible areas of future donor programming. The team recommends that the HEIs consider the following: 1. Improved management, empowerment, and stakeholder accountability • Involvement and empowerment of local industry experts in program design, development, and management is essential to local ownership and success. • Broad-based advisory board representative of stakeholders – e.g., business/ industry, local community and provincial leaders, academics, associations, and government. 2. Strengthened linkages - cooperation and partnerships • Line-item training provider position with responsibilities for Cooperation and Outreach Services; e.g., contacts with government, industry associations, donors, community, industry, student work experiences, employment information/job placement, on-campus visits/interviews, and follow up of graduates. • Business/industry to provide joint corporate social responsibility opportunities in response to local educational social needs. • Joint cooperation and partnerships with other training providers – shared courses, exchanges, sharing of facilities, joint offering and sponsorship of short-term training opportunities. 3. Strengthened program quality and relevancy • Joint responsibility of Ministries, National Boards, industry associations, and training institutions in setting and ensuring competency-based program skill standards for trainees relevant to the workplace with workplace skill-certification and workforce competency-based assessment. • Full-time (2-4 month) work experience for students; coordinated and monitored by provider and industry; MOUs to clarify roles and responsibilities. • Professionally trained/qualified staff with work experience. • Strong English program component, computers sector-integrated, entrepreneurship experiences/courses, and employability/life skills integrated with the program (e.g., use of active learning teaching pedagogy). 4. Enhanced local responsiveness–labor demand and economic development • Institutional strategy that sets targets, monitors performance (budget, program and employment); flexibility to offer demand-driven courses/programs deemed necessary by regional/national industry and for development of local economy. • Priority given to gender-balance and opportunities for needy/low-economic students. 5. Strengthened resource mobilization Joint responsibility of donor partners, industry (e.g., scholarships), government/ budget, student/fees, and institution (e.g., production training center, short-term upgrade training for industry offered for a fee, project contracts) in generating resources for the institution. VII. Leadership, Governance, and Management Evaluation of USAID Higher Education Portfolio 126 Academy for Educational Development August 29, 2008 This is an area that has yet to be a major focus of the HEC although some progress has been made over the last five years. There have been several workshops on leadership, as noted earlier, including one about transformational change. The selection of Vice Chancellors at public universities has been substantially improved with the introduction of selection procedures84 which have depoliticized the process significantly and improved the quality of the candidates. Vice Chancellors of provincial universities are still chosen by the governors, but from a short list of candidates who have been vetted using these procedures. Private institutions continue to have their own procedures for appointments. Efforts are being made to encourage universities and DAIs to carry out strategic planning. Their finances are also under careful scrutiny by the HEC. Nonetheless, these are areas in which a great deal more work is needed. In that regard, we note the following: A. Leadership training • HEC programs are a promising beginning but much more needs to be done. Middle level management training in particular is weak. • The HEC might consider developing regular programs, including budget and strategic planning training, for VCs, Deans, and Department chairs. • Expand education leadership program for women in higher education leadership positions. Excellent examples of what can be done are found at Fatima Jinnah Women University, which has developed a program for women leaders in higher education. B. Governance • HEC policy for nomination of Vice Chancellors has helped improve the choices and reduce the political influence in the process. • There is a need to strengthen faculty governance input into the development of academic programs with international standards as the benchmarks. • The nature of the Council of the HEC might be rethought so that it serves not only as a governing body, but also has the ability to defend the HEC in times of crisis and serve as a buffer between the HEC and government. Many of its current members have such extensive responsibilities in other Ministries or as Vice Chancellors that the Council does not serve as an effective advocacy body when that is needed. The addition of several retired Vice Chancellors, professionals, or businessmen might provide a core of individuals who could lead such efforts when needed. • While the confusions that existed three years ago between the HEC, Provincial, and institutional roles in governance has lessened, in part as an indirect consequence of the expansion of effective HEC control of policy and finances, the confusion needs to be reduced through clear policy directives. • Management: While management systems have been substantially improved through the HEC in terms of MIS availability to institutions and departments, there is a need 84 See, Procedures for the appointment of Vice Chancellors of Federal Public Sector Universities, Presidential action, January 26, 2005. Evaluation of USAID Higher Education Portfolio 127 Academy for Educational Development August 29, 2008 to provide management training for institutional staff and faculty administrators – most of whom do not come from academic backgrounds. VIII. Improving the Physical and Technical Infrastructure. A. Context The crisis in higher education facing Pakistan in 2002 was in part a consequence of low budget allocations by both the federal and provincial governments, which remained largely stagnant for many years at about 0.1 percent of GDP. Laboratories, libraries, and IT infrastructure were in especially poor condition. The major contributing factors included limited funds, which in turn was an expression of a lack of political will and commitment to the higher education sector, as well as bureaucratic inertia, and a failure to keep up with international standards. These issues were highlighted in a number of reports on the state of higher education in Pakistan, the most significant being the report of the Task Force on the Improvement of Higher Education in Pakistan. The introduction to this report highlights the problems facing Pakistani HEIs in the following words: “The research base in the universities is weak, and inadequately equipped libraries and laboratories and a shortage of qualified teachers continue to hinder the progress of higher education towards excellence."85 B. Development Funding Major increases in the higher education funding enabled the HEC to embark upon an ambitious program of infrastructure and technology development in higher education institutions across the country. The HEC increased its development budget for HEIs from Rs. 4.2 billion in 2002-2003 to Rs 18 billion in 2007-2008.86 The amount for approved projects grew Rs. 34.3 billion for 108 development projects by 2006-07, with 33% of the budget allocated for infrastructure development. The funds are being used to establish 13 new sub-campuses of 9 public sector universities, establishment of seven new public sector universities, introduction of 9 new disciplines in universities, and implementation of 9 development projects across the country.87 C. Academic and Research Infrastructure One of the important steps taken by the HEC to improve the academic and research infrastructure was to focus on improving the technical universities of the country. The HEC helped engineering HEIs to improve their design capacities. Through a Rs. 37.6 million project, 9 engineering universities/DAIs in the 4 provinces were equipped with advanced designing software, including MATLAB, ANSYS, Pro-Engineer, and CATIA.88 The new design software enabled faculty members to both improve their design skills and to provide faculty and students the opportunity to conduct research. 85 Report of the Task Force on the Improvement of Higher Education in Pakistan. March, 2002 86 HEC, June 2008, Figures in 2007-8 are estimated. 87 Higher Education Commission. Annual Report, 2006-07, page 137. 88 Higher Education Commission. Annual Report, 2006-07, pages 10-11 Evaluation of USAID Higher Education Portfolio 128 Academy for Educational Development August 29, 2008 The HEC has supported HEIs to develop capacities in new and emerging technologies, particularly in the fields of biology and chemistry. A Rs. 153.4 million project to strengthen HEIs in new and emerging technologies helped in the establishment of a National Centre of Biotechnology, a Bio-Organic Chemistry Laboratory, and a Research and Development facility for exotic cut flowers and value added products.89 Research from these institutes has great potential for industrial and other applications. Access to scientific equipment and maintenance of such equipment has been particularly weak, preventing scientists and faculty (and students) from conducting quality and meaningful research. The acquisition of equipment entails huge costs, primarily because most of it has to be imported from developed countries. The HEC came up with an innovative solution to make the equipment purchased as widely available as possible. The Access to Scientific Instrumentation Program designated 12 high technology research institutes or laboratories as repositories of instrumentation in specific fields of science and equipped them with state of the art instrumentation. Free access to these institutions allowed some 1,300 researchers and scientists from public sector universities and research to use the equipment. In addition, some 38 grants of up to Rs. 1 million were provided to HEIs for maintenance of scientific equipment through a Rs. 37.4 million program.90 D. Access to Information The need to establish a knowledge economy if development was to occur had become evident to leaders in higher education by 2000. Building bigger and better libraries could not keep pace with the global information explosion, even if funds could be found for that expansion. The only viable option available was to take advantage of information technology. In that context, the HEC began to create a digital library, with its server housed in the HEC, designed to provide access to academic journals, books, and other information around the world to higher education institutions throughout Pakistan. In a short period of time this became one of the best and largest digital libraries in the developing world, bigger than most in Europe. Presently the National Digital Library Program provides free access to HEIs to over 25,000 research journals and 40,000 e-books. Over 2 million articles were downloaded between 2004 and 2006 alone.91 The implications of such a library are immense. Faculty, researchers, and students now have access to the best research journals in the world in every field of knowledge. Traditionally Pakistani HEIs have been extremely isolated from each other, with little opportunity for interaction between faculty and students of various universities, except for occasional conferences and seminars. The HEC created the Pakistan Education & Research Network (PERN) and connected 97 HEIs with a bandwidth of 310 Mbps. Plans are being made through PERN2 to improve the connectivity using upgraded technology and establishing metro networks in major cities.92 89 Higher Education Commission. Annual Report, 2006-07, page 12 90 Higher Education Commission. Annual Report, 2006-07, pages 9-10 91 Higher Education Commission. Annual Report, 2006-07, pages 61-62 92 Higher Education Commission. Annual Report, 2006-07, page vii. Evaluation of USAID Higher Education Portfolio 129 Academy for Educational Development August 29, 2008 Many researchers in Pakistan are not able to get their research papers published in a timely fashion and in the most appropriate journals. HEC has recently introduced a Higher Education Scholars Publication web-site where the outputs of such scholars will be available; 25 publications have been launched so far. With the growing number of HEC publications a HEC Book House/Publications with an online shopping cart has also been developed. E. Development of HEC Infrastructure With the expansion in its roles and functions, it was necessary for the HEC to streamline its own internal systems and build the technological capacity to handle the large number of projects it was monitoring. It was also faced with the challenge of handling large numbers of files manually, which was a slow and inefficient process. A number of initiatives were implemented in this respect. A Rs. 22.9 million project has helped HEC to update its IT infrastructure to meet the growing demands of various programs and HEIs. A File/Document Tracking System has been developed to automate HEC’s internal business processes. An Enterprise Resource Planning (ERP) Solution is being implemented to improve processes and streamline activities of the HEC.93 F. Relevance, Efficiency and Effectiveness The HEC has adopted a holistic approach to infrastructure development. For the first time the HEIs have been encouraged to develop long-term institutional visions and objectives and identify all their infrastructural development needs for the next 5-7 years. These projects are reviewed by the Executive Committee of the National Economic Council (ECNEC), which approves all mega-projects in the country. ECNEC determines the relevance and value of the project in relation to its costs and the higher education needs of the country.94 The HEC has put in place a rigorous system of monitoring by the Planning and Development Division to keep track of all infrastructure development projects. Institutional and thematic quality was achieved through the setting of performance and specific time-based targets for all programs, and subjecting them to quarterly performance reviews. Slow performing projects were encouraged to improve their performance, and warned of penalties in case they failed to meet their targets. In some cases funds were shifted from slow-performing to fast track projects. Upgrading science laboratories with new equipment and training of technical staff has greatly facilitated student and faculty research. One indicator of improved research output is the 360 per cent increase in publications from Pakistan in research journals of international repute in the last four years.95 In our site visits we received positive feedback from faculty about the increased research opportunities due to well-equipped laboratories. G. Conclusions 93 Higher Education Commission. Annual Report, 2006-07, pages 57-60 94 Higher Education Commission, ibid. (June 08, 2008) 95 Higher Education Commission. Annual Report 2006-07, page vii. Evaluation of USAID Higher Education Portfolio 130 Academy for Educational Development August 29, 2008 The HEC has had a significant impact on the improvement of the physical and technological infrastructure of the higher education infrastructure in Pakistan. There is overwhelming evidence of this in the new buildings, laboratories, IT facilities, the digital library and a host of other projects. Despite these major efforts, considerably more needs to be done to improve the physical and technological infrastructure to make up for years of neglect. The development of the physical and human resources must go hand-in-hand if optimal use is to be made of the newly acquired physical and technical resources. That process is underway. Its continued support will be critical to the long-term quality improvement of higher education. H. Recommendations • Maintain the HEC-established fund for repair and maintenance of equipment open to public and private universities. • Maintain the HEC-established program to train scientific and technical staff both in Pakistan and abroad; 38 people have already been trained under this program at a cost of Rs. 36.661 million.96 • The HEC web-portal is one of the most used Pakistani sites with about six million hits per month. However, the contents of a number of departments/sections need to be updated. • While considerable effort has been made to obtain and authenticate data from HEIs, the HEC is still behind in authenticating and updating data in key areas like enrollment of students (updated and authenticated through 2004-05). A new Director General is now in charge of the statistics division and with more staff it is hoped that the division will catch up with the backlog. It is vital for the HEC to have its critical data up to date. IX. Public-Private Higher Education Cooperation and Development The private higher education sector in Pakistan dates to the late 1800s. In 1864, Lahore Mission College was established, later changing its name to Forman Christian College in 1894, and awarded degrees through Calcutta University. In the 1980s, the Aga Khan University (1983) and the Lahore University of Management Sciences (1985) were established. The development of private higher education institutions increased as a result of the 1979 National Education Policy, which reversed the ban on higher education institutions introduced by the 1972 National Education Policy. One of the goals of the Draft National Education Policy: 2008 is to encourage private provision at all levels including higher education.97 The HEC has begun to do that over the last two years. The private sector has become an especially active player in higher education in the last few years, and now includes 15 percent of total enrollments. While private sector enrollments have increased by 75 percent in four years, their share of total enrollments has actually declined from a high of 17 percent in 2003 to 15.1 percent in 2006, primarily due to the faster expansion in public sector intake (See Table 20 below). 96 Ibid. Annual Report 2007-08, pp. 16-18. 97 Ibid., Draft National Higher Education Policy: 2008, chapter 8. Evaluation of USAID Higher Education Portfolio 131 Academy for Educational Development August 29, 2008 A number of factors have contributed to the increasing role of the private sector, including the inability of the public sector to meet the growing demand for higher education, legislation supporting private sector involvement in higher education, a growing middle class, more job relevant curriculum in private HEIs, and the declining quality of the educational and political environment of public sector HEIs.98 Beside the issue of quality, a major area of concern has been the high fees of the private higher education sector, which is affecting its growth. Many private HEIs have developed a variety of scholarship programs to off set this disadvantage for some students. For example, the evaluation team saw a very elaborate range of scholarships offered by the Institute of Management Sciences in Peshawar, focused on the neediest students. Some private universities and colleges have made major efforts to provide financial assistance to poor students and those from disadvantaged areas. For example, IBA/Karachi, LUMS and FCCU in Lahore have policies whereby no student gaining admission is deprived of availing the opportunity due to financial constraints. In the last few years, the HEC has made a deliberate effort to attract the private sector to assist in meeting the increasing demands of the student population for higher education. The number of private HEIs (both universities and degree awarding institutions) has increased from 2 in the mid-eighties to 19 in 1999/2000. The total had grown to 57 by 2007/2008. Currently, there are 45 private universities and 12 private degree granting institutions serving approximately 80,000 students, which represent about 15 percent of the students enrolled in HEIs in Pakistan. Table 24: Enrollment in HEIs, 2001-2006, Illustrative Public-Private Ratio Annual Increase (percent) Public percent Private percent Public Private 2001-02 230,722 83.7 44,981 16.3 2002-03 274,688 83 56,398 17.0 19.1 25.4 2003-04 360,387 85.3 62,248 14.7 31.2 10.4 2004-05 404,011 85.6 67,953 14.4 12.1 9.2 2005-06 442,539 84.9 78,934 15.1 9.5 16.2 Total Increase 91.8% 75% Source: HEC, 2008 A. The HEC Guidelines and Regulatory Framework Currently, the HEC is working to provide a consistent regulatory environment for private education. In response to the growing number of private HEIs, the HEC, based on the revised criteria as approved on February 27, 2002, and amendments approved on October 4, 2006, has set forth a regulatory framework of governance for the establishment of new public and private HEIs in Pakistan called: “Guidelines for the Establishment of a New University or an Institution of Higher Education.” Currently, 24 of the 57 private sector universities and degree awarding institutions meet the major requirements of the criteria, 27 have minor shortfalls, and 6 are seriously deficient. 98 World Bank, Higher Education Policy Note, 2006, p.41 Evaluation of USAID Higher Education Portfolio 132 Academy for Educational Development August 29, 2008 In addition, the HEC has been working to develop consensus on the issue of providing government assistance to private sector universities. Based on the 2006 regulatory framework criteria, along with a set of 8 additional selection criteria, the HEC has identified 9 private sector universities and 4 degree awarding institutions that are eligible for funding as of March 14, 2008 (see Appendix XV). The HEC’s intention is to develop a working combination of public and private sector higher education institutions that are complementary to each other and ensure a diverse but dual quality higher education system that is responsive to the needs of Pakistan. B. The Challenge Private sector institutions vary in their quality, as do public institutions. They tend to charge more tuition, pay their faculty members better (although the tenure-track pay scale has altered that for many private institutions), have more part-time faculty, have faculty with lower credentials, have a more flexible curriculum, and are more independent. They are often less research oriented, provide some demand driven-courses (especially in the computer, technology and business areas), and tend to offer a high percentage of the lower cost general education courses. Some are beginning to establish linkages to industry (e.g., LUMS). C. HEC Support to Private Higher Education Institutions In accordance with the HEC regulatory framework and a set of selection criteria, 9 private universities and 4 degree awarding institutions have been declared eligible for HEC funding. Included are some initial efforts by HEC to provide support to approved private HEIs in order to ensure a dual system with quality assurance and responsiveness to the needs of Pakistan. • Access to Digital Library – Digital Library access for country wide resources has been provided to each of the 9 approved private universities. • Pakistan Education Research Network (PERN) Connectivity – GIK Institute of Engineering and Technology, Lahore University of Management and Science, National University of Computer and Emerging Sciences, and Aga Khan University have been connected to PERN. • Faculty Research Projects – A number of private universities have had access to HEC research funding including: Aga Khan University; Ripha International University; National Textile University; LUMS, Lahore; GIKI Topi; and FAST. • Development Projects – One institution, LUMS, has been approved for a development project; i.e., lab equipment and human resource development in support of the establishment of a LUMS School of Science and Engineering. • Merit and Need-Based Scholarships – Lahore University of Management Sciences and Aga Khan University are eligible for the HEC/USAID undergraduate and master’s-level Agriculture and Business Merit and Need-based Scholarships. Separately, USAID provides undergraduate scholarships support to FCCU merit and need scholarships. D. Public-Private Partnership Proposal Status The HEC has begun to implement several of the Public-Private Partnership policy initiatives recommended in the World Bank Higher Education Policy Note – Pakistan, June 28, Evaluation of USAID Higher Education Portfolio 133 Academy for Educational Development August 29, 2008 2006, including improving the regulatory environment for private HEIs, access to information technology, and improved linkages between the HEC and private higher education.99 E. Conclusions The level of demand for access to higher education, as well as the need for highly trained and well qualified employees, is beyond the capacity of the public education sector and government to provide, even if higher education enrollments continue to grow at the current rate of 14.7 percent per year. Private provision of higher education is essential to meet both demands for access and for national development. Our evaluation suggests: • A growing demand on private education for access, job training, specialized training (e.g., medicine), and cultural values related to religion, social, and moral concerns; • A need to guarantee the quality of private education as well as public education; e.g., the HEC has started working to provide a consistent regulatory environment for private HEIs and to eliminate diploma mills and low quality institutions; • Work needs to be done by the HEC and others to develop consensus on mechanisms and conditions for government assistance to private HEIs; and • The benefits of public-private partnership initiatives; such as those begun by the HEC and those suggested in the 2006 World Bank Report, should be clear from early successes in Pakistan and the success of the US model. X. HEC Goals in the Context of International Standards and Expectations From the outset the HEC has set international standards and expectations as the benchmarks for quality improvement. In that sense, Pakistan is not unlike many other countries which have formally set their objectives using international standards. However, very few have actually utilized those standards in assessing institutions, faculty hiring, promotion of faculty members, tenure decisions, recognition of journals, and other areas in which quality assessments are essential to being competitive in a globalized world. Only a few developing nations such as South Africa, Madagascar, and India, have actually used those benchmarks in decision-making and evaluation of quality. Pakistan has done so across the breadth of their programs. In accreditation, for example, they have built their standards for quality in the sciences and technology on the pre-eminent accreditor for engineering and computer science in the world, ABET.100 Their use of international peer reviewers for faculty promotions, approval of PhD and Master’s theses, and for recognition of journal articles in approved journals, are additional examples of using international standards. No other developing country has done this on such a broad scale. Using such high standards poses risks of making the hurdles for achievement in higher education too high, but people are meeting these standards and quality is improving 99 Ibid. Policy Note, pp. 45-53. Each of these goals is spelled out in much more detail in the Policy Note which could serve as a useful guide for future planning and project funding. 100 ABET, Inc., is the recognized accreditor in the United States for college and university programs in applied science, computing, engineering, and technology. It is a federation of 28 professional and technical societies representing these fields. It is among the most respected accreditation organizations in the World and has been involved in accreditation and quality assurance in higher education for over 75 years. It is part of what is called, The Washington Accord, a coalition of ten countries which agree to mutual recognition of the accreditation decisions of each other and thus the graduates of all their accredited universities. Evaluation of USAID Higher Education Portfolio 134 Academy for Educational Development August 29, 2008 Pakistan. Adhering to these high standards, in the face of some efforts to weaken standards (as in the case of tenure requirements), is an excellent measure of the self-confidence of the higher education leadership in Pakistan and the commitment of the HEC, the Government of Pakistan, and its universities to quality improvement and high quality. XI. General Assessment of the Higher Education Commission at the Portfolio Level One of the most striking aspects of HEC since its inception is the emphasis on excellence and high quality in every sphere of its activities. Expectations were set high from the beginning. Quality targets were established as international standards and expectations. Faculty promotions, publications, PhD dissertations, research grants, and many of the HEC programs were subject to these standards, including evaluation by external peer reviewers. In addition, standards for admission to PhD programs were increased in 2003. This was in spite of the need for more graduates to teach the growing number of students. In keeping with its focus on quality, the attitude of the leadership of the HEC was that “quality is much more important than quantity.” In the short run, the higher standards substantially reduced the number of students admitted to PhD programs in Pakistan – by 31percent in 2003-2004, the first year it was enforced. The total number of students admitted to graduate programs increased somewhat the next year but was still 21 percent lower than in 2002-2003.101 Faculty recruitment requirements were raised in 2007 with standards spelled out in the HEC publication Eligibility Conditions for the Appointment of Faculty Members in Universities and DAIs. 102 They included a PhD from an HEC recognized institution for appointment as an assistant professor and publication requirements for associate and full professor. From the outset, the HEC made substantial amounts of funding available to universities to improve their infrastructure, for faculty development, research, professional meetings, travel, and a variety of other programs. In each case, appointments, access to programs, grants for equipment, infrastructure, and research were competitive. Most involved external review. What was striking about the HEC was that in spite of the breadth and magnitude of its programs and the extensive changes involved, implementation of all these programs was undertaken with great care to ensure quality improvement. In spite of rapidly increasing budgets, the HEC managed to keep the focus on quality improvement with the goal of meeting international standards in higher education. There was substantial resistance to the HEC and many of its programs from its establishment in 2002, especially from some of the older universities such as Quaid-e Azam University and the University of Punjab. Among their criticisms was the assertion that the creation of the HEC was a violation of university autonomy. Some also argued that its powers infringed on those of the regions. Indeed, several universities refused to apply for infrastructure and research funding and were unwilling to participate in the tenure track system in spite of salaries two to three times higher than those then in place. Some administrators and faculty members also suggested that 101 Calculations based on “HEC, Enrollment at Universities and DAIs during 2001-06.” Figures for Years 2004-05 and 2005-06 are provisional. July 2007. 102 HEC, Eligibility Conditions for the Appointment of Faculty Members in Universities and DAIs, Islamabad, January 2007. Evaluation of USAID Higher Education Portfolio 135 Academy for Educational Development August 29, 2008 the Accreditation system was suspect103 and in several cases suggested that their particular institutions were of such high quality that no outside body was qualified to evaluate them. In recent years, the leaders of the HEC have recognized the need to encourage private education. The HEC recently added 9 private universities104 to the list of those eligible for public funding for 3 types of programs, the National Research Program for Universities, Foreign Faculty Hiring, and major development projects for infrastructure, academic and research facilities (up to 50 percent of the cost).105 The first major construction project under this facility for private universities is underway at LUMS. By 2008, in contrast to 2005, the general attitude toward the HEC by almost all the nations’ universities, public and private, was very positive. In many respects it can be summed up by the comments of one vice chancellor who noted: “We have done very well under the HEC. It has helped us tremendously to develop. We have a success story to tell of which the MTDF and HEC are a central part.”106 The evaluation team completes it assessment of the Higher Education Commission at the portfolio level and its review of achievements to date on the Medium Term Development Framework 2005-2010 with the overall conclusions that the HEC is well on its way to achieving its goals, that its progress to date has been remarkable – indeed, in terms of value added, better than any other developing country this team has reviewed over the last two decades. Our overall assessment of USAID or USG funded projects is very positive. USAID funding has been effective, its administration effective both in terms of cost and use of human resources. The goals of USAID, and thus it programs have been extremely relevant to the aims and objectives of the GoP as reflected in the MTDF in particular (See Appendix XVI for a discussion of “USAID Contributions to the Government of Pakistan Higher Education Policy and Goals: Effectiveness, Efficiency, and Relevance”). USAID and USG support for the HEC has played an important role in the successes of the HEC in a number of critical areas of the MTDF including: improved quality of faculty members through Master’s and PhD training; improved access to higher education – especially to needy and disadvantaged students; excellence in teaching and research through enhanced scientific research, technology transfer, and applied research which fosters national development; and improved relevance of higher education programs (See Appendix XVII, “USAID Contribution to the GoP Higher Education Goals”). The MTDF, which elicited strong opposition from a number of major institutions and higher education organizations a few years ago, is now receiving approval. We heard almost unanimous praise for the HEC and the progress made in upgrading and reforming higher education from 103 Notes of the author, initial meeting of the Teacher Education Accreditation Council, December 6, 2005. 104 They are: Aga Khan University, Karachi; Ghulam Ishaq Khan Institute, Topi; Lahore University of Management Sciences; Riphah University, Islamabad; Foundation University, Islamabad; Institute of Business and Technology, Karachi; National University of Computer and Emerging Sciences, Islamabad; National Textile University, Faisalabad; and Indus Institute of Higher Education, Karachi. 105 HEC, Annual Report 2006-2007, p. 138. 106 Interview by the authors, May 8, 2008. Evaluation of USAID Higher Education Portfolio 136 Academy for Educational Development August 29, 2008 almost every institution visited. There were a few detractors worried about university autonomy, Provincial authority, academic freedom, and the impact of these changes on older employees and faculty. Many of these concerns represent legitimate issues on which there are different points of view. What has changed is that the overall attitude toward the HEC is positive with some comments especially laudatory and eloquent. We heard, for example, from one vice chancellor that: “The coming of the HEC was a boon for the university. This university was starved for funds and totally devoid of infrastructure prior to that.” This is not to say that the HEC has silenced its critics. We found several who had strong criticisms of some HEC programs – especially as they were felt to impinge on what was seen as the autonomy of the university. At the same time most of these critics showed respect and appreciation for the work of the HEC in general. As another vice chancellor put it: “We have been uneasy about some of the PhD and M.Phil criteria. But I have said, if the HEC programs are good ideas, we shouldn’t oppose them. For that I have been branded as an agent of the HEC by some faculty members.”107 Such debate is healthy and important for the future. We note uncertainties we found regarding the governance and location of the HEC. At the time of writing (June 2008), there was a great deal of discussion in government, the press, higher education institutions, and among the public about the status of the HEC – whether it would continue to be an autonomous semi-ministerial institution (similar to a Ministry of Higher Education in many countries), whether it should be placed under the Ministry of Education (now responsible for primary and secondary education), or whether some other governance arrangement should be put in place. We think it would be a major education policy mistake to weaken the autonomy and independence of the HEC, which has served it so well, or to limit its budget freedom which has allowed it to allocate, monitor, and spend its funds so effectively. In our judgment, the current structure and organization should be maintained as it is – as an autonomous Higher Education Commission. While we believe the HEC needs to improve its links with the Ministry of Education and continue its cooperation with the Ministry of Education, the Ministry of Science and Technology, Ministry Health, and Ministry of Food, Agriculture and Livestock, this can be done on a bi-lateral bases or through an inter-ministerial committee or committees. The successful organizational structure of the HEC should remain untouched. Cooperation with other Ministries is already improved over even three years ago and has borne fruit in several joint projects such as the very successful Pakistan-US Cooperative Research Project with the Ministry of Science and Technology. XII. Analysis and Review of the Effectiveness of Monitoring and Evaluation (M&E) Tools The HEC has an extensive system of evaluation and monitoring in place for all its projects. We reviewed that of the Need and Merit-based scholarship program in particular and have commented on it in an earlier chapter. During our evaluation, budget controls and monitoring were noted by USAID as problems as a result of a Control Environment & Risk Assessment. 108 We discussed these findings with Senior HEC management. They were making a concerted 107 Interviews by the evaluation team, May-June 2008. 108 Controller, USAID, “Control Environment & Risk Assessment – Higher Education Commission,” February 4, 2008. Evaluation of USAID Higher Education Portfolio 137 Academy for Educational Development August 29, 2008 effort to respond to all recommendations from the risk assessment and appear to have succeeded. A USAID report on July 22, 2008 indicates they have succeeded.109 Overall, we are impressed by the level of review and control over both the recurring and the development budgets at senior management levels and think the new budget control system will resolve the problems noted. We discussed USAID monitoring with the HEC and members of staff of USAID. There is regular contact between the leadership of the HEC and USAID and the relationship seemed to be a good one. The Risk Assessment, noted above, is an indicator of ongoing monitoring by USAID. XIII. Effectiveness: Achievement of Project Objectives Against Planned Targets and Intermediate Results Increase knowledge and training in higher education (Strategic Objective 3). USAID goals and those of the MTDF for higher education are congruent in their focus on increased knowledge and training, which was one of the three major goals of the HEC. It is best exemplified in several USAID/HEC programs including the Fulbright Program with 420 fellowships to date and 196 selected for 2008 (to which the HEC was a major contributor), the Merit and Need-based Scholarship Program with a total of 906 scholarships, and the Pakistan-US Cooperative Research Project with the Ministry of Science and technology and the American Academy. That effort focused on expanding state-of-the-art research in Pakistan, training of faculty and staff, and expansion of research efforts. Increased infrastructure development of higher education through scholarships, research, and linkages (Strategic Objective 3). The HEC fulfilled this objective through the Fulbright Program, the Merit and Need-based Scholarship Program, and the Pakistan-US Cooperative Research Program. The linkages with 46 US partners are a good example of how infrastructure can be developed through partnerships. What was particularly impressive about these cooperative plans was the mutuality involved in the research with substantial benefits going to both the US and Pakistani partners as spelled out in detail in Chapter 6. Infrastructure development was a central feature of the MTDF of the HEC with a number of their own programs adding to the infrastructure development of USAID supported and administered programs. Deepen the capacity of key social institutions (Investing in People). The HEC MTDF is designed to improve the quality, relevance, and access to higher education. In the last five years, the HEC has succeeded in moving Pakistan from among the worst in higher education to the forefront of change and as such expanded the capacity of higher education in ways that are already making a key contribution to national development, social institutions, and the creation of a knowledge society in Pakistan. Establish roles of key public and private delivery of education services (Investing in People). All three major USAID projects carried out in conjunction with the HEC involved both public and private higher education institutions. In the last few years, with encouragement from 109 USAID, “Status on Control Environment & Risk Assessment (CERA) – Higher Education Commission,” July 22, 2008. A report that notes that previously identified weaknesses have been corrected or resolved and the HEC meets required standards or is in the process of meeting them. Evaluation of USAID Higher Education Portfolio 138 Academy for Educational Development August 29, 2008 USAID and the World Bank, the HEC has moved to assist private higher education recognizing that the state on its own could not meet national higher education needs or demands for access. For the first time, the HEC has provided funding to private institutions for infrastructure development – a program that will be expanded over the coming years. Address the root causes of instability in Pakistan (Strategic Plan 2006). A key to providing stability in Pakistan is increased access to higher education. In this regard the HEC has been far more successful than its own targets of 10% growth a year increasing access by more than 14% per year for the last five years. Part of that success is attributable to the contributions of the USAID/HEC Merit and Need-based Scholarship Program which has allowed more than 900 students from disadvantaged families to attend higher education institutions. As the team learned from interviews, most of these students would not have been able to attend higher education institutions and would have ended up unemployed, or as one of them put it, gone to work as a laborer in Kuwait. Several of the institutions involved made a special effort to attract students from the border areas. The Fulbright Program has also made a contribution to this effort and has been involved with an impressive outreach effort in the border and disadvantaged area reaching more than 40,000 students. Increase access to higher education (IR 3.4). One of three major goals of the MTDF is to increase access to higher education – a goal totally in accord with USAID’s objective IR3.4. Over the last five years access has increased by 89%. This has been done at the same time that quality has been improved. The student-teacher ratio has decreased during this time, curriculum is being revised, the HEC has embarked on major efforts to encourage and reward good teaching, and the first of the Masters and PhD students sent abroad as Fulbrights and through other program are returning to enhance the quality of teaching. They are being eagerly snapped up by universities. In addition, the improved conditions of service at the universities are attracting Pakistani expatriates back to Pakistan and encouraging many to come to Pakistan to teach during their sabbaticals – all contributing to improved conditions for access with high quality. Strengthen and improve education policy making (IR 3.1). The HEC has embarked on a major effort to improve governance in higher education, including the selection of university Vice Chancellors, strengthen requirements for promotion, PhDs, and publications. They have succeeded in implementing these changes. Though not a direct consequence of USAID funding, they reflect the support of USAID and other donors and the influence of American higher education on the kinds of transformation taking place in Pakistan which are very much based on the US model. In addition, since the US give some direct budget support to the GoP, it can take some credit for the more than 350% increase in the higher education budget over the last five years. Improve the growth and development of the economy (Strategic performance goal 5). Part of the impetus for the MTDF is to develop the economy of Pakistan. As noted elsewhere in this report, World Bank research has shown that no nation reaches the stage of a developing nation without a high quality higher education system. That goal has not yet been met in Pakistan, but the efforts of the HEC have put it well on its way to achieving this goal. The HEC efforts focus on quality at every level and in every aspect of their development effort. They are insisting on external review of promotions, hiring, publications, and PhD theses, among others. They have Evaluation of USAID Higher Education Portfolio 139 Academy for Educational Development August 29, 2008 established a quality assurance and accreditation process and will soon have institutional accreditation of public and private universities in place. Pakistan is one of the most improved higher education systems in the developing world. Increased participation, representation, and accountability, and strengthened democratic values (Strategic Objectives 4, 6, & 7). The HEC has been especially insistent on accountability and has demanded it of all its programs. In a few cases, we think its concerns have been excessive (and have so noted). Where implementation has been slow, the HEC has shifted resources to other programs. Where institutions falter, they have been punished. The HEC, which originally operated on a top-down strategy has shifted to a much more consultative approach and decentralized many of its programs. They have also encouraged more democratic governance in Universities and in so doing improved participation and accountability. Democratic values, as we noted elsewhere are likely to become a byproduct of more the more creative, problem solving approach to teaching and learning being encouraged by the HEC. Only time will tell if that outcome is realized, but our visits to campuses throughout the country suggest that things are moving in that direction. Encourage the GoP to spend more on education (Performance Goal 5). USAID and other donor support have encouraged the GoP to increase the higher education budget by 350% of the last five years. Those increases are currently under threat by cuts made by the new coalition government. We hope they will be reversed in the near future because they put both the progress made to date, and the hope of a high quality education system in Pakistan, in jeopardy. XIV. Conclusions • For the first time since 1947, higher education in Pakistan has undergone significant transformation and improvement that was long overdue. Higher education in Pakistan remains well behind international standards; it is behind most of its neighbors in the region. On the other hand, progress has been made. The HEC has put in place a number of impressive programs over the last five years in terms of both the history of higher education innovation in Pakistan and on a comparative level internationally. That progress needs to be sustained. • The HEC focus on quality and quality improvement with international standards as benchmarks has been critical to their success to date. There is a need to continue development of effective policies for quality improvement, access, and relevance including Standards for Institutional Accreditation. • The emphasis on improving faculty quality with a broad range of programs is not only appropriate but critical. The efforts include: Master’s, PhD training abroad and in country; salary increases coupled with accountability; research support; conferences, short courses, start-up funding; tenure track system with external review; higher standards for employment and promotion. In addition, policies need to be continued to attract and retain the best and brightest faculty members by providing good salaries, equipment, funding for research and travel for newly minted PhDs. This makes university teaching, research, and service attractive to young people who might otherwise leave the country or go into other areas of employment. Evaluation of USAID Higher Education Portfolio 140 Academy for Educational Development August 29, 2008 • The introduction of the tenure tack coupled with major increases of salaries has improved the attractiveness of the academic profession to bright young men and women, encouraged them to plan for careers in higher education, attracted many Pakistani expatriates to return, and already shows evidence that it is enhancing the quality of teaching and research. It provides important mechanisms for holding faculty members accountable, and provides rewards for those who are productive, creative, and dedicated. It will be important for the HEC to maintain the high standards set for hiring and promotion and to insist that all faculty members on tenure track have regular professional activities reviews of their teaching, research, and public service. • The HEC has demonstrated flexibility and responsiveness to institutional concerns in the implementation of programs in general. There is a need to continue to encourage buy-in from the universities and degree awarding institutions to HEC policies. The HEC needs to continue its on-going dialogue with the universities and degree granting institutions and strengthen the bonds between them. In the last year they have begun the decentralization and dispersal of authority to institutions and other appropriate organizations. This has been important to success to date and is essential in the long run. • We want to emphasize the importance of high-quality higher education to democratization and political development. The emphasis on creativity, active learning, critical thinking, student outcomes, high-quality research, and a higher education system relevant to the needs of the nation, is already paying dividends in the growing level of confidence of students, faculty members, alumni, and citizens in the higher education system. It would be tragic if the significant progress made to date were blunted or reversed because of lack of badly needed funding or technical assistance. XV. Recommendations In the light of the conclusions drawn from the team’s evaluation of the Higher Education Commission, its portfolio, its budget, and funding prospects, we make the following recommendations: • Continue joint HEC/USAID programs, and expand the Fulbright, the Merit and Need￾based Scholarship, and the Pakistan-United States Science and Technology programs. • Consider three new programs (see also Appendix II for details): o A major new teacher training program for primary and secondary teachers including a major distance education initiative. The program would be inaugurated and piloted at existing teacher training institutions. This program would focus on subject specific training and pedagogy. Too many teachers have very little solid knowledge of any subject area – which is reflected in the poor performance of many of their students in demonstrating subject knowledge. Math, science and technology areas must be emphasized to lay the groundwork for more advanced work. Teachers should also be trained on how to update their own subject matter Evaluation of USAID Higher Education Portfolio 141 Academy for Educational Development August 29, 2008 teaching material on a regular basis. Pedagogy specific modules which include teaching methods to help them impart skills of critical thinking and problems-solving would be included. Practice teaching would also be enhanced and extended from its current short duration of several weeks. The program would include efforts to improve language and communication skills, provide training about measurement and evaluation of students, and help foster a commitment to quality improvement. A new distance education teacher education program, built on the model piloted at existing institutions (noted above), which could reach as many as 5,000 teachers a year. It could be carried out after class hours so that it does not disrupt on-going teaching. Successful completion would result in a diploma (perhaps from a US institution as a major incentive for teachers). This program would include face-to-face teaching carried out at training out-posts – about 60 of which exist at the present time throughout Pakistan. (See Appendix III for details). o Improvement of higher education social sciences. A program designed to significantly improve the quantitative social sciences in Pakistan so that they can make a needed contribution to policy-analysis, policy-making, higher education teaching, service, and research. There are very few people in Pakistan who can carry out policy analysis, conduct quantitative analysis of education data, carryout systematic surveys, analyze economic data, assess costs and risks, and make effective assessments of government policies. This program would 1) send a number of promising PhD students to US universities to work in the social science with the goal of having them return to a center of excellence in the social sciences; 2) bring some of the best social scientists from the US, Europe, and elsewhere to provide two-week seminars in quantitative and policy related social sciences. (See Appendix III for details). o Training for technicians and middle level administrators. Fund the development of a program to provide training for technicians and other mid-level administrators to overcome shortages in these areas and bring this sector up to international standards. There is a critical need to train high quality technicians and specialists to run and service the equipment being purchased in the sciences, quantitative social sciences and technical fields like engineering. Similarly, well-trained mid-level administrators are needed who understand academic programs and are trained in the latest techniques of administration, accounting, purchasing, and have the requisite leadership skills and training to work as colleagues with faculty. (See Appendix III). • Provide higher education scholarship support for people from poor, underdeveloped, and impoverished areas of Pakistan. In addition to scholarships for tuition, room and board, the HEC might consider other strategies for supporting disadvantaged students. Evaluation of USAID Higher Education Portfolio 142 Academy for Educational Development August 29, 2008 • That HEC focus on institutional needs and program requirements including: o To insure that funds for programs such as the Merit and Need-based Scholarships and the Science and Technology Cooperation Project are released in a timely fashion. o To insure that USAID reporting requirements for the Science and Technology Project are carried out not only as a grant requirement but also as a means of documenting program success. o To improving gender equity in all programs including management in the universities and the HEC. o To build stronger links with business and industry, establish incubator programs, and improve relevance of university programs in science, business, engineering, and technology areas. o To identify, hire, and train more high quality mid-level staff at the HEC, especially women. There is need to plan for succession at the HEC with a more active internal training program as well as an effort to identify star talent to groom for leadership positions. • Maintain the autonomy of the HEC, regardless of the governance arrangement it chooses. Much of the success of the HEC is attributable to its autonomy, its control over funding once allocated, its senior administration and staff, and its success in working with higher education institutions, both public and private, throughout Pakistan. We believe a major change in the structure or autonomy of the HEC would be detrimental to the successes in higher education to date and to the long term prospects of higher education in Pakistan. Evaluation of USAID Higher Education Portfolio 143 Academy for Educational Development August 29, 2008