A MULTI-CASE STUDY EVALUATION OF FOUR HIGHER EDUCATION PARTNERSHIPS: Applying the HED Theory of Change in a retrospective regional evaluation in Albania, Croatia, Kosovo, and Macedonia August 2015 AEG-A-00-05-00007-00 DISCLAIMER This publication was produced was prepared independently by Jessica M. Bagdonis and Anne Guison￾Dowdy of Higher Education for Development and Chrystal George-Mwangi of the University of Massachusetts-Amherst. The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Table of Contents SECTION I: INTRODUCTION Background ............................................................................................................................................... 4 Purpose of the Study ................................................................................................................................. 4 Methodology ............................................................................................................................................. 6 Design ..................................................................................................... Error! Bookmark not defined. Instrument ............................................................................................. Error! Bookmark not defined. Participants ............................................................................................ Error! Bookmark not defined. Analysis .................................................................................................. Error! Bookmark not defined. SECTION II: CASE STUDIES ALBANIA ........................................................................................................................................................ 8 Overview ................................................................................................................................................... 8 Findings ..................................................................................................................................................... 8 Context .................................................................................................................................................. 8 Partnership Management ..................................................................................................................... 9 Intended Higher Education Strengthening Outcomes ........................................................................10 Unintended Higher Education Strengthening Outcomes ...................................................................12 Impact Beyond Higher Education .......................................................................................................13 Sustainability of Partnership between Institutions ............................................................................14 CROATIA ......................................................................................................................................................15 Overview .................................................................................................................................................15 Findings ...................................................................................................................................................15 Context ................................................................................................................................................15 Partnership Management ...................................................................................................................16 Intended Higher Education Strengthening Outcomes ........................................................................17 Unintended Higher Education Strengthening Outcomes ...................................................................19 Impact Beyond Higher Education .......................................................................................................20 Sustainability of Partnership between Institutions ............................................................................21 KOSOVO ......................................................................................................................................................22 Overview .................................................................................................................................................22 Context ................................................................................................................................................22 Partnership Management ...................................................................................................................24 Intended Higher Education Strengthening Outcomes ........................................................................26 Unintended Higher Education Strengthening Outcomes ...................................................................29 Impact Beyond Higher Education .......................................................................................................30 Sustainability of Partnership between Institutions ............................................................................31 MACEDONIA ................................................................................................................................................33 Overview .................................................................................................................................................33 Findings ...................................................................................................................................................33 Context ................................................................................................................................................33 Partnership Management ...................................................................................................................35 Intended Higher Education Strengthening Outcomes ........................................................................36 Unintended Higher Education Strengthening Outcomes ...................................................................40 Impact Beyond Higher Education .......................................................................................................43 Sustainability of Partnership between Institutions ............................................................................45 SECTION III: CONCLUSIONS AND RECOMMENDATIONS Stakeholder/Partner Engagement and Perceptions ...............................................................................47 Higher Education Strengthening Outcomes ...........................................................................................47 Impact Beyond Higher Education ...........................................................................................................50 References ..................................................................................................................................................53 SECTION I: INTRODUCTION Background Higher Education for Development (HED) was founded in 1992 by six major U.S. higher education associations1 to advance the engagement of the higher education community in addressing development challenges around the world. HED does this by managing innovative partnerships that join U.S. universities and colleges with higher education institutions in developing countries. HED supports complex, consortia partnerships in a country or region, clusters of partnerships that address a more narrowly defined development challenge, or the traditional partnership involving a single U.S. and single host-county institution of higher education. HED operates with funding from USAID’s Bureau for Economic Growth, Education and Environment, Office of Education (E3/ED) through a Leader with Associates (LWA) cooperative agreement. Under the LWA Cooperative Agreement and at the request of USAID, HED also designs and establishes higher education partnerships through Associate Awards. In such cases, E3/ED may elect to match funds from USAID Missions and Bureaus to catalyze investments. HED’s model is based on its theory of change, which posits that higher education institutions are key to economic growth and the advancement of societies. Higher education contributes to creating new bodies of knowledge and bringing innovative solutions to market, engaging active and emergent leadership, and building a competent workforce. By promoting a culture of continuous learning and improvement within and outside the walls of the institutions, these elements can support policy changes and create enabling environments to facilitate development. Purpose of the Study HED’s theory of change is grounded in an extensive literature review of policy and research papers on higher education and global development and consultation with experts in higher education global engagement. However, it had not been applied to a retrospective evaluation of any of its partnerships. Thus, the purpose of this study was to further test the theory of change by applying it to a retrospective evaluation and to learn about the impact of the partnerships. The HED Theory of Change emphasizes both process and results and posits that higher education institutions are key to economic growth and the advancement of societies. Attaining sustainable human and social development goals through higher education can only be attained, however, by starting with a solid global engagement management foundation. A strong management foundation takes into consideration and applies best practices in strategic planning 1 The six higher education associations include the American Council on Education (ACE), the American Association of Community Colleges (AACC), the American Association of State Colleges and Universities (AASCU), the Association of American Universities (AAU), the Association of Public and Land-grant Universities (APLU), and the National Association of Independent Colleges and Universities (NAICU). and results-based management. With regards to strategic planning a strong foundation reflects having conducted rigorous institutional needs and capacity assessments and appropriately aligning institutional capacity and strengths with pursuits to contribute to solving global development challenges. With regards to results-based management a strong foundation reflects having put into place comprehensive, efficient, and effective project management information system (PMIS); clearly articulated processes that the organization and individuals abide; comprehensive and detailed monitoring and evaluation plans, sustainability planning integrated from project start-up through implementation and post evaluation; and the establishment of collaborative relationships, local ownership and strategies to ensure partnership resiliency. If a solid global engagement management foundation has been put into place, then activities to support higher education strengthening are more likely to be successful. Higher education strengthening includes enhancing the capacity of research, teaching, and extension; organizational and institutional transformation; and growing strategic alliances with other higher education institutions or consortia, the private sector, local civil society organizations, as well as government. By continually making investments to enhance each of these areas, higher education can contribute in the short-term to creating new and shared bodies of knowledge, active global citizens, and a competent workforce. Together, shared knowledge, active citizenship and a competent workforce can contribute to long-term societal impacts such as bringing innovative solutions to market and good governance. Innovative solutions to global development challenges coupled with good governance can support economic growth for sustainable human and social development. By promoting a culture of continuous learning and improvement within and outside the walls of the institutions, each of these elements can support policy changes and create enabling environments to facilitate development. Methodology Higher education partnerships are complex. Measuring the results and evaluating the impact of higher education partnerships contributions to solving global development challenges is mired with complexity. Cause and effect relationships are difficult to measure as a result of many confounding variables. In addition, HED partnerships are only funded for up to three years, thus often not providing adequate passing of time to measure longer-term outcomes and impact. Therefore, we adopted an explanatory case study approach to further test the theory of change by applying it to a retrospective evaluation and to learn about the impact of the partnerships. . In 2013 two teams of two HED staff traveled to Albania, Croatia, Kosovo, and Macedonia to conduct interviews and focus groups with host country partners and stakeholders engaged in past HED partnerships in each of those countries. A total of 40 stakeholders including past and current faculty and teaching staff, former students, current higher education institutional administrative leaders, and other relevant stakeholders were interviewed. Individual interviews and focus groups were conducted using a semi-structured interview guide designed to solicit information about activities and perceived results at each level of the HED Theory of Change. Interview lengths ranged from twenty minutes to over an hour. All individual interviews and focus groups were transcribed. These transcriptions were analyzed using NVivo software as a tool to organize, review, and identify patterns in the data. Merriam’s (2009) constant comparative method of case study analysis, which is based on Glaser & Strauss’ (1967) use of constant comparative in grounded theory, shapes the data analysis. This analytic approach was selected to move the analysis from description to interpretation. It reflects a comprehensive analytic process, which is described in detail below. Although this analysis is described linearly, the process itself was iterative and overlapping, which is a characteristic common of qualitative data analysis (Creswell, 2008; Merriam, 2009; Stake, 1995). For example, while the connstant comparative method suggests a three stage coding process, open and axial coding stages occurred concurrently in order to make adjustments to the analysis as new codes and categories were developed. The data was read multiple times for comparative examination. This included reviewing the interview protocols, interview transcripts, and interviewee demographic data. The approach to open coding was both inductive and deductive. Inductive analysis was used to remain open to new and emerging themes in the data (Stake, 1995). During this early stage of analysis, transcripts were openly coded for “data that strike as interesting, potentially relevant, or important” (Merriam, 2009, p. 178). In reading through transcripts, annotations were made at lines in the text that appeared to describe how partnership and partnership results were developed, maintained, and sustained. A list of open codes was developed from these annotations and corresponding nodes for these codes were created in NVIVO. By identifying sensitizing concepts, a deductive approach was also incorporated (Merriam, 2009). These sensitizing concepts included key terms from the HED Theory of Change Model and interview protocol. The interview protocol guided initial coding in that codes were developed that aligned with major themes in the protocol/interview questions (e.g. relationship building, institutional conditions). After developing an initial list of open codes, the HED Theory of Change was used to compare and contrast codes with the model. The open code list was further refined using language and concepts from the Theory of Change model as applicable. The next stage in the constant comparative method was axial coding (Merriam, 2009), which occurred both during the open coding process and after initial open codes were developed. This stage in the process includes comparing and connecting emerging codes into categories (Merriam, 2009). NVIVO was used to group together data by open code in order to reassemble the data and view patterns and themes within the cases (Merriam, 2009). Through axial coding, the coding system was developed and refined. Open codes were connected to three broad categories that comprised recurrent patterns within the data and reflected the HED Theory of Change to the extent that the data permitted (Merriam, 2009). These three categories are 1) global engagement management 2) higher education strengthening and 3) higher education impacts. See Table 1 for sub-categories within each of these top levels. In NVIVO, categories were organized into hierarchies, moving from general categories at the top (the parent node) to more specific codes (child nodes). Table 1. Top Level Categories and Sub-Categories of Coding Top Level Category Sub-Category Global Engagement Management Partnership Creation Relationship Management between Individuals Partnership Management between Institutions Higher Education Strengthening Organizational and Institutional Systems Development Educational Effectiveness Strategic Partnerships and Alliances Higher Education Impacts Active Citizenship Competent Workforce Shared Knowledge Good Governance Innovation SECTION II: PARTNERSHIP CASES ALBANIA Overview Between August 2008 and September 2012, the University of Hawai’i at Manoa (UH Manoa) and the Agricultural University of Tirana (AUT) worked together to enhance the capacity of AUT to contribute to economic development in Albania by strengthening the teaching, research, and outreach capacity of its Faculty of Economy and Agribusiness (FEA). Specific partnership objectives were to 1. co-develop a state-of-the-art Master's degree curriculum, graduate-level courses, and instructional approaches in agricultural economics; 2. strengthen the applied research and policy analysis capacity of the faculty and students in agricultural economics; and 3. co-develop materials and teaching training modules on data collection and dissemination, farm management, and market and trade policy analysis for agricultural extension and businesses. The partnership received a total of $ 99,994 in funding. Together, UH Manoa and AUT contributed $103,489.82 in cost-share. Findings Context External conditions. Participants explained that a major external condition impacting the partnership was the organization of extension work in Albania. An administrative leader at the host-country institution explained, “Regarding the extension service, in Albania there are several centers for extension services located in different areas in Albania, but within the umbrella of Minister of Agricultures [research/extension centers], while the applied research is done in universities.” Because the Ministry of Agriculture controls extension, partnership goals related to extension were difficult to achieve. The administrative leader went on to explain, “Linkages between university and research centers – extension centers are weak. It has been a problem.” Similarly a lecturer at AUT explained, “We didn’t have direct linkages with these centers…our university, faculty with these centers.” Still, while this disconnect existed between the Ministry and the university, the administrative leader expressed that, “The ministry has been supportive all the time…with people coming here when we organized the workshops and seminars, with the people doing the…with meetings and everything else.” Institutional conditions. Overall, participants expressed that institutional conditions at AUT made a positive impact on partnership implementation and that the administrative leader was particularly instrumental in this area. Additionally, AUT was committed to meeting the infrastructure needs of the partnership. A lecturer, who was heavily involved in the partnership explained, So we arrange[ed] everything and according to project needs. We immediately after the project start[ed] created an office for the project. Appointed an assistant on the Albanian side and we supplied the office with necessary things. An associate professor of economics and agribusiness at AUT during the partnership cited a similar opinion regarding institutional conditions surrounding the partnership, “In addition to the strong personal commitment of the staff, there was a flexible or positive institutional framework behind it [partnership].” A University of Hawaii faculty member, expressed that her institution helped to increase the technology infrastructure and Internet access, while AUT provided additional resources that benefited the partnership, With electronics and Internet, we [U. of Hawaii] were able to purchase them with this electronic database so they [AUT] could access that. We were able to provide that and they were also able to provide some basic infrastructure so that all faculty and perhaps an area where students could have access for Internet, and I think that was really a key thing. That they [AUT] provided an infrastructure. The provision of Internet access, physical spaces and personnel to support the project positively impacted communication between the partners and productivity of partnership implementation. Partnership Management Dynamics. In regards to the dynamics of the partnership, participants described equal and collaborative engagement. An AUT assistant professor reiterated this point, “I would say that we shape[ed] together this project…it was very collaborative as an effort.” The assistant professor explained that throughout partnership development and implementation, both partners remained in consistent communication and together created a set of goals and processes, So there's a process of discussion, all together discussing about setting up the priorities…Since the project began, for each step, they were gathering together. So Hawaii University and our representative together. And they decided for the next time. So it was a process of collaboration. They decide what priorities for everything, for activities together. I can say that it was equal. Participants at AUT and the University of Hawaii expressed that the Albania partners were heavily engaged in the partnership and were clear in their desire for local ownership in the process. One lecturer expressed that it was a “strong and very active relationship during the implementation period of [the] project.” Contributions. AUT contributed a number of infrastructure needs to the partnership including physical space and personnel. On faculty member also explained that, “So our institution [AUT], faculty of economy and agribusiness, brought to the table the willingness to participate and the willingness to be involved, our willingness to learn…especially in the research.” In describing the contributions of the University of Hawaii he went on to explain, “Hawaii brought to the table the research experience, research methodologies, the experience in writing proposals. And generally an experience in scientific writing.” A different faculty member described the University of Hawaii’s contributions this way: “The contribution of Hawaii University was technical, in the technical level. Developing the curricula and assisting us in improving the research projects.” A contribution of both partners was a collaborative spirit and commitment as yet another faculty member explained, What struck me was involvement and capacity building, so the focus was not on delivering outputs simply, but rather working together. I’ve seen many projects that take the shortcut of producing outputs, studies, reports, but without making a lot of effort to involve people, because it is sometimes difficult and challenging. It consumes a lot of time, just the number of humans for example. So what I found extremely useful and special compared to other projects is that there was a strong eagerness to involve and where interest was on the ground, and to do things together, and that’s how we did. Intended Higher Education Strengthening Outcomes Educational offerings. One of the primary outcomes of the partnership was curriculum development and reform. Graduate programs were developed at AUT at both the Masters and PhD level, with an emphasis on applied economics and integrating research methodologies. An AUT faculty member described this impact as the “development of a modern curricula.” In explaining the impact of the partnership on curriculum development a different AUT faculty member stated, Maybe by oursel[ves] we might have changed the curricula, arranged some in it. But the project brought to us a consolidated experience of teaching them curricula…I think without [the] project, maybe we would not have the curricula…that we have now. A USAID agricultural specialist also expressed that the partnership had a major impact on curriculum development, The University of Hawaii brought a new way of doing things with the Agricultural University of Turana and especially in the area that AUT wanted, which is strengthen the curricular, especially the master of science programs. And they helped, and they did in my understanding a very nominative way…And so there was this communication between the Albanian students, Albanian professors, and their counterpart, the American professors from Hawaii University. Now that the partnership has ended, an AUT faculty member explained, “we’re now applying in the Ministry of Agriculture for getting two other branches in our faculty business informatics and applied economics.” As AUT continues to expand its academic offerings, participants expressed a desire to use partnership models and practices to build curriculum for these new programs as well. Additionally, the partnership had a significant impact on teaching methods and student-faculty engagement within the affiliated academic programs. For example, a faculty member from the University of Hawaii explained, What I learned in the U.S., I would show them [AUT faculty]. That’s how we develop courses. That's how we interact with students. Instead of just training the trainer, we actually have the student there. They see our style and actually we were more interactive with students….So we just did that and….I think their willingness to also embrace these things makes a difference. Participants expressed that the partnership influenced AUT faculty to increase their engagement with students in the classroom and beyond the classroom, which traditionally was not a part of their pedagogical practices. An assistant professor at AUT explained, “Here in Agriculture University [there is] something different between the relationship of the student and the professor…We are more open in the discussion with the problems…it is more free to talk.” Participants found that faculty who taught with or were trained by University of Hawaii faculty now get better student feedback, which they attribute to these more interactive teaching methods. Research. A lecturer at AUT described the positive impact of the partnership on building research capacity, “I think that without the AHEED project, those research capacity that we got from the AHEED project would not be in this level that we have now.” A research grant program was developed as a component of the partnership, in which the lecturer described, The faculty members were encouraged to write the proposal for doing research. And then a committee [reviewed] the proposal[s] and they select[ed] five or six…the project funded the mini grants, [which] were 3,000 euro per research proposal. So the second year was the same, and the third year was the same…team[s] who won funding [were] obliged to write the research paper and to participate in [an] international conference. This grant program was particularly helpful as participants expressed that one of the major setbacks in terms of conducting research is lack of financial support/funding. Additionally, the program supported faculty to share their work at conferences and publish in academic journals. As a result of the program, an AUT faculty member explained, Regarding the small grant scheme, we managed to participate in conferences, publish papers, and…a majority of those who did during the project, they continue to do this even afterwards… they managed to conduct research, present and publish. This was something we haven’t done in the past and we say among us that with [this] modest amount of money, we managed to do a lot of things. We have had much larger projects but they have not [had] such an impact. Another major outcome of the partnership was the development of AUT’s own Annual Agricultural Economics Conference. An AUT administrative leader explained, So despite the fact that we’ve been participating in the international conferences outside, and in Budapest, in Frankfurt, in Boston and so on, every year, since the project start[ed], we’ve been organizing our conference here in faculty. So it is the fourth conference that we’re going to organize…So for four consecutive years we’ve been organiz[ing] this conference, supported by the project in the beginning, and three first years, supported by the project, the AHEED project. A professor from UH also commented on the success of this conference, The mere fact they continue this annual conference just to elicit their own faculty work, the student work because we also have student competition. I understand they are partnering with Croatia. So it's a regional conference…so those are the things that very obviously is connecting beyond their own institution. So showing the leadership in more the academic aspect. Strategic alliances. Through the partnership, AUT has developed relationships with other universities in Albania and in the larger region as an AUT lecturer described, “So we have alliances with institution[s] within Albania. And with outside Albania, of course…We’re now participating in a cross-border collaboration…We’[ve] now applied for [the] Tempus project [as] an example in this context.” A lecturer involved in the partnership similarly explained, “Our objective of the works and in our field of interest [is] to collaborate with other institutions and so experiences from this project contributed more in this intensity.” Unintended Higher Education Strengthening Outcomes Extension. Although the partners intended to engage in extension work as part of the project, they experienced challenges in this area due to the organization of extension services in Albania through the Ministry of Agriculture [see External Conditions]. Participants described the extension component of the partnership as, “the weakest spot,” and “not [completed] as effectively as we would wish.” However, participants explained that attempts were made to involve the Ministry by inviting them to conferences and workshops as well as including Ministry officials as co-authors on research proposals and for a book project. Still, as an AUT professor expressed, “we are still struggling to establish this link…between our faculty and Ministry of Agriculture.” The professor futher explained, They [government] moved the research component to the Agriculture University…but they didn’t care too much about supporting this function with budget. And what we thought was that [the] Ministry of Agriculture [would] budget some money for research and channel this money to Agriculture University. We produce results and then it goes to the field… this was nice in theory but it didn’t work so much. Research. As a result of the project, AUT faculty were inspired to develop a research institute called the “Institute of Economic Study and Knowledge Transfer” in which faculty act as research consultants, develop grant proposals and build relationships with external stakeholders such as farmers, the Ministry, and agribusinesses. Additional plans include building partnerships with other universities in the region. A dean at AUT believes “it will be a kind of bridge between a research institution and the universities, and the business [sector].” Institutional reputation. Participants expressed that the partnership had a positive impact on the reputation of AUT and particularly the academic programs associated with the partnership. Engaging in activities such as writing a book, publishing academic papers, and presenting at conferences provided exposure to AUT on regional and international levels. For example, an AUT dean explained that the partnership also helped AUT gain acknowledgement in rankings, Two years ago the Minister for agriculture carried out ranking of universities. The faculty of economy and agribusiness, our faculty within the public universities, is in first place. Of course the partnership helped a lot in this context…in terms of scientific work and research in agriculture, that’s…the main impact. We have maximal grade in publications. And in this context the AHEED project helped a lot. We have the highest points in research and in the national reputation…and in faculty infrastructure. Impact Beyond Higher Education Contributions to local market and workforce development. Participants described the important impact that AUT has on the agricultural sector of Albania. By strengthening AUT and its educational as well as research offerings, the partnership is also making a contribution to the local Albanian market. For example, a faculty member at AUT explained, Agriculture [is] 1/5th [of] the GDP and almost more than half of the employed people are employing in the agriculture sector, almost half still live in the rural areas. So not just agriculture but rural development is a wider concept and is very important for Albanian socioeconomic development, and in that sense the Agricultural University has been crucial during communism and after communism. An administrative leader expressed the impact of graduating AUT agricultural students on the market, In all Albanian-speaking areas there’s only one agricultural university on the faculty of economy and agribusiness – our faculty. This product, our students, are unique in [the] Albanian speaking area and the market…In 2007 we had around 2600 students. And today we have 6000…this product – these students – the market needs this kind of product that we produce. Participants explained that graduates of AUT’s academic programs fill an important labor force need in Albania, particularly in rural agriculture as one lecturer explained, Rural agricultural areas need new professions and with our university is the main contributor in this field…rural areas are facing now with a lot of problems of transition. Every transition country has more or less the same problems about the development of markets, of economy, but especially these problems are greater in rural areas, and I think that our institution is contributing in these difficult areas for development because farms, rural economies, rural areas have necessary new professionals, new people with knowledge, with skill, with competencies, and our university is offering specific fields of study to solving these problems. Policy. Some participants explained that Albanian policy makers have referenced research conducted through the partnership on agriculture and rural development in order to develop economic strategies. Project replication and spin-offs. An AUT faculty member provided a number of examples highlighting replication of partnership approaches in subsequent projects including, an “FOU project and we are more or less applying the same approaches we developed during the project.” Additionally he described, [There] has been a request by the Ministry of Agriculture and Development in Kosovo and Forestry to have an extensive market study on consumer behavior in Kosovo, and they expressed their interest to have me on board, and as they were interested to replicate what we have done in Albania, to replicate in Kosovo. So the spillover effect can be identified, can be observed even on a region scale. Furthermore, participants discussed spin-off projects that occurred as a result of the partnership including engaging in a series of consumer studies, preparing a joint USAID research proposal, and additional research collaborations Sustainability of Partnership between Institutions None of the participants stated that the involvement between AUT and the University of Hawaii was discontinued at the end of the formal partnership. Conversely, one lecturer expressed, “The connections, the network created between us and Hawaii is still working.” In addition to continued engagement between the partners, Albanian participants suggested that although the formal partnership has ended, the capacity building and other gains made continue to be sustained. For example, an AUT faculty member explained that a partner from the University of Hawaii played a significant role in the partnership but due to her collaborative approach, That was the best way we [AUT partners] learned and the best way to become sustainable. And now our partner left Albania, but we continue. Certain things we keep on going, like working with her on an individual basis because there are no more projects to support us, and on certain things we go it on our own. If this is what’s left behind by [the] project, it’s called [a] success story. CROATIA Overview Between November 2001 and November 2004, Montana State University (MSU), the University of Zagreb (ZU), and Osijek University (OU) sought to build on a growing cooperative business culture in Croatia to institutionalize and increase community capacity in the agricultural sector. Specific objectives of the partnership were to 1. develop academic curriculum on cooperative business principles and practices for use at the university level in Croatia; 2. provide training to agricultural economics faculty at the University of Zagreb and Osijek University that focuses on the methods and techniques most applicable to delivering cooperative business course material to university students; and 3. develop an outreach program aimed at improving the strategic planning skills of Croatian agricultural cooperative directors. The partnership received a total of $99,675 in funding. Together, MSU, ZU, and OU contributed $109,775 in cost-share. Additional partners involved in the project included Rocky Mountain Supply Company, Darigold, Inc., Farm Credit Systems Office in Bozeman, Flathead Farmers Universe, and Bozeman Community Food Cooperative. Findings Context External conditions. Croatia becoming a part of the European Union was an external condition that directly and indirectly impacted the partnership. For example, it played a role in the organization of extension work in Croatia, which a University of Zagreb (UZ) professor explained, “extension service by us is organized by [the] government. It was something like obligation ten years ago in our process to be ready to be [a] member of [the] European Union.” Because of this structure, it was difficult for partners to formally engage Croatian farmers or apply their research to practice. The professor went on to state “We [UZ faculty] are not satisfied with extension service in Croatia because we’re on faculty on the university. We work pretty hard on our projects, but results usually stay in booklets or in our rooms.” Furthermore he discussed initial distrust from farmers regarding the university and their research, “Unfortunately people don’t, in the practice…agriculture people, they don’t trust so much formal education.” Conversely, becoming part of the EU had a positive impact on the partnership financially. Montana State University professor, Martin Frick expressed that prior to Croatia’s participation in the EU, the Croatian partners had few financial resources, but, When they became a candidate and then started working toward it, it appeared to me, as someone just as an observer without being able to look at any books or anything, that things became better in terms of resources, monetary resources to do things. Institutional conditions. An administrative leader at UZ expressed that the institutional climate at UZ and the UZ partners were very open to working on an global scale, Today, the world is quite global and the borders are not so important as as they were before. And in order to, to become a member of a broader scientific community in the world, we [UZ] need this international cooperation and this partnership. And of course, I mean United States universities may be quite far away from us…in distance. But we very often are sharing similar tasks and way how we are solving things, and especially in agriculture. Additionally, in terms of UZ institutional leadership, the majority of participants expressed good experiences. A professor described a positive experience with institutional leadership, “I have to say that our former dean, Professor Havranek, is now chief director of agency and she supported very strongly and strictly such co-work and also idea[s] to unlock this project.” Martin Frick also described initial positive support, but explained that with changes in leadership, this support diminished for the partnership, At the dean level we had one dean who was super cooperative and would help speak for us any time, and then the next dean that came in was not – I would consider at least neutral. Wasn't really pro or for our project, for what we were doing…that became a little more cumbersome than at the beginning…they [UZ] change deans every on average four years. So at the beginning things were great and then things were okay. It established a bigger challenge. We got a little more traction out of the Ministry of Agriculture than we did with the administration after the change in leadership at that institution. While the participants generally discussed positive experiences with institutional leaders and climate, they expressed lack of concrete forms of institutional policies or resources for partnership beyond verbal support. Martin Frick explained, “As soon as they [UZ administration] get a grant they're supportive. It's not out of their pocket. They're behind it, not that they are spiritually or enthusiastically behind it or really engaged in it, but they are supportive of it,” However, over time these conditions improved. For example, participants described the development of UZ’s Center for International Relationships, which acted as a source of support for the partnership. This center provided both financial and administrative support. Partnership Management Dynamics. Participants expressed that both partners engaged equally in the partnership. One Croatian student explained that the partnership began through a joint concept by partners from the United States and Croatia, It was really cooperation because there [was an] idea from a professor from MSU, from Martin Frick, and also from [a Croatian] professor here…so they really give some suggest[ions] to both. So it was [an] interactive process, and then also when it [was] introduced the first generation of students always were asked to give some suggestions and what they think it could be useful for improvement. The same question was for our students, the Croatian one, but also for MSU students…so it was really a cooperation. Therefore, there was a high level of engagement of individuals from both the United States and Croatia, which led to the development of a strong relationship between the partners. An administrative leader at UZ found that for the UZ partners, “it’s [a] good relationship and good partnership because I know that they are quite active with MSU and in exchange visits, scientific collaboration…and in developing curricula.” All participants indicated a strong partnership relationship. Contributions. Participants did not discuss tangible contributions of the partners (e.g. physical space, personnel); however, they did suggest that partners contributed skills. For example, a professor discussed having the U.S. partners provide feedback and revisions on their curriculum development proposals due to the U.S. partners’ expertise in this area. A stakeholder in the agricultural sector explained that the Croatian partners contributed their regional connections and networks, “we have been a valuable partner because we have [a] network that can go to every region, even to the very sensitive regions, which are war torn…that gives you a picture of how we established this network of collaboration.” One participant explained that U.S. and host country stakeholders even at the student level brought in differing perspectives that provided ways to teach one another, Croatian [students], but also for MSU students, it's a completely…different way of thinking. Because you know they [U.S. students] are used to thinking extension service because they already have this incorporating in their teaching; we haven’t [Croatian students]. So our [Croatian] students are more theoretically oriented, and now they explain them why it's important, and the way of thinking of solutions when you have some kind of problem. So it was really cooperation. This more micro/student level example highlights that having partners and stakeholders from two different parts of the world offered different worldviews and perspectives that could make a positive and more holistic impact on the development and implementation of the partnership. Intended Higher Education Strengthening Outcomes Educational offerings. Participants explained that two courses were developed as a result of the partnership, a case study in agriculture course and a cooperative business course. A UZ professor elaborated on this curriculum development, They [courses] are completely incorporated…that was actually one [of our] goals to have such courses and I know that very often students, they came from abroad. In [a] few cases, Slovak Republic, Czech Republic, Austria and Hungary…Students pick up this course for credits…just because of [the] possibility also to learn more about extension work, about business decision for farms, preparing of business plans and everything. These courses not only enriched the curriculum at UZ, but also became a draw for students from other countries who were interested in gaining skills and knowledge that these innovative courses could provide. Exchange. A number of exchanges between the U.S. and Croatia occurred as a result of the partnership. This included students and faculty as well as exchanges from the U.S. to Croatia and vice versa. Much of the exchange was part of the two courses added to the UZ curriculum (see intended higher education outcomes – educational offerings), as a UZ professor explained “We are very satisfied because if we compare all [of] our exchange programs, we’re talking about students and professors too… More than 50 percent actually is part of our activities with USA course just because of such synergistic activities.” He went on to express that a number of Croatian scholars also engaged in other exchange activities, “We are satisfied in the last two years, five or six [of] our young researchers and assistants from agricultural economic departments visited MSU.” Additionally, faculty from the United States came to Croatia to teach courses at UZ. As a result of these exchanges, Croatian scholars were exposed to a number of new pedagogical techniques (see unintended higher education outcomes – educational offerings). Martin Frick explained that many students from the United States did exchange visits to Croatia as well through a formal program developed during the partnership, You know kids in Montana who've never been there [Croatia], they’re somewhat isolated. It's cool in a way, but it's a long way from [home]. I mean and actually the kids are thirsty for that experience. So because of my interaction with my Croatian partners, I was able to then foster that on my home turf with the kids and say "You know we ought to go there. It would really be a great experience for you", and they all said it was. So that would be the biggest thing, one of the neatest contributions that they [UZ] provided. You know on our campus we're two hours, roughly an hour and a half north of Yellowstone Park in the middle of you could call it nowhere compared to the rest of the United States. Extension. Although external conditions led to challenges with partnership engagement in extension work, participants described developments in this area. For example, the partners used the academic courses they developed to incorporate engagement opportunities with farmers and cooperatives. A participant explained that as part of the final exam for one of the courses, students were expected to, Target the [agricultural] problems, the main problems…so they [students] were well prepared for this and then they give some kind of solution, some kind of suggestions, and they present this at class…when they finish all analysis. And there were also farmers present at [the] presentation, so he can say if something [i]s not as he think[s] it is. And it was really useful because now he [farmer] said that there are two or three ideas that he get[s] from their [students’] work. So it's really extension service work. Strategic alliances. Throughout the project, participants expressed that the partners were able to establish and/or strengthen relationships with external stakeholders. An administrative leader at UZ explained, “Some of us are developing quite good partnership[s] with the local community.” This included the U.S. embassy in Croatia, the Croatian commercial chambers of commerce, the Croatian government (specifically the Ministry of Agriculture), local farmers, and food and agriculture organizations. One participant, further described the development of new external relationships with as a result of the partnership also impacting opportunities for students, [For example] we have one farmer…he was also at MSU. So these contacts with these people give us opportunity to use them as [connections]. So we now through this good context can take our students to their farms. So it's contact that really useful, so we have lots of fieldwork, students going to do real farm in real life, and I think it's mostly because of this good cooperation because there is a contact established, so it's easier. You just call them because you know that they will understand and they are really open for this cooperation with students. Unintended Higher Education Strengthening Outcomes Educational offerings. One positive, but unintended outcome of the project was professional development regarding teaching methods and pedagogy. One participant explained, “I think the cooperation with MSU helped improv[e] [the] quality of teaching [be]cause now we are focused on small groups and interaction with students and more practical work, not just talking and teaching, but tak[ing] some examples and study[ing] these examples.” Martin Frick, agreed, We worked on methodologies in the classroom as well and what we could do with students other than just plain lecture, [we] did problem solving, discussion, other stuff. So that was an ancillary kind of contribution I think the whole engagement made and that's where they [Croatian faculty] saw some benefit to me giving them some ideas on how they could do that [be]cause they wanted to be a little more progressive in the classroom. A UZ professor explained that this change in engagement between students and faculty and the approach to pedagogy was a direct influence of the partnership, particularly from Croatian faculty interacting with U.S. faculty, I know that sometimes our old professors said to us, “Hey guys, when you come back from the USA, you completely change your way of [ad]dressing and talking to students”…that means we [Croatian faculty] started to work with students like with colleagues. More directly. In the past, by us it was common, you can see on the door in which time you can visit your professor…He never spend time with you on really practical topics. Today it’s common [student and faculty engagement]…that is only influenced from exchanging of colleagues from USA and us. Extension. Participants explained that the work of the partnership encouraged both farmers and the Croatian government to be more supportive of cooperatives. A stakeholder at the Croatian Agricultural Agency, expressed that historically in Croatia farmers saw cooperatives negatively because in the 1960s and 70s they were forced to join them. Yet the presence of the partnership, Opened views and expanded our approach to the farmers. Yes. And I think it helped in this part with the discussions we are often involved in as consultants from in the Ministry of Agriculture. Some of these issues maybe wouldn't even be brought up or discussed if it [partnership didn’t exist]…like farmer education, farmers’ organizations, and cooperative relationship[s]. I think after that [partnership] it was a lot more discussion on cooperatives. Institutional reputation. Participants suggested that the partnership had a number of indirect, but positive impacts on the reputation of UZ at the institution and faculty level, which was due to the recognition that UZ received as a result of engaging in this international partnership. One participant explained, It’s really important to have a partner strong as MSU is. It's really a good thing. For example, if we compare our [agriculture] faculty with other from the University of Zagreb, we are at high ranking because of this international cooperation…In this comparison always there is a[n] impact that's pointed out [that] this international cooperation is really something important. It's University of Zagreb but also Croatia…because when we're speaking about universities we like to say that we are the most important university in this region. So it's important of having a good international cooperation, it's leading in region. An administrative leader at UZ explained that this international partnership helped to give UZ agriculture faculty more recognition and influence among other stakeholders, It [partnership] empowers us in our connections with the government, with university…we actually, as the faculty, are second or third place at the university level according to our international relations. And then you become more [of a] strength factor when you are discussing about the different things at [the] university, they appreciate much more your opinion. Impact Beyond Higher Education Contributions to local market and workforce development. Participants explained that the partnership influenced the development and strengthening of cooperatives in Croatia. A stakeholder in the agricultural sector stated, “So now we have cooperatives in agriculture. I can't say if they are all [a] result of this [partnership]. But in some parts, some of them, yes. I think yes.” Martin Frick similarly expressed that without the partnership, “in all modesty there would've been probably a few less cooperatives in Croatia that exist today.” He also described the partnership helping to give farmers the knowledge about cooperatives that can lead to future cooperative development, I think planting the knowledge and giving them the experience, you know, who knows? Maybe tomorrow someone will start a cooperative that was somehow touched by this project and I mean that sincerely. Maybe it's next year, but these people now know about it, understand what a cooperative could do for them and who knows when they might end up doing something like that? A stakeholder in the agricultural sector expressed that the partnership contributed to both knowledge transfer and policy development at the institutional and societal levels, [Contributions] in knowledge. In research. In new ideas. New projects. New in a transfer of knowledge from university to the field. I think one of major issues in Croatia will be lifelong learning for…professionals in agriculture. Let's say like people from agency. But they need also to keep u with current knowledge. What is going on in any other field that is related to agriculture. Everything is related to agriculture. So we have to keep up with the new things. And we have turn[ed] to your educational institutions…[Additionally] the partnership had an effect on the policies. So the partnership affected the policies here, at the institution and at the agency [level]. So both institutions here, host country institutions, had as a result of this partnership had changes at the policy level. Participants also explained that the partnership had an impact on Croatian workforce development. Many of the Croatian students who participated in the partnerships’ academic offerings are now employed and as one participant described, “Now they are like leaders in their areas, so at Ministry, at local government, at big company, Agrokor, it's the most important company here in Croatia, some cooperatives. So I think they incorporate this new knowledge in their everyday work.” Project replication and spin-offs. Participants explained that there have been attempts to replicate aspects of the partnership in other contexts. For example, an administrative leader at UZ expressed that UZ is trying to implement similar curriculum development and exchange programs with other universities. Another UZ professor also explained that there have been attempts to replicate the format of the partnership’s academic courses in other UZ departments, but financial support for this is lacking and therefore there has not been much success. Additionally, this professor continued to express that he has attempted to establish regional partnerships similar to the U.S. partnership, but he has not succeeded in obtaining financial support for this endeavor. Both Croatian and U.S. members of the partnership have continued to look for additional ways to further collaborate and remain in communication. However, to date those attempts have not been actualized in a formal way, as the professor explained, We are now not only colleagues, we are friends. And we try to exchange our booklets, some lectures. And I’m not sure, but I hope in the next future we will authorize more exchange programs between us by students or we are in better position of using some funds from our university for young researchers Sustainability of Partnership between Institutions Since the partnership ended, there has been some contact and continued commitment between the partners as a UZ professor explained, “I’m thankful because Professor Martin Frick and Professor Douglas Steele supported us and also people from management from MSU, two times covered our cost of our accommodation and transport there [U.S.]…after the project.” However, participants also suggested that there lacks sustained activities between the partners such as formal exchange programs or consistent communication. KOSOVO Overview Between February 2007 and December 2010, Arizona State University (ASU) and University of Pristina (UP) collaborated to transform the relevance of accountancy training offered by the University of Pristina. Specific objectives of the partnership were to 1. establish a new baccalaureate degree in accountancy within a new University of Pristina Department of Accountancy, 2. train new accountancy instructors for Kosovo higher education, and 3. translate key western accountancy text materials. The partnership received a total of $463,080 in funding. Together, ASU and UP contributed xx in cost-share $384,683.43. Additional partners involved in the project included Gateway Community College, CARANA Corporation, and the Society of Certified Accountants and Auditors of Kosovo (SCAAK). Context External conditions. One of the external conditions impacting the partnership was Kosovo’s need for congruency regarding career preparation within the field of accountancy. Prior to the partnership, Kosovo lacked a university-based accounting program or preparation process for accountancy certification. An executive at SCAAK (Society of Certification of Accountants and Auditors of Kosovo) described this context, As a[n] accounting and auditing association, and the only one in Kosovo, we have started a certification program, which the certification program is very long for those who are finishing the university degree. Because in the University of Pristina, the curricula in the university was not in such level that we thought that we need for professional accountants and auditors. The accounting certification was conducted by SCAAK (an NGO) and there lacked a relationship between the university’s academic preparation of future accountants and the accountancy certification process. An ASU professor discussed the indirect impact of the Bologna process on the partnership. Prior to partnership implementation, the University of Pristina (UP) engaged in efforts to become compliant with Bologna. However, the professor described this as, Fraught with disaster and it still is a disaster. Faculty members didn't know how many credits to assign to their course. They were not guided in that process. There were a whole lot of rapid attempts to show that they could conform to Bologna, which may have set back the process of more genuine capacity building and reform at University of Pristina. Because of the need for UP to align their academic process with Bologna, UP students completed a three-year bachelor’s degree program, rather than a four-year program. This created a challenge for partners as they sought to implement an exchange program between UP and ASU. An American project facilitator assigned to UP, explained, Due to the fact that University of Pristina functions based on the Bologna Memorandum, which is basically [a] three-year degree for a bachelor. They [UP students] were missing a number of credits…in order to fulfill requirements set by [the] admission office at Arizona State. One student defined this issue as incompatibility of policies, “so you have on one hand one institution, which has different policies and maybe different…targets as compared to the local one here at Pristina University….Incompatible with the policies over there.” Due to this incompatibility, partners had to spend time and resources to develop a strategy that would align the needs of ASU’s admissions policies with the partnership goals (see Unintended Higher Education Outcomes – Exchanges). Institutional conditions. One institutional condition that impacted the partnership was UP’s context of governance. Specifically, when leadership changed within the university, shifts in priorities and support towards different initiatives also occurred. As an executive at SCAAK explained, The mentality of Kosovo is that if you change the management, all the partnerships will need to start from the beginning… And the [new] management, that are coming in…. have the [perspective that] “I am here and I'm going to change everything.” Because of this, partners discussed worry that the institutional support from leadership might not be sustained throughout the partnership. Stephen Bataldan further stated as a comparison between the United States and Kosovo, These rectors [in Kosovo] have two- or four-year mandates and are renewable once and that's it. So if a rector is there for only four years by the time he's in his third year nobody is really paying much attention to him or her. And so there's always the concern that you have to make adjustments to new leadership and that typically isn't the case here [in the U.S.] because the programs are run at a lower level. You know our [ASU] university partnership programs are not run out of [the president’s] office or some high level official in the vice presidential level. And so there's an unequal kind of status and relationship that we always have to be mindful of and it's part of the centralization of authority that has always been a part of Balkan universities in the post-WWII period. Additionally, institutional policies impacted the partnership. For example, there is a policy at UP that only individuals under the age of 35 can apply to be instructors. An associate at SCAAK, who participated in the Master’s of accountancy program, explained the impact that this policy had on him, In order to get in the process of teaching and being [an] assistant teacher at Pristina University, you need to be younger than 35. It’s now, personally for me, meant that I am not qualifying for this…I would say [it was] the main reason why I couldn’t get through the process. Although I qualified, on the other hand, as my qualifications and as being part of the project. Participants did not specify whether this policy was known during partnership implementation; however, it appeared that all of the students who participated in the program were assured that they would be hired at UP once they returned from ASU. Yet, when they returned, students over the age of 35 were not hired by UP. Another institutional policy acting as a barrier to the partnership was as Stephen Bataldan explained, “the regulation in the University of Pristina and…all the new universities established in Kosovo…that ultimate responsibility for all courses must be in the hands of faculty members with a PhD degree.” He went on to express that the partners “were only marginally aware of the dimensions of this barrier when we launched the project.” Therefore, students who participated in the Masters of accountancy degree program could not be hired to teach undergraduate courses at UP as originally intended. Additionally a senior administrative leader at UP, identified, “maybe main obstacle, institutional obstacle is the small budget.” Participants discussed financial challenges to the partnership in part due to the lack of institutional fiscal resources as well as due to other financial constraints. For example, Stephen Bataldan explained, “if we'd had another $200,000 or $300,000 we probably would've put a program officer in residence.” This lack of staffing was another condition that impacted the partnership, as the USAID activity manager for the partnership described, “ASU had no permanent presence in Kosovo throughout project implementation to manage the complicated relationships.” Infrastructure constraints related to physical space and technology was also present. A US partner expressed, You have to keep always in mind that the infrastructure that they have here is so that they cannot actually offer all the quality and other resources that we can offer to our students in the US. Technology, for example. Technology in classroom. Yes, they [UP faculty] use the same textbook as the professor [of] accountancy use at the Arizona State University, but at the Arizona State University, there is a Blackboard. There is a Canvas. There is an iPad. There is iPhone. There is email. There is interactive tools. Unfortunately that doesn’t exist [at UP] right now. Partnership Management Dynamics. Participants from UP did not provide much description of the partnership dynamics. Conversely, U.S. partners expressed that the dynamic was unequal, with ASU taking on the primary leadership role. An ASU professor described, Obviously, because this was designed to introduce [an] American degree in accountancy into a Kosovo institution, we're obviously setting forth the terms of the partnership and in that sense ASU was probably more dominant…the initiative came from ASU and from the design of the project itself by HED and so it has a distinctly American character about it. A USAID employee explained that the dynamics were, “Overall good…however, there was a somewhat difficult interaction with UP. Throughout project implementation ASU just assumed that UP was doing the work they agreed to, but this was not always the case.” Both a representative from USAID and an ASU professor found that UP did not have equal levels of participation in the partnership with ASU. While an ASU professor expressed that this may be due to the way in which the partnership was developed (via ASU and HED), some of this was also reflective of the change in UP institutional leadership during partnership implementation (see Institutional Conditions). There was a mix of perspectives on the strength of the partnership. Some participants described the partnership as strong, with UP obtaining a number of benefits from working with ASU. For example, an associate at SCAAK stated that during his involvement the relationship was “very strong and very productive,” and a UP faculty member, stated “It is my opinion clearly that the partnership and cooperation with Arizona State University is very useful for us.” However, other participants described the partnership as weak. One student who participated in the accountancy program, A former ASU student explained, “I would characterize it as loose, a very loose relationship, because I didn't have much information. I don't know who was in charge for what part. I couldn't see that someone was leading something.” Lack of communication between partners and during the partnership negatively impacted partnership dynamics. One example of this was miscommunication between ASU and UP regarding implementing the accreditation process for the new UP accounting department that was developed through the partnership. While there were differences in opinion regarding the dynamics of the partnership, students who participated in the accountancy program described acting as a bridge between UP and ASU. A former UP student and now associate at SCAAK explained, During the implementation…we were in direct contact with the dean of the faculty here in Pristina and selected members of the faculty here were involved in the entire process. So we communicated with them in order to harmonize the needs of the university and the needs for the texts and the needs of the University of Pristina with what we can provide from ASU to UP. So it was kind of a two-way communication. And most of or the entire…our [students] engagement there was designed to meet this need. Another student expressed that the students played a primary role in ensuring that the accounting department became accredited, When we [students] got back [to UP from ASU], we found out that the accreditation wasn't taken. They [UP] didn't even apply for accreditation. So then we – all five of us, we gathered all together [and] got all the documentation done…We interacted with all [of the] stakeholders, so we could get the accreditation for the accounting department and the business school. Contributions. Participants primarily described ASU contributions more so in terms of tangible resources and processes and UP contributions in terms of climate/attitude towards the partnership. For example, a former UP student explained, “The contribution of ASU [is] obvious [in] that they provided the opportunity for us [students] to study and to get qualified there.” This opportunity required ASU to contribute a number of resources and organizational infrastructure to the partnership as a project manager from ASU described, [ASU’s] Melikian Center basically hosted the five students at ASU. And they [Melikian Center] were in charge of finding an accommodation for them [students]. They were in charge of making sure that they have all [of] the textbook[s] that they needed. They have the bikes to go from their apartments to the campus…also the staff of Melikian Center helped us [students] logistically and technically To sum up ASU’s contributions, an ASU professor explained, “We trained some outstanding students in accountancy and so there was a skill contribution. The capacity building that [is] associated with the training of these five really great graduate students.” For UP’s contributions, a former student and now an associate at SCAAK expressed, “The UP contribution is in terms of taking the courage to make the change. So the faculty of economics made the change in department organization.” Similarly an ASU professor described, “On the University of Pristina, level their contribution of course was the initial willingness to undergo major curricular reform.” A UP faculty member indicated that, “The University of Pristina didn’t have too much to give to this project but the university was ready to implement [the] project in meaning of offering professors [and] offering classes for courses.” Thus, UP presented an openness to the partnership goals and initiatives, which required the faculty to change their program and course structure. Intended Higher Education Strengthening Outcomes Educational offerings. There were a number of positive, intended outcomes of the partnership related to educational offerings. The primary outcome was the development of an academic degree program in accounting at UP. Some from USAID explained that, “The development of the UP Accountancy curriculum can be entirely attributed to the partnership.” An ASU professor expressed a similar perspective, The program is now a regular part of the University of Pristina faculty of economics. There is now a baccalaureate in accountancy and there was not at the beginning of the project, and students are advancing in that program and the curriculum is the curriculum that was fashioned in collaboration with ASU and the School of Accountancy here. An ASU partner described the development of the accountancy department being the result of the partnership and a number of stakeholders, I would say that one of the greatest outcomes was the fact that the new accountancy department exist[s] right now and functions right now at the University of Pristina. Before this project, that department did not exist. And that would not [have] happened unless we had the support from [the] rector and from [the] faculty of economy. And I must say the same time that the rector had basically pushed, very much put pressure on econ faculties and leadership to make sure that the department is functioning. The other institution that also helped in achieving [the] outcomes is the SCAAK. An ASU professor went on to explain that he sees the curriculum reform reflecting part of a larger transformation at the institution, [The] University of Pristina is undergoing a major kind of transformation and that process had begun under [one UP leader} and it now is really mobilized under [another UP leader] and the rector's office, and all of those changes are associated with the need for capacity building. And so curricular reform is a part of that capacity building process, and…I think we have to credit the University of Pristina with the ability to pull off some of that even given the profile of some of their older scholars who may not be so interested in new blood appointments or curricular reform, but it's happening and it's happening very quickly. Another major contribution of the partnership was the translation of an ASU accounting textbook from English to Albanian that is used as the primary textbook for the UP accounting program. This textbook was translated by the first five UP Masters of accountancy students who participated in an exchange program to ASU. One of these former students explained, We [students] also were involved with translation of some U.S. accounting books, for Kosovo students, so they could use on undergrad studies…That went on for a year after we came back, so we got it done. We translated the books. We put them on CD because printing was quite expensive. It was a good idea to put them on the CD, so all the students could use them. The translation of books went very good, because it was only the five of us [students]. Participants explained that the textbooks are still being used at UP as well as at other institutions in Kosovo. An ASU faculty member stated, “They are still in use. And they’re being used from generation to generation. The curriculum that we established in 2008 or ’09, it’s still working and functioning.” Additionally a UP senior administrative leader found the textbook to be, The best book on accounting. And this book was printed here…and was given to students [for] free. And as I am informed…the book is still being used by students…because it was very complete. I saw the photocopies of this book in different private universities, in private colleges all over Kosovo. An ASU professor explained that the textbook has become even further reaching than Kosovo, The whole generation in the last five years of accountancy students not just in Kosovo, but in Albania and in Macedonia have gotten access to these diskettes that contain five major textbooks in accountancy. We spent a lot of time and effort and even managed to get a waiver of a lot of the royalties from the presses to get this done, and so the impact has now reached well beyond the University of Pristina to those who are in the world of accountancy throughout the Albanian-speaking world. A third education-related contribution of the partnership is alignment between the UP accountancy program and the SCAAK certification process. One accountancy student, Arber Hoti explained, If an accountancy department student finishes all the [requirements and] graduates from that department, then he will be exempt for the first level of certification, which is provided by SCAAK, which is accountancy technician. Because they believe that the level of teaching and the expertise and the materials which are being taught at the University of Pristina economic faculty department of accounting are relevant to what SCAAK requires for the certification. And what SCAAK requires is also in compliance with international educational standards. An associate at SCAAK also described, We have an agreement or the memorandum of understanding with the University of Pristina and department of accounting to have harmonized the curriculum with the professional requirements. So now the bachelors [students] who successfully finish at the department of accounting will not need to take the first four exams of the professional certification process. Participants, particularly the Masters of accountancy students, expressed gaining exposure to new methods of teaching and building their own teaching skills as a result of the partnership. The use of technology was a major teaching tool that students were able to gain experience with in the classroom. Even beyond these students, senior administrative leader at UP explained that technology is now heavily integrated into the classroom experience at UP, which is a direct partnership outcome. Furthermore, a distance learning center was developed through support of the project that, according to this UP administrative leader, Is very useful, because our Ph.D. and master's students many times are attending lecture through this center directly from faculties at ASU. When they [ASU faculty] are offering lectures there for their students, our [UP] students are distance attendees of these lectures…which is a new methodology of teaching for [the] first time in our university. Exchanges. Five students were selected from UP to participate in a year-long program at ASU to complete their Master’s degree in accountancy. During the program, the students translated an accounting textbook from English to Albanian and engaged in teaching opportunities. A senior administrative leader at UP was pleased with skills these students gained as a result of the program, “We sent some of our students over [to ASU], and they came back with more advanced knowledge in field of accountancy, and accounting, and other economics issues.” However, while one UP faculty member also was pleased with the outcomes of the program, he believes the program would have been more successful with a more intentional or rigorous selection process for students. Additionally, a number of UP faculty and senior administrators have also participated in short-term exchange visits to ASU during the partnership period. Extension. Participants explained that an initial needs assessment was conducted during partnership development, in which partners reached out to potential employers of UP accounting graduates and interviewed/surveyed them. An ASU professor further described that this was, To try to identify what future employers of accountants and auditors most needed in Kosovo…and that came up with some very interesting findings about the importance of written work, the importance of basic computer skills and so forth, but in the process we identified the real stakeholders for this project, namely the end users of the baccalaureate graduates of the accountancy program. While this outreach to industry in Kosovo occurred during the initial stages of the partnership, participants explained that further outreach and extension work did not occur later on in partnership implementation. Strategic alliances. According to participants, SCAAK played a major role in partnership implementation. Students who participated in the Masters in accountancy program found SCAAK staff members to be supportive, particularly in assisting them communicate with UP administration. One former student expressed that SCAAK was instrumental in pushing the partners to recruit five students to the program, rather than two or three as initially intended, “SCAAK said that it should be better and the funds should be used to fund more people because SCAAK promotes [the] accountancy and audit profession in Kosovo…and then they wanted to have more professionals in Kosovo.” A current USAID employee found SCAAK’s involvement in the partnership to be positive, describing their involvement as “facilitat[ing] the recruitment of partnership scholarship candidates and facilitat[ing] related conversations with UP.” Participants explained that SCAAK helped mitigate some of the challenges to the partnership by acting as a catalyst in the accreditation process for the UP accounting department and assisting some of the Masters of accountancy students find employment opportunities when they were denied employment at UP. An executive at SCAAK explained, “We are using them [students] as much as they have time to help SCAAK on development, because we have seen that they are really capable of doing and developing everything, but I think [the] university hasn't seen that.” Unintended Higher Education Strengthening Outcomes Educational offerings. The partnership was negatively impacted by a shift in leadership at UP. This led to the students who participated in the Masters of accountancy program not being hired at UP as originally promised during early partnership development. One former student explained, The dean of the faculty, which is not the current dean, and the rector, which is not the current rector, they signed a letter or promised that after – immediately after we [students] came back [to UP from ASU], we will become teaching assistants….so this is like a support from them that, “We guarantee…you’ll be employed at the University of Pristina.” An ASU faculty member described this support changing with new UP leadership, Here in Pristina, I would say the rector at…the University was very, very helpful. And then by the end of the project, there was [a] change in leadership at [the] University of Pristina. And at that time, we could not find [it] very easy to communicate with them [new UP leadership]. Similarly, the SCAAK Director expressed, “None of them [students] was hired by the University of Pristina. I know that the project was supposed to have them for two years…but even that didn't happen…the management of [the] University of Pristina changed and everything collapsed.” One of the five students who participated in the program explained the lack of support for the partnership as a result of the new leadership, “He (new Dean) was so negative about everything we [students] developed. So he simply didn't want us there [at UP].” Exchanges. While the Masters of accountancy exchange program was an intended outcome of the partnership, this program created an unintended outcome as well. None of the students selected for the program met ASU admissions requirements. Therefore, the partners had to develop an alternative strategy to meet the needs of ASU, while also ensuring that the students would be able to participate in the program. This resulted in the development of a “bootcamp,” which an ASU professor described, We negotiated what I think [it] is called provisional or probationary admission for each of the five finalists, and established with his [ASU admissions administrator] approval and payment to him a boot camp for two weeks prior to the regular academic year in which they [student finalists] did nothing but anticipate major kinds of issues that might be coming forward in the first semester of their school work, and they actually I think fulfilled the requirements for one of the prerequisites that was needed. They met for three hours every day. I don't think that it's possible to get Kosovar students into first tier American university graduate programs without that kind of boot camp experience. The total scores sometimes aren't quite there. Institutional policies and processes. Participants explained that UP’s engagement with ASU increased UP’s efficiency regarding decision making processes. For exampled, an ASU faculty member described, In terms of this policy and procedures, I can tell you that there is a pretty tedious procedure if you would like to establish a new department…The process is very long, and I know that in [the] case of accountancy, we [ASU [partners] asked the rector and the leadership to make sure that this process is [a] speedy process. I think in this case, we kind [of] show[ed] them [UP leadership], “Hey, you know, you can do this in [a] much more efficient way.” We show[ed] them you can do this without actually jeopardizing any of your policies. A senior administrative leader at UP also expressed that working with ASU partners and other U.S. stakeholders positively impacted UP management and organization. Impact Beyond Higher Education Contributions to local market and workforce development. Participants explained that students who participated in the Masters in accountancy program have had a positive impact on the accounting industry in Kosovo. One former student stated of himself and his peers, “Our personal commitment of all five of us is to the profession. So we remained all in the profession… So it [partnership] had a positive impact to the community of professionals.” A UP faculty member expressed, “So I think that having five students finishing their master’s studies, it’s a good asset. You spoke about role in society in general and we have those students involved in [the] business field and they give contributions” Some of the students created a company called Arizona Partners of Accounting and Consultancy Ltd. One of them further described this development, The project produced some good, skilled labor force, and brought it here in Kosovo…We [students] started this company, and we are creating new jobs, and this is just the start of the company. We have developed a strategy where we could develop this company into a very important company in Kosovo…. We provide financial, accounting, tax consulting services [and] internal controls to domestic companies and international companies as well. Project replication and spin-offs. One type of project replication that occurred was the development of an accounting program in another institution of higher education in Kosovo. An associate at SCAAK explained, Part of the results of the contribution is that at the second public university in Prizren, there is also a department of accounting there…it was created in 2009…It was [a] result, which was not planned originally, but…the project contributed there…in three years, we managed to have two universities having compatible programs with international standards…So this is what I say [is] the replication Sustainability of Partnership between Institutions After the change in UP institutional leadership and the challenges in getting original goals of the partnership implemented (e.g. hiring of Masters in accountancy students at UP), there was a lag in the relationship between partners. An executive at SCAAK explained, “I think after the change of the management in the University of Pristina something is missing between these two [partners]. It was not like [a] continuation of a relationship.” She went on to explain that because there was no staff person for the partnership in residence at UP, it was hard to maintain a connection between the two partners. Arben Demaku also described a lack of partnership engagement after the accounting program was established, “In department of accounting we didn’t have too much support after we started with department. Maybe it was the role of Arizona State University only to begin with work of [the] department but after that we didn’t have too much support…” However, participants expressed that recently the partners have reestablished communication and a relationship. An executive at SCAAK attributed this to a recent change in UP institutional leadership, “We have rebuilt the partnership this year again with the new management.” Similarly an ASU faculty member described this rekindled partner engagement, It [partnership] does exist [and] the current leadership at University of Pristina is very much interested to expand activities here involving other universities, especially U.S. and also European…We had actually a visit from [the] University of Pristina over at the ASU in January of this year. As a result of that meeting and operation and the history that we have between the ASU and UP we were here in March with another project. And then at that time we also signed this memorandum of agreement between ASU and UP to enable other programs and eventually [to] apply for grants in [an]other partnership that eventually may come through. And I think right now with the leadership at UP, we have pretty good relations. I could not say that with the previous leadership at UP. We did not – we were not able to establish some kind of communication with them, although we tried. An ASU professor also attributed the reestablished relationship to new leadership, The partnership now is at an entirely new level because of the fact that one of the former ASU Sun Devils who was here for a semester in residence program on another project is now the [UP] rector, and he is really a great friend of Arizona State University…So we're at an entirely new level now and it documents I suppose how oscillating or how variable is any kind of an exchange given rather frequent turnover of administrative leadership at the University of Pristina. MACEDONIA Overview Between October 2001 and September 2006, Indiana University (IU) and South East European University (SEEU) worked to develop South East European University as a new multilingual, multiethnic institution offering a Western-style education. Specific partnership objectives were to 1. develop up-to-date curricula, teaching methods, administrative system at SEEU; 2. establish aggressive outreach programs, including a service-learning program to link SEEU students into mutually beneficial relationships with other Macedonians; and 3. give SEEU an independent identify as an Albanian language university within Macedonia The partnership received a total of $2,325,000 in funding. Together, IU and SEEU contributed $968,899.56 in cost-share. Additional partners involved in the project included the Indiana Consortium for International Programs, the Organization for Security and Cooperation in Europe, and Sabre Foundation. Findings Context External conditions. The external conditions in Macedonia impacting the partnership were primarily economic or political. For example, Macedonia was suffering an economic crisis during partnership implementation, which made it difficult for citizens to pay tuition at South Eastern European University (SEEU), as it is considerably higher than tuition other universities within the country. However, as a private institution, SEEU does not receive government subsidies. Exval Rakipi, a former staff member at SEEU described, “The social circumstances in the country, as we have a crisis here. S[EE]U is private-public [with] still very high tuition fees. We’ve experienced that enrollments are dropping… The problem, is financing…people don’t – they cannot afford it.” Similarly, a SEEU administrative leader explained, “every place in the university, now we have [a] decreased number of students. The economic crisis, so families, they cannot pay. On the other side, we don’t have any support from the government.” A SEEU student affairs administrator agreed that the institution suffered due to other local higher education institutions with a lower tuition prices or complete student subsidization, “Yes, it’s fine, you have the quality and the services…but what about money? Who are going to pay the tuition fees when I have the neighborhood [university] for free?...This is a huge challenge for the university.” A SEEU faculty member described the challenge with tuition dependency on the institution, Students go to study [at a public university] for free, almost, or 50 to 100 Euros per semester which is a lot less than what we charge here and so there's this constant issue that we need to look at. Assuming and understanding that 90 percent of our income is based on student tuition. This is a large amount. And so there's not a lot of diversification of our assets. If we don't have students, we have a problem. Macedonia also experienced political conflict prior to and during the partnership, which impacted the goals of partnership stakeholders. For example, one Indiana University (IU) faculty member explained, There was a particular branch of the EU that deals with minority affairs issues and the head of that at the time was [name of individual]…the creation of the university was originally his brainchild, but its focus was really much more of a political one and it was born during the active conflict in Macedonia, and so part of it was to increase access for ethnic Albanians in Macedonia to higher education, but the other part of it was to drain the life out of the University of Pedavo, which was a kind of nationalist radical institution…And so a lot of their focus was basically trying to attract students away and faculty away from University of Pedavo to kind of diminish its impact in radicalizing the ethnic Albanian minority in Macedonia. Some partnership stakeholders were initially unaware of this political climate surrounding the development of the university. An IU administrator went on to state that some of the EU stakeholders, “did sort of hamstring the university in some ways in terms of getting quality programs launched, but on the other hand they had a sort of utility in terms of the national political agenda.” This left some U.S. partners feeling less empowered during the initial stages of the partnership; however, participants explained that this EU involvement waned as the partnership progressed over time. Institutional conditions. In the early phases of the partnership, European Union (EU) stakeholders had a major role in directing the development of SEEU. This created tension between the U.S. partners and the EU stakeholders, but did not have a negative impact on the relationship between U.S. partners and Macedonian partners. An IU administrative leader explained that the EU, Brought a lot of people into the [SEE]U - administration and faculty for essentially political purposes, people who didn't really have the appropriate administrative and academic backgrounds. A lot of faculty were hired based on their political party affiliation. So for us and for our European counterpart faculty, we saw this as a problematic approach. Our focus was really more on building a quality institution…[It] became a sort of political issue between the academics, both European, American, and the EU bureaucracy from Brussels…The [university] administration was very supportive of some of the things that we wanted to introduce…and without an administration that was willing to do that and make those changes, it would've been really hard to accomplish much. Participants indicated mixed opinions about the university’s multi-ethnic composition and it’s impact on the partnership. Exval Rakipi was positive, explaining, “We never had intercultural conflicts or ethnic tensions, relations. And the relationship between staff, Macedonians and Albanians have been always perfect…that’s extremely unique. At least because it started immediately after the war.” However, participants also described initial tensions between ethnic Albanians and ethnic Macedonian students and faculty during the first few years of SEEU’s development, which was directly connected to this larger political issue. There was also a high level of pushback from older, established faculty at SEEU, who were less open to partnership approaches to the university. For example, Exval Rakipi explained that because they lacked English proficiency or proficiency in using technology in their classroom teaching, which were related to two primary initiatives of the partnership, “there were certain professors who didn’t support it [partnership] very much.” Similarly a SEEU administrative leader found, People arguing whether it should be like this, like that. We were very frequently put at a situation between American and our old professors who had some cemented ideas about universities…we had old professors here saying, “Well, we are not sure if these people who come from IU know much about how to set up a university.” Partnership Management Dynamics. Participants described the dynamics between partners evolving over time. Initially, as a SEEU associate professor explained, “Most of the initiatives were from IU…They did the actions. Sort of leading… And then because of the lack of experience from our university and our staff, I think that we [we]re not so very initiative[-driven] in the beginning.” Other participants described this leader-follower dynamic as well. For example, one faculty member who directs the SEEU E-Learning Center expressed, “from the perspective of Indiana University, I think that our University for them was as a child, which grow[s] up in [a] good and healthy environment…Give [them] this basic education as a new institution. Being the mother…” Similarly a SEEU administrative leader expressed, “we [SEEU] had a role then to be disciples….So in some cases we were disciplined disciples and we worked with our mentors…and we produced with joint efforts, with interaction and understanding mutually each other.” Over time, SEEU became more active in their partnership engagement, but the dynamic was never fully equal with the U.S. partners. One SEEU associate professor expressed, Basically it [partnership] was collaborative…most of the time, it was equal. However, the university – the SEE University always relied on IU because of their experience….but, from their side [IU], it was equal. And a partner. However, we always give them preference whenever they start because, of course, their experience. And they have come here to help…But they were very careful not to impose something…[We] started in partnership and then independently [grew] up. However, participants did describe the use of a “train the trainer” model within the partnership, that assisted SEEU in becoming more independent of Indiana University. For example, a SEEU administrative leader explained that IU trained SEEU staff on use of classroom technology tools and then the SEEU staff trained additional SEEU staff. Another SEEU faculty member further explained, “They [IUPI] had some academic staff and administrative staff here training the local staff, so once they leave…[we] could function independently.” Contributions. Participants described the contributions of IU in a number of ways. A SEEU student affairs administrator expressed that IU’s main contributions were “support,” “expertise” and “training” of SEEU staff. One SEEU faculty member found that, “IU brought to the table their experience, so the way they were functioning.” He went on to explain that in terms of information technology (IT) education at SEEU, IU contributed, “all the procedures…all the regulations. So basically we functioned administratively as [a] remote campus of IU.” A SEEU administrative leader also commented that, The main contribution of Indiana…there were core contributions….Curriculum development and learning. But also the other contributions, which were…maybe intangibles…were very important too. They mentored the creation and the foundation, the putting up of our university system… Research, it has been helped [and] also development of faculties has been helped. Participants expressed that SEEU contributed their willingness and openness to change. For example, Exval Rakipi explained, “S[EE]U, from the beginning, brought a different kind of university, not in terms of as a structure…but as a mentality, as an attitude, it’s an approach.” Another SEEU faculty member further described, “SEEU brought into the table something new and putting into question everything they had. And they basically were filtering their regulations…And they also agreed and changed a lot of things.” Similarly a SEEU faculty member stated, We [SEEU] were very open to the dynamics of higher education and the need for reforms. Since we were a new university, we had the benefit of starting with new ideas in reforms in higher education whereas some other institutions are still not moving towards this direction. An IU faculty member found that the SEEU staff members were particularly committed to partnership goals, “[SEE]U recruited some really good people for us to work with…so that was instrumental to our being able to accomplish much in that regard.” Additionally, participants explained that both IU and SEEU partners were highly committed to and invested in the partnership, which was also a major contribution and helped to set the partners on equal footing. A SEEU professor explained, “There was a strong motivation, strong involvement. There was personal involvement to the activities. And this was the main result of the success that we had reached. There was equal involvement from the two parties.” Intended Higher Education Strengthening Outcomes Educational offerings. Curriculum development was a major educational outcome of the partnership. Curriculum development occurred in a number of different programs. An IU faculty member explained, Part of what we tried to do was to help develop a curriculum that was more contemporary with respect to the way business administration is taught elsewhere in Europe and in the U.S…. And then we also really updated the computer science curriculum and the infrastructure there, including a computer literacy program for faculty and students, which, they hadn't had before. A SEEU professor spoke of specific contributions to the curriculum such as a “computer gaming [course], there was not such a course in the region. And we were privileged to have it in our curriculum.” One component of the curriculum development was offering courses in English as an IU administrative leader explained, We established probably the best English language center in the Balkans…Less emphasis on translation and formal grammar instruction, and the faculty took to that approach very well and employed it in the English language instruction that students get. As you know it's a multi-lingual institution. And so even though not 100 percent of the instruction is in English, the students pretty much have to interrelate in English. So that's been a very important part of the identity that the university has developed. The language center is still moving along pretty much as we left it in terms of the level of professionalism, the approach to language instruction. A SEEU faculty member explained that, "moving towards teaching students an English language and students understanding the need for English is very important for the Faculty of Business – these were based on the partnership we had with IU… And I think it was a good idea.” Participants generally shared Luan’s positive perspective on the curriculum development. For example, one former Masters student stated, “In the aspect of the developing and improving the curriculum, the cooperation was very successful.” A SEEU professor shared, I have been involved in all activities of the project that related to IT… It included the reconstruction of the computer science curriculum from three years to four years studies. It included the proposition and the building of the certificate program in business for computer science students and in computer science for business students. So, my general experience with this project was that it was [a] great project and great experience. In addition to revising academic programs’ curriculum, partners also focused on curriculum development in SEEU’s Career Center. Exval Rakipi described, This was to introduce career courses within the curriculum. [We] provided two career courses as electives. And only the first semester we had around 700 students enrolling. So it was one of the most popular elective course[s] that we had at the time. We have two courses. One of them was for junior students…and at a senior level there was career search strategies. Technology integration within educational offerings was another major outcome of the partnership as Exval Rakipi explained, “We went from traditional classroom teaching and students learning by remembering –to interactive teaching using technology at the highest level.” A SEEU professor also expressed the impact of using technology as “a way of teaching and approach to teaching, it was completely something new that we have learned through this collaboration.” This included tools such as videoconferencing for distance education courses and a course management software platform. Vladmir Radevski went on to explain the dynamics around greater implementation of technology at SEEU, particularly the use of course management software among the faculty, The course management system ANGEL was very successfully implemented at the university. [A] big effort was made to overcome the resistance on the professors and assistants, specifically in the fields that are not technological…It was a miracle. But I remember that there were specific training sessions for staff, that our staff from our faculty was supported to disseminate the experience for ANGEL using to other faculties. This was in continuity three or four years. And part of the project elements of the second partnership was this introduction to course management system. And in the same time, introduction to a similar system for a career center. Participants explained that using a “train the trainer” approach was useful in convincing professors to integrate technology. For example, one participant described, “It was like a network, a few people were trained, and then the[y] trained other people, and other people spread it like a real network…and now the whole University applies this learning, uses this learning management system deliberately.” Exchanges. Participants discussed a number of SEEU student, faculty, and administrator exchanges to IU. A SEEU administrative leader explained, “You can see that about 40 of our staff somehow have been educated in United States.” This academic program allowed SEEU students to pursue a Master’s degree from the School of Business or computer science department, which included an exchange to IU. A SEEU professor explained, “In the agreement was stated that our staff will be sent there [IU], will get their degrees, and will come back to work for our university. All staff that went there for Master’s studies returned back with their degrees.” Participants explained that students who participated in this exchange program returned to SEEU with a higher quality skill set and knowledge base. One faculty member described, So those [students] who were sent at the time…they have this capital investment…They have the education from Indiana University, and now they can easily continue. They apply for projects, but it is on their CV, and not just in letter; it is in their –personality, professional beat…They are in terms some are different. Extension. Participants described outreach to local business as a key outcome of the partnership. Exval Rakipi stated, “And so this company/university business relationship was something that I'm quite proud of and we invested a lot of efforts in…and this is the same thing they’re actually still doing.” There were two main forms of extension that occurred as a result of the partnership. The first was through the SEEU Career Center, which an IU administrative leader described as important, Because part of our task was to develop academic programs which had a better school to work fit than was common at…the other institutions in Macedonia, and so that involved outreach to various business entities and communities, and we through the project held the first career counseling fair and first job fair ever held in Macedonia at a university. A SEEU professor also expressed that the SEEU Career Center was critical to building relationships with Macedonian businesses and “nobody in the region, not in Macedonia, have ever heard about career center[s] before we visited the [IU] Career Center, sent our people to be trained there, and opened our Career Center at the [SEE] university. These were pioneering steps.” As a result of this initiative, some local businesses have become engaged in SEEU curriculum development. For example, another SEEU faculty member discussed Ecolog, a company that often hires SEEU graduates, They [Ecolog] were receiving graduates that spoke English, that had some IT skills and some other skills that other universities didn't provide and now this company is working with us to create a new Master's program. So, they're even now heavily involved in the curricula. So, they started out accepting our students the way they were and now they are looking at how we can even better scale our students…So, this is a very American sort of approach to higher education. The second form of extension was the SEEU Business Development Center (BDC), which one faculty member described, “Is an American model. So we use[d the] Indiana University approach on how they do outreach with business community and how they create auxiliary income for the University so we were able to realize that.” Participants explained that the BDC was created in part to generate income for SEEU through philanthropy from local businesses; however, it has not generated much income according to participants. However, having the BDC has had a positive impact on SEEU’s reputation as one former SEEU Masters student expressed, “As a business center, we also [were] able to be recognized by all the business community in the western part of Macedonia, which is a very powerful tool, not just for my whole faculty, but also for our university.” The BDC also provides courses and training programs to the community and staff of local Macedonian businesses focused on areas such as information technology. Additionally, the faculty member explained “Some faculty are now becoming more involved with the Business Development Center to create some applied research. I think, that [is] where we're lacking as a university is research.” Student services. The partnership influenced the development of student services and programming at SEEU. A SEEU student affairs administrator described, “We created registrars. We created bursars and continue with the financial aid and career services, and academic planning.” Offices such as the SEEU Career Center provided job support and career development for students. Exval Rakipi explained how the Career Center directly connected SEEU students to employers via a data management program in which students could upload a career profile for employers to view. A SEEU student affairs administrator further explained that SEEU endeavored to be “supportive to the students” and a “service provider.” He expressed, This is the reason why the students choose our university…Otherwise why to pay, 10 or more times [the] tuition for education? [It is for] the system they get, the support they get. And I hope all the knowledge they get from the university…To me this is direct reflection of the student services. Strategic alliances. Participants described a number of strategic partnerships that were developed during the project. Many of these partnerships represented donor relationships that helped to create student scholarships, fund SEEU institutional projects, and create an endowment fund. A SEEU faculty member described, About half of the money that was used for the establishment of the university came through the grant that you managed and the other half came from other sort of donations…but even then, it was the Americans that were approaching the European colleagues to sort of give some money and [specific SEEU individual] was very important. One partner was the Soros Foundation/Open Society Institute. This organization became a major donor to SEEU. A SEEU student affairs administrator explained that through funding from this organization, SEEU was able to establish a financial aid office as well as student scholarships. While a SEEU faculty member reiterated, “We have some grants from the Soros Foundation and things like that that probably looked at us as being very important, a qualitative university because of the partnership we had with IU. So, that was very important to have.” An IU administrative leader explained that partners also developed a relationship with organization World Learning. World Learning provided training and funding support for establishing the SEEU career center and English language center. However, participants had mixed reactions about the sustainability of these relationships in the long-term as an IU administrative leader described of the Soros Foundation/Open Society Institute, We worked with SEEU to try to get a bit more of the commitment from the Soros Foundation, Open Society Institute…we were really hoping to help them forge a more substantial partnership with the Open Society Institute, but that's just never happened…so the Society Institute wasn't interested in making any bigger commitment than they already made. Yet, participants described mixed opinions on this. A SEEU faculty member explained, “So there were some other donors…that supported the university and some are still continuing today. The Soros Foundation and the Dutch government is continuing to give about 500,000 Euros of money in scholarship money for students.” Additionally, the partnership allowed SEEU to partner with local business such as Ecolog (see section on Intended Higher Education Outcomes – Extension) as well as with other universities in Europe. A SEEU faculty member explained that IU “helped us in creating our own partnerships” by introducing SEEU to other stakeholders. For example, a different SEEU faculty member described that during the second grant period, “There were some other universities…some of them were brought by IU…So we are now working together on different projects, so the network has grown.” An SEEU administrative leader also expressed, “Especially with the reforms we’ve been doing with you…I signed maybe three, four agreements: Turkey, Bulgaria, Albania, Kosovo, all the universities. Regional cooperation. And I think we have an impact, b[y] having our strategic partner Indiana University.” Unintended Higher Education Strengthening Outcomes Educational offerings. There were a number of education-related procedures and processes that were developed at SEEU as a result of partnership engagement. This included course evaluations and curriculum review as one SEEU faculty member explained, “Every year, we do curricular review, something that most universities don't do here. But this is an American approach to the higher education.” SEEU professor described another new curriculum-related process that was adopted, The course offerings are renewed on a regular basis. That is something that we have learned that should be done…it was not something that we were obliged to do, but we loved, the functions like this ….we have learned that the planning should be done in advance, we learned from this project. If we have learned that, this year, you have this set of courses with your offering; next year, you can have five more or you can change three of this, and we have learned from this partnership. Additionally, SEEU originally used ANGEL course management software during partnership implementation; however, this became financially unfeasible. One SEEU faculty explained, “ The entrance of ANGEL changed completely the way, made the significant difference between us and the universities around. And after the project [wa]s finished we were in a situation where we needed to pay on our own ANGEL, which was very expensive. I remember that there was a period that we could cover was for a year or two, but for longer periods, we couldn’t cover the expenses. Similarly, another SEEU professor expressed, ANGEL…it’s not open source; it’s a for-payment system. And when I saw that there is a part of implementation of ANGEL consist[ing] of buying a service…and there is subscription…When we saw the amount of…if you pay subscription on the basis of the number of our students, the price was too high. And I was pretty much sure that when the project will finish and when the university will have to pay for it, we’ll forget about ANGEL. As a solution, SEEU partners developed their own course management platform, which one SEEU faculty member called an “in-house solution.” A different SEEU professor went on to describe, A solution had to be found at the time…as soon as we were working on ANGEL, we put a group of our IT students to study the system and to build a homemade system similar to ANGEL. So while we were learning to work on ANGEL, a part of the students was developing free of charge our system for ANGEL. And interestingly enough, we have now the system. The system is called LIBRI. The LIBRI functions very much same as ANGEL. It is not copied and stolen. There are many functionalities that were not in ANGEL, but the main motivation for building and for using such a system was the experience that we got through this partnership to use a course management system. Exchange. Although the exchange programs developed through the partnership were intentional, the continued “spinoff” exchanges, particularly among faculty since the partnership ended was not intended. An IU faculty member explained, “Because we helped develop a number of the programs, a number of the other faculty who have found their own resources in one way and decided to come to IU because of the sort of historical relationship.” Institutional resources. During the partnership, SEEU asked IU to manage their newly created endowment of one million dollars. An IU administrative leader expressed, “And when the project finally concluded…[SEEU] had accumulated about $1 million in donations and grants from people in the U.S. to support the university and approached us [IU] about managing the funds on SEEU's behalf.” However, because of the economic crisis, their endowment decreased in value. A SEEU administrative leader explained, “So we transferred our money from our income to Indiana University Foundation…later on, of course, it happens, this crisis, you know? Endowment lost a little bit.” Still, IU continues to manage the SEEU endowment as an IU administrative leader expressed, We do have an endowment in the IU Foundation that we manage for Southeast European University, and the main purpose of that really is to support faculty development activities. In our experience doing this many years it's the first time our partner has said, "Gee, we've got some money. Would you take care of it for us and help us use it?"…Then they've used their income from the endowment to fund at least portions of a couple of those fellowships and to fund some additional exchange. Student services. One SEEU student-related event that became established as a result of the partnership was a graduation ceremony. Macedonian partners saw this ceremony at IU and as one SEEU faculty member expressed, No other university was doing such a graduation ceremony in Macedonia, in the region. And now, every university [is] trying to copy us on how we do the graduation ceremony…We have about 3,000 to 5,000 people every year. This is the biggest marketing tool that we have and it's a wonderful experience. An SEEU administrative leader agreed, “Those graduation ceremonies…I think that it’s pretty good marketing. They [students] are coming with parents. They can see our campus, and always we are inviting some speakers special, especially from [the] United States.” Institutional reputation. Participants described the positive impact that the partnership had on SEEU’s institutional reputation. A SEEU administrative leader explained, People were amazed, but not only locals, internationals. When they came here [SEEU] and they saw, they were amazed, the standards that we were using. And [the] introduction of these standards, to a large extent, can be attributed to this project…to the mentoring institution, to Indiana University. A major goal of the partnership was to have SEEU become a high quality institution, which a SEEU student affairs administrator expressed, “make[s] a distinct competitive advantage for [SEEU] compared to other universities and institutions.” Many of the partnership initiatives including technology integration, increased student services, and a multi-ethnic campus population contributed to the positive reputation as a SEEU associate professor described, The technical support and the support for students increased rapidly at that time. And something new which was not in the region. And so the level of services and different services was quite unique at that time for the country. And that helped the [SEE] university a lot in creating an image in the public that the university is offering more services to students and…it gave a unique opportunity for people to study at that time. And to have communication, especially with the multicultural approach. Exval Rakipi agreed explaining, “English and the computer skills….these are two things that S[EE]U has distinguished itself from –any other university in the country.” An IU faculty member emphasized the IT initiatives’ impact on SEEU’s reputation and how this has led to more institutional support for these initiatives, The information technology environment, they've [SEEU administration] just come to accept how critical that is to the institution's mission and how much it distinguishes them and it sets them apart from other institutions in the Balkans and so they've continued to support in it and invest in it. Participants explained that without the partnership, SEEU would not have developed such innovative practices in terms of curriculum and services the IU faculty member explained, “I think the care of the institution would've been a lot different and not as cosmopolitan and not as unique as it is within Macedonia. There are relatively few other institutions like it in the Balkans.” Another SEEU faculty member agreed, explaining that the partnership gave SEEU more respect among external stakeholders and without it, would not have received their recognition, The partnership was very important for the recognition of our university as an international one so that our European partners would sign agreements with us. They wouldn't in the beginning. I think that if we didn't have such a partnership, we would be stuck with a local sort of approach to higher education…We would not have a good ranking. This same SEEU faculty member explained that SEEU is ranked number two out of Macedonian universities on the Shanghai Jiao Tong University Index Ranking, which she attributes to partnership influence. She went on to express, “I don't think we would have this international collaboration that we have with other universities. So, it has been the major push of quality for our university.” A SEEU administrative leader also acknowledged SEEU’s ranking and attributed it to the partnership. He described SEEU as “a trendsetter university” and a “third generation university… a piece of U.S. or EU in the middle of the Volgas…A modern university….which is oriented more outwardly than inwardly.” Exval Rakipi expressed, “I actually believe that S[EE]U is a model in the region for higher education.” However, although the partnership has improved SEEU’s reputation as an institution, participants expressed that SEEU has not gained significant recognition locally or from the Macedonian government. The SEEU administrative leader explained, After 12 years of existence, we are still among the only two institutions in the country that are operating with mixed nationalities in one classroom or in one institution…[yet] we have not been afforded or recognized by society in any formal way… Impact Beyond Higher Education Contributions to local market and workforce development. An IU administrative leader expressed that SEEU’s curriculum provides students with quality workforce skills that then strengthen the competitiveness of the Macedonian economy, The country as a whole is going to find it hard to attract foreign investment unless they have a workforce that has the kind of skills that in particular Western Europe needs…A more contemporary curriculum and informatics and business management in entrepreneurship that improves the school to work fit is a benefit to the country as a whole, and I think [SEE]U has been able [to as] an innovator introduce some of those ideas to other institutions in Macedonia. Through the creation of the Career Center and Business Development Center, the partnership provided opportunities for local businesses to build relationships with SEEU, receive professional development training, and have a platform to recruit employees. These partnership initiatives benefitted both the businesses and SEEU. Exval Rakipi explained, So we brought companies in [to SEEU] every week to explain recruitment procedures, how they employed people, what they are looking for in people. A lot of these companies used this to promote themselves, but for us it was still okay, because it gave us a chance to for them to come to [the] university, see the university, talk to our students. And they came in, came afterwards, [as] participants in the career fair. They were bringing actual active recruiters at certain points. Former Masters students who participated in SEEU academic programs described the advantages they possessed in terms of skills and workforce preparation, “when [the students] got back [from IU], immediately we felt the system, and it help[ed] us understand it much better. And…all of us got strong, good positions…we were immediately advanced in our position[s]…it did impact our careers strongly.” A student affairs administrator at SEEU explained of SEEU and workforce development, “Because of SEEU diploma…[we have] graduates with different profiles, with different aspect of being more open and competencies. And skills linked with [what] a labor market requires.” Student gained skills in information technology and English, which made them competitive in the workforce. Additionally, a SEEU associate professor expressed, “It gave a unique opportunity for people…to have communication, especially with the multicultural approach…. So it joined together also intercultur[al] negotiations.” Overall, one SEEU professor finds that the partnership has made a major contribution to workforce development in Macedonia, In Macedonia…you will see announcements for job positions, they say, “Okay, we are asking for a business administration to be tra, la, la. But we are not accepting people from this and this university." SEE university was never on these lists. It's…well accepted. So this is [a] great contribution. Some of the business partners will say, “Okay, when I see your students, these are the only students that can write in English, that can express – that have skills that the other students doesn’t have.” This is [a] unique contribution. Project replication and spin-offs. Participants noted that many of the SEEU services and programs developed during the partnership are now being replicated at other universities in the region, which an SEEU student affairs administrator refers to as “copy-pasting.” He explained that other universities are, Trying to implement it in our way and then copy-pasting. But imagine if somebody is trying to copy-paste from us then we are happy, we are proud of that. If they see [in] us an idea [that is] good. You will see in a lot of other universities, they developed [our model] in terms of services. It is important to note partnership project replication did not always include partnership stakeholders. However, in some cases, partners were involved in replicating SEEU partnership activities at other universities. For example, one SEEU professor explained, We had various other institutions and universities with which we were trying to develop partnerships as we developed the partnership between Indiana and this university. There was the French university of Nantes that tried to build a master program in public administration. They had a head of project sitting at our campus. SEEU Career Center activities were often replicated at other Macedonian universities as a SEEU administrative leader noted, We were taught by Indiana [how to] organize [an] open house or open day. And like three or four years after we started organizing, we had open days in all of the institutions in Macedonia. We were taught through Indiana, by the USAID grant how to organize a career office…And nowadays, career offices…and career fairs are organized, in all the country, each of the universities. This led to the eventual development of a national career counseling association in Macedonia, which is a cooperative effort between Macedonian universities and Macedonian businesses. Participants also described the development of a new initiative between IU and SEEU to create greater opportunities for faculty and student exchange. One SEEU faculty member explained, During our 10 year anniversary, we communicated with IU and they were willing to sign an MOU – a Memorandum of Understanding…it’s not specific with any financial details but it's just an understanding between two partners on potential mobility of students and staff in the future with no commitments…And the new rector is now in IU today with a new provost and we're looking at ways to fund some new staff members, some academic staff members, to be part of each faculty to reintroduce an international experience for our students and all the faculties. However, there's no funding for this. Other spin-off projects include plans to develop additional SEEU Masters programs (e.g. a project management program) using the partnership model, the creation of SEEU’s in-house course management system (LIBRI) and the development of a distance-learning center. Sustainability of Partnership between Institutions There were mixed perspectives from participants regarding partnership sustainability. Original projects developed through the partnership (e.g. SEEU student services, SEEU career center) are “stagnating” since the partnership ended according to Exval Rakipi. Conversely, a SEEU student affairs administrator shared that since the partnership ended, some of these initiatives are operating without a need for formal assistance from IU, which led to less communication between the partners. However, he explained that this is, “A negative aspect because as a project, this ends. And, in the certain period…you still have a need of support…or cooperation or getting new ideas.” Initially, after the partnership ended there was no major communication between the partners. A SEEU faculty member described, “After the partnership closed, immediately after, we had a new grant. But even after the new grant closed, there was a couple of year period where there was no real sort of interaction.” Additionally, a SEEU administrative leader explained, “I’m afraid that I must conclude that we have not used entirely the opportunities of this friendship and this collaboration, this experience, to build more tight links even after the project.” However, participants expressed that informal relationships between individuals participating in the project continued to exist once the formal partnership closed. It can be noted that in recent years the partners reconnected as an IU administrative leader expressed, We have an MOU with them [SEEU] to collaborate. We are managing their endowment on their behalf. We have had a member of the governing board, since about 2005 or 2006 and I served in that capacity until last year and now our vice president David Cera is filling that position. So we actually have a membership on the [SEEU] governing board and when they had their tenth anniversary celebration they gave an honorary doctorate to IU's president, Michael McRobby, recognition of the relationship…there's the formal connection between our having a voting member on their governing board and managing their endowment and then the number of faculty who've had, advanced study here at IU who are back teaching. So those are kind of the three pillars of the continuing relationship. SECTION III: CONCLUSIONS AND RECOMMENDATIONS Stakeholder/Partner Engagement and Perceptions While HED requires partners to develop a proposal outlining goals as well as a final report outlining outcomes, it is not required that all partnership stakeholders contribute to these documents. Because of this, these reports may not give voice to differing opinions, evolving relationships, or unintended results of the partnerships. However, each of these factors may provide important lessons that can be gleaned from the partnerships and their lack of acknowledgment could result in overlooking key partnership processes and outcomes. To mitigate this issue, this report considers the diverse range of stakeholders engaging in the partnership in order to acknowledge the variety of voices that participated in the projects. The data collected on the involvement and perceptions of these various partnership stakeholders provides a more holistic understanding of the partnership context, engagement, and outcomes. During our analysis, we compared different partnership factors across the project proposal document, final report document, and interview transcripts. We found a number of inconsistencies between proposals/reports and the interview transcripts. For example, while a final report might not describe the achievement of an outcome related to curriculum development, we would find evidence in the interviews that this type of outcome was achieved during the partnership. Thus, using multiple sources of data provided a means of triangulation during our analysis that illustrated a richer understanding of the projects and increased the quality of our findings and analysis. Our findings suggest that it is critical that each stakeholder should be fully engaged in the project, meaning that they are each aware of the project objectives, processes, and outcomes. If a stakeholder is not involved or aware of each of these factors, their perceptions and understanding of the partnership may be quite limited. In order to address this issue, HED has since developed a number of protocols to ensure greater project engagement across stakeholders that are involved in the partnership. Higher Education Strengthening Outcomes Higher education strengthening refers to capacity building within the host country higher education institutions. Stakeholders addressed higher education strengthening in diverse ways; however, across partnerships, there was emphasis related to teaching, research, and extension. For example, shifts in teaching methods and the development of new types of curricula were primary themes related to the development of undergraduate and graduate programs at the host institutions. When discussing teaching, both U.S. and host country stakeholders across partnerships described improvements as a shift from rote learning and lectures to greater engagement between students and faculty in the classroom as well curriculum designed with a clearer connection to market needs. Words like “modern;” “contemporary;” “applied;” “practical;” “interactive;” and “progressive;” were used repeatedly by Eastern European stakeholders to describe the changes and reform efforts related to educational offerings. U.S. teaching and curricular methods were described as best practices and models for this shift. Improved educational effectiveness also appeared to have a strong link to institutional development in terms of human resources (e.g. faculty development), access to material resources (e.g. textbooks), and improved technology resources/infrastructure. While some partnerships focused on change primarily within the program/department that was directly linked to the partnership, it is important to note that there were instances where these practices flowed over into other programs, departments, or even other institutions within a host country. The Theory of Change Model captures this spillover effect as the development of a “culture of continuous learning and innovation” and “evolution of cultural and social capital.” For example, in Albania, stakeholders used the development of a graduate program through the partnership as a springboard for developing similar programs in other academic areas. Positive engagement in the partnership (Theory of Change level one) could provide Eastern European stakeholders with the skills, resources, networks (capital) and predisposition towards additional capacity building beyond the partnership and single institution (e.g. moving from Theory of Change level two – Higher Education Strengthening to level three – Higher Education Impact). This combined with the changing nature of higher education in Eastern Europe allows for what a number of stakeholders described as an opportunity for sustained growth and reform within the higher education sector (movement into level four of the Theory of Change). Improved research capacity was also present, although the emphasis on this area of strengthening varied more across partnerships than educational effectiveness, with Albania partner stakeholders engaging in the most discussion about this area of strengthening. Additionally, this area appeared to reflect research engagement of individual stakeholders (particularly between U.S. and host country stakeholders) more so than at a program, department, or institution level. Stakeholders often discussed their own individual conference presentations and published books/articles. Thus, while educational effectiveness regarding teaching and curriculum appeared to have a diverse impact on institutional development, stakeholders linked research capacity building primarily to improved reputation and recognition among individual faculty at the host institution and subsequently increased faculty development. For example, one faculty member from Albania explained that since the partnership, “We are well-known now as good researchers –we have a economic faculty in Tirana and they recognize that we are more skillful in researching than they are.” Research strengthening was also linked to the development of strategic alliances, often with other higher education institutions locally and internationally as well as with the local government and NGOs. In Albania, host country stakeholders hold an annual conference to showcase faculty and student work, which led to research collaborations with other higher education institutions in the region. Thus, these alliances led to the development of spin-off research projects, consulting work, publications, and the development of research centers, institutes, and/or new academic departments/programs. Patterns in the data illustrated partnership research engagement fostering institutional development and strategic alliances, which in turn led to greater opportunities for additional research engagement even beyond the host country higher education institution (moving into level three of the Theory of Change – Higher Education Impact). Extension and outreach emphasized non-formal education within local communities. Across partnerships, this form of strengthening appeared to be the most complex, as it required engagement and alliances both within and beyond the walls of the host country higher education institution. For example, while some stakeholders described success in teaching research related activities, they found extension work challenging due to government stakeholders and policies acting as barriers. In order to engage in extension work effectively, stakeholders across partnerships described the need for these alliances to be developed at the initial stages of partnership development, which is different from forging alliances as result of research engagement later in the partnerships. Extension work also required a strong awareness of the characteristics and needs of local communities. Thus, partners discussed the importance of strong partnership development and initial strategizing in order to have success in extension and outreach, which reflects the need for strength at level one of the Theory of Change before moving into level 2. While some partnerships struggled with their extension work, stakeholders also described this work as positive in the development of strategic alliances with government, NGOs, and local organizations/communities. For example, in discussing Croatian farmers who participated in a training program, one stakeholder expressed, So th[ese] contacts with these people [farmers] give us opportunity to use…this good context [and take] our students to their farms. So it's contact that it's really useful, so we have lots of fieldwork, students going to do real farm in real life, and I think it's mostly because of this good cooperation because there is a contact established, so it's easier. You just call them because you know that they will understand and they are really open for this cooperation. Thus, extension work was also tied back to both educational effectiveness and research in that the community links partners made could create opportunities for student internships and other practical experiences as well as research projects. In addition to teaching, research, and extension, stakeholders discussed higher education strengthening as institutional development. Specifically they discussed:  The development of academic/research centers, departments or institutes  Qualitative or quantitative changes in host country higher education faculty or staff as a result of the partnership. For example, hiring additional personnel or staff training.  Impact of partnership on institutional policies, processes (such as processes for curriculum review), and strategies for creating institutional systems  Impact of the partnership on institutional reputation, which can include prestige/reputation￾building, university rankings, and local/national/international recognition.  Impact of partnership on institutional governing boards, senior leadership and administration.  Partnership impact on procurement or development of material and technical resources, which included funding, supplies, and technology.  Impact of partnership on services offered to students such as academic advising and career development. Institutional development was often discussed a mechanism for improving teaching, research and/or extension work rather than as a standalone objective or outcome of the partnership. As aforementioned, faculty development was linked to the improvement to teaching and research. Similarly institutional reputation was perceived to be an outcome and driver of research engagement. However, while both host country and U.S. stakeholders discussed institutional development as a component of higher education strengthening, the host country stakeholders often discussed it in more detail. This may relate back to how partnership motivations and benefits were perceived by different stakeholders. In this case, host country stakeholders having a greater interest in institutional capacity building as an objective connected to larger international development outcomes. Lastly, strategic alliances were emphasized as a component of higher education strengthening. Strategic alliances could be formed through increased reputation and awareness of host country institutions (institutional development); however, as aforementioned they were also developed intentionally at the onset of partnership development. In the Kosovo partnership, “the stakeholders in a broad sense were identified at the very earliest needs assessment stage and were the government and private sector.” Yet, even with early planning, external conditions could challenge the development of strategic alliances. For example, most of the partners described experiencing tensions that existed between host country government agencies and host country universities, at least initially due to a lack of prior collaboration. However, high levels of partnership collaboration and equal power dynamic/local ownership appeared to also foster openness to involving external stakeholders. In addition, across partnerships, strategic alliances allowed partnerships to have a more lasting impact through spin-off projects, project replication, professional opportunities for students (workforce development), and policy change/reform. Thus development of strategic alliances explicitly highlights the connection of higher education strengthening (second level of the Theory of Change) to level one (global engagement management) because of its emphasis on partner dynamics and contributions, as well as to level three (higher education impact) regarding the impact of the alliances in supporting both short- and long-term outcomes in the host country. Impact Beyond Higher Education The development of institutional capacity building at host country institutions provided stakeholders with the ability to also make contributions to larger society within host countries in both the short and long-term. The Theory of Change model refers to this level of engagement as higher education impact. However, as partners described higher education impact, they often did so in conjunction with descriptions of higher education strengthening, which suggests that the two are interdependent. For example, partners discussed the creation of a highly skilled workforce as being driven by a strong academic curriculum. Similarly, partners identified the implementation of innovative practices in local communities stemming from faculty development related to improving research skills and extension work. Therefore, stakeholders perceived the ability to make a positive societal impact through the partnership as being linked to a strong host country institution. At the same time, when the work of the universities was not recognized by powerful external entities in the host society, the reach of the partnership was limited. A Macedonian partner expressed, Th[ere] is an inefficient administration, inefficient and lower-level political leaders and social leaders. After 12 years of existence, we are still among the only two institutions in the country that are operating with mixed nationalities in one classroom or in one institution. We have not been afforded or recognized by society in any formal way, but there is strong contribution to the everyday life, to the reality, and to the state of the country and of the society. But what I would like to see even more is that the Ministry of Education takes this example into consideration. But this is not done. That is why I’m saying the impact is half of the potential of what should be. Regarding short-term outcomes, partners discussed workforce development and building a competent workforce most often. One stakeholder in the Albania partnership described, Our students, our product are unique. In all Albanian-speaking areas there’s only one agricultural university on the faculty of economy and agribusiness – our faculty. And the students – I mean, this product, our students, are unique in Albanian speaking area and the market. Which has – this product – these students – the market needs this kind of product that we produce. Likewise, a stakeholder in the Croatian partnership explained, At the beginning of this project there were…[students] going there at MSU, so now they are learning about cooperative business and now they are like leaders in their areas, so at ministry, at local government, at big company, Agrokor, it's really the most important company here in Croatia. So I think they [students] also incorporate this, new knowledge in their work, everyday work. This development of a competent workforce was typically tied to the development of greater educational effectiveness at the host country institution, research engagement opportunities students had due to the partnership, and strategic alliances built with local organizations/communities that allowed students to get hands on experiences. Stakeholders also expressed an outcome of the partnership relating to active citizenship of the host country institution and its members. At the institution level, host country institutions became more involved in acting as a resource in local communities and at the individual level, participants described a greater commitment to their communities and professions. However, active citizenship was discussed much less than workforce development. This may be because active citizenship requires greater connection back to extension work and strategic alliances with local organizations, which was a weaker facet of the Eastern European partnerships. Shared knowledge is another outcome of the partnership, which was discussed in two primary ways. First was the application of research to practice, described by a Croatian partner in Croatia, “In knowledge, in research, in new ideas…in a transfer of, knowledge from university to the field…People from agency need to keep up with current knowledge…And we have [to] turn to your basic educational institutions, so [that’s] our faculty of agriculture here.” The second way was in project replication and spin-off projects. In Albania, one stakeholder explained, We are considering to extend this new knowledge to Kosovo and there has been a request by the Ministry of Agricultural and Development in Kosovo and Forestry to have an extensive market study on consumer behavior in Kosovo…and as they were interested to replicate what we have done in Albania, to replicate in Kosovo. So the spillover effect can be identified, can be observed even on a region scale. This form of shared knowledge was also discussed alongside partnership and relationship sustainability as it related back to the maintenance of stakeholder communication and a continuation of project/partnership activities. Shared knowledge could also contribute back to higher education strengthening as it created opportunities for reinforcing research skills, building institutional reputation, and strengthening/creating strategic alliances. In addition to direct human capacity building through workforce development, a number of stakeholders expressed that the partnerships created new products and processes, which impacted the larger host society as a form of long-term innovation. For example in Macedonia, as a result of the partnership, the host county university developed career fairs and open house events as well as career offices as a type of student service. However, this has trickled down to become institutionalized across higher education institutions in Macedonia. One stakeholder involved in the partnership explained, “Nowadays, career offices are even in the law and [back] then, they were as good as non-existing. Not anywhere. So now career fairs are organized in all the country, each of the universities.” It is important to note that this type of innovation often required buy-in from additional stakeholders and strategic alliances beyond that of the host country partner and U.S. partner. In Macedonia, buy-in from the business community was essential to the success of their impact on the higher education sector within the region. The Macedonian example also illustrates the indirect impact of the partnership outcomes and innovations on local communities as one stakeholder explained, In terms of community…a whole new culture was brought here. Here in Tetovo, and it was a small city with a few high schools only and CU University has brought to this community a completely new brand. Now when they say data, well, they say the CU University. And we have youth from the region coming here. We have the life and data in the communities, and due to the students who come the city is much changed. It’s, , the life in the city, to the community a lot of students, professors coming from abroad. An additional long-term outcome of the partnership was good governance, which was discussed less often than many of the other outcomes. This may relate back to the political context of Eastern Europe and the struggles many partners faced in working with government bodies in the host countries. However, there were instances of positive outcomes in this area as well. In the Albania partnership, policymakers have referenced research conducted through the partnership on agriculture and rural development in order to develop economic strategies. Partners did work hard and intentionally to involve the Ministry in the Albania partnership activities even through setbacks and barriers, which may indicate the need for resilience and perseverance in achieving this particular partnership outcome given the political climate of the region. The Theory of Change model illustrates that as partners achieve higher education impact, they will begin to move into sustainable human and social development. However, when partnership stakeholders were asked to comment on the feasibility of partnerships achieving this level of development, the results were mixed. For example, at the time of the stakeholder interviews, many of the participants did not feel as if enough time had passed since the formal partnership closing to assess whether their partnership had made this sort of impact. Additionally, some stakeholders were more interested in specific goals or outcomes that were more targeted at the institution level and so did not emphasize this type of outcome. This may relate back to the fact that not all stakeholders interviewed were involved in the larger goals of the partnership, but instead included faculty/staff who were working in specific areas of the projects or that were students and local individuals who perceived themselves to be recipients of partnership outcomes rather than participants in the partnership. Therefore, while these stakeholders believed sustainable and human development to be positive and important outcomes of a partnership, they often did not feel capable of assessing whether the partnership was able to achieve this objective. References Creswell, J. W. (2008). Qualitative inquiry and research design: Choosing among five approaches, (2nd Ed.). Thousand Oaks, CA: Sage. Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. New York: Aldine. Merriam, S.B. (2009). Qualitative research: A guide to design and implementation. San Francisco: Jossey-Bass. Stake, R.E. (1995). The art of case study research. Thousand Oaks, CA: Sage.