Evaluation of HED/USAID Women's Leadership Program-Rwanda (2012-2015) External Evaluators: Illana Lancaster, PhD Justine Mirembe Editor: Noopur Vyas September 4, 2015 2 Table of Contents ACRONYMS 3 EXECUTIVE SUMMARY 4 INTRODUCTION/PURPOSE 6 PROGRAM OVERVIEW 7 LITERATURE REVIEW 8 AGRICULTURE, GENDER, AND EDUCATION 11 EVALUATION METHODS AND LIMITATIONS 15 METHODS 15 DATA COLLECTION 15 DATA ANALYSIS 15 MEMBER CHECKING 17 DATA LIMITATIONS 17 PRESENTATION AND DISCUSSION OF FINDINGS 18 EDUCATION PARTNERSHIP 18 1. EFFICIENCY 18 2. RELEVANCE 20 3. EFFECTIVENESS 21 4. IMPACT OF INTERVENTION 26 5. SUSTAINABILITY 28 AGRICULTURE PARTNERSHIP 31 1. EFFICIENCY 31 2. RELEVANCE 33 3. EFFECTIVENESS 37 4. IMPACT 40 5. SUSTAINABILITY 40 RECOMMENDATIONS 43 RECOMMENDATIONS FOR WLP EDUCATION AND AGRIBUSINESS 43 RECOMMENDATIONS FOR WLP-EDUCATION 43 RECOMMENDATIONS FOR WLP- AGRIBUSINESS 44 CONCLUSION 45 WORKS CITED 46 ANNEX 1: LIST OF PARTICIPANTS 47 ANNEX 2: DOCUMENTS INCLUDED IN DESK REVIEW 51 ANNEX 3: SEMI-STRUCTURED INTERVIEW PROTOCOL QUESTIONS 58 3 Acronyms ACE American Council on Education AAU Association of African Universities ANSTI Africa Network of Science and Technology Institutions CAVM University of Rwanda’s College of Agriculture, Animal Husbandry and Veterinary Medicine EFA Education for All FAWE Forum for Africa Women Educationalists HEC Higher Education Council HED Higher Education for Development HEI Higher Education Institutions IARD International Agriculture Research ICT Information and Communication Technologies IUCEA Inter-University Council for East Africa ISAE Institute of Agriculture and Animal Husbandry KIE Kigali Institute of Education MSU Michigan State University NUR National University of Rwanda PEARL The Partnership for Enhancing Agriculture in Rwanda Through Linkages SPREAD Sustaining Partnerships to Enhance Rural Enterprise and Agribusiness Development TTC Teacher Training College UCLA University of California- Los Angeles UR University of Rwanda URCE University Of Rwanda College of Education USAID U.S. Agency for International WLP Women’s Leadership Program WSU Washington State University 4 Executive Summary This report presents the findings of an evaluation of the Women’s Leadership Program (WLP) in Rwanda, which began in November 2012 and ended in June 2015. The WLP pairs two U.S. universities with two colleges in the newly formed University of Rwanda system to develop institutional capacity in the fields of education and agriculture. It includes partnerships between: 1.) Michigan State University (MSU) and University of Rwanda (UR) College of Agriculture, Animal Sciences, and Veterinary Medicine (CAVM); and 2.) University of California- Los Angeles (UCLA) and the University Of Rwanda College of Education (URCE). Higher Education for Development (HED) is implementing these partnerships. The $1.2 million dollar partnership between Michigan State University (MSU) and University of Rwanda’s College of Agriculture, Animal Sciences and Veterinary Medicine, initiated in November 2012 and ending June 2015, aimed to strengthen the human and institutional capacity of UR in teaching and applied research in agricultural sciences, promote and support women's access to graduate education in agricultural sciences, and extend UR’s knowledge about and women’s expertise in agricultural sciences to the community. The nearly $1.1 million dollar partnership between UCLA and URCE sought to increase the number of highly qualified and gender-sensitive teachers in Rwanda by promoting and supporting women’s access to URCE and affiliated Teacher Training colleges (TTCs), better inform URCE’s institutional policies and practices through gender-focused research and capacity building, and improve the capacity of URCE to prepare teachers to address gender equity in the classroom through pedagogy, course content, and classroom management techniques. The purpose of this evaluation was to determine to what extent objectives of the WLP were achieved. In order to make this assessment, the evaluation employed mostly qualitative data collection methods, although the evaluation team also consulted quantitative data from a review of program results framework and progress reports. The evaluation team interviewed partnership staff, beneficiaries and coordinators in Rwanda and the U.S and conducted focus groups with students in the e-learning program in the College of Education, instructors at a TTC, as well as district and sector-level education administrators. Purposive sampling was employed to select study participants and documents for review. Following the OECD’s Principles for Evaluation of Development Assistance, this evaluation examined the efficiency of management processes and systems, relevance of partnership design, effectiveness of partnership implementation, impact of intervention, and potential for partnership sustainability. This evaluation exercise, however, had some limitations. Evaluators were unable to make conclusive evaluative statements about efficiency since the merger of five public institutions in Rwanda significantly delayed the implementation plan. Thus, this evaluation only makes tentative claims about impact. Also, the evaluators had difficulty finding pertinent documents in the desk review of partnership materials. The MSU –CAVM Agriculture Partnership was found to be highly responsive to the needs of beneficiaries. The partnership activities and interventions were also clearly aligned with USAID funding strategies in the agriculture and food security sectors, education, and gender equality and women’s empowerment. However, the merger of five public universities in Rwanda affected decision making with regard to the use of human, financial and non-financial resources. For example, MSU was excluded from making some decisions that UR considered internal, though they were engaged later. 5 In terms of partnership fidelity, evaluators observed efforts to reduce barriers to women pursuing advanced degrees in agricultural sciences. The College is now offering a gender-sensitive Master of Science (MSc) program that prepares students for careers and leadership roles. A gender-sensitive curriculum has been established and increased research opportunities for agricultural science students and academic staff members have been achieved. Although late, the MSc in Agribusiness which began in March 2015--just two weeks before the start of the evaluation exercise--has increased self-efficacy of the beneficiaries with regards to their capacities to achieve personal and professional goals. Participants have gained the theoretical, practical, and soft skills that international companies expect of their employees. Research and teaching faculty members have also gained research and pedagogical skills from attending capacity-building workshops offered by the partnership. The MSc in Agribusiness seems to be sustainable as it has been firmly integrated into the College of Agriculture’s graduate offerings and as long as there is continued support from college leadership. In terms of financial and non-financial resources management, the evaluation team did not have access to financial documents (e.g. audit reports). Therefore, no evaluative statements on the sustainability of financial resources management could be made. The URCE and UCLA partnership was found to be effective in its implementation of the partnership. Regular correspondence between both partners enhanced the efficiency of partnership planning and implementation. The Partnership Management and Implementation Plans enabled implementers to capture developments of partnership activities. However, a change in leadership at the College of Education coupled with perceived incompatible financial systems resulted in significant inefficiencies in management systems and decision-making processes. Administrative bureaucracy at URCE delayed requisitions for partnership activities. The partnership design consistently linked activities and outputs to outcomes and objectives. Activities were also aligned to Rwanda’s development strategy, particularly of promoting gender equality in all spheres of life including education, namely Vision 2020, both EDPRS1 and EDPRS 2. The partnership was also relevant to the research agenda and needs of the University of Rwanda, which has prioritized research as one of the ways through which socio-economic development can be achieved. An institutional review of URCE before the initiation of the partnership identified gaps in overall research capacity and gaps in gender-focused research in particular. The partnership has implemented activities aimed at increasing access, retention, and completion of female students and those from under-represented schools at URCE through its outreach activities. Other activities implemented under this yardstick include: workshops in conducting outreach in junior and senior high school for URCE staff, workshop on adolescent health, and training on mentoring for URCE staff. The partnership also established an e-learning platform for student teachers. However, while a new gender sensitive curriculum was developed under the program, it has yet to be approved. Increased capacity to conduct and disseminate gender-focused research was also enhanced for faculty through training on gender focused research methodology. Five additional research projects than planned in the first round were funded by the time of evaluation. A number of other workshops and trainings on gender awareness, MS Access, MS Excel, and Relational Data and planning were conducted for respective stakeholders. Delays caused by the merger of universities took almost a year off implementation period. Therefore it is too early to make conclusive evaluative statements about outcome level results and expansive effects. 6 However, some activities that have been implemented under the partnership have had some noticeable impact. Workshops on gender focused research improved female faculty self-efficacy by 42.89 percent (WLP-Rwanda Educ-APR (Oct, 1, 2013-Sept, 30, 2014). Also, 83.4 percent of the 36 teachers trained in mentoring skills such as role modeling, guidance, and counseling reported improvement in mentoring capacities (Program-Level Self- Efficiency-Survey Results-FY13-14). E-learning was found to be a very sustainable aspect of the partnership because a platform called UR Connect is expected to enhance e-learning in URCE. Also, the outreach initiative is owned by the community and leaders, from the lower levels of administration up to the decision making body (ministries). In addition, UR prioritizes research, so the research skills gained by faculty during research trainings and workshops under this partnership will continue to be practiced. The evaluation recommended a number of initiatives with regard to the partnership including an extension, increased funding, and gathering of gender disaggregated data by UR, and modifications of UR’s procurement regulations, development of human capacity in the field of Agribusiness to ensure sustainability among others. Introduction/Purpose Higher Education for Development (HED), a program of the American Council on Education (ACE), operates with the advice and counsel of six U.S. presidential higher education associations and with support from the U.S. Agency for International Development (USAID) to manage higher education partnerships that address global development challenges. These partnerships are usually between higher education institutions in the U.S. and in developing countries. Through these collaborations, the expertise of higher education is mobilized to address development challenges in areas such as agriculture, the environment, economic and workforce development, education, and health. HED relationships enable higher education institutions to address faculty and student training needs, conduct applied research, improve academic program offerings, and engage the institutions in community outreach. These initiatives are aimed at improving the capacity of the host country’s higher education institution in order to contribute to local and national development goals. This report presents the findings of an evaluation of the Women’s Leadership Program (WLP) in Rwanda. This program was expected to begin in November 2012 and end in June 2015. The WLP includes partnerships between two U.S. universities and two colleges in the newly formed University of Rwanda system. The partnerships are between: 1.) Michigan State University (MSU) and University of Rwanda (UR) College of Agriculture, Animal Husbandry, and Veterinary Medicine (CAVM); and 2.) University of California- Los Angeles (UCLA) and the University Of Rwanda College of Education (URCE). The purpose of this evaluation is to determine the extent to which the objectives of the WLP in Rwanda have been achieved. Where specific objectives have not been achieved, it will explore reasons why objectives were not met. Special attention will be paid to the sustainability of program goals. The evaluation was conducted following the OECD’s Principles for Evaluation of Development Assistance, which identifies critical partnership variables and evaluation criteria against which the variables can be examined. The major issues of consideration in this evaluation framework include:  Efficiency of management processes and systems,  Relevance of program design,  Effectiveness of program implementation,  Impact of intervention, and  Potential for program sustainability. 7 This evaluation report will showcase the achievements of the program in relation to these benchmarks. Program Overview Under Higher Education for Development (HED), WLP implemented five partnerships in Armenia, Paraguay, Rwanda, and South Sudan. Two of these are in Rwanda. WLP Rwanda pairs two U.S. universities with two colleges in the UR system to develop institutional capacity in the fields of education and agriculture. The goal is to foster the advancement of women and girls as articulated in USAID’s Gender Equality and Female Empowerment Policy, released March 2012. The following section is a brief description of these partnerships. Women’s Leadership Program –Agriculture The partnership between Michigan State University (MSU) and University of Rwanda’s (UR) College of Agriculture, Animal Husbandry and Veterinary Medicine began in November 2012 and ended in June 2015. The total funding for this partnership is $ 1,280,000. Under this partnership, MSU and UR aimed to achieve the following objectives: 1. To strengthen the human and institutional capacity of UR in teaching and applied research in agricultural sciences; 2. To promote and support women's access to graduate education in agricultural sciences; and 3. To extend UR’s knowledge about, and women’s expertise in, agricultural sciences to the community. The expected results were: 1. Increased applied learning and research opportunities for students and faculty at UR. 2. New opportunities for women to become leaders in the field of agribusiness. 3. Development of a mentorship program and internships for students with local businesses and NGOs. Partnership achievements include: 1. Developed and launched the Masters of Science in Agribusiness, which includes new curriculum that incorporates gender-sensitive admissions policies. 2. Conducted an academic exchange between UR and MSU. 3. Implemented workshops in the areas of pedagogical techniques and leadership development. Women’s Leadership Program –Education The partnership, between the University of California Los Angeles (UCLA) and the University of Rwanda’s College of Education (URCE) was planned to last from November 2012 to June 2015. The institutions came together to foster strategies that promote gender awareness in classrooms through targeted outreach and mentoring programs, conduct institutional capacity building activities, create gender￾sensitive curricula for teacher trainers, build capacity to conduct gender focused research, and promote gender-sensitive teaching approaches. The total budget for this partnership was $1,079,996. The goal of this partnership was to achieve the following objectives: 8 1. Increase the number of highly qualified and gender-sensitive teachers in Rwanda by promoting and supporting women’s access to URCE, Rwanda Teachers’ College, and affiliated Teacher Training colleges (TTCs). 2. Better inform URCE’s institutional policies and practices through gender-focused research and capacity building. 3. Improve the capacity of URCE to prepare teachers to address gender equity in the classroom through pedagogy, course content, and classroom management techniques. The expected results were: 1. Establishment of a gender-equitable mentoring and recruitment strategy and diversified course delivery. 2. Increased research and policy advocacy capacity. 3. Updated curricula and teaching techniques at UR—College of Education and TTCs. Partnership achievements include: 1. Development of curriculum for a primary education degree targeting those with certificates who are interested in earning a degree. 2. Provision of workshops on adolescent reproductive health, using technology in classroom, conducting outreach activities, and integrating gender into research and teaching. 3. Organization and delivery of mentor training for mentor supervisors. 4. Development of an e-learning platform and the launch of a facility utilizing online distance learning. 5. Launch of a small grants program in support of gender-related research. Literature Review Rwanda, a former Belgium colony, received her independence in 1962. During colonialism, Belgium introduced formal education as a way to help colonial masters train workers in order to retain control of colonies. In most colonies, girls were not allowed to go to school. In Rwanda, for example, girls were not allowed attend schools until 34 years after boys (Allison et al, 2008). Even after gaining the right to attend school, girls generally spent less time in school than boys (ibid). After gaining political independence, most African countries re-focused their educational programs on educating citizens who were expected to contribute to the economic, social, and political developments of the young nations. This function of education was well articulated by Julius Nyerere (first president of the United Republic of Tanzania) in the 1960s: Education is viewed as an instrument for preparing the community for life. It must foster the social goals and encourage the growth of the social attitudes and values. These include equality and respect…. Education must encourage the development of an enquiring mind, a scientific outlook at issues; creativeness, problem solving, the ability to think for one self, to interpret decisions, to learn from others and reject or adopt it in accordance with particular needs and circumstances. Education … must impart knowledge and skills needed for family life and for participation in the development and maintenance of the community (Nyerere, 1968: 274) While education in post-colonial Africa was aimed at creating a pool of human resources with knowledge and skills to drive economic, political and social change in their respective societies, women were still discriminated against in education. In many sub-Saharan African countries (pre- and post- 9 independence), few spaces were available for girls and women in schools and universities (Masanja, 2010). Many of these countries, including Rwanda, are now collaborating with development partners, local and international non-governmental organizations (NGOs) such as the Forum for Africa Women Educationalists (FAWE), and civil society organizations, to increase gender equality in education. This is largely aimed at meeting the Education for All (EFA) goals, Millennium Development Goal 3 (MDG3), and poverty reduction strategies designed by various countries. In Rwanda, both Vision 2020 and Economic Development and Poverty Reduction Strategy (EDPRS 1 and 2) explicitly emphasize the acceleration of education for all people in Rwanda as well as gender equality in the provision of educational services. The African Union (AU) has also established mechanisms and special ministerial-level groups that are responsible for monitoring progress towards attainment of EFA and MDG3 (Masanja, 2010). Education organizations, especially the higher education institutions (HEI), are undertaking various initiatives to promote women’s access, retention and completion at the tertiary level. Higher education initiatives are being organized at both the institutional level and at the regional level, where the Association of African Universities (AAU), the Inter-University Council for East Africa (IUCEA), and the Africa Network of Science and Technology Institutions (ANSTI) are leading the way. These initiatives have helped increase gender parity in primary and secondary school enrollment. However, gender inequalities in access to and participation in tertiary education persist in most African countries, including Rwanda. Although gender equality in the higher education sector has seen some improvements in the enrollment statistic, average female enrollment for countries in sub-Saharan Africa continues to be about 30 percent of total students except for in the disciplines under the humanities and social sciences, where gender parity is observed. Moreover, occupational and disciplinary stereotypes persist. For example, nursing and social work programs tend to have large proportions of women, up to 95 percent, while science, technology, engineering, and mathematics (STEM) fields have low proportions of women, below 10 percent in some cases (Masanja, 2010:6). While Rwanda has witnessed great achievements in gender parity on the political front, girls continue to be underrepresented in government schools, although they dominate enrollment at private universities (Huggins and Randell, 2008). Most of these universities are more expensive, provide lower quality education, and offer few or no science and technology programs. In Rwanda, girls’ participation in public HEI ranges between 25 percent and 27 percent. Participation in STEM subjects at the secondary school level ranges between 5 percent in technical subjects to 40 percent in biological and chemistry subjects, while in overall STEM subjects, girls account for 35 percent (Masanja, 2010:8). The enrollment statistics for tertiary education by exit ward in 2013 indicate that the percentage of male students enrolled in each exit award is considerably higher than the percentage of female students, except in the bachelor’s award. For example, the Ph.D. award had only two female students out of the 12 who enrolled for the award in 2013. 10 Figure 6: Enrolment by Exit award by Gender 2013 Source: MINEDUC Statistics 2013 That only 12 students total enrolled for Ph.D. study in Rwanda in 2013 is a clear indicator that the rate at which leaders in academics and research, for whom a Ph.D. is a requirement, are being trained is exceedingly slow. However, it is even more alarming to note that only two of the 12 Ph.D. candidates were female. This is one of the major issues that the WLP has attempted to address. One of the aims of the Girls’ Education Policy in Rwanda is the progressive elimination of gender disparities in education and training as well as in management structures. The profile of primary, secondary, TVET and tertiary education teaching staff by gender and the enrollment in tertiary education by exit award by gender in 2013, all mirror gender disparities in education and training, as well as in management structures in Rwanda. In primary schools, the percentage of certified male teachers is marginally higher than the female counterparts (97 percent males, 94 percent females). Administrative staff members in primary level are predominantly male (70 percent male, 30 percent female), while at TVET level males form 74 percent of teaching staff compared with females at 26 percent. The situation is worse at the tertiary education level where over 80 percent of academic staff at universities in Rwanda is male and a mere 19 percent is female (MINEDUC Statistics 2013). Data from Rwanda shows that females form only 19 percent of academic staff in both public and private tertiary institutions (MINEDUC Statistics 2013). In order to increase the number of female academic leaders in universities in Rwanda, the National University of Rwanda (NUR), which has since merged with other universities in the country to form the University of Rwanda, initiated a flexible program to encourage women to earn a Ph.D. This was aimed at encouraging women who have been stuck at the master’s level for many years to earn a Ph.D. and take on leadership roles in academia. The Swedish International Development Agency (SIDA) awarded funding to the first cohort of seven women in 2009. Six enrolled at universities in neighboring countries where they took their young children and their house helpers with them. Many wished to remain close to their families and spouses, who visit on weekends. One registered in Sweden for a sandwich program, which only required her to be in Sweden one month per academic year. Soon after the program’s inception, SIDA revised its regulations and insisted the beneficiaries of their funding take courses in Swedish universities on full-time basis. This deterred many of the prospective students from enrolling in Ph.D. studie. Here, we see a development intervention that failed to address the local needs of 11 beneficiaries, resulting in a failure to achieve intended objectives. It is also a clear case where the concept of gender division of labor diminishes women’s opportunities for self-development (Walby, 1990). Although some of efforts were initiated in the 1970s, focus on gender equality in HEI intensified in the 1990s. These efforts have resulted in gender policies at the country level and at HEI level, as well as special policies and programs focusing on girls and women’s education, policies focusing on specific gender-related issues such as gender-based violence, re-admission of girls and female dropouts due to pregnancy in order to create a gender-friendly environment for both men and women. Even with these policies in place, women continue to lag behind men in higher education, particularly in STEM education. Outside of academia, 80 percent of Rwanda Education Board (REB) inspectors are male, 73 percent of 416 sector education officers are male, and parent-teacher associations are almost exclusively chaired by men who have the final say on committee decisions (MINEDUC Statistics, 2013). Female underrepresentation in administration and management roles diminishes their active participation in major decision-making. There appears to be invisible, but real discriminatory barriers that prevent women from rising to positions of power or responsibility within institutions. Male dominance on selection committees has been noted as one prominent barrier. Researchers at the University of Dar es Salaam conducted analyses of academic staff statistics that show that the success rates of women are as good as those of men, yet women tend to apply in smaller numbers for academic and senior positions. Women also tend to take two to ten years longer for promotion than their male counterparts (Masanja, 2010). This means that while women’s higher educational attainment should make them leaders in their respective fields, it does not often do so because of socio-cultural and structural barriers, particularly in patriarchal societies such as Rwanda. Agriculture, Gender, and Education Rwanda Vision 2020 (MINECOFIN, 2000), the Economic Development and Poverty Reduction Strategy 2 (MINECOFIN, 2013), the Higher Education Policy (MINEDUC, July 2008), the Education Sector Strategic Plan 2013/2014- 2017/2018 (MINDUC, October 2013), and the Strategic Plan for the Transformation of Agriculture in Rwanda – Phase II (PSTA-II) (MINIAGRI, 2009) are five key policy documents that articulate a comprehensive, consistent, and aggressive development strategy. This strategy places agriculture, gender, and education at the center of the Government of Rwanda’s development efforts. Rwanda’s national policies consider agriculture to be the springboard for the country’s fight against poverty. Agriculture contributes to about 39% of GDP in current prices, employs about 88 percent of the economically active population, and is the main earner of foreign exchange, supplying up to 80 percent of exports (MINIAGRI, 2009, p. 1). The main objective of the agriculture sector is the sustainable increase of diversified incomes for rural households (MINECOFIN, 2013). The broader objective of the Government of Rwanda’s national development plan is the creation of a more diversified and competitive economy that will “transform Rwanda into a middle-income country by the year 2020” with per capita GDP growing from US$220 to US$ 900 (MINECOFIN, 2000). By 2020 the population of Rwanda is expected to double to around 16 million (MINECOFIN, 2000). To become a middle-income country, Rwanda’s sustained average GDP growth must be at least 11.5% (MINECOFIN, 2013). To aid in the development of this new economy, the government must invest private sector and donor aid in the crucial sectors of education, agriculture, and gender. Human Resource Development and a Knowledge- 12 based Economy and Productive and Market Oriented Agriculture are two of the 8 pillars of Vision 2020, while Gender Equality is one of the three cross-cutting areas. Table 3: Pillars of the VISION 2020 and its Crosscutting Areas (MINECOFIN, 2000, p.13) Pillars of the VISION 2020 Cross-cutting areas of VISION 2020 1. Good governance and a capable state 1. Gender equality 2. Protection of environment and sustainable natural resource management 3. Science and technology, including ICT 2. Human resource development and a knowledge based economy 3. A private sector-led economy 4. Infrastructure development 5. Productive and market-oriented agriculture 6. Regional and international economic integration. Vision 2020 calls for the “transformation” and overhaul” of agricultural policy. With the population doubling, land productivity must increase, as subsistence farming cannot sustain such population expansion. Growth rates of 4.5 to 5 percent per year were set alongside commitments from the Government of Rwanda to transform the agriculture sector from subsistence to fully monetized and commercialized (MINECOFIN, 2000, p. 20). While this fully monetized and commercialized agriculture sector requires the production of high value crops and modern livestock management, it also requires the development of the “spin-off effect” of agro-businesses that will spill over into other sectors of the economy (MINECOFIN, 2000, p. 20). The private sector and regional and international economic integration are central to this new approach toward productive and market-oriented agriculture and its spillover into other sectors of the economy (MINECOFIN, 2000, p. 3). The Strategic Plan for the Transformation of Agriculture in Rwanda – Phase II (PSTA II) articulates the specific objective for the Government of Rwanda’s strategy as the “increase output of all types of agricultural products with emphasis on export products, which have high potential and create large amounts of rural employment”… under sustainable modes of production” (MINIAGRI, 2009, p.vii). While land is a “scarce commodity” in Rwanda, labor is Rwanda’s “most abundant factor endowment” (MINIAGRI, 2009, p. iv). Vision 2020 articulates a human capital approach that moves the economy from subsistence-based agriculture to a knowledge-based one. A knowledge-based economy requires highly skilled scientists and technicians, and the Government of Rwanda recognizes the present shortage of such human capital “at all levels” (MINECOFIN, 2000, p. 22). The generation, dissemination, and acquisition of scientific skills and the development of technological innovations are policy priorities. Toward this end, the Government of Rwanda has prioritized science and technology education and ICT skills. Women and girls are at the center of these human resource development policies and strategies. Women participate in subsistence agriculture more than men and they are responsible for the majority of food crop production (MINIAGRI, 2009, p. vi). Recognizing that “until recently, girls were the minority in secondary schools, women had little access to the opportunities available to men and they were poorly represented in decision-making positions”, Rwanda will “support education for all, eradicate all forms of discrimination, fight against poverty and practice a positive discrimination policy in favor of women (MINECOFIN, 2000, p. 21)”. While women participate in subsistence agriculture more than men and are responsible for the majority of food crop production, women are grossly underrepresented in decision and policy-making positions, and in public and private agricultural development services in Rwanda and in sub-Saharan Africa more 13 broadly (Acker, McBreen & Taylor, 1998). Gender is a core principle of the 2009 Strategic Plan for the Transformation of Agriculture in Rwanda – Phase II (PSTA-II). PSTA-II calls for increased numbers of women extension agents (30% of cooperative management committees are required to be women) and the promotion of commodity chains and agribusiness in which young women farmers are targeted (MINIAGRI, 2009). Not only must women be a part of agricultural development services, there is a critical need to build a pipeline for women to generate, disseminate, and acquire scientific skills and develop technological innovations in Agriculture. At present few women are in university-level agricultural science programs in Rwanda. Internationalization of Higher Education in Rwanda After years of neglect and divestment, African universities have been experiencing a renaissance and revitalization over the past 15 years. Higher education has become central to the development agenda of African nations along with that of international development banks, donor countries, and intergovernmental organizations. Despite the African delegation’s numerous calls for investment in higher education, universal primary education became the priority of the global education development agenda in the 1990s. World Bank economic orthodoxies around rates of return to investment1 dictated that priority be given to investment in the expansion of primary education over higher education because the rates of social and economic returns from investments in primary education were believed to be higher than the rates of return from investment in higher education. As a result, higher education institutions in Africa experienced great financial neglect from the international community (Hoosen, Butcher, & Njenja, 2009). Today, new economic orthodoxies recognize that knowledge societies require a highly educated citizenry to create and sustain knowledge products. With high rates of return on knowledge creation and production, there has been a re-centering of higher education within the development agenda. African governments are revising national policy frameworks and in concert with the IMF Poverty Reduction Strategy Papers (PRSP). Rwanda’s 2013 PRSP recognizes education, skills, and technology as key drivers of economic productivity (p. 57). Vision 2020 (Government of Rwanda, 2000) articulates a long-term development plan that transitions Rwanda from an agrarian economy to a knowledge-based economy with heavy investment in education and information and communication technologies (ICT). With Singapore as a model, Rwanda’s development agenda transitions the country from low income to middle income by 2020. In order for Rwanda to successfully manage this transition, it has embarked on a program of institutional capacity strengthening, human resource development, and public debt reduction. WLP-Rwanda contributes to this agenda by aiming to strengthen institutional capacity and develop human resources through its work with the University of Rwanda system. Internationalization and its related discourses on globalization and knowledge societies are conceptually fluid and open to interpretation (Dale, 2007; Knutsson, 2012; Singh, 2010). We are urged to see internationalization as a process, not a product. When understood as a process, definitions of internationalization tend to be more dynamic. Internationalization then becomes broad enough to have analytical and conceptual relevance across geographical contemporary and historical contexts, which is 1 It is important to note that the earlier orthodoxies on investments in primary education were challenged by African delegations during meetings for the World Conference on Education For All in Jomtien in 1990.The marginalization of African contingencies’ request to multilateral conference organizers to prioritize investment in higher education has led some scholars in the field to claim that today’s education development agenda is part of a long legacy of the imposition of colonial agendas. 14 critical to a broader and more inclusive understanding of the process (Knight, 2008; Jowi, 2009). Singh (2010) urges us to think of internationalization as “a suite of policy framing vocabularies” that include “knowledge societies, entrepreneurial universities, widening participation, academic mobility, cross￾border education, higher education cooperation, partnerships and networks, third stream income; and regional integration” (p. 269). One such set of policy framing vocabularies is the power asymmetries and inequalities that characterize today’s higher education scene in Africa. It is argued that multilateral bodies, international aid agencies and foreign foundations continue the work of earlier colonial projects to exert influence over higher education in Africa (Brocke-Utne, 2000; Samoff and Carrol 2004; Sawyerr 2004; Singh, 2010; Teferra 2008). These power asymmetries can be traced to the historical power asymmetries and inequalities present during the development of higher education institutions in the 19th and 20th centuries. The earliest African institutions of higher education were founded during the height of colonial rule2 . In 1948 the Asquith Commission on Higher Education convened to investigate higher education in the British colonies and recommended the establishment of an “intimate relationship between university institutions in Britain and university institutions in its colonies” (Hussey, 1945, p. 166). The Inter￾University Council for University Relationships in the Colonies was established to assist universities in the colonies with “periodic visits to the Colonies, in regard to interchange of staff as between this county and the colonies, in the placing of colonial students at the undergraduate and postgraduate stages in British Universities and so on” (Hussey, 1945, p. 166). As a result of the 1948 Asquith Commission on Higher Education, Makerere University in what is today Uganda affiliated with the University College of London in 1949. Known as the Empire University, the University College of London, was selected by the Inter-University Council for University Relationships, a council created by the Asquith Commission, to be the university to which all colonial universities would affiliate. The University of Ghana and Ibadan University in Nigeria were such affiliates and were founded in 1948. Periodic visits, interchange of staff, and placement of students in undergraduate and postgraduate programs in northern institutions continue to characterize the processes of African internationalization efforts today. African nations primarily send students and staff to consume higher education in the Global North and receive cross-border provisions of higher education through partnerships. Most initiatives originate in the north and are often directed or managed by northern institutions. For many in the field, African countries are simple objects of the internationalization ideologies of the north and have developed accommodative and imitative behaviors (Singh, 2010) that have left African universities dependent on external support, advice, and personnel (Teferra, 2008). For others, internationalization processes in Africa offer opportunities and benefits for individual citizens, institutions, and society at large. While these scholars recognize the asymmetry of the relationships between northern and African universities, they note that African institutions are not docile recipients of northern agendas. Rather, African universities, and the University of Rwanda in particular, are actively negotiating and translating policies and practices and in the process are exercising agency and local control (Knight & Sehoole, 2013). 2 Fourah Bay College, an Anglican missionary school founded in 1827 in what is today Sierra Leone served the “native” populations of the colonies of British West Africa, while the University of Cape Town, founded in 1829, served the white colonialists. Founded as the South African College, a high school for boys, greatly expanded its tertiary education in the 1880s as a result of the tremendous growth in the diamond and gold industries requiring workers with higher skills. Makerere University in Uganda was founded in 1922 as a technical school for “native” populations in the trades of building, carpentry, and mechanics. 15 The complex nature of the University of Rwanda’s partnerships with UCLA and MSU is best understood through exploring the dynamic process of who controls the agenda. If viewed solely through the lens of a rigid colonial reproduction model, we miss opportunities for identifying creative local engagement by the University of Rwanda and its local partners. We are urged to explore how the local benefits of internationalization are being distributed, to whom, and with what impact (Singh, 2011; Dale, 1999). This evaluation will pay particular attention to understanding how the WLP-Rwanda distributes program benefits to women and girl and what are the impacts to women and girls. Evaluation Methods and Limitations Methods In assessing WLP-Rwanda, the evaluation team took an evaluative case study approach in order to examine the partnerships’ contribution to gender equality and female empowerment in teacher training and agriculture. USAID’s Technical Note on Evaluative Case Studies (2013) recommends using this strategy of inquiry when seeking to convey an in-depth understanding of the partnerships and the context in which they are implemented. In addition, case study evaluations are especially useful when studying partnerships that undertake multiple interventions, as is the case with both the Education and Agriculture partnerships. Specifically, the team used interviews, focus groups, and document analysis to understand the partnerships on five criteria: efficiency, relevance, effectiveness, impact, and sustainability. Data Collection The evaluation team employed purposive sampling techniques to select participants. This method enabled the evaluators to choose participants most likely to provide information that could best address the evaluation’s objects because of their backgrounds and involvement with the partnerships. Participants included partnership implementation directors and staff, professors, instructors, students, and community members. Their different roles offer different perspectives, and thus create a richer sample. These participant interviews were conducted in person and in a semi-structured manner. Such an asymmetrical approach helped the evaluation team to elicit detailed narratives from the respondents on their individual experiences with the respective partnerships. It also allowed the team to use probes and adjust wording for clarity. The interview protocol is attached as Annex 3. In total, forty-five individuals gave interviews. See Annex 1 for the complete list of participants interviewed by partnership. In order to increase the validity and reliability of the evaluation and provide a better understanding of the partnerships as a whole, the research team also conducted document analysis to triangulate the findings from the interviews and focus groups. Documents consulted include the applications for funding, monitoring and evaluation plans, progress reports, and correspondence. See Annex 1 for a complete list of documents analyzed by partnership. Data Analysis Qualitative data analysis involves coding of themes, patterns, and categories of evidence (Bryman, 2007). For this study, the evaluation team conducted analysis using five externally derived criteria established by the OECD’s Development Assistance Committee’s Principles for Evaluation of Development Assistance (1991): efficiency, relevance, effectiveness, impact, and sustainability. They then manually coded data based on the sub-criteria and categories defined by HED. 16 The first criterion, efficiency, measures the efficiency of management processes and systems. Sub￾criteria and categories defined by HED are listed below. In order to generate information that would show whether the partnership was efficiently implemented the evaluators interviewed partnership implementers and beneficiaries and reviewed partnership documents. Sub criteria Categories Management Systems Results-based management Decision-making processes Timeliness of activities Use of resources (human, financial and non-financial) Feedback loops Using interviews and focus group discussions, the evaluation team also explored relevance, whether local and national development priorities were identified, how the partnership responded to the needs of beneficiaries, and how the partnership reflected funder strategies. Sub criteria and categories under relevance are outlined below. Sub criteria Categories Contextualization Local and national development priorities Responsiveness to beneficiaries needs Reflection of funder strategies Logic Causal linkages in the results framework Single/multiple pathways The third criterion under consideration was the effectiveness of partnership implementation. Effectiveness of partnership implementation can be seen through partnership results and the extent to which the partnership was carried out in accordance with its implementation plan and results framework. Sub criterion Categories Results Progress vs. targets Program fidelity The fourth criterion, impact, explores the partnerships’ effect on beneficiaries and whether those effects could be attributed to the partnerships’ implementation. Sub criteria Categories Outcome level results Intended Unattended Positive Negative Results directly attributable to intervention Results not directly attributable to intervention Expansive effects Local ownership Regional institutions Businesses affected Government agencies Other entities outside HCI 17 Finally, evaluators analyzed data related to the sustainability of the partnerships’ results. The evaluation team considered the evidence on whether the results would continue over time after funding had ceased and if so, what aspects of the partnership had the best chance of continuing Sub criterion Categories Program Effects Financial resources management Non-financial resources planning and management Results based management Institutionalization Stakeholder engagement Continued relevance of program design After data collection, the evaluation team manually coded them under the categories outlined above. They were then analyzed based on the frequency of ideas under the specific themes and presented using narrative style. Qualitative data taken from partnership documents, such were used to enhance the views of participants and strengthen the assertions made by evaluators. Member checking The evaluation team also conducted member checking to help improve validity of the study. Five to six members of each partnership’s implementation team were given their respective data analysis sections for review. Evaluators received feedback on the education partnership from UCLA’s Dr. Azeb Tadesse and Ben Musuhuke and feedback on the agriculture section from Dr. Jean Chrysostome Ngabitsinze. Data Limitations Due to time limitations evaluators were unable to observe teaching and learning take place for both the education and agriculture partnerships. The majority of fieldwork took place over the course of 11 days and since both the MSc in Agribusiness and the Diploma in Education Studies are primarily taught through distance learning modes, convening for in-person instruction on weekends, the evaluators were unable to observe learning in action. Evaluators attempted to conduct classroom observations, but the Friday evening evaluators had scheduled to observe a class being taught as part of the Diploma in Education Studies program (March 27, 2015), no direct instruction was taking place; rather, students were working in small groups to complete an assignment. Similarly, evaluators were unable to observe teaching and learning in the MSc in Agribusiness classroom. The evaluators began fieldwork only three weeks after the program started. At the time the fieldwork was being conducted, students had just completed the second weekend of face-to-face instruction. The Sunday the evaluators were on campus meeting with students (March 29, 2015), soft-skills were being taught. Students had yet to begin the academic course work. Because key university partnership personnel and university administrators shifted over the life cycle of the grant, some key insights into earlier phases of implementation and challenges associated with that time period is missing. Additionally, because of time limitations evaluators were unable to secure interviews with key university administrators and Higher Education Council decision-makers, limiting key insights on the efficiency of management processes and systems and effectiveness of partnership implementation. Other limitations to this evaluation include the lack of probing questions posed to program managers, implementers and beneficiaries on certain evaluation sub-criteria and categories, including management systems, logic, expansive effects, and impact. Because of time constraints or a misunderstanding of the 18 evaluation criteria, the evaluation team was left to draw conclusions on these criteria based on a desk review of the partnerships’ documentation. Such a misunderstanding may have also led to inter-coder reliability issues when interpreting the data. Presentation and Discussion of Findings In this section evaluators will present and discuss the findings from the review of partnership documents and interview data collected during fieldwork March 23 - 30, 2015 as well as interviews taking place virtually and in person April 14 - 21, 2015. Because this evaluation employs qualitative methods, data will be presented and discussed together. We will first present and discuss data from the education partnership before presenting and discussing data collected from the agriculture partnership. Education Partnership 1. Efficiency The results described in this section are related to the performance management, decision-making processes, and feedback loops utilized by the partnership in order to maximize partnership’s efficiency. Overall, the evaluation team finds the partnership to have been implemented efficiently, despite the delays caused by the university system merger. 1.1.Performance management Results-based management The partnership was largely designed and implemented following USAID and HED’s results-based management system. HED helped the partnership establish monitoring and evaluation processes at the onset of the grant in October 1, 2012. Results-based management protocols observed include: 1.) submission of annual implementation plans for FY2014 and FY2015; 2.) baseline assessment data collected in August 2013 and August 2014; 3.) Semiannual and annual HED verification of data quality in FY2013 and FY2014; and 4.) submission of custom indicator definitions along with up-to date revised monitoring and evaluation plans. Tools such as the Partnership Monitoring Plan and Partnership Implementation Plan were also used to ensure proper implementation of the partnership both in terms of monitoring achievements and respect of schedule. Reporting To ensure consistent monitoring and reporting on the partnership’s intended results, the partnership regularly submitted semiannual and annual progress reports over the course of its life cycle. Although some partnership activities were still ongoing at the time of this report, partnership implementers have aimed all activities toward achieving the outcomes established in the results framework. 1.2.Decision Making Processes 1.2.1.Timeliness of activities In September 2013, the Government of Rwanda initiated the process of merging five public institutions into one university system. Due to the complications resulting from the merger, the decision-making process around the timeline of activities was significantly delayed. This in turn resulted in significant delays to approvals and implementation of activities, which are discussed in detail throughout this section. 19 1.2.2.Use of resources Financial Resources Two factors led to significant inefficiencies in management systems and decision-making processes: leadership changes among the College of Education implementers and perceived incompatible financial systems. In one example, URCE did not make arrangements to have refreshments served for several hundred guests and UCLA was left to directly pay for them because they had to be ordered with a few days’ notice. The rates that URCE had negotiated were significantly higher than the rates UCLA eventually secured. In other instances, the partnership utilized URCE vendors at higher costs and generally of lower quality. Some participants, however, viewed this incident cited above as an example of the disregard of established procurement procedures at URCE by UCLA. URCE’s bureaucracy also posed a challenge in the disbursement of funds. Requisition for funds was often delayed, sometimes leading to an intervention from the Principal of the College of Education. In some cases, UCLA would make payments directly from their account to ensure activities could be completed. At the time of evaluation, the situation had improved compared to when the partnership started because the long list of signatories for disbursement had been reduced to only the Directors of Finance and Planning. Procuring supplies for the partnership was also difficult because of lengthy processing times. URCE’s procurement process requires that the institution advertise a tender which items are sought along with the requirements to win the tender. Companies or individuals then submit proposals, followed by the selection of a qualified bidder. The selection phase is often when delays occur because of the need to gain buy-in from many signatories. Human Resources Following the mentorship workshop for mentors from TTCs affiliated with URCE, the individual in charge of the mentorship component left URCE and mentors did not receive follow-up instructions before his departure. The workshop had intended for the mentors to report back to URCE on the mentoring and counseling activities at their respective TTCs and mentors were supposed to be supervised through this process in order to be oriented for better performance. When the evaluation team followed up on this partnership component, URCE communicated that a mentoring program existed UCRE before the partnership and this component was no longer a priority. Because the mentors were not told that their mentorship program was no longer a priority, they believed that their URCE supervisors were neglecting them. Some mentors did report on their post-workshop, but stopped after not receiving a response. Though the workshop participants did gain mentorship and counseling skills, the activity detracted from the partnership’s overall efficiency because funds were expended on a de-prioritized partnership component. . 1.3.Feedback loops Regular correspondence between UCLA and URCE and other stakeholders’ enhanced the efficiency of partnership planning and implementation (Data Quality Verification Notes: WLP-Rwanda-Education, 2014), though partnerships did experience some challenges during implementation, especially at the startup phase. Some of these challenges were communication delays caused by the merger of several universities into one. All partnership activities were planned in collaboration with the stakeholders, enabling an easy flow of information and communication between the partners. URCE sent partnership progress reports to UCLA, who then submitted them to HED for consideration. URCE then received feedback based on the reports where necessary. 20 The partnership also put in place internal strategies to strengthen feedback loops. One example of such a strategy was when the URCE program coordinator received a report from the administrative assistant and sent it to the liaison officer who then sent it to UCLA. Other avenues of communication are implementation update reports delivered to the Principal by the Director of Planning and when necessary, the Coordinator and Principal organize in-person meetings. The URCE internal reporting system has kept everyone on the implementation team aware of the progress of the partnership. To prepare for site visits, HED, UCLA, and USAID held a number of telephone conversations to determine implementation progress. In one example of an efficient feedback loop, HED and UCLA pushed up the site visit dates to early March 2014 in response to USAID's urgent request for their presence on the ground to help rectify partnership implementation and facilitate the transition in programmatic and administrative leadership. HED also shared the trip objectives with and conducted in- and out-briefs with the USAID/Rwanda Activity Manager. 2. Relevance Partnership activities and interventions were both relevant and logical for the Rwandan context. All activities respond to the need for gender equality and women empowerment, which is one of the Government of Rwanda’s development priorities. Partnership design consistently linked activities and outputs logically to outcomes and objectives. 2.1.Contextualization 2.1.1.Local and National Development Priorities The partnership enhanced its relevance by aligning activities to Rwanda’s development strategy, especially the promotion of gender equality. Promotion of gender equality by integrating gender in all of the Government of Rwanda’s development programs is one of the major pillars of Vision 2020, while gender equality is recognized as one of the drivers to economic development and poverty reduction in both EDPRS1 and EDPRS 2. Also, partnership activities and interventions were relevant in that they supplemented the efforts of the Government of Rwanda to promote gender equality and women’s empowerment by increasing women’s access to, and participation in, education. For example, Rwanda’s 2008 Girls Education Policy, the Education Sector Strategic Plan, the Educational and Vocational Training Policy, and the Early Childhood Development Policy all focus on gender equality in access, retention, and graduation at all school levels. This partnership therefore directly supported Rwanda’s national development priorities because it sought to increase access to higher education in Rwanda, retention as well as completion as stated in partnership Outcome 1.1: Increase in access, retention, and completion of female students and those from underrepresented schools at URCE. 2.1.2.Responsiveness to Beneficiaries Needs URCE administration and faculty The partnership was also relevant beneficiaries’ needs because it aimed to improve the research needs of URCE. The University of Rwanda has prioritized research in general as one of the ways through which socioeconomic development can be achieved. In addition, an institutional review of URCE before the initiation of the partnership identified both general research capacity gaps and gender-focused research gaps among faculty. In order to achieve outcome 2:1: Increased capacity to conduct and disseminate gender-focused research, the partnership rendered technical support to research institutions through training in research, as well as provided funds for faculty to conduct gender focused research. Faculty at URCE who received research grants were expected to have finalized their research and be in position to disseminate the findings at the end of June 2015. 21 Diploma students Primary school teachers understand the importance of upgrading skills and embrace the in-service diploma as a mechanism to do so. URCE as an institution was also impressed by the diploma initiative and the support provided by the partnership. Beneficiaries expressed that the partnership came at the right time when there was need. The partnership showed responsiveness to the needs of beneficiaries in the in-service diploma in education by developing an e-learning platform. Diploma candidates were teachers who had face-to￾face sessions with their instructors on weekends only. The e-learning facility gave them the opportunity to learn throughout the week whenever they had time by sharing information with their instructors and among themselves hence bridged the gaps in in-person instruction. Some of these students also lived far from the study center so the e-learning facility saved them the trouble of having to travel to Kabgayi every time they wanted advice from their instructors. Raising Gender awareness was also relevant to the needs of diploma students because they needed to understand gender concepts in order to be able to develop and apply gender sensitive pedagogy in the classroom. With gender awareness and the need to make gender an integral part of teachers’ pedagogy, there was need to design a gender sensitive curriculum which would work as a guide for teachers on classroom management, and teaching among others in a gender sensitive way. A Gender sensitive curriculum was therefore of great importance to URCE faculty and student teachers. 2.2. Logic: Causal Linkages and Single/Multiple Pathways The causal linkages articulated in the partnership results framework are sound. Though partnership does not have an explicitly stated theory of change, an examination of the partnership’s results framework document shows a hierarchy of results, along with indicators, and assumption statements for each outcome and output. At the highest levels of the framework, the partnership also set objectives and a goal. The lower-level outputs support the mid-level outcomes, which in turn support the broader objectives of 1.) Strengthening the human and institutional capacity; 2.) Promoting and supporting research 3.) Extending URCE’s knowledge into the community. There are multiple pathways to meeting objectives. Objective 1 has one outcome and three outputs. Objective 2 has two outcomes with three outputs each. Objective 3 has one outcome and three outputs identified. Based on an interpretation of the results framework, the underlying theory of change states: If... 1) Institutional capacity in recruitment and teaching for underserved female students at KIE is strengthened; 2) Gender-focused research capacity among faculty members is enhanced, and 3) Faculty and students develop awareness of gender issues Then…the capacity of the University of Rwanda College of Education to advance women's leadership in the field of education will be strengthened. The degree to which the partnership has put the links in place to achieve this overarching goal is examined in the next section. 3. Effectiveness This section explores the effectiveness of the partnership, as measured by the progress it made toward achieving targets set for each output in its results framework and its fidelity in content and delivery. 22 With the exception of Outcome 2.1, the evaluation team found the partnership made good progress in reaching its targets with a high degree of fidelity. 3.1.Progress vs. Targets After initial delays in getting the right URCE leadership team in place, the partnership made significant progress toward achieving its intended outcomes and outputs. Outcomes under this category have been achieved with a relatively high degree of implementation fidelity. Outcome 1.1: Increased access, retention, and completion of female students and those from underrepresented schools at Kigali Institute of Education (KIE). To accomplish this outcome, the following outputs were implemented:  Output 1.1.1: Outreach and recruitment programs targeting young women, students from underserved schools, and those interested in staying in or entering the teaching profession are developed  Output 1.1.2: Mentoring and counseling programs are established for female teacher trainees, individuals from underserved areas, and those interested in staying in or entering the teaching profession  Output 1.1.3: Flexible options for program delivery are developed (e.g., distance learning, evening and holiday classes, etc.) This outcome has been achieved with a relatively high degree of implementation fidelity as follows: Under Output 1.1.1, KIE Staff were trained to conduct outreach in junior and senior high schools. They held a workshop on adolescent health and 28 participants (11 females and 17 males) attended. Eleven females were empowered to lay strategies to reduce adolescent reproductive health. Second, a workshop on adolescent Reproductive Health Services was conducted and Paula Tavrow, UCLA presented. The workshop considered specific reproductive health issues facing adolescents in East and Southern Africa and examined school-based approaches to reduce adolescent reproductive health risks. These school-based approaches were developed based on data from Kenya and elsewhere. The partnership also conducted outreach presentations in communities aimed at keeping girls in school. This activity began in March and was to end in May 2015. Participants in the meetings were local leaders, church leaders, civil society organizations, sector, and district-level heads of schools, education officers and girls. Interviews with a church leader, a head of school, and educational officers in Kayonza highlight the information shared at meetings, including the effect of early pregnancies, the importance girl’s education, and need for the participation of girls and women in the development of the country. The meetings employed tools such as success stories from empowered women, written materials, and videos to raise awareness of the importance of girl’s education. Meeting attendees made resolutions such as educating parents and the community in general, encouraging girls to go to school and punishing men who prey on girls. There is hope that this mobilization will be sustained even when the partnership ends because the community owns it. Clubs aimed at raising awareness about girls’ education were to be established and an active database with students from clubs has been set up. The head teacher in one of the schools in Kayonza said: We met as stakeholders and each has a role to play. Our school has clubs through which we deliver the importance of girls’ education. We also have to discuss with parents whenever we have meetings about hindrances to girls’ education-early pregnancies is a common problem that needs to be addressed. 23 The partnership also trained URCE staff members in mentorship techniques. Charles Alexander of UCLA presented at partnership-organized workshops entitled “Developing Outreach Strategies" and "Successful Mentoring Strategies". The workshops focused on how to reach out to underserved students and how to utilize mentors to support and retain students in an educational program. The workshop also focused on the essential components of a student outreach plan and peer mentorship programs. Mentors were recruited and trained. Twenty-six participants (18 females and 8 males) from an advanced mentor workshop at URCE and its TTCs were trained on how to administer their mentoring program as well as on advanced skills in counseling and advising students on both academic and personal matters. One of the mentor and counselor said, As mentors we attend to students from different social background, there those that need guidance in their career but also others have got huddles in their lives that can stop them from attaining their education. We are able to deal with such issues because the partnership has equipped us with skills. As described above under the Efficiency criterion, TTC staff gained knowledge in counseling and mentoring but it was de-prioritized to avoid duplication, given that there were other similar initiatives already in place at URCE. A manual for TTC operations was not developed because TTCs became part of the School of Education after the merger. They are now administered according to the policy of the School. The overall policy direction is not settled and awaits finalization and procedural announcements from UR. Under the WLP, short-term training as an activity targeted 38 participants but this was exceeded due to local demand for the trainings. The training sessions included over 60 participants from TTCs who felt that these sessions were important to their professional development. In relation to output 1.1.3, the new gender-sensitive curriculum developed under the partnership has not yet been approved. URCE was asked to wait for impending changes in the national curriculum, which MINEDUC is designing, before the curriculum can be submitted to National Council for Higher Education of Rwanda for approval. At the time of evaluation, it was uncertain when the national curriculum was to be adapted. Implementing team members interviewed are optimistic that the gender-sensitive curriculum will be adapted even if the partnership ends because there is a URCE committee in place that will follow up and also the trained staff will be present to integrate the changes. The E-Learning platform to facilitate student teachers to learn without the presence of the trainers was established and is functional. As described above, it has enabled in-service teachers enrolled in the diploma program to learn online and only attend class in person in the evenings and weekends. All these initiatives have been aimed at increasing access to education for Rwandans generally and empowering women specifically. Outcome 2.1: Increased capacity to conduct and disseminate gender-focused research The outputs under this outcome as listed below:  Output 2.1.1: Gender-focused research initiatives are jointly developed between KIE and UCLA.  Output 2.1.2: Workshops on gender-focused research methodologies, exchange training program and orientation are offered to URCE faculty. 24  Output 2.1.3: An institutional review of URCE’s departments and administrative units is conducted to identify opportunities for enhancement and improvement). To achieve this outcome, the partnership conducted a workshop on gender-focused research methodology for faculty. Thirty-five senior lecturers attended this workshop and are now able to integrate gender into their respective fields and research. One of the lecturers who participated in the workshop mentioned that much will be changed in policies that are aimed at gender equity and empowering women because of their new research capacities: We are now implementing research partnerships that will uncover the obstacles to girls’ education. We will use the new skills and knowledge to conduct research partnerships that address gender and produce information that can impact policy. Senior lecturers were also able to initiate studies that have potential to be published in professional journals and presented at conferences. The partnership planned to fund between five and six research partnerships with five researchers each. The first round of proposals was weak, so they conducted more workshops in order to improve the quality of research work. After the workshops, the next call for proposals was successful and effective. At the time of evaluation, six research projects3 had been funded. However, there was a concern that since the partnership was to end in June 2015, a few months from the time of this evaluation, faculty members who were still conducting research at the time of the interviews did not feel that they had enough time to finish the research. In addition to the gender focused research workshop, the partnership conducted a gender awareness workshop attended by 49 URCE faculty staff (26 female and 23 males), as well as 43 non-URCE participants. Output 2.1.3: An institutional review of URCE’s departments and administrative units is conducted to identify opportunities for enhancement and improvement). The partnership targeted three reviews for FY2014. Due to the University of Rwanda reorganization, institutional reviews of URCE's departments and administrative units are no longer within the mandate of the College of Education. This process is now determined by policy and hence beyond the scope of this partnership. This output was replaced with the development of a plan for capacity building of URCE administrative and operating systems in order to enhance efficiency and strategic decision-making. The partnership developed a plan for capacity building of URCE administrative and operating systems to enhance efficiency and strategic decisions. This was achieved by training targeted administrative staff and those who managed operating systems. The administrative staff members were trained in data management to enhance their efficiency at work. A workshop on Access, Excel, and Relational Data and planning, was organized and attended by 8 participants including staff of the department of planning and its aligned offices at URCE. During the training, staff members were taught new skills in streamlining data management, developing targeted reports, and using data for more efficient planning and management. 3 (1) An Investigation into the Gender Responsiveness of the Upper Primary Social Studies Curriculum in Rwanda. (2)Gender Analysis of Research at Higher Learning at Higher Learning Institutions in Rwanda: Looking Back and Thinking Ahead. A case Study of UR-CE. (3) Girls in Technical and Vocational Education and Training (TVET: Towards Alleviating Gender Based Violence Gender Based Challenges Faced by Female Students in Rwanda TSS. (4) Effects of Single Parenthood on Academic Achievements of their Children: Case Study of Gasabo District, Rwanda. (5) Teenage Motherhood and the Right to Education: Experience of Teenage Mother School Dropouts in Rwanda. (6) Gender Differences in Education Outcomes: A Situational Analysis of Enrollment and Graduation and Rates between Public and Private Higher Learning Institutions in Rwanda. 25 Given this data, the evaluators conclude that outcome 2.1 of the partnership has been achieved with a relatively high degree of implementation fidelity. Outcome 2.2: Enhanced visibility of KIE's gender-focused research This outcome had a number of outputs namely:  Output 2.2.1: An online platform is developed to disseminate gender-focused research  Output 2.2.2: Articles, papers, and policy briefs on gender-focused research are published (paper and digital/web-based publishing)  Output 2.2.3: Gender-focused research is disseminated through participation in conferences and workshops. This outcome was to be fully achieved if five publications in form of articles, research papers and policy briefs on gender-focused research conducted at UR-CE were published. A second target was holding two conferences or workshops where gender-focused research would be disseminated. As mentioned above, gender-focused research findings have not yet been published. In addition, this outcome has suffered the delays caused by the merger. Participants also noted a delay in receiving research grants. Because of these factors, it is too early to assess fidelity. However, there is likelihood that this outcome will be achieved with a high degree of implementation fidelity once research projects are finalized. Outcome 3.1: Increased gender awareness of KIE and TTC faculty and in-service teachers This outcome had the following outputs:  Output 3.1.1: Gender-sensitivity and lessons on female empowerment are integrated through curricula re-design  Output 3.1.2: New tertiary education programs are developed: for primary school teachers who already have a certificate, to get a diploma; and - for secondary school teachers who have a diploma, to get a degree.  Output 3.1.3: New teaching materials that incorporate gender-sensitive concepts are developed. The partnership improved the capacity of URCE faculty to train teachers in the integration of gender equity in the classroom. Primary school student teachers at URCE are now trained in skills to address gender equity in the classroom through pedagogy, course content, and classroom management techniques. In addition, in September 2013, the partnership developed a set of modules for a new Diploma in Primary Education program. A curriculum expert at UCLA reviewed the modules and provided detailed feedback to the curriculum committee. The committee incorporated the suggestions and began the approval process, which included a review by the URCE Faculty of Education and a second URCE approval committee before presentation to the National Council for Higher Education of Rwanda, which has a government mandate to approve all new courses before the HEIs can offer them. At the time of the evaluation, the modules were being reviewed by the final URCE approval body before submission to the National Council for Higher Education of Rwanda. The approval process has been long and frustrating for UCLA partly because URCE was not providing UCLA with regular updates on the 26 progress of the approval process. However, with the intervention of the Principal of UCRE, the process began to move faster than before. It stalled again though due to the national curriculum re-design and the need to align the modules with the new national curriculum. A new national curriculum was announced in March 2015, and URCE has begun reviewing the modules of the Diploma in Primary Education curriculum in order to ensure alignment. 4. Impact of Intervention Although the merger of universities caused delays that took almost a year off implementation period, the partnership has had some short, mid-term and long-term impacts. This impact has been at individual, institutional, and national levels. However, while some impact of the partnership was visible, it is too early to make evaluative statements about outcome-level results and expansive effects. The following are partnership activities, which different stakeholders testify have changed their lives. 4.1.Outcomes 4.1.1. Gender focused research capacity Workshops on gender-focused research, exchange training programs, and orientation were offered to URCE faculty. At the individual level, faculty and all trainees who participated in the gender and research methodology workshops testify that their way of thinking about girls’ abilities have changed positively. One female faculty said: Before the gender awareness workshop, I believed in women cannot but today I know women can. I believed that girls should behave differently from boys because they are girls. These attitudes keep girls behind in the way they think and do things. I believe even some other participants have changed their negative attitudes on girls. Change is a gradual process, but when we combine our efforts, gender equity and empowerment of girls in education and other spheres of life will be attained. The increased capacity does not stop at the individual level but extends to the college and community. Female self-efficacy improved by 43 percent (0.38) out of 0.588 targeted. The self-efficacy survey results from 2013-2014 did have limitations, however. Some participants were unwilling to complete survey. Others could not understand information on the form because of the language barrier. Translating the survey into Kinyarwanda could have avoided this problem. Enhanced skills in gender research also impacted the lives of individuals. Because faculty promotions are based on publications, they are eager to complete their research and publish their work in order to be considered for promotions and salary increases. One faculty staff member said: I am sure we shall be promoted after a number of publications. Some faculty could not publish because they lacked sufficient research skills. The partnership has left us more prepared, and equipped with skills in research methods, gender-focused research and knowledge in gender concepts. There are six research projects to be conducted. Each group is composed of both female and male faculty. We have to be exemplary before we sensitize others about gender equity. At the national and institutional levels, the increased capacity in gender research for faculty members has positively impacted URCE’s practices. Faculty who were interviewed during the evaluation exercise said that research also greatly contributes to national and institutional development, because it is through research that impediments to national and institutional development are identified and solutions sought. 27 In an anecdote shared by one interviewee, a male faculty assisted a female student who felt sick while taking an exam. This student was pregnant and needed fresh air. He assisted her in getting open space and she was able to complete her exams. The male faculty member feels that although he believed in justice even before the gender research workshop, he may not have allowed the woman to get out of an exam room because he felt all students should be in one room while sitting for their exams. The faculty member believes that it would have been difficult for the woman if she missed an exam. “Being pregnant does not deprive women their right to complete their education or other rights whether social, economic and political,” he explains. All the modules URCE will develop in future should have gender components, such as those that have been developed during this partnership. As a result of this partnership, URCE’s overall capacity to conduct and disseminate gender-focused research has increased. 4.1.2.Gender-focused teaching capacity Generally, gender awareness of URCE and TTC faculty and in-service teachers has increased. Specifically, 42.89 percent (0.38) reported self-efficacy, although this figure fell short of the target of 0.588. In addition, 83.4 percent of participants in the de-prioritized mentorship-training program reported improvements in their mentoring capacities (WLP-Rwanda Educ-APR, Oct, 1, 2013-Sept, 30, 2014). Mentors from TTCs mentioned that partly as a result of the mentorship, which took place from July￾November 2014, 120 students were registered but some then dropped out of school for different reasons. Some of the constraints to their good performance were lack of enough time to revise, long distances to places of study and nurturing roles that made it difficult for women, especially those with small children, to concentrate. For example, this category of women had to move out of class frequently to breast-feed the babies. An academic staff at a TTC who was interviewed for this study said: Because of early pregnancies, there are special categories of girls who want to attain their education but do not have who to leave their babies with. Their families may not accept them. It is difficult to mentor and counsel such girls. In most cases they get low grades because they have not been able to accept their situations as young mothers who are in school. I believe more women/girls would get better marks if they had no such huddles. 4.1.3.Outreach capacity The partnership conducted outreach presentations in communities aimed at keeping girls in school, however it is difficult to measure the impact of this output on access, retention and completion of females in educational institutions. From the stakeholders in Kayonza where one of the mobilization campaigns took place, clubs were established to sensitize the community about the importance of girls’ education and the fight against all the hurdles to attaining their education. One of the stakeholders, a head teacher in Kayonza said: “We do not known how many girls or community members were reached. We were mobilized to take part in this campaign but we do not know how the success will be measured. 4.1.4.Institutional capacity building The planning office is able to gather and disseminate data that can be used for institutional decision￾making. Capacity for URCE staff especially line managers increased through institutional capacity building workshops conducted under the partnership. Training in research, strategic planning and data management skills were essential for institutional capacity building because it helped URCE academic, administrative, and ICT staff of URCE bridge gaps in their respective domains. 28 4.1.5.Student outcomes E-learning The partnership developed flexible options for program delivery for teachers who were enrolled for the diploma in education. The e-learning platform described above, named “Brain-Share,” was developed and is operational. Students who are using the Brain-Share platform were excited. The majority of these students were not trained teachers with appropriate qualifications, but at the time of the evaluation, those interviewed said that they were equipped with all the knowledge needed for one to teach. Students are proud to say that there have been no dropouts and women are very active in class despite responsibilities at home. The students have gained classroom management and computer skills such as using the Internet to search for information. “We save soft copies of our modules on computers, which is more convenient for us than keeping hard copies” one of the students testified. These students have encountered some challenges though in the course of learning, such as lack of Internet access in remote areas, and lack of money to buy airtime to load on the modems that URCE has provided. At the beginning of the course, many of them were not familiar with computers, but they received assistance from an IT person when problems arose. Students who do not work for the government as teachers have no indefinite contracts, making it difficult for them to access credit services, since most financial institutions are reluctant to offer credit to people with temporary contracts. Consequently, this category of students finds it difficult to pay tuition. Distance is another factor. These students come from different districts, some far from the learning venue (Kabgayi). Some students therefore find it difficult to meet transportation costs. 5. Sustainability 5.1.Financial resources management Sustainable reform within universities requires a number of years to take hold. The merger negatively impacted institutional capacity building, as the time required to work through and establish these processes was shortened. Implementers managing the partnership and HED staff expressed their frustrations with UR’s financial administration systems. The merger brought together different institutional cultures and structures. One stakeholder interviewed estimates that a year and a half were lost in learning who to go to, what forms to fill out, and how to get the requisite permissions. There is very little flexibility on the part of HED/USAID with its financial systems, and there also seems to be little flexibility on the part of UR/UCLA. Though both financial management systems lack flexibility, there is an expectation that the partner institutions need to reform some of their financial management systems in order to run the partnership adequately. 5.2.Non-financial resource planning and management URCE technical staff members were trained in data management skills. This is a sustainable outcome of the partnership because of the multiplier effect it has had and will likely continue to have. The Director of Planning is training UR staff at headquarters to enhance their skills in data management. These staff will train others and pass on the baton to others. Capacity building in strategic planning conducted under the partnership laid a foundation for proper planning at URCE. The beneficiaries of the strategic planning training at URCE have been assigned to train other colleges in UR on strategic planning to ensure successes of UR as a whole. To ensure proper 29 strategic planning, a committee has been established at URCE to oversee strategic planning for the college. These initiatives point to a sustainability of the strategic planning for URCE and UR generally. 5.3.Results-based management It is difficult to make evaluative statements about the sustainability of results-based management. It is too early to determine if institutional capacity efforts have created a management culture that is focused on results. 5.4.Institutionalization Training in research, strategic planning and data management skills have potential to help in strengthening of institutional capacity. However, it is difficult to make evaluative statements about institutionalization because of delays in partnership implementation and the institution itself undergoing possible further restructuring. 5.5.Stakeholder engagement Although the outreach program has lagged behind schedule, the community has been mobilized. URCE reached different stakeholders including local leaders, church leaders, civil society organizations, sector and district-level heads of schools and education officers and girls to sell the idea on the importance of education of the girl child. Some respondents believed that the outreach is sustainable because it is owned by the community and leaders from the lower levels of administration up to the decision making body (ministries). One respondent said: When a campaign to raise awareness of the importance of girls’ education starts in a district, authorities are engaged at every step. They invite other stakeholders in the same district to participate in the campaign. Clubs aimed at raising awareness about girl’s education were to be established and active databases with students from clubs were set up. There is hope that this mobilization will be sustained even when the partnership ends because the community owns it. The partnership also engaged other stakeholders such as Women for women Rwanda. Members of Women for Women Rwanda took part in gender awareness sessions. The partnership reached out to the NGO and visited their women's opportunity center in Kayonza. As result of this linkage other future activities between the partnership and NGO are being planned. This is a good sign that programs aimed at gender equality and women’s empowerment, which is the major target of WLP in Rwanda, will continue to be implemented by stakeholders such as women for women international Rwanda even when the WLP closes. GirlHub The partnership reached out to GirlHub Rwanda as a collaborative partner for the screening of "Girl Rising" at the partnership launch. This was because the movie aims at empowering young women, which is the same mission of the WLP in Rwanda. Since GirlHub is an established NGO in Rwanda, and continues to engage in activities aimed at empowering women in education and gender equality, the gains of the partnership will be sustained through GirlHub initiatives. Regional university engagement Also, staff and faculty of private colleges and universities were offered an opportunity by the partnership to participate in training workshops on gender equality and women empowerment. It is therefore highly likely that these faculty members will keep putting in practice the knowledge and skills 30 gained in these workshops to advance gender equality and women empowerment in their work. It is therefore most likely that the private universities as stakeholders who were engaged in the WLP activities will sustain issues of gender equality in classroom pedagogy. Rwanda Education Board (REB) The partnership has an existing relationship with REB. Board members were part of the curriculum development workshop held in September of 2013. During training workshop REB leadership took part in various workshops. As a body charged with curriculum development and teacher inspection, it is hoped that REB staff will apply the knowledge and skills learned in the workshop in their work especially curriculum development and support other initiatives aimed at promoting gender equality and women empowerment in education through curriculum design and pedagogy. Microsoft UCLA has existing relationship with Microsoft. As part of that relationship Microsoft became a technology partner. Microsoft was represented at the launch, participated in the technology workshops, and consulted on the technological needs of the partnership. 5.6.Continued relevance of program design The partnership objectives will have continued relevance for years to come. Gender is a crosscutting issue in development strategies of all sectors in Rwanda. Gender development is a cornerstone of current economic development orthodoxy, and is reflected in Rwanda’s Vision 2020 (MINECOFIN, 2000) and Economic Development and Poverty Reduction Strategy 2 as well as donor funding strategy. The demand for well-trained professionals in gender studies is still high in Rwanda where every Ministry and other sub-sectors have a position for a gender-focused person. It is therefore vital that professionals of all disciplines, more so teachers are versed with gender concepts and basic skills of gender analysis and application in their work. The curriculum and module delivery pedagogy developed under the partnership are consistent with those of Rwanda MINEDUC. This consistency implies that these elements will be carried on after the closure of the partnership. Where there is inconsistence, the curriculum developed by the partnership was to be aligned with Rwanda’s national education curriculum to avoid any clashes, which is a good foundation for sustainability of its goals. The program developed UR Connect platform, which should also promote sustainability. This platform was launched April 22, 2015. Unlike the Brain-Share platform, which requires Internet connection every time the user tries to access materials, UR Connect platform works differently. Users can download materials and save them for later use when they are not online. This platform enables students from rural areas or those that have no fulltime Internet access to e-learn. This is a sustainable venture because the program and the Government of Rwanda have put in place mechanisms to sustain the initiative. For example, under the program, staff of URCE working in ICT were trained and equipped with skills required to use the platform. To ensure effective and appropriate use of the platform, a user manual was developed and given to staff. A follow up meeting between ICT staff and a representative of the company, which developed the UR Connect platform, was held in March 2015 to enable ICT staff to know more about the new platform. UR Connect is in line with learning and teaching innovations in Rwanda. For instance, the Government of Rwanda’s seven-year program emphasizes growth of ICT in education. Therefore government policy is that ICT use should be an integral part of teaching and learning in all education institutions. The UR 31 Connect platform will likely be sustained after the close of the partnership. However, during the focus group discussion, student beneficiaries of e-learning underlined the need for computers if e-learning was to take root. There were concerns that the majority of the Diploma students would not afford to buy computers on teacher salaries if they were to hence failure to engage in e-learning. However, some respondents from URCE administration said that the University has the capacity to buy computers, implying that the component will be sustainable since URCE has both human and financial resources necessary for e-learning and teaching. The partnership’s increased research capacity also appears sustainable. There is recognition on the part of UR that research is integral to the University’s growth and the nation’s development priorities. Partly with the research skills gained during research trainings and workshops under this partnership, faculty will be able to engage in gender-related research. In UR, as in most academic institutions the world over, academic staff promotion is based on the number of published works among other factors. Therefore, it is anticipated that faculty will publish periodically for purposes of promotion, and also with an aim of enhancing knowledge of gender and reforms in the educational sector especially those focusing on improved educational opportunities for the girl child. In addition to research funding and conduct being a priority on URs agenda, another key strategy is that UR has been assisted by UCLA on how to find and reach donors. Objective 2- better informed institutional policies and practices through gender-focused research and capacity buildings- seem to have made greater strides toward sustainability. Agriculture Partnership This section discusses findings from data collected through document review and during fieldwork conducted in Rwanda from March 23 to March 30, 2015 and in the U.S. via virtual interviews with MSU and WSU implementers from April 14 to April 21, 2015. 1. Efficiency 1.1.Performance management Results-based management The partnership was largely designed and implemented following USAID and HED’s results-based management system. HED helped the partnership establish monitoring and evaluation processes at the onset of the grant in October 1, 2012. Results-based management protocols observed include: 1.) submission of annual implementation plans for FY2014 and FY2015; 2.) baseline assessment data collected in August 2013 and August 2014; 3.) Semiannual and annual HED verification of data quality in FY2013 and FY2014; and 4.) submission of standard indicator definitions along with up-to date revised monitoring and evaluation plans. Tools such as the Partnership Monitoring Plan and Partnership Implementation Plan were also used to ensure proper implementation of the project both in terms of monitoring achievements and respect of schedule. Reporting To ensure consistent monitoring and reporting on the partnership’s intended results, the partnership regularly submitted semiannual and annual progress reports over the course of its life cycle. Although some project activities were still ongoing at the time of this report, project implementers have aimed all activities toward achieving the outcomes established in the results framework. 1.2. Decision Making Processes 32 Inclusive decision-making processes began at the initial stage of the partnership. During information sessions, MSU did preliminary assessments of NUR’s needs. Originally, a team of faculty and staff from MSU, WSU, and NUR collaboratively developed a proposal for a MSc in Crop Protection and Phytosanitation. However, the decision to develop a gender-sensitive MSc in Agribusiness instead of Crop Protection and Phytosanitation seems to have been exclusionary- with neither NUR nor MSU included in the process. Though USAID approved the jointly developed proposal for a Crop Protection and Phytosanitation master’s program, stakeholders from the MSU-NUR partnership team indicated in interviews that they were excluded from the decision-making process that shifted the program focus from crop protection and phytosanitation to agribusiness. This decision seemed to have been made by USAID-Rwanda, independent of stakeholder input. Stakeholders interviewed from MSU, HED, and CAVM could not offer any insight into how the decision to switch the program focus from Crop Protection and Phytosanitation to Agribusiness was made. MSU was asked to rework their proposal to accommodate this change, which resulted in some work needing to be done to get NUR on board. Toward this end, according to MSU stakeholders, MSU leveraged its extensive alumni network to identify Eastern and Southern African MSU Ph.D. graduates teaching in post-graduate Agribusiness programs in African universities. MSU invited these alumni to share their perspectives and experiences with Agribusiness program design with their NUR colleagues. 1.2.1.Timeliness of Activities University system merger In September 2013, the NUR ceased to exist as a legal entity, and the process of merging five public institutions into one university system was initiated. Due to the complications resulting from the merger, the decision-making process around the timeline of activities lost much of its earlier inclusive nature. While UR moved through its merger process, joint MSU-NUR partnership decision-making processes seemed to halt. Though the MSc in Agribusiness was submitted and approval was received in August 2013, NUR did not successfully forward the proposal to the Higher Education Council (HEC) at the Ministry of Education before the merger process began. Once the merger process began, the status of the proposal was ambiguous. The MSU team was unclear as to who was authorized to forward the proposal to the Ministry of Education. On December 6, 2013, the WLP partnership leadership team met with UR Vice Chancellor, Dr. James McWha, and the Principal of the newly formed CAVM, Dr. Laetitia Nyinawamwiza, to discuss the status of program approval. During that meeting the WLP partnership leadership team learned that the proposal had not been forwarded from NUR to the HEC, and that the UR Deputy Vince Chancellor for Academic Affairs would need to review the proposal before it could be forwarded to the HEC. By February 14, 2013, the WLP partnership leadership team received word that the UR Deputy Vince Chancellor for Academic Affairs had reviewed the proposal and forwarded it to the HEC. By February 21, 2013, the HEC had received the proposal. In March 2013 the proposal was still with the HEC pending approval. The launch of the MSc in Agribusiness finally took place on January 22, 2015. The first cohort of 23 students began classes in March 2015, well after the original target date. The restructuring resulted in the formation of CAVM and MSU lost some of its decision-making counterparts as CAVM’s leadership team was reconfigured. The partnership lost its UR Partnership Director, Dr. Solange Uwituzi, who was also the Director of the Department of Agroeconomics and Agribusiness and Dr. Masanja Grace Verdiana, the UR Gender Expert, before the MSc launched in March 2015. According to the October 2014 MSU trip report documenting partnership activities that took place in August 2014, the MSU team learned then that Dr. Solange Uwituzew would be leaving UR in the middle of October 2014 to join RUFORUM at the Makarere University. Dr. Jean Chrysostome Ngabitsinze, then Project Coordinator, assumed the leadership role after Dr. Uwituzew’s departure and became the UR Partnership Director. Dr. Wanjiku Chiuri was hired as the gender expert. As the new UR 33 leadership team took shape, and the “bureaucratic tangles associated with the merger” (Mendendorp, Decemmber 2013 Trip Report) were worked out, the implementation timeline picked up considerably, increasing the efficiency of the partnership’s activities. 1.2.2.Use of Resources (Human, financial and Non-Financial) Many human resource decisions and processes are internal to institutions. The merger necessitated internal human resource decisions and processes that excluded MSU. The MSU team reengaged once these internal decisions were made and collaborative decision-making resumed between the partner institutions, albeit with a newly composed leadership team and expanded roster of beneficiaries for capacity-building workshops. Through a collaborative negotiation process, the team made decisions about human, financial, and non-financial resources at the partnership level, which then required higher-level approvals within each partner institution. The partnership leadership team was excluded from the decision-making processes behind the approvals. According to interviews, members of the partnership team, especially from MSU, expended quite a bit of energy trying to negotiate relationships and influence decision-making at higher levels. 1.2.3.Feedback Loops Because of the shifting roster of stakeholders, feedback loops were constantly being established, reestablished, and strengthened. Progress reports provided a more formal mechanism for feedback, and this mechanism was used efficiently. The more informal feedback loops of in-person and virtual meetings, phone calls, and emails in place at the time of fieldwork seemed increasingly more solidified, fluid, and well functioning. 2. Relevance Program activities and interventions were adapted to meet the local context and program design consistently linked activities and outputs logically to program outcomes and objectives. 2.1.Contextualization 2.1.1.Local and National Development Priorities The activities of this partnership are aligned with Rwanda’s national development priorities as articulated in the most recent and relevant policy documents: Rwanda Vision 2020 (MINECOFIN, 2000); Economic Development and Poverty Reduction Strategy 2 (MINECOFIN, 2013); Higher Education Policy (MINEDUC, July 2008); Education Sector Strategic Plan 2013/2014- 2017/2018 (MINDUC, October 2013). From the beginning of partnership implementation, local and national development priorities were central to the work of the partnership. The curriculum writing workshop held April 2-3, 2013 brought together NUR faculty and external stakeholders including the director of the Higher Education Council, a representative from an NGO, a representative from the private sector, and representatives from other universities. MSU Partnership Director, Dr. John Mendendorp, began the workshop with a brief presentation on the stages of curriculum development before presenting the objectives, pillars, and cross-cutting issues of Rwanda Vision 2020. Dr. Mendendorp then posed the questions: 1.) “How ought this program support the Rwandan development goals and 2.) What ought the contents of this program be in order to accomplish this?” (WLP Annual Progress Report October 1, 2012 to September 30, 2014, p. 5). 34 USAID is not the only donor organization assisting in building University of Rwanda’s capacity. The Swedish National Development Agency (SIDA) has been working with University of Rwanda/ National University of Rwanda for over ten years. In August 2013, SIDA announced the third phase of their 10- year bilateral relationship and the development of three new MSc and three new Ph.D. programs in the areas of Animal Science, Plant Breeding and Genetics, and Plant Protection and Phytosanitation through a collaboration with the Swedish University of Agricultural Sciences (SLU). The curriculum for the MSc in Agribusiness designed by MSU and NUR/ CAVM was designed to complement the MSc in Agricultural Economics, Agroforestry and Soil Science developed and funded in collaboration with SLU. Students in both MSc programs would take their first year coursework together before students went off to complete their internships during the second year of the program. The shared courses for Semester 1 include: 1.) Microeconomics; 2.) Macroeconomics; 3.) Econometrics; 4.) Economics and Institutions Analysis; and 5.) Global Food Systems Analysis. However, when the MSc in Agribusiness students started their coursework in March 2015, the MSc in Agricultural Economics, Agroforestry and Soil Science students were not present. It is unclear the degree of progress CAVM has made with this MSc funded by SIDA. 2.1.2.Responsiveness to Beneficiaries Needs The two sets of beneficiaries interviewed, MSc students and CAVM faculty, expressed the relevance of partnership activities to their personal and professional development. Two of the MSc students interviewed are also CAVM tutorial assistants and attended the pedagogy and gender workshops. A two￾day workshop took place on December 4, 2013, entitled “Creating a Gender Inclusive Classroom”. It included a morning session facilitated by Dr. Jolly Rubagiza, Director of the Center for Gender, Culture, and Development Studies at UR and Dr. Amy Jamison, MSU’s gender expert. An afternoon session entitled “Improving Student Learning Through Innovative Teaching” also took place, facilitated by Dr. David Wong from MSU’s College of Education. The partnership also organized a three-session workshop dedicated to mentoring on December 5, 2014. Ms. Eugénie Mukanoheli, National Coordinator of the Forum of African Women Educationalists-Rwanda Chapter (FAWERWA), facilitated two presentations on the importance and relevance of a gender-sensitive mentoring program for university students and the tools needed for effective gender-sensitive mentoring. Next, Dr. John Medendorp facilitated a session on how to build an effective mentoring system within the administrative structure of UR. One MSc student and tutorial assistant mentioned that while he has a BA in Agriculture Economics and Rural Development, he felt unqualified to teach a course assigned to him. He felt his content knowledge was not strong enough nor did he feel he was pedagogically and methodologically prepared. By attending pedagogy workshops and absorbing the content of the modules in the MSc program, he felt better equipped to teach more effectively. He shared, “In area of research and teaching, the program will enhance what I already know.” He also expressed the differences in approach between his BA program and the MSc program: If you compare learning environments, the BA level is heavy and hard. [The MSc program] is simple and real. For example, the way the lecturer [at the BA level] teaches, [he] is accustomed to use[ing] chalk and blackboard. [This] time is the first time to see projections. This way you gather information as to how the teacher is selling it. It’s much easier than previous training. Learning is much simpler and easier. Prior, you read and read and read. 35 All students appreciated the practical nature of the MSc program, especially the training in soft skills provided by Kepler and the link with the private sector. MSc students feel confident that they will have no problems finding employment upon completion. Many students remarked that the market demand for well-trained professionals in agribusiness is high, while the supply is low. One MSc student remarked, “Concerning market needs, we’re not enough …. We will choose masters in Agribusiness even if we come from Animal Sciences because there are more opportunities.” The delivery model of the program is also responsive to student needs. Most students appreciated the alternative delivery systems. The format of face-to-face meetings all day on Saturday and for half a day on Sunday accommodates students’ professional and family obligations. A few of the MSc students live and work outside of Kigali and commute to Rubilizi campus for weekend modules. While they found the transportation costs to be a financial burden, they seemed to be willing to take them on. They also appreciated having digitized content accessible on tablets provided by the program, though students expressed that some of their peers have limited access to wireless Internet, and this presents some problems. CAVM faculty and teaching staff interviewed praised the relevance through of the capacity-building workshops. One CAVM teaching staff member who attended workshops on pedagogy, inclusion of gender in the curriculum, and e-learning was effusive about her professional development. Though she has content knowledge in agriculture economics and agribusiness, prior to the workshops, she had not received training in pedagogy. Today she has a better sense of how to prepare her course to be more student-centered. Her awareness on gender issues in her classroom has also grown. She comments, `“As they started, they opened my eyes about how gender matters and how to include gender in the classroom… how women and girls are involved, practical works on how to integrate, assess participation of girl students… how they are attending, how they are behaving vis a vis are they on time… Are they late? Are they absent. Now I am taking care.” This stakeholder anecdote is consistent with data collected pre- and post-event evaluations carried out during workshops. The results of these evaluations showed some minimal progress in understanding, though it was difficult to know which respondents were present for the entire event, which left early, and which arrived late. MSU also acknowledged that the time between pre- and post-test was too short, thus any expectation for significant changes in attitude is unrealistic. Nevertheless, the partnership team did detect slight positive changes in gender attitudes (MSU Trip Report December 2013). The program was also relevant to the research agenda and needs of CAVM. The University of Rwanda has prioritized research in general as one of the ways through which socio-economic development can be achieved. In order to achieve Outcome 1.3, increased applied learning and research opportunities for agricultural science students and academic staff at CAVM, the project rendered technical support to CAVM through training in research, as well as provided funds for faculty to conduct research in agribusiness. Faculty at URCE who received research grants are expected to have finalized their research and be in position to disseminate the findings by the end of June 2015. The project was therefore relevant to the needs of URCE because it has improved the capacity of faculty to conduct research. The partnership research efforts are also responsive to the Agriculture sector. Topics that are being pursued include: 1.) A nationwide crop insurance exchange for coffee farmers; 2.) New terracing practices for farmers; 3.) A change of food safety practices in a limited number of food processing industries; 4.) Changes in animal nutrition practices for increased dairy production; and nutritionally improved strains of wheat for Rwandan farmers. 36 Another beneficiary of the partnership is the agribusiness sector itself. In April and May 2013, Dr. Colleen Taugher (WSU) and Dr. Luis Flores (MSU) collected stakeholder baseline data through a series of meetings with a network of private sector, NGO and government collaborators identified by NUR/CAVM and MSU. The data collected offered the partnership team insights into the knowledge, skills, and attitudes potential employers believed MSc graduates would need to possess. The results of this effort have guided the curriculum development efforts. These meetings were also instrumental in establishing a network of partners and collaborators who would serve an advisory role, host interns, participate in experiential learning programs and hire graduates. On December 5, 2013 the inaugural meeting of the Advisory Committee took place. 2.1.3.Reflection of Funder Strategies The partnership is aligned with USAID funding strategies in the sectors of education, gender equality and women’s empowerment, and agriculture and food security. Partnership interventions align with Goal 2 of the USAID Education Strategy by supporting CAVM’s improved ability to produce a workforce of agribusiness professionals with relevant skills to support country development goals The partnership also aligns with two out of three goals in USAID’s 2012 Policy on Gender Equality and Female Empowerment: 1.) Reducing gender disparities in access to, control over and benefit from resources, wealth, opportunities and services - economic, social, political, and cultural; and 2.) Increasing capability of women and girls to realize their rights, determine their life outcomes, and influence decision-making in households, communities, and societies. Third, the partnership’s objectives have also been designed and implemented to align with USAID’s agriculture and food security sector funding priorities, which are captured in the Feed the Future initiative, and include 1.) investing in cutting-edge scientific and technological agricultural research to develop stronger seeds and greener fertilizers so farmers can grow more; 2.) developing agricultural markets, expanding trade and using mobile phones to provide real-time prices, so farmers can sell what they grow at a profit; 3.) offering extension services, so farmers can learn the best techniques to grow and store their crops; and 4.) developing sustainable agriculture strategies, so countries can feed their populations without depleting their natural resources. USAID’s former commitments to development priorities in the agriculture sector also underpin the partnership. The Partnership for Enhancing Agriculture in Rwanda through Linkages (PEARL) and Sustaining Partnerships to enhance Rural Enterprise and Agribusiness Development (SPREAD) programs were previous USAID programs in which MSU was an implementing partner. PEARL was a six-year project (2000-20006) which partnered MSU and Texas A&M University with NUR to provide human and resource development for Rwanda’s agricultural institutions and establish community development outreach programs (Chemonics 2006). SPREAD, a $5 million grant between the Borlaug Institute for International Agriculture and the Rwandan National Development Board, NUR, Radio Lifeline, Population Media Center, Ritchey Design Inc. focused on increasing incomes and improving standard of living through value chain development from 2006-2012. Key program components included off-farm development, capacity building in management, quality improvement and control, product marketing, coffee quality research and development, educational outreach, access to credit, and health and HIV/AIDS awareness (http://borlaug.tamu.edu/projects-by-region/sub-saharan-africa/rwanda-spread/). 37 2.2. Logic: Causal Linkages and Multiple Pathways The causal linkages articulated in the partnership results framework are sound. Though program does not have an explicitly stated theory of change, the results framework appears as part of “MSU-NUR ME Plan” dated 29 October 2012, in which a hierarchy of results, along with indicators, and assumption statements are offered for each outcome and output. At the highest levels of the framework, the partnership also set objectives and a goal. The lower-level outputs support the mid-level outcomes, which in turn support the broader objectives of 1.) Strengthening the human and institutional capacity; 2.) Promoting and supporting women’s access to graduate education in agricultural sciences; and 3.) Extending CAVM’s knowledge about, and women’s expertise in, agricultural sciences to the community. There are multiple pathways to meeting objectives. Objective 1 has three outcomes and six outputs. Objective 2 has one outcome with three outputs. Objective 3 has a single pathway with one outcome and one output identified. Based on an interpretation of the results framework, the underlying theory of change states: If... 1.) institutional capacity in teaching and applied research in agricultural sciences is strengthened; 2.) women's access to graduate education in agricultural sciences is promoted and supported; and 3.) the College of Agriculture and Veterinary Medicine’s knowledge about, and women's expertise in, agricultural sciences is extended to the community... Then…the capacity of the University of Rwanda- College of Agriculture and Veterinary Medicine to advance women's leadership in the field of agriculture will be strengthened. The degree to which the partnership has put the links in place to achieve this overarching goal is examined in the next section. 3. Effectiveness This section explores the effectiveness of the partnership, as measured by the progress it made toward achieving targets set for each output in its results framework and its fidelity in content and delivery. After substantial delays in the approval process for the MSc in Agribusiness, the partnership made significant progress toward achieving its intended outputs and outcomes. Program fidelity was high and was demonstrated through the delivery of activities that adhered to the partnership’s plans. 3.1.Progress vs. Targets and Fidelity Outcome 1.1 A gender-sensitive Master of Science (MSc) program in Agribusiness that prepares students for careers and leadership roles is established and offered. To accomplish this outcome, the partnership implemented activities under the following outputs:  Output 1.1.1. A gender-sensitive curriculum in agribusiness that meets Rwandan and international requirements for a master's degree program is developed)  Output 1.1.2. Gender sensitive course modules on multidisciplinary professional skills which prepare women students for leadership roles are created Because of the results of these two outputs, this outcome has been achieved with a high degree of fidelity. The gender-sensitive MSc program that prepares students for careers and leadership roles has been established and is being offered at present. The inaugural cohort of 23 students began coursework 38 on in March 2015, a year and a half after the launch date of December 2013. The partnership leaders developed a gender-sensitive curriculum that met Rwandan and international requirements. The program was put together through an inclusive and collaborative development process that included a curriculum writing workshop in April 2013 and a peer review process involving various Rwandan and international and participants. The MSU and NUR implementing team faciliated the curiculum writing workshop, which was attended by 23 NUR faculty, civil society members, agribusiness professionals, a Ministry of Agriculture representative, and the Institute of Agriculture and Animal Husbandry (ISAE). During this two-day workshop the implementing team collected a wide-ranging input participants and was captured their feedback through the collaborative process of curriculum design. The team delegated responsibilities and established a timeline for submission. The new curriculum was submitted to the NUR Senate in July 2013, approved, and then forwarded to the Vice Chancellor’s office where it sat unattended through the merger process. Eventually the curriculum was reviewed and approved by the Higher Education Council and Cabinet in November 2014. Outcome 1.2. Enhanced pedagogical skills of faculty to teach the gender-sensitive agricultural sciences curriculum and supervise agricultural sciences research. To accomplish this outcome, the partnership team implemented activities under the following output:  Output 1.2.1 Faculty development training about agricultural sciences teaching methods delivered The results demonstrate that this output has been achieved with a high degree of fidelity. The partnership offered capacity-building trainings in pedagogical skills. In addition, ad ten-day study tour in June 2014 brought seven faculty members and CAVM’s finance staff member to MSU in East Lansing and the annual meeting for International Agriculture Research Development (IARD). During this visit, CAVM teaching faculty participated in workshops that included modules on backward design and syllabus design, writing learning outcomes, assessing student learning, and designing active and experiential learning activities. These workshops have continued. Knowledge exchange meetings at MSU for CAVM faculty to build new content knowledge and build collaborative research relationships with MSU faculty were not included on the study tour agenda, though such meetings may have taken place informally. Relevant content on gender was incorporated into the pedagogy workshops. Other faculty training-related activities under this output include developing a plan for improving institutional infrastructure for research and teaching by increasing access to e-resources from MSU’s library pending approval from library administrators. In addition, an internship handbook, developed by Dr. Colleen Taugher, Dr. Daniel Rukazambuga, and Mathias Ndizhiwe, Director of Career Development, Continuing Education, and Faculty Development, is currently under review by the partnership’s advisory board. Outcome 1.3 Increased applied learning and research opportunities for agricultural science students and academic staff. To accomplish this outcome, the partnership team implemented activities under the following outputs:  Outputs 1.3.1 Increased applied learning and research opportunities for agricultural science students and academic staff  Output 1.3.2 Joint UR and MSU/WSU applied research initiatives that address practical problems in agriculture and economics conducted 39  Output 1.3.3. Establishment of an advisory committee, including university and external stakeholders, for the development and delivery of the agribusiness master's program All three outputs have been achieved with a high degree of implementation fidelity. In September 2013 the partnership hosted an academic exchange between MSU and CAVM faculty. Prior to the exchange, Researchers were grouped according to similar research interests and then collaboratively drafted proposals. Three proposals in dairy nutrition, soil health, and coffee value chain were successfully submitted in October and November 2013. However, the partnership ran into repeated difficulties in implementing the grants and asked the three research teams to resubmit their proposals. Five proposals submitted in the second round that were considerably stronger were awarded grants of US$20,000. Research teams were working during the time of fieldwork and are expected to finish in June 2015. Outcome 2.1 Barriers to women pursuing advanced degrees in agricultural sciences are reduced. To accomplish this outcome, the partnership team implemented activities under the following outputs:  Output 2.1.1 Flexible delivery options that make master's coursework accessible to working women and women with family responsibilities are identified and institutionalized  Output 2.1.2 Affordable educational financing options for women are made available),  Output 2.1.3 Mentoring programs for women pursing a graduate degree in agricultural sciences are established Activities aimed at accomplishing all three of the above are in progress with a relatively high degree of implementation fidelity. Out of 23 students in the inaugural MSc cohort, 11 are women. All 11 women received full scholarships. As mentioned in the section discussion partnership relevance, the delivery of the MSc program has been structured to be flexible, with face-to-face instruction taking place on weekends to increase enrollment and retention of students who have family obligations and professional responsibilities outside the program. Finally, the partnership has taken measures to boost the mentorship capacity for women pursing advanced degrees in agricultural sciences by providing workshop training to faculty members. Outcome 3.1 Extend CAVM's knowledge about, and women's expertise in, agricultural sciences to the community. To accomplish this outcome, the partnership team implemented activities under the following outputs:  Output 3.1.1 New extension activities that improve technical knowledge of external stakeholders are in progress. It is too early to the degree of fidelity to the partnership’s plans for this outcome. A seven-person advisory board representing both public and private sector interests has been established. Members represent the Parliament, the Rwanda Agriculture Board, the Rwanda Export Board, Chamber of Industry, an international farmer entrepreneurship company, and a local NGO. It remains to be seen whether the advisory board’s connections will allow the partnership to extend its reach beyond the CAVM walls in order to bring technical knowledge into surrounding communities. 40 4. Impact The significant delays caused by the merger complicated the approval process for the MSc in Agribusiness. The first cohort of students began coursework just two weeks prior to the evaluator team’s field visit in March 2015. Thus, conclusive submitted to HED in progress reports does suggest that the partnership was successful in laying the foundation for long-term change. Overall, gender awareness grew as a result of the partnership’s activities. After each of the workshops held on pedagogy and gender, gender-sensitive curriculum development, and mentorship for female graduate students, the partnership implementation team carried out surveys in order to measure changes in women’s perceptions of self-efficacy and all participants’ views on gender equality. The results of these surveys showed that at most of the workshops, the majority of women reported an increased belief in their own self-efficacy. The pre-and post-workshop surveys also showed that the majority of participants’ views either changed favorably in regard to gender equality or remained the same (in some cases with no possibility of improvement because these participants had already scored the maximum on the pre-test). The MSc students interviewed were confident in their abilities to secure a well-paying job in Agribusiness after finishing the program. They also expressed confidence that they would be able to excel in their future jobs because they would be well trained theoretically, practically, and would possess the soft skills international companies expect of their employees. Research and teaching faculty also displayed self-efficacy. Faculty who conduct research spoke of the value of the research projects that they were undertaking and the value of the collaborative research process. They recognized the need to publish in top tier journals and the value of publishing for tenure and promotion. Teaching faculty interviewed spoke confidently about their increased pedagogical skills and increased gender awareness resulting from program activities. The partnership implementation team did, however, encounter some data collection challenges. It was difficult to control for participation in events where attendance was fluid. As a result, it is that some survey takers did not take stay for the duration of the workshops. Also, the time between pre- and post￾test was often short, making the expectation of significant changes in attitude, perhaps, unrealistic. 5. Sustainability This section discusses the partnership’s sustainability on six dimensions: financial resources management, non-financial resources management, results-based management, institutionalization, stakeholder engagement, and continued relevance of program design. Despite the lack of faculty capacity with expertise in the discipline, the MSc in Agribusiness seems to be firmly integrated into the College of Agriculture’s graduate offerings and will remain so as long as there is continued support from college leadership. The growing relationships between CAVM and the private sector interests as mediated through the Advisory Board are an asset and are integral to the success and sustainability of the program. It is imperative that the college develops its human capacity in the field of agribusiness to ensure sustainability. Overall, the partnership relationship between MSU and UR-College of Agriculture is strong and future collaborations with a variety of funders are in development. 5.1.Financial resources management Sustainable reform within universities requires a number of years to take hold. The merger negatively affected the partnership’s institutional capacity building momentum, as the time required to work through these processes was truncated. In interviews, partnership and HED staff members expressed 41 their frustrations with UR’s financial administration systems. The merger brought together different institutional cultures and structures, and one interviewee estimated that it took nearly a year and a half to learn which financial staff members to consult, which forms to fill out, and how to get the requisite permissions. There is very little flexibility on the part of HED/USAID to adapt its financial systems to the partnership’s context, and there also seems to be little flexibility on the part of UR/CAVM, but it appeared that UR/CAVM would have to be the one to make accommodations to its systems in order to make progress. There have also been expenditure problems on the CAVM side of the partnership. During the merger, the partnership could not access award money for over six months. When the account was finally accessible, the implementing partners found that management systems and personnel did not operate efficiently. Invoices and time sheets fail to be submitted in a timely fashion and CAVM and UR requisition and vendor awarding systems are slow and inefficient. In order to complete activities, MSU had to take lead on the management of the finances. The MSU leadership team often traveled to Rwanda with thousands of dollars in cash to pay for goods and services, resulted in MSU spending down its portion of the overall award, while CAVM’s award remained relatively untouched. 5.2.Non-financial resource planning and management There has been some capacity building around planning and reporting. A planning and reporting retreat in the first two quarters of FY2014 and the CAVM MSc partnership team’s capacity building trip to MSU in June 2014 helped to grow non-financial resource planning and management capacity. The CAVM faculty’s enhanced pedagogical skills to teach a gender-sensitive agricultural sciences curriculum and supervise agricultural sciences research bode well for sustainability providing the right support remains in place. The pedagogy and gender workshops have been followed up with a series of individual and small group meetings in which faculty have submitted course syllabi to Dr. John Mendendorp and Dr. Wanjiku Chiuri to assess the integration of learner-centered, experiential pedagogies and gender sensitivity. At the time of fieldwork, Dr. John Mendendorp and Dr. Wanjiku Chiuri were in the process of visiting all of CAVM courses. Based on past experiences with uneven support from higher-level administrators, partnership team members expressed concern in interviews for sustaining initial program successes if such support is not in secured. 5.3.Results-based management It is too early to determine if institutional capacity efforts have created a management culture that is focused on results-based management of programs. 5.4.Institutionalization Since it was implemented in a shortened timeline, the evaluation team cannot make conclusive statements regarding the institutionalization of the partnership’s results. 5.5.Stakeholder Engagement The partnership has worked hard to engage stakeholders, especially the private sector. Institutional and individual relationships have been built through listening meetings and through the advisory board. The creation of a director position for Continuing Education, Faculty Development, and Student Support demonstrates commitment on the part of CAVM to maintain private sector engagement and sustain the experiential and practical nature of the MSc curriculum. This aligns with industry need. In order to 42 maintain the progress made in capacity building around gender sensitivity, however, a full-time gender expert is required. 5.6.Continued Relevance of Program Design The program objectives will have continued relevance for years to come. The field of agribusiness is experiencing tremendous growth and agribusiness development is a cornerstone of current economic development orthodoxy, and is reflected in Rwanda’s Vision 2020 (MINECOFIN, 2000) and Economic Development and Poverty Reduction Strategy 2 as well as donor funding strategy. The demand for well￾trained professionals in agribusiness is high and will remain so as Rwanda’s economy develops. As the country shifts from female-dominated subsistence agriculture to male-dominated commercial agriculture, current gender imbalances and inequalities may deepen. Transforming gender roles and gender understandings in agriculture and more generally is crucial to economic success and societal cohesion. 43 Recommendations Recommendations for WLP Education and Agribusiness 1. University of Rwanda procurement regulations should be revised with an eye to expediency and efficiency. University and college financial systems and finance personnel must be flexible enough to accommodate the demands and requirements of various regional and international partners. However, donor support of University of Rwanda needs to be better coordinated. The Directorate of Planning and Development can play an instrumental role in building systems for more harmonized support, especially in financial management and reporting systems. 2. Institutional support at various levels across the colleges and University must be secured and maintained. 3. UR should collect gender-disaggregated data in whatever they do. This will help it to gauge its gender sensitivity in service delivery. 4. Gains in institutional capacity have a better chance of being sustained with broad-base support from faculty, staff, and administration. While considerable effort has been made to earn buy-in from all faculty, staff, and administration from the outset, it must remain a priority. Recommendations for WLP-Education 1. The URCE partnership implementing team requested an extension of the partnership because there are critical activities pending due to the delay caused by the merger of universities into one university. For example, the outreach program has had little time for implementation. URCE would have benefited from seeing the implementation of all phases of the program: community mobilization, recruitment phase and follow-up. Also, the online platform is a pilot. Implementers and all stakeholders wish to see the outcome, and use lessons learned to improve the developed UR Connect platform. 2. Mentoring content should be strengthened and more time allocated to the training component. In addition, mentors need motivation to do their job efficiently. They should make themselves known to the college community for students to access them. 3. When setting agendas for partnership activities, consider narrower scope of activities. 4. Improved infrastructure can ensure sustainability of online learning. The government should provide electricity to most remote areas without access. 5. TTCs are affiliated with URCE. There are should be strategies where URCE can foresee the mentoring and counseling activities. The support may not be monetary. Orientation and a template to track what they are doing may be enough to improve their overall performance. Mentors and counselors should be facilitated by URCE to achieve their goal. Although they are rendering services (recruiting, mentoring and counseling) to students, they are handicapped. 44 Recommendations for WLP- Agribusiness 1. Growth of the program must be controlled to ensure quality. The program needs to grow strategically and incrementally. The College will be under pressure to grow the program quickly due to high demand and revenue generation potential. If the program grows too quickly for institutional capacity and, more especially, human resource capacity, the quality of the program will be severely compromised. 2. A plan for faculty development that allows for more advanced training of its current human resources (lecturers, tutorial assistants, etc.) for higher-level degrees in the field of Agribusiness is crucial. While faculty and staff are pursuing more training, adequate highly-qualified teaching staff must be in place to continue teaching in the program. CAVM should consider regional recruitment for qualified teaching staff to fill in the gaps. Students, especially females, who are excelling in the program should be identified at the end of the first year, and receive extra support and leadership development opportunities. These women could be prepped to pursue doctoral programs abroad, perhaps through the Borlaug program. 3. As Agribusiness grows in Rwanda, so will employment opportunities for those with appropriate qualifications. CAVM must remain competitive with the private sector at all levels. Teaching faculty must be ensured favorable working conditions, such as competitive wages, manageable teaching load and reduced commuting between campuses. The public sector must also remain competitive with the private sector. If not, graduates of the MSc Agribusiness who are working in the public sector will migrate to private sector companies, essentially resulting in a brain drain in the public sector. 4. The program of study should be revised so that there are more practical classes in the first two semesters before students begin their 200 hours of internship. Courses such Strategic Management in Agribusiness, Agribusiness Value-Chain Management, and Agribusiness Marketing should be offered during Semester 1 and Semester 2. 5. Maintaining engagement with the private sector is crucial to ensure that the quality of the program remains high. The relationship between CAVM and the advisory board must continue to be cultivated. The link among private sector, public sector, and the University is crucial to maintaining the quality and relevance of the Agribusiness program, as well as the nation’s development agenda. 6. The internship and thesis should be integrated. This would raise the overall quality of the program. 7. A gender specialist within the college is necessary. Ideally, gender specialists would be found on each campus. Resources must be spread across all five campuses. Existing gender committees at Nyagatre and Busgogo must be nurtured. Develop best practices or a handbook for how to integrate gender into the curriculum. 8. Continued resources must be dedicated to building capacity in career development. The hiring of a career development specialist is a great start. This person will need support staff in order to succeed. 45 Conclusion Today new economic orthodoxies recognize that knowledge societies require a highly educated citizenry to create and sustain knowledge products. With high rates of return on knowledge creation and production, there has been a re-centering of higher education to the development agenda. Vision 2020 (Government of Rwanda, 2000) articulates a long-term development plan that transitions Rwanda from an agrarian-based economy to a knowledge-based economy with heavy investment in education and Information and Communication Technologies (ICT). Rwanda’s development agenda transitions the country from low-income to middle income by 2020. In order for Rwanda to successfully manage this transition, Rwanda has embarked on a program of institutional capacity strengthening, human resource development, and public debt reduction. The WLP-Rwanda worked to strengthen institutional capacity and develop human resources through its work with the University of Rwanda system. Central to institutional capacity building and human resource development is the promotion of gender equality. The integration of gender into all of the government of Rwanda’s development programs is one of the major pillars of Vision 2020. Gender equality is one of the drivers of economic development and poverty reduction. Rwanda’s Girls Education Policy, the Education Sector Strategic Plan, Educational and Vocational Training Policy, and Early Childhood Development all codify gender equality in access, retention, and graduation at all school levels. In September 2013, both National University of Rwanda (NUR) and The Kigali Institute of Education (KIE) ceased to exist as legal entities, and the process of merging five public institutions into one university system was initiated. The revitalization and reformation of higher education is a complex process with a never-changing set of actors. Partnerships must be seen as a process, not a product, so that attention can always be given to the nurturing of the process. The nature of these partnerships must constantly be negotiated and renegotiated so that program goals, priorities, and responsibilities are clearly articulated and space created for local interpretations and innovation. In spite of complications resulting from the merger, both partnerships have made substantial progress toward achieving its core objectives and intended outcomes and outputs. For URCE the greatest success has been in meeting its second and third objectives: better informed URCE’s institutional policies and practices through gender-focused research and capacity building and improved capacity to prepare teachers to address gender equity in the classroom through pedagogy, course content, and classroom management techniques. For CAVM the greatest success has been in meeting its first and second objectives: strengthened human and institutional capacity of UR in teaching and applied research in agricultural sciences promotion and support women's access to graduate education in agricultural sciences. 46 Works Cited Bryman, A. (2007). Barriers to integrating quantitative and qualitative research, Journal of Mixed Methods Research, 1(1), 8-22. Chemonics. (2006). Assessing USAID’s Investments In Rwanda’s coffee sector: Best practices and lessons learned to consolidate results and expand impact, Washington, DC. Huggins, A., Randell, S. (2008). Gender equality in education in Rwanda: What is happening to our girls? Paper presented at the South African Association of Women Graduates Conference on “Drop￾outs from School and Tertiary Studies: What is happening to our Girls?” Cape Town. Masanja, V. G. (2011). Increasing women's participation in Science, Mathematics and Technology education and employment in Africa. Expert Group Meeting. Ministry of Finance (2000). Rwanda vision 2020. Kigali: Ministry of Finance and Economic Planning. Ministry of Finance (2013). Poverty reduction strategy paper. Kigali: Ministry of Finance and Economic Planning. Nyerere, J.K. (1968). “Education for Self Reliance” in Freedom and Socialism. Oxford: Oxford University Press. Program Level Self- Efficiency-Survey Results-FY13-14. Republic of Rwanda. (2007). Economic Development and Poverty Reduction Strategy (EDPRS) 2008 – 2012. Kigali: Rwanda. Republic of Rwanda. (2013). Economic development and poverty Reduction Strategy (EDPRS 11) 2013 – 2018. Kigali: Rwanda. United Nations. (2000). Millennium development goals. New York: United Nations. WLP-Rwanda agriculture annual progress report. (October 1, 2013 to September 30, 2014). WLP- Rwanda education annual progress report. (October 1, 2013-Sept, 30, 2014). Walby, S. 1990. Theorising patriarchy. Oxford: Basil Blackwell. 47 Annex 1: List of Participants Participants from the Education Program Stakeholder Type Beneficiary Name Position Date/Time Location of Meeting Individual/Group Admin No Dr. Alfred Otara Partnership Director Monday, 3/23/15 KIE Individual Admin No Apophia Program Assistant Tuesday, 3/24/15 KIE Individual Admin No Dr. Azeb Tadesse UCLA Partnership Director Tuesday, 3/24/15, 1pm Aroma Café Individual Admin Yes Francois Partnership Finance Monday, 3/29/15 KIE Individual Former Admin Yes Dr. Jolly Rubagiza Professor, Former Partnership Director Monday, 3/ 30/15, 2pm KIE Individual Admin Yes James Director of Planning Monday, 3/ 30/15, 2pm KIE Individual Admin No Principal Principal of College of Education Tuesday, 3/30/15, 1pm KIE Individual Admin No Dr. Cyprian Dean of College Tuesday, 3/30/15, 3pm KIE Individual Faculty Yes Dr. Mike, Ph.D Professor Monday, 3/23/15 KIE Individual Faculty Yes Margaret Instructor, Sociology of Education Monday, 3/23/15, 5:30pm KIE Individual Faculty Yes Raymond Mwesigye Online Instructor Tuesday, 3/24/15, 10am KIE Individual Faculty Yes Dr. Gaspard Gaparayi & Dr. Celestine Kyanga Professors, Foundations & Demographer Tuesday, 3/24/15, 11am KIE Group 48 Faculty Yes Dr. Claudine & Chantal Kyaboyu Lecturer & Assistant Lecturer Tuesday, 3/24/15, 2pm KIE Group Faculty Yes Ali Lecturer￾Online Tuesday, 3/24/15,4pm KIE Individual Students Yes Celestine (47, M); Germain (27, W); Yvette (34, F); Ferdinand (28, Male); Denise (28, F) Diploma in Education Studies Friday, 3/27/15, 5pm Distance Learning Center, Muhanga Group Community No Bishop John Outreach facilitator Sunday, 3/ 29/15, 3pm Serena Hotel Single Community Yes Charles (District Ed Officer, M); Benita (Section Ed Officer, F); Hormides (Headmaster, Secondary School) Kyonza District Outreach Tuesday, 3/30/15 Kyonza Municipal Office Group Admin No Ben Musuhuke Liaison Officer Aromas Coffee Individual 49 Participants from the Agriculture Program Stakeholder Type Beneficiary Name Position Date/Time Location of Meeting Individual Group Admin No Dr. John Mendendorp MSU Partnership Director Sat., March 28th @ 9:30am Serena Hotel Individual Admin/ Support No Gertrude Secretary Thurs., March 26 @ 12:15pm Huye/ Butare Campus Individual Admin￾Implementer Research Training & Grantee Dr. Daniel Rukazambuga Career Development Director Thurs., March 26 @ 12:15pm Huye/ Butare Campus Individual Admin/ Implementer Research Training & Grantee Dr. Jean Chrysostome Ngabitsinze UR Partnership Director Sat., March 28 Rubirizi Campus, Kigali Individual Beneficiary Curriculum & Research Team Sandrine Urujeni Asst. Lecturer in Agribusiness Sun., March 29th @3pm Sun., Hill Top Hotel, Ramera, Kigali Individual Beneficiary MsC in Agribusiness Students Evrine (F, 26), Maurice (M, 29),Alice (F, 26), Mary (F, 27), Frank (M, 31) MsC in Agribusiness Students Sun., March 29th &12pm Rubirizi Campus, Kigali Group Advisory Board Yes Ivan Twasirashema President of Chamber of Industry 15/4/15 @2pm Ivan's Office Individual Admin/ Implementer Yes Dr. Colleen Taugher MSU- Career Development Expert 4/14/15 at 10am Skype Individual Admin/ Implementer Yes Dr. Gretchen Neisler MSU Principal Investigator 4/21/15 at 10am Skype Individual Admin/ Implementer Yes Dr. Amy Jamison MSU Gender Coordinator 4/15/15 at 10am Skype Individual Participants from HED Stakeholder Type Beneficiary Name Position Date/Time Individual/Group Partnership Management No Marilyn Crane Program Specialist 4/14/15, 11:30am Individual 50 Partnership Management No Amy Regan Senior Program Specialist 4/14/15, 12:30pm Individual 51 Annex 2: Documents Included in Desk Review 1. RFA- Request for Application - Design Assess o Background Docs  Rwanda EDUCATION_POLICY_July 2003  GMO_Annual_Report_2009-2010  Government of Rwanda ESSP 2010 – 2015  Higher Education Policy July 2008  National Gender Policy-FINAL VERSION JULY 2010 cabinet  Teacher Development and Management Policy in Rwanda 2007 o United States Agency for International Development (USAID) Gender Policy Papers  Gender Equality Policy March 2012  Gender Education Policy One-Pager o RFA Framework - RFA Women's Leadership Program in Rwanda (Agriculture) 2. Application materials - Michigan State University (MSU) Abstract - MSU HED (Higher Education for Development) Rwanda Monitoring and Evaluation (2) - MSU Narrative Rwanda Ag 3. M_E_ docs- Monitoring and Evaluation - Annual Implementation Plan (AIP) o MSU_AIP_ Fiscal Year (FY) 2014 o MSU_AIP_FY2015 – revised - Baseline Assessment o Final_Baseline_Report  WLP Rwanda Ag_MSU NUR (National University of Rwanda- College of Agriculture, Animal Science, and Veterinary Medicine)_Baseline Tool_08_26_2013  August 2014 WLP Rwanda Ag_MSU NUR_Baseline Tool_FINAL - HED Data Quality Verification o FY13  Annual  WLP Rwanda Ag_1April_13_30Sept_13  Semiannual  PRIME Report_1Oct12_31March13 o FY14  Annual report  WLP Rwanda Ag_1Apr_14_Sept_14  Semiannual  WLP Rwanda Ag_1Oct_14_30Sept_14 - M&E Plan - Results Framework and Scope of Work (SOW) o Final_SOW  Standard Indicator Definitions  Standard_ Indicator_ Def o WLP Rwanda Ag. M&E Plan  Original M&E Plan  MSU-NUR ME Plan 10 29 12  Revised M&E Plan Jan. 2015  Revise_Justification 2015 Annual Plan and Activity Calendar Narrative for change of focus on Rwandan WLP partnership 52 Revised FY2015 Annual Implementation Plan 9.15.14  MSU-UR ME Plan 1_1_15 - Progress Reports o FY13  Semiannual Progress Report (WLP) Oct 2012 to Mar 2013  WLP Rwanda Ag - Annual Progress Report (WLP) Oct. 1, 2012 to Sept. 30, 2013 – FINAL o FY14  MSU_Semiannual Progress Report (WLP) Oct 1, 2013 to Mar 31, 2014  MSU-UR Annual Progress Report FY14-Full o Reports Documentation  FY13  DAIRY_concept note_Def_18 Oct 2013  final proposal_Coffee1 (2)  List of members of advisory committee  Rwanda Seed Grant proposal_Soil Health Conservation project  UR CDO Internship Handbook DRAFT  FY14  2014.07.22 Draft Study Tour Report  Access-Policies - Admissions Policy  Acess-Policies - Scholarship Policy  April 2014 Workshop  August 2014 Workshop (2)  August 2014 Workshop  Awarded Proposals  Custom Indicator 5 - Awarded Proposals  Custom Indicator 5 - Reviews 2013  Custom Indicator 8 - Scholarship Policy  December Board Meeting Signature  Faculty Behavior Baseline Report  In Country Sessions  Journal_MSU_Martin_neisler_edits  Masters_agribusiness_gender approved (2)  Masters_agribusiness_gender approved  Program Level- 4-16-14  Program Level- 12-6-13  Registration AIARD  Reviews 2013  Scholarship Policy  September 2014 Workshop  Training- Short Term- In Country Sessions 4. Management - Contact List o WLP Ag. Contacts - MSU UR  Contact Information_UR and MSU_revised_2_5_15  WLP-wide Contact List  WLP Partnership Contact Information - Correspondence o Call Notes  December 2013  USAID-Rwanda and HED  July 2014 53  Call Follow Up (Memo from UR & MSU)  March 2014  HED and MSU - Revising Indicators  October 2014  HED and MSU 10.6.14  HED and MSU and Rwanda o Memo - Launch Delay  Memo - Delay of Launch at UR and Indicator Revision Request o Study Tour Challenges  FW URGENT! Women's Leadership Program Study tour the U S - HED Partners Mtg (Kigali) o WLP Meeting Evaluation Report Draft - Feb 2015 o Womens Leadership Program Partners Meeting – Agenda o Womens Leadership Program Partners Meeting - Participants List - Media o Draft Press Release - MSc program at UR Launch version 2 o Flash_News_29_August_2014 o Fwd Youth & Universities in Food Security Feed the Future Newsletter - November 2014 o Re Rwanda Launch - Domestic stories o RE WLP Partnerships Featured in USAID Global Waters Magazine - HED Monitoring Visits o January 2015 Visit Notes Draft  Monitoring Visit Notes 2_2_15  WLP_UR_Monitoring_Visit  January 2015 Visit Schedule  Schedule_Rwanda_ WLP_Jan_2015  January Workshop 2015  January leadership development agenda 1-9-15late pm  MSc Launch Jan. 2015  Talking Points USAID UR MSU MSc Ag Launch USAID Talking Points  Rwanda Launch Program Final (1) - MSc in Agribusiness Curriculum o Masters_agribusiness_gender approved - MSU Trip Reports o Post Trip WLP November 12 14 12 Final o Post Trip WLP Report 0413-0419-2014 o Post Trip WLP Report 0811-0831-2014 o Post Trip WLP Report 0922-0929-2013 o Post Trip WLP Report 1204-1206-2013 o Post Trip WLP Report 03302013 o RW WLP Stakeholder Contacts o Trip Report - Stakeholders Needs Assessment June 13 - Webinars o Fall 2014  WLP Webinar Draft- Nov 18th Final  WLP Rwanda Ag Summary o Summer 2014  WLP Summer Webinar - Presentation - July 28  WLP_Summer_Webinar_Summary 5. Subaward 54 - Agreement o WLP Rwanda AG - MSU Subaward - Fully Executed 11.28 - Modifications o Modification 1 - Award Date  MSU - Modification 1 - Revised End Date - FULLY Executed o Modification 2 - Whistleblower clause]  Final_Signed_Mod_2_MSU_UR_Whistleblower o Modification 3 - Budget Revision and Obligated Funds  Final_Mod_3_MSU_UR_Ag_Budget - Sub-Subaward o Sub-Subaward - MSU and UR 6. Other - HED Info o HED PPT Template-HE audience 8-2011 - WLP Info o HED_FactSheet_WLP_20140721 o ProjectFactSheet - WLP Agriculture – USAID o Women’s Leadership Focus of Five New University Partnerships _ NewLiberian.c - WLP_Rwanda_Ag_FY14_Narrative_Final Rwanda- Women’s Leadership Program in Education 1. RFA- Request for Applications - RFA Assessment  WLP RFA report - WLP Rwanda RFA document 2. Application materials - UCLA (University of California- Los Angeles) Rwanda Ed Narrative - UCLA_HED Rwanda Abstract 3. M_E_docs- Monitoring and Evaluation - Annual Implementation Plans:  AIP_2015  Partnership Implementation Plan (PIP)_2014 - Baseline and SOW o Baseline  Final_Baseline_M&E (Oct 2013)  Initial_Baseline_M&E (Apr 2013) o SOW April 2013  SOW (Apr 2013)  Rwanda Education ME Documents -4-03-2013_UCLA  SOW (Jul 2014)  UCLA Rwanda Education M&E Documents - Revised SOW (7.31.2014)  USAID Approval - Revisions to Results Framework Partnership Management Plan (PMP) and PIP  WLP-Rwanda Ed - Revisions to Results Framework_Response  SOW (Oct 2012)  HEDRWANDA_ScopeOfWork_REVISED  UCLA Rwanda Education M&E Documents - FINAL 10 31 12 - HED DQV-Data Quality Verification o FY13  Annual  WLP Rwanda Ed_1Apr13_30Sep_13  Semiannual 55  WLP Rwanda Ed_1Oct_12_31March_13 o FY14  Annual  WLP Rwanda Ed_1Apr_14_30Sept_14   Semiannual  WLP Rwanda Ed_1Oct_13_31March_14 - Progress Reports o FY13  Annual_Apr_Sept  Annual Progress Report (WLP) Apr. 1, 2013 to Sept. 30, 2013 (final)  Semi_Annual_Oct_Mar  WLP Oct 2012 to Mar 2013 (final) o FY14  Annual  WLP-Rwanda Ed - APR (Oct. 1, 2013 to Sept 30, 2014)  Substantiating documentation  Additional Questions-Implementation Progress- PIP_2015  Beneficiaries - Direct – AttendenceGenderResearchWorkshop  Beneficiaries- Direct- PlanningTrainingSigninSheet  Call Notes from the DQV  Curricula - New - Diploma Primary Education Program  Custom 9 - Teaching Materials Incorporating Gender  Custom 13 - SignInSheet_AccessExcelData  DirectBeneficiaries_FY2013-14  ST Training Activity 3 – AttendenceGenderResearchWorkshop  ST Training Activity 4 - MentorWKSP_STIP_2  UCLA - URCE_Target_and_Actuals_Summary revised 11.13.2014  UCLA- Rwanda Ed Post DQV Meeting Notes  Semi-Annual  Semiannual Progress Report (WLP) Oct 1, 2013 to Mar 31, 2014_REVISED  Semiannual Progress Report (WLP-Rwanda Ed) Oct 1, 2013 to Mar 31, 2014  Substantiating Documentation  AIP_2015  Custom Indicator 2 - PGDE Program Specification  LaunchImageGroup2013  LaunchImageGroup2013  Marilyn - Double Check of Duplicates  Outreach Indicator - November2013_Lis_CERT  Outreach Indicator - November2013Workshop_PUBLIC_OUTREACH  Outreach Indicator 3 - Integrating Technology Workshop Sign￾in sheets  Outreach Indicator Activity 1 - Gender Workshop Sign-in sheets1  Outreach Indicator Activity 1 - Gender Workshop Sign-in sheets2  Outreach Indicator Activity 1 - Gender Workshop Sign-in sheets3 56  Outreach Indicator Activity 1 - Gender Workshop Sign-in sheets4  Outreach Indicator Activity 2 - Mentoring Strategies Workshop Sign-in sheets1  Outreach Indicator Activity 2 - Mentoring Strategies Workshop Sign-in sheets1  Program - Level SelfEfficacy_SurveyResults_FY13-14  ST(Short Term) Training Indicator - November2013_Lis_CERT (1)  ST Training Indicator - November2013Workshop_GenderandMentoringURCE (University of Rwanda College of Education)  ST Training Indicator Activity 1 - Mentoring Strategies Workshop Sign-in sheets2  ST Training Indicator Activity 1 - Mentoring Strategies Workshop Sign-in sheets3  ST Training Indicator Activity 2 - Gender Workshop Sign-in sheets1 (1)  ST Training Indicator Activity 2 - Gender Workshop Sign-in sheets2  ST Training Indicator Activity 2 - Gender Workshop Sign-in sheets3  ST Training Indicator Activity 2 - Gender Workshop Sign-in sheets4 4. Management - Call Notes o Call with USAID -UCLA-HED - 1.17.2014 o Monthly Conference call for UCLA-KIE partnership_Feb26 o Notes _March_20 o WLP- Education (1.27.2014) Call o WLP- Education (6.232014) Call o WLP-Rwanda (Ed) - Webinar on PRIME (Partnership Results & Information Management Engine) Training o WLP-Rwanda -Call Notes 9.17.2014 o WLP-Rwanda -Call Notes 10.29.2014 o WLP-Rwanda -Stakeholder Call Notes 11.13.2014 o WLP-Rwanda -Stakeholder Call Notes 12 3 2014 - Full Partnership Contact List o WLP Partnership Contact Information - HED Partners Mtg (Kigali) o WLP Meeting Evaluation Report Draft - Feb 2015 o Womens Leadership Program Partners Meeting – Agenda o Womens Leadership Program Partners Meeting - Participants List - HED Site Visits o July 2013  18_07_13_Monitoring Visit o March 2014  Agenda_March10-14_SiteVisit_draft  SiteVisit_Meetings  WLP Site Visit Report o January 2015- Coming Soon  Schedule_Rwanda_ WLP_Jan_2015 57 - Webinars o Fall 2014  WLP Webinar- Nov 18th Final  WLP Rwanda Education Summary o Summer 2014  WLP Summer Webinar - Presentation - July 28  WLP_Summer_Webinar_Summary o NUR Restructure o Staffing Chart for USAID-Rwanda 5. Subaward - Modifications o Modification #1 - Full Funding  WLP-Rwanda (education) - Mod 1 - Full Funding and Change in Contact o Modification #2 - Whistleblower Protection  WLP-Rwanda (Ed) - Mod#2 - Whistleblower Protection o Modification #3 - Budget and Work Plan Realignment  Attach_A_ Revised SOW (7.31.2014)  Signed Mod #3 WLP-Rwanda Edu UCLA (executed) - Sub-Agreement o WLP Rwanda Ed UCLA - Fully Executed Subaward (confidential) - Sub-Sub-Agreement_MOU o Fully Executed_Subagreement o KIE-UCLA_MOU 6. Other - Fact Sheets o ProjectFactSheet - WLP Education – 2015 o ProjectFactSheet - Women in Development (Education) - 4.2013 - General Updates o WLP-Program - FY14 Updates o WLP-Rwanda_Education_AchievementsUpdate_October2014 o WLP-Rwanda_Education_ProjectDescription_APRIL2014 (2) - Press o allAfrica.com_ Sudan_ USAID and HED Announce Women's Leadership Program for o Empowering Girls to Achieve in Rwanda, UCLA International Institute o USAID and HED Announce Women's Leadership Program for Armenia, Paraguay, Rwa o USAID and Higher Education for Development Announce Women's Leadership Progr o Women’s Leadership Focus of Five New University Partnerships _ NewLiberian.c 7. Targets - FY15 Targets in PRIME - UCLA - University of Rwanda College of Education_Target_and_Actuals_Summary revised - UCLA-University of Rwanda Kigali Campus_LEVEL_Targets-and-Actuals-Summary.revised 58 Annex 3: Semi-structured Interview Protocol Questions Partnership Implementers- MSU, UCLA, URCE, and CAVM 1. Get biographical data. 2. Could you talk about your involvement with the program? 3. Can you tell us a little about your experiences with the results-based management system? Please name the systems and describe your experiences with them. Have the management systems helped you to do your project work more efficiently or less efficiently? Please explain. 4. Tell us about how decisions are made on project implementation. Think of a recent example of a decision that was made about the project. Can you walk us through the process of how that decision was made? 5. Can you list a few of the project activities or interventions? Is there any one activity or intervention that stands out to you? Talk a little about this one activity/ intervention. 6. Do you feel that this activity was relevant to the local context? How was this activity/ intervention adapted for the local context? In general do you feel program activities and interventions are relevant to the local context here? Say more. 7. Please list the program outcomes and objectives. 8. Do you feel that overall the outcomes and objectives match the activities and interventions? In what ways? What outcomes and objectives were being met with the activity you just mentioned? 9. Did the match between the activity and outcome seem logical to you? Overall, how would you characterize the logic used to match the program activities/ interventions with the program outcomes/ objectives? 10. Now that you are at the end of the project, can you look back and assess the overall effectiveness of the project. What are your thoughts on the overall program effectiveness? At the very beginning, when the program was just getting started, what did the program set out to achieve? Now that the program is coming to a close, do you think the program achieved what it set out to do initially (Note the project objectives if necessary-Improve female recruitment and develop gender research capacity for the education program; and strengthen human and institutional capacity in teaching and applied research, promote and support women’s access to graduate education, and extend UR’s knowledge about and women’s expertise in agricultural sciences for the agriculture program)? 11. Let’s talk a little bit about impact. How would you characterize the overall impact of the program? 12. Can we get more specific? What are some of the outcome level results? Give me one or two examples of outcomes. With this one or these two examples in mind, to what degree can you attribute the outcome results to the implementation process? 13. Again, with this example you just shared with me, is there evidence that this outcome had a wider effect than what you anticipated? Tell me more. 14. How do you feel about the University of Rwanda’s ability to continue building the capacity to advance women’s leadership in education/ agriculture once the program wraps up? What particular aspects of the program do you think will continue? What factors would influence whether or not these aspects you mention would continue? 15. Is there anything else you think is important for us to know about the program? 59 Workshop Attendees 1. Get biographical data. 2. Could you talk about your involvement with the program? 3. What workshop(s) did you attend? 4. When did you attend the workshop(s)? 5. Where? 6. How did you learn about the workshop? 7. Did you attend the workshop in its entirety? 8. What was covered in the workshop/ training? 9. Can you talk about the structure of the workshop/ training? What was the agenda? 10. What was your experience in the workshop/ training? 11. What did you learn? What did you walk away with? 12. How did your thinking change as a result of the workshop? 13. Have you used what you learned in the workshop? How so? Students 1. Get biographical data. 2. Have you used what you learned in the workshop? How so? 3. How did you hear about the program? 4. Tell us about the program. What has your experience been in the program? 5. What do you like about the program? 6. What are some of the challenges you have been facing in the program? 7. How does gender appear in the program? Talk to us about our your understanding of gender. 8. What advice do you have for UR about sustaining the program? Community Stakeholders and Other Beneficiaries 1. Get biographical data. 2. Could you talk about your involvement with the program? 3. Explore the particulars of the involvement/ engagement with the WLP. 4. Ask more specific questions about some of the data that arose from other stakeholders that needs more explanation or clarification. 5. What has been successful? 6. What have been the challenges? HED Staff 1. What is your involvement with the program? 2. What has been successful? 3. What have been the challenges to the program? 4. Pursue the challenges to the program. 5. Ask more specific questions about some of the data that arose that needs more explanation or clarification.