USAID Sustainable Higher Education Research Alliances (SHERA) Final Evaluation Cooperative Agreement No: AID-497-A-16-00004 Prepared for: Thomas Crehan, Director Jalu Cahyanto, AOR Human Capacity and Partnerships USAID/Indonesia Prepared by: Susan Buck Sutton Emerita Associate Vice President of International Affairs, Indiana University On behalf of the Institute of International Education (IIE) August 2020 List of Acronyms, Abbreviations, and Terms Affiliate HEIs participating in a CCR, other than the Lead ANBIOCORE CCR Animal Biotechnology & Coral Reef Fisheries ARI CCR Acute Respiratory Infections ARF Activity Results Framework of the SHERA MEL BAN-PT Indonesian National Board of Higher Education CCR Center for Collaborative Research CDCS Country Development Cooperation Strategy CDSR CCR Development of a Sustainable Region FY Fiscal Year GDA Global Development Alliance GOI Government of Indonesia HEI Higher Education Institution IIE Institute of International Education IIEF Indonesian International Education Foundation IPB Institut Pertanian Bogor/Bogor Agricultural University ITB Institut Teknologi Bandung/Bandung Institute of Technology Kemenristek/BRIN Ministry of Research and Technology/National Research and Innovation Agency KEMRISTEKDIKTI Ministry of Technology, Research, and Higher Education KPI Key Performance Indicator Lead The Indonesian HEI leading each CCR MEL Monitoring, Evaluation, and Learning Plan MIS Management Information System MIT Massachusetts Institute of Technology MOU Memorandum of Understanding NCSTT CCR National Center for Sustainable Transportation Technology NGO Non-governmental Organization PIRS Performance Indicators Review Sheets PPP Public Private Partnerships R&D Research and Development RFP Request for Proposals RIRN Rencana Induk Riset Nasional/Master Research Plan SDG United Nations Sustainable Development Goal SHERA USAID Sustainable Higher Education Research Alliance SMART CITY CCR Scientific Modeling, Application, Research, and Training for City-centered Innovation and Technology S&T Science and Technology UGM Universitas Gadjah Mada/Gadjah Mada University UI Universitas Indonesia/University of Indonesia UNPAD Universitas Padjadjaran USAID United States Agency for International Development USG U.S. government White Paper SHERA Baseline Assessment Preliminary Findings August 2016 Table of Contents EXECUTIVE SUMMARY..................................................................................................................................1 A. THIS EVALUATION: ITS PURPOSE AND BASIS ........................................................................................3 B. USAID SHERA OVERVIEW .............................................................................................................................5 C. PROGRESS TOWARD USAID SHERA’S FUNDAMENTAL GOAL: AN OVERALL ASSESSMENT...5 THE GOAL: DEFINITION, RATIONALE, AND KEY TARGETS................................................................................................5 ASSESSING PROGRESS: IMMEDIATE RESULTS AND LONG-TERM CAPACITIES ....................................................................7 TARGET 1: FOCUSING THE RESEARCH ON INNOVATIVE SOLUTIONS TO INDONESIA’S DEVELOPMENT CHALLENGES ...........7 TARGET 2: BROADENING S&T RESEARCH HORIZONS TO INCLUDE WORLD-CLASS STANDARDS, RECOGNITION, AND ENGAGEMENT. ............................................................................................................................................................8 TARGET 3: ELEVATING THE RESEARCH INFRASTRUCTURE AND CULTURE OF INDONESIAN HEIS ....................................10 TARGET 4: PROVIDING ON-GOING PROFESSIONAL DEVELOPMENT AND HANDS-ON RESEARCH OPPORTUNITIES FOR FACULTY, STUDENTS, AND POSTDOCTORAL RESEARCHERS...........................................................................................10 TARGET 5: BUILDING THE COLLABORATIVE LINKAGES AMONG HEIS NEEDED FOR HIGH-LEVEL, MULTI-FACETED RESEARCH, BOTH WITHIN AND BEYOND INDONESIA.....................................................................................................11 TARGET 6: BUILDING SIMILAR RESEARCH LINKAGES WITH A RANGE OF PUBLIC/PRIVATE STAKEHOLDERS ......................12 TARGET 7: SPREADING HIGH-LEVEL RESEARCH CAPACITY TO MORE INDONESIAN UNIVERSITIES IN MORE DIVERSE LOCATIONS ...............................................................................................................................................................13 TARGET 8: DIRECTING SPECIAL ATTENTION TO TAPPING AND DEVELOPING THE RESEARCH CAPACITIES OF INDONESIAN WOMEN INTERESTED IN S&T .....................................................................................................................................15 COMPARISON OF THE CCRS ON SHERA’S OVERARCHING GOAL .................................................................................15 D. FURTHER DISCUSSION OF TARGET 4: PROVIDING ON-GOING PROFESSIONAL DEVELOPMENT AND HANDS-ON RESEARCH OPPORTUNITIES FOR FACULTY, STUDENTS, AND POSTDOCTORAL RESEARCHERS ....................................................................................................16 E. FURTHER DISCUSSION OF TARGET 3: ELEVATING THE RESEARCH INFRASTRUCTURE AND CULTURE OF INDONESIAN HEIS...............................................................................................................18 F. FURTHER DISCUSSION OF TARGETS 5 AND 6: BUILDING COLLABORATIVE LINKAGES WITH HEIS AND PUBLIC/PRIVATE STAKEHOLDERS..........................................................................20 COLLABORATIONS AMONG INDONESIAN HEIS ............................................................................................................20 COLLABORATIONS WITH INDONESIAN PARTNERS OUTSIDE ACADEMIA ..........................................................................21 COLLABORATION WITH U.S. HEIS .............................................................................................................................22 G. FURTHER DISCUSSION OF TARGET 8: DEVELOPING THE RESEARCH CAPACITIES OF THE NUMBER OF INDONESIAN WOMEN INTERESTED IN S&T.................................................................23 H. EVALUATION OF USAID SHERA’S DISTINCTIVE APPROACH ..........................................................23 LOCAL OWNERSHIP ...................................................................................................................................................24 COLLABORATION AND ENGAGEMENT .........................................................................................................................24 LEARNING AND OVERSIGHT........................................................................................................................................25 I. SUSTAINABILITY AND LONG-TERM IMPACT .......................................................................................26 J. FINAL THOUGHTS..........................................................................................................................................28 ANNEX I. THE CENTERS FOR COLLABORATIVE RESEARCH AND THEIR INITIAL GOALS...........30 ANNEX II. USAID SHERA TRAINING TRACKER (CUMULATIVE THROUGH Q3 FY20) ......................31 ANNEX III. KPI DEFINITIONS.............................................................................................................................51 ANNEX IV. USAID SHERA KPI RESULTS (CUMULATIVE THROUGH Q3 FY20)....................................54 ANNEX V. CCR RANKING PER PERFORMANCE INDICATOR...................................................................60 ANNEX VI. USAID SHERA CCR LEAD AND AFFILIATE INSTITUTIONS………………………………..62 List of Figures Figure 1. USAID SHERA Activity Results Framework 4 Figure 2. CCR Map 14 List of Tables Table 1. USAID SHERA Results: innovative, research-based solutions to the challenges affecting Indonesia’s development 8 Table 2.KPIs: USAID SHERA Goal - All CCRs as of Q2 2020 9 Table 3. Overarching USAID SHERA Goal: Research partners beyond academia 12 Table 4. USAID SHERA Products Being Put to Use 13 Table 5. Geographical Distribution of Indonesian CCR HEIs 14 Table 6. KPIs: USAID SHERA Goal - CCR Comparison FY18-FY20 15 Table 7. Number of KPIs for which Each CCR Came in 1st, 2nd, 3rd, 4th, or 5th 16 Table 8. KPI Outcome 1: Improved Capacity of Scholars - All CCRs FY18-FY20 17 Table 9. KPI Outcome 1. Improved Capacity of Scholars - CCR Comparison FY18-FY20 17 Table 10. KPI Outcome 2: Strengthened, inclusive environment for research and management at Indonesian HEIs - All CCRs FY18-FY20 18 Table 11. KPI Outcome 2: Strengthened, inclusive environment for research and management at Indonesian HEIs - CCR Comparison FY18-FY20 19 Table 12. KPI Outcome 3: Enhanced collaboration - All CCRs FY18-FY20 20 Table 13. KPI Outcome 3. Enhanced collaboration - CCR Comparison FY18-FY20 21 Table 14. Female Participation in USAID SHERA 23 Table 15. Key Lessons Learned from Managing USAID SHERA (as articulated by SHERA and CCR Staff) 26 Table 16. 2020 THE Global Impact Rankings – Top 100 Placements of Indonesian HEIs 28 1 Executive Summary USAID SHERA is a United States Agency for International Development (USAID)-sponsored project to bring the research capacity of Indonesian Higher Education Institutions (HEIs) in priority, nationally￾relevant S&T (Science and Technology) topics to sustainable, world-class levels, through a process of both domestic and international collaboration. Operating from 2016-2020, SHERA established five Centers for Collaborative Research, that brought together more than 30 Indonesian HEIs, 12 U.S. HEIs, and 41 governmental agencies, private companies, and local communities to focus on such capacity￾building. In so doing, SHERA aimed at eight targets identified as most critical to advancing S&T research at Indonesian HEIs: 1. Focusing the research on innovative solutions to Indonesia’s development challenges. 2. Broadening S&T research horizons to include world-class standards, recognition, and engagement. 3. Elevating the research infrastructure and culture at Indonesian HEIs. 4. Providing on-going professional development and hands-on research opportunities for faculty and students. 5. Building the collaborative linkages among HEIs needed for high-level, multi-faceted research, both within and beyond Indonesia. 6. Building similar research linkages with a range of public/private stakeholders. 7. Spreading high-level research capacity to more Indonesian universities in diverse locations. 8. Directing special attention to developing the research capacities of Indonesian women interested in S&T. This evaluation, conducted toward the completion of the program, concludes that USAID SHERA has both produced remarkable immediate results in S&T research and built capacity for significant future work across a wide range of Indonesian institutions. SHERA’s HEIs and scholars uniformly testify to how they have been changed by the program, and the research they conducted has already resulted in over 500 peer-reviewed publications on priority development issues. These results reflect the way IIE structured the program: the strong, collaborative partnerships that were built, and the continuous mentoring and guidance given by SHERA staff to the CCRs. The program has demonstrated that Indonesian HEIs have the capacity for high-level, world-class, development-oriented S&T research and illuminated the conditions that activate this capacity. SHERA’s bold move of having lead Indonesian HEIs manage program development and finances, while also asking them to actively engage other HEIs across the country, has enhanced the capacity of all to undertake similar projects in the future. SHERA also opened more research opportunities for the high number of Indonesian women interested in S&T; created mutually beneficial relationships between universities and various public and private organizations; and engaged U.S. scholars in capacity-building for their Indonesian counterparts, setting the stage for more comprehensive partnerships in the future. In reaching these results, SHERA operated as a learning laboratory for what works and what does not in building research capacity on a national scale. It did this by creating a feedback loop between training, infrastructural development, and network building, on the one hand, and real-time research and publication, on the other. These two elements played off each other in a process of continuous learning and improvement. Many lessons were learned, shared, and used to strengthen the CCRs’ operational capacity. All CCRs experienced significant, immediate results in their work, which then energized the research ambitions of participating scholars. All also built capacity for further such work that will tap the potential for nationally-relevant research across a range of Indonesian HEIs. Whether the CCRs continue 2 in their current structure or transition to another format, their work in researching and identifying innovative solutions to Indonesia’s development challenges will continue well beyond the SHERA program. Indeed, SHERA has played an important role in moving Indonesian universities into a new, globally￾recognized phase of development-oriented research, and its projects were specifically recognized in Indonesia’s remarkable placement in the 2020 Times Higher Education Impact Rankings. “Indonesia’s stellar performance in this year’s Times Higher Education Impact Rankings reflects the burgeoning capability of the archipelago’s top institutions … Claiming three top 100 spots in any global ranking – and a handful of top 40 entries in key areas – is unheard of for Indonesia ... Japan is the only Asian country to match Indonesia’s efforts in the Impact Rankings.” John Ross, Times Higher Education, May 18, 2020 https://www.timeshighereducation.com/world-university-rankings/why-indonesia-makes￾impact-sdg-focused-ranking 3 A. This Evaluation: its purpose and basis This final evaluation assesses the extent to which the USAID Sustainable Higher Education Research Alliances program (SHERA) has achieved its goals, while also providing feedback on the effectiveness of the strategies SHERA used to move toward these goals. The evaluation provides an overarching view of what SHERA has accomplished, what worked well, what challenges arose, and what continuing impact SHERA can be expected to have in the future. This evaluation is based on analysis of the following materials: • CCR progress reports, both quarterly and (where available) final • SHERA staff written identification of the key elements in the above reports • CCR quarterly Success Story reports • SHERA annual reports to USAID • Final tabulation of the KPIs (Key Performance Indicators) from the Activity Results Framework of the SHERA MEL (Monitoring, Evaluation, and Learning Plan) • CCR responses (both verbal and written) to additional questions posed by the reviewer and SHERA staff in spring 2020 • SHERA Training Tracker, a listing of the mentoring and coaching activities provided to the CCRs by SHERA staff, cumulative through Q3 FY20 (see Annex II) • SHERA foundational documents (RFP, Proposal, White Paper, MEL Framework) • SHERA CCR Sustainability Assessment Out-Brief 2019 • Consultation on key questions with SHERA staff The Activity Results Framework is an essential part of this review. The Framework contains 22 KPIs spread across 1 goal and 3 outcomes deriving from this goal (see Figure 1 for the Framework and Annex III for KPI definitions). The CCRs and SHERA staff have collected a wealth of data on these KPIs for FY18, FY19, and FY20. Annex IV presents cumulative totals for this three-year span that are a rich record of SHERA’s output. As important as the KPIs are, they are not all that is needed to assess SHERA. They must be placed within the broader context of what was intended, what actually happened, and what impact such activities have had or are likely to have in the future. As discussed in Section C, this requires assessing long-term impact as well as immediate output. And it asks that all of SHERA’s foundational targets be evaluated, including those not fully captured mathematically. To do this, the narrative information listed above is equally useful. There are a few caveats to this evaluation. The first, and most important, is that it is too early to assess the full impact of SHERA. This is true for two reasons. First, research, publication, and application are lengthy processes, even at well-resourced institutions. For example, the major international scientific publisher, Elsevier, describes a typical review and revision process that often takes two years from manuscript submission to publication. Second, SHERA was directed toward capacity-building for the future as much as toward immediate scientific advances. The degree to which new capacity created by SHERA is effective cannot yet be fully known. We can determine how much growth has occurred since the start of SHERA, but not what is yet to come. Yet another caveat is that despite the rich record compiled by the CCRs and SHERA staff, there are occasional gaps in information, such as perspectives from U.S. partners and subawardees (a consequence of SHERA’s design, which assigned partnership management to CCR Leads rather than SHERA staff). 4 Finally, the progress made by SHERA must also be understood within the context of its early closure due to unexpected financial limitations. In-progress activities were curtailed, and the fifth and final year of activity was cancelled. Figure 1. USAID SHERA Activity Results Framework Outcome 1. Increased capacity of faculty, PhD students and postdoctoral researchers in target Indonesian universities 1.1: % of scholars who present at conferences 1.2: % of scholars who participate in collaborative research Outcome 2. Strengthened, inclusive institutional environment for research and management in target Indonesian universities 2.1: % of Indonesian institutions with improved research policies & system in place 2.2: % of CCR members with improved administrative & management systems in place Outcome 3. Enhanced collaboration in CCR/consortium for collaborative research 3.1: % of institutions engaged in joint research projects 3.2: % of partnerships created to support joint research projects Output 1.1. Increased research capacity in research design, methodology, analysis, report writing and presentation skills 1.1.1: # of short￾term training courses held 1.1.2.: # of researchers participated in short￾term training courses held 1.2.3: # of U.S. scholars visiting Indonesian institutions to train Indonesian partners Output 1.2. Increased professional knowledge exchange collaboration 1.2.1: # of Indonesian scholars who participate in in￾person faculty exchanges held in the U.S. 1.2.2: # of Indonesian scholars who received ongoing mentoring from U.S. partners universities 1.2.3: Deleted Output 2.1. Institutional policies in support for research and management implemented in CCR members 2.1.1: # of institutions with improved research and management policies developed Output 2.2. SHERA best practices documented and disseminated to Indonesian higher education institutions 2.2.1: # of CCR websites that are operational and being used regularly 2.2.2: # of CCR knowledge products created 2.2.3: # of program￾wide SHERA knowledge sharing events on best practices & lessons￾learned held 2.2.4: # of people participated in activities addressing gender equality or female empowerment in science & technology research Output 3.1. Institutional relationship established and strengthened in related to collaborative research capacity of Indonesian universities 3.1.1: Deleted 3.1.2: # of scholars at CCR institutions with increased access to external research engines SHERA Goal: Sustainable Improvement in Quality and Quantity of Science and Technology Research in Indonesian Higher Education Institutions Indicator 1: # of peer-reviewed scientific publications resulting from USG support to research and implementation program [STIR-12] Indicator 2: Ratio of citation to publications produced by Indonesian researchers [IR 4.1-3] Indicator 3: Ratio of academic research initiatives whose findings are utilized to address development challenges 5 B. USAID SHERA Overview USAID SHERA is a United States Agency for International Development (USAID)-sponsored project which uses collaboration (both domestic and international) to move the research capacity of Indonesian Higher Education Institutions (HEIs) in priority, nationally-relevant areas of ST&I (Science, Technology, and Innovation) to world-class levels. The Institute of International Education (IIE) has been the implementing partner (Cooperative Agreement AID-497-A-16-00004) since May 2016, with IIEF (Indonesian International Education Foundation) as its in-country partner, and with continuous, significant input from the GOI, especially the Ministry of Research and Technology/National Research and Innovation Agency (Kemenristek/BRIN). After several months of research and program design, IIE issued a fall 2016 RFP that resulted in the establishment of five (5) CCRs (Centers for Collaborative Research) in 2017 (see Annex I). Each CCR contains at least five partners –one Lead, top-tier Indonesian HEI that hosts the CCR, at least three (3) Affiliate Indonesian HEIs, and at least one (1) U.S. university. The decision to have Indonesian HEIs be the Leads was an unprecedented step in capacity-building, matched by a continuous mentoring and process-improvement program on research management, financial oversight, and monitoring provided by IIE. Much of FY2017 was thus devoted to the careful preparation and construction of the CCRs so as to be most effective. By FY18, all CCRs were actively pursuing programs of collaborative research, outreach, and capacity￾building, with significant results in Scopus-indexed publications already emerging. This continued into FY19, when funding limitations at USAID began to reduce IIE’s ability to implement SHERA, as well as the CCRs’ ability to conduct planned activities and expand new linkages with HEIs and other partners alike. Ultimately these funding issues led to the decision to conclude SHERA in September 2020, seven months earlier than originally planned. In one form or another, however, the work of the CCRs will continue even after conclusion of the program. C. Progress toward USAID SHERA’s Fundamental Goal: an overall assessment A single goal has driven SHERA and unified its many activities. This section gives an overall assessment of the progress SHERA has made toward this goal and is followed by sections that go into greater detail on some of the points made here. Put another way, this section yields a broad, synthetic view of how SHERA’s different elements combined to address its driving goal. Later sections elaborate on several of these individual elements as well as some cross-cutting issues. All reach the conclusion that SHERA has achieved much of that it set out to do, even with its truncated time span. The program’s positive impact should be felt for some time to come. The goal: definition, rationale, and key targets The Activity Results Framework (ARF) gives a clear statement of the fundamental goal driving SHERA: Sustainable improvement in quality and quantity of science and technology research in Indonesian Higher Education Institutions. This goal derives from the integration of USAID Indonesia’s Country Development Cooperation Strategy (CDCS) with the 2017-2045 Master Research Plan (RIRN/Rencana Induk Riset Nasional) of the Indonesian Ministry of Research, Technology, and Higher Education (now Kemenristek/BRIN, the Ministry of Research and Technology/National Research and Innovation Agency), as well as GOI initiatives more generally. At its outset – and before establishing the CCRs - SHERA thus devoted 6 considerable attention to defining the key elements of this goal and the key actions needed to achieve it. Conversations between IIE, USAID, and what is now Kemenristek/BRIN identified the S&T research topics of greatest importance for Indonesia’s advancement; extensive desktop research explored the current state of S&T research at Indonesian HEIs, especially in these topics; and interviews with several HEIs added nuance and depth to what was learned. The results were captured in the August 2016 White Paper (SHERA Baseline Assessment Preliminary Findings). This initial work brought together several initiatives. It reflected 2013 CDCS Development Objective (DO) 4 (collaborative achievement in science, technology and innovation increased, leading to evidence￾based decision-making and innovative approaches to development). 1 It funneled the RIRN’s 13 priority S&T research topics important for Indonesia’s developed into SHERA’s five focus areas of Food Security and Self-Sufficiency; Environment, Energy and Maritime Sciences; Public Health and Infectious Disease; Urban Development and Planning; and Innovative Technologies. It intersected with the National Board of Higher Education’s (BAN-PT) recent efforts to set standards, grant accreditation, and raise the level of research at Indonesian HEIs. The White Paper identified 50-100 Indonesian HEIs ready to move towards more robust S&T research. These institutions are heavily concentrated on Java, however, with some other regions having none. Research funds are limited, administrative requirements cumbersome, and project management procedures nascent or non-existent. The curriculum in many Indonesian HEIs, even some top institutions, is heavily theoretical, with little problem-based learning or research experience. Access to international journals is also low, with some 7,000 Indonesian journals, many not refereed, filling this gap. And, as discussed in Section G, while female Indonesian secondary school students have an unusually high interest in S&T, they comprise only one-third of the tertiary enrollments in these fields, with little access to research. In short, the S&T research potential of Indonesian HEIs was barely being tapped, hence also their ability to contribute to the nation’s advancement. As discussed further in Section H, a needs assessment conducted with the CCR Leads, once they were selected, confirmed many of the above points even for these top HEIs. Only one had been the direct recipient of USAID funding before, although several had been subawardees. Few women were involved in the initial CCR teams. Project and grant management skills and resources were not well developed. Few had active relationships with HEIs in other parts of Indonesia. Taken together, these initial strands of research and discussion revealed the key elements needed to meet SHERA’s overarching goal. Sustained improvement in quality and quantity of S&T research in Indonesian HEIs required attention to each of following targets. 1. Focusing the research on innovative solutions to Indonesia’s development challenges. 2. Broadening S&T research horizons to include world-class standards, recognition, and engagement. 3. Elevating the research infrastructure and culture at Indonesian HEIs. 4. Providing on-going professional development and hands-on research opportunities for faculty and students. 5. Building the collaborative linkages among HEIs needed for high-level, multi-faceted research, both within and beyond Indonesia. 6. Building similar research linkages with a range of public/private stakeholders. 1 SHERA fits equally well with the current CDCS DO 2.3 (capacity of educational institutions improved) and CDCS Cross-cutting Theme 3 (advancing mutual prosperity and security through science, technology, and innovation). 7 7. Spreading high-level research capacity to more Indonesian universities in diverse locations. 8. Directing special attention to developing the research capacities of Indonesian women interested in S&T. Meeting all these targets required, in turn, the distinctive approach adopted by SHERA, giving CCR Leads oversight responsibilities but also much mentorship, working broadly across Indonesian HEIs rather than building up only one or two, and combining immediate hands-on research experience (from data collection to publication) with capacity-building for even more in the future (see Section H for more details). This approach asked much of everyone involved, but did so with the goals of spreading research growth broadly across the archipelago, engaging participants fully in that it takes to carry out developmentally-oriented research, and yielding the capacity to continue such work in the future. Assessing progress: immediate results and long-term capacities Assessing progress on SHERA’s driving goal requires looking at each of these eight targets. Most are represented in the Key Performance Indicators (KPIs) of the ARF, but some are not. Furthermore, no dimension is fully captured by the KPIs. Evaluating the extent to which there has been sustained improvement in S&T research, for example, requires not only counting the publications already produced, but also what foundation has been laid for more to be published in the future. SHERA is about capacity￾building which requires identifying what resources and supports have been created that were not there before, and how widely these resources and supports have been spread, in terms of both geography and gender. While the KPIs greatly inform this evaluation, they are thus not its organizing principle. The eight targets identified above provide a better structure for assessing how well SHERA has performed on its overarching goal – with KPIs from across the ARF brought into the discussion when relevant, and still other material and analyses used to identify what capacity for the future has been built. Target 1: focusing the research on innovative solutions to Indonesia’s development challenges SHERA hit this target head on. Its defining structure insured a focus on priority development issues recognized by both GOI and USAID-Indonesia. This was done to meet national needs, but also because linking research to real-world issues can be a catalyst for engaging faculty and graduate students, while revealing the importance of S&T to the public. Toward these ends, after the initial planning period, SHERA limited CCR applications to the following five focus areas: Food Security and Self-Sufficiency; Environment, Energy and Maritime Sciences; Public Health and Infectious Disease; Urban Development and Planning; and Innovative Technologies. Under these headings, the selected CCRs pursued the following specific issues. • Acute respiratory infections: Embedding university research into clinical hospital settings to address the burden of disease, use of immunizations, and prevention of viral and bacterial respiratory illnesses in highly impacted populations, including young children, pregnant women, and the elderly (ARI) • Animal biotechnology and coral reef fisheries: Advancing food security by 1) using animal biotechnology to develop functional genomic markers for bull fertility and economically important traits for cattle production, and 2) investigating the impact of different fishing and management strategies on the biodiversity of coral reef fisheries, using ecosystem food web models to assess production potential (ANBIOCORE) • Sustainable regional development: Enabling Indonesia to maximize its many potential sources of renewable energy by developing hybrid energy system prototypes to support near zero-energy 8 buildings in tropical urban areas and by creating a network of renewable energy stakeholders within the country (CDSR) • Addressing urban challenges: Using multidisciplinary systems modeling to advance the adoption of green technology in cities; use of IT in urban administration, services, and governance; and reliability of urban infrastructure (SMART CITY) • Sustainable transportation technology: 1) Developing advanced technologies for producing the next generation of electrical vehicles in Indonesia and 2) creating integrated transportation solutions that address technology, policy, infrastructure and planning development (NCSTT) As shown in Table 1, significant energy was generated on each of these research topics. 64% of the scholars affiliated with the CCRs (faculty, graduate students, postdoctoral researchers) joined in this research (numbering 618 in total), 250 scholars gave conference presentations, and 534 publications came out. As discussed in Section I, all five CCRs are continuing the research started under SHERA even after the program ends. Table 1. USAID SHERA Results: innovative, research-based solutions to the challenges affecting Indonesia’s development CDSR ANBIOCORE NCSTT ARI SMART CITY Research projects pursued Projects to develop energy sustainability methodology, and explore decentralized photovoltaic & bioenergy systems Field projects on ecosystem impact of different fishing systems, and genomic markers for the specific cattle in different regions Projects to advance electric vehicle technology, develop e￾Trike, and produce ethanol from invasive water plants Projects assessing respiratory disease burden at hospitals, and build ongoing research into clinical practice Development of 2 prototypes: Smart Land Surveillance System, Adaptive Traffic Management System # scholars involved in collaborative research 215 83 128 89 103 Resulting publications 93 26 174 4 237 # scholars giving conference presentations 72 31 63 8 76 Target 2: Broadening S&T research horizons to include world-class standards, recognition, and engagement Significant progress was also made on this target – in at least two ways. The first is SHERA’s mentorship of CCR HEIs and scholars in pursuing international research standards, as also discussed under the next two targets below – something that has laid a secure foundation for such high-level work to continue in the future. The second, as shown in Table 2, are SHERA’s successful efforts to garner greater global awareness of the work being done by Indonesian HEIs. The program resulted in 534 peer-reviewed publications, 94% of which occurred in Scopus-indexed outlets. This stands in marked contrast to the 9 more common Indonesian pattern mentioned above and is the direct result of SHERA’s guiding CCR scholars toward such outlets. While citation rates are dependent on many variables (academic field, journal circulation, etc.) and generally get larger over time, the fact that 43% of CCR publications emerging in FY18 have already been cited by other researchers is another sign of the recognition CCR research is receiving. So, too, is Indonesia’s noteworthy 2020 move upward in the Times Higher Education (THE) Impact Rankings. THE developed the Impact Rankings to recognize those HEIs doing exceptional work in addressing the United Nations Sustainable Development Goals (SDGs), something masked in their more general rankings. While the number of Indonesian HEIs placing in THE’s general Top 1000 grew from one in 2016 to six in 2020 (and also went up in the QS rankings), it is Indonesia’s tremendous growth in the Impact Rankings that has drawn international attention. The THE Impact Rankings also relate directly to SHERA’s focus on innovative solutions to Indonesia’s development challenges. As discussed further in Section J, all five CCR Leads, as well as two Affiliates, appear in the Impact Top 100 institutions worldwide on one SDG or another, a feat characterized as “almost unheard of” by THE.2 In fact, the work done by the CCRs appears to have been part of THE’s high assessment of these institutions. THE indirectly notes this by highlighting the ways Indonesian HEIs are forming international partnerships to tackle such issues as energy, sustainable transport, food security, environmental sensing, and infectious disease, and then directly illustrates this point by describing UI’s partnership with MIT (hence recognizing the SMART CITY CCR). Table 2. KPIs: USAID SHERA Goal - All CCRs through FY2020 Performance Indicators FY18 FY19 FY20 Grand Total 1. Number of peer-reviewed scientific publications resulting from USG support to research and implementation program3 51 127 95 273 Journal Articles 37 66 39 142 Proceeding Articles 43 182 167 392 Indexed in Scopus 71 230 206 507 Not indexed in Scopus 9 18 0 27 2. Ratio of citation to publication Percentage of cited publications 43% 21% 13% 21% # of Cited Publications 34 52 24 110 # of authors 91 149 34 274 Male authors 69 99 24 192 Female authors 22 50 10 82 Percentage Male/ Percentage Female 76%/24% 66%/34% 71%/29% 70%/30% 2 John Ross, Times Higher Education, May 18, 2020, Why Indonesia Makes an Impact in SDG-focused Ranking. https://www.timeshighereducation.com/world-university-rankings/why-indonesia-makes-impact-sdg-focused￾ranking. 3 In calculating the numbers in this row, SHERA counted 3 proceedings publications as the equivalent of 1 journal article. 10 Target 3: elevating the research infrastructure and culture of Indonesian HEIs This target corresponds to Outcome 2 of the SHERA Monitoring, Evaluation, and Learning Plan (MEL): strengthened, inclusive institutional environment for research and management in target Indonesian universities. Such increases in world-class research and recognition reflect SHERA’s constant and effective efforts to enhance the practices, procedures, and institutional culture for research at CCR HEIs (discussed in more detail in Sections E and H). CCR Reports uniformly state that the research environment at participating HEIs has changed as a result of the trainings, guidance, and modeling they encountered on best practices in managing and supporting research. Such efforts were built into the very structure of SHERA. Breaking with much past practice in U.S.-Indonesian higher education partnership programs, SHERA assigned responsibility for program development and management to the Indonesian partners. SHERA’s designers chose this structure, despite knowing that it would be challenging, because they wanted to build capacity at Indonesian HEIs. They wanted SHERA to have an impact on institutions as well as individual scholars. This structure demanded much from SHERA staff. Only one CCR Lead had previously been the direct recipient of USAID funding. As detailed in Section H, SHERA also operated as a learning laboratory on collaborative research program management. SHERA staff provided 146 training sessions and meetings on grant management; research protocols; reporting, monitoring & evaluation; creating knowledge products; connecting to national goals/needs; publicizing successes; financial management; invoicing and paying subawardees; closing the gap between U.S. and Indonesian practices; and providing updates and mentoring on current activities. They also established a monitoring, evaluation, and learning (MEL) system in which CCRs submitted (and received extensive feedback on) Quarterly and Annual Reports, noting their progress on the KPIs. Plans and activities were regularly modified as a result. This capacity-building required a lengthy start-up period for the CCRs, as they and SHERA staff built the scaffolding needed for program success. Multiple challenges arose, especially around financial management and completing reports. There was much to be worked through, and many lessons were learned. Ultimately, however, these efforts produced not only significant research and publications, but also a cadre of Indonesian HEIs with deep experience working with large, collaborative research projects and international grants. Target 4: providing on-going professional development and hands-on research opportunities for faculty, students, and postdoctoral researchers This target corresponds to Outcome 1 of the SHERA Monitoring, Evaluation, and Learning Plan (MEL): increased research capacity of faculty, Ph.D. students and postdoctoral researchers. SHERA’s capacity-building (and move toward world-class research) also included active, repeated, and successful research mentoring of Indonesian scholars. As discussed in more detail in Section D, CCR Reports repeatedly mention the value Indonesian scholars derived from the SHERA training sessions they attended and from their interactions with U.S. scholars, both face-to-face and virtually. These activities elevated the way Indonesian scholars thought about their research while also entering them into global research conversations. Such guided learning was constant throughout SHERA, with roughly 600 Indonesian scholars attending training programs each year and 289 becoming active researchers on CCR projects. U.S. scholars also paid 43 visits to the CCRs, 28 Indonesian scholars traveled to CCR partners in the U.S., and 81 Indonesian scholars received on-going mentoring from U.S. scholars. 11 Such efforts were designed and conducted by each CCR for its own scholars. These activities immediately led to an increase in peer-reviewed publications and conference presentations. The beneficial effect of encouraging scholars to publish in Scopus-indexed outlets, expanding their research networks, increasing their capacity to write in English, enhancing their skills of data collection and analysis, and broadening their view of research will stay with SHERA participants long after the program is over. So will the 531 Knowledge Products (research reports, training modules, guidelines, and other educational materials) created by the CCRs. Target 5: building the collaborative linkages among HEIs needed for high-level, multi-faceted research, both within and beyond Indonesia This target corresponds to Outcome 3 of the SHERA Monitoring, Evaluation, and Learning Plan (MEL): enhanced collaboration in CCR/consortium for collaborative research. In the interviews conducted to establish SHERA, Indonesian HEIs responded that other than additional funding to support S&T research, what they most needed were greater opportunities for research collaboration. Linkages to HEIs in other countries open pathways for sharing laboratories, knowledge, methodologies, policies, and procedures, at the same time they introduce what is happening in Indonesian research to the world. It is difficult to conceive of world-class research occurring anywhere without worldwide connectivity. It is similarly difficult to conceive research relevant to Indonesian development occurring without its HEIs working with government agencies, research institutes, NGOs, local communities, and the private sector. Such linkages engage relevant stakeholders, increase the resources and knowledge available to HEIs, and tie their projects to real-world issues. As detailed in Section F, the linkages created among CCR Lead and Affiliate HEIs are one of SHERA’s most important outcomes. While there was initial concern that meaningful connections could be formed between more and less prestigious Indonesian HEIs, that is indeed what happened. Despite some of the normal management issues that can arise in multi-institutional arrangements, scholars from Lead and Affiliate institutions collaborated with some frequency, articles and presentations were co-written, research sites were scattered across institutions, events rotated among HEIs, U.S. scholars visited affiliates as well as leads, and CCR-wide delegations traveled to the U.S. All plan to continue these collaborations in one form or another after SHERA (see Section I). The linkages with U.S. HEIs took on a somewhat different character. While there were initial concerns that U.S. HEIs might not embrace the capacity-building aspect of SHERA (focusing instead only on specific research projects), these concerns also turned out to be unfounded. U.S. scholars made 43 visits to their partner CCRs (some visiting two or three times), giving training sessions, mentoring Indonesian researchers, and joining in research projects. The engagement of these U.S. scholars was indeed a great success. What was less successful with the U.S. partners was the development of full-bodied institutional partnerships going beyond these particular individuals. More comprehensive bi-national partnerships, the kind that generate multiple projects across multiple fields, often take several years to develop. SHERA, especially in its truncated form, did not provide sufficient time for this to happen. Conversations toward this outcome have occurred, and there are several possibilities on the horizon, but more work will need to be done. 12 Target 6: building similar research linkages with a range of public/private stakeholders This target corresponds to Outcome 3 of the SHERA Monitoring, Evaluation, and Learning Plan (MEL): enhanced collaboration in CCR/consortium for collaborative research. As discussed further in Section F, for Indonesia’s research profile to grow, HEIs must work with other entities involved in either conducting or applying research. As with their university partners, such collaborations enable the sharing of resources, information, and expertise. In this case, they also enable university researchers to match their work to what is both needed and feasible, to tailor it to the needs and specifications of those who might use it, and – in turn – to influence how it is used. Table 3 reveals the many such linkages made by SHERA’s CCRs, which range across industry, government (national, provincial, local), hospitals, public transportation, and research centers. In some cases, these partners provided training sessions for CCR scholars. In others, they were more directly involved in collaborative research. Table 3. Overarching USAID SHERA Goal: Research partners beyond academia CDSR • 6 local agencies in Bangka Belitung • PT Korintiga Hutani • Shimizu Corp • Govt. of Gorontalo Region ANBIOCORE • BET (Livestock Embryo Center) Cipelang Bogor • BIB (Artificial Insemination Center) Lembang Bandung • BBIB (Artificial Insemination Center) Singosari Malang • Lolitsapi (Cattle Research Center) Grati-Pasuruan • BTPU HTP (Livestock Breeding Center) Indrapuri NCSTT • Ministry of Transportation • P.T. Bakrie & Bros. • BPPT (Agency for Assessment and Implementation of Technology) • PT PLN (State Electricity Co.) • Palembang Dept. of Transportation • Jakarta Rapid Mass Transit Corp. ARI • 12 hospitals spread across Bandung, Banda Aceh, Banjarmasin, Mataram and Ambon • 1 pharmaceutical company SMART CITY • PT Telkom • PT Mass Rapid Transit Jakarta • PT Pangripta Geomatika • Panasonic • Agencies for Planning & Develop for Depok, Serang, Sukabumi, & Bandung Table 4 demonstrates that some of these collaborations have already resulted in tangible outcomes. More can be expected in the future, as several agreements and projects already testify (see Section I for details). 13 Table 4. USAID SHERA Products Being Put to Use4 CDSR Sustainability energy methodology adopted by United Nations Development Program (UNDP) to use in Bali, East Nusa Tenggara, and Southeast Sulawesi NCSTT Appointed by Ministry of Transportation to the team designing the transportation system for new capital of Kalimantan ARI • District Health Office of Bandung using ARI mortality research results to develop policy • Developed the application AMARI-COVID-19 to help with assessing the pandemic in Bandung and West Java5 SMART CITY • Smart Land Surveillance System, web-based application developed with PT Pangripta Geomatika being tested in Depok City • SMART CITY Governance in Indonesia, instrument developed with Ministry of Internal Affairs to assess regional government readiness for smart city development • Produced the TaLis (Electric Tube), a sustainable energy storage system that has been applied in Depok; Pertamina Gas Station in Cicoko South Jakarta; Palu￾Donggala Post-Tsunami Settlement; Lombok Post-Earthquake Settlement; and Banten Post-Tsunami Settlement • Working with Ministry of Internal Affairs to develop a SMART CITY Assessment for ASEAN Sustainable Development Goal (SDG) target • Successful patent entitled “Production Method through Wet and Dry Extraction, and Vegetable Oil Extracellular Lipase” Target 7: spreading high-level research capacity to more Indonesian universities in more diverse locations Maximizing S&T research in Indonesia requires unlocking the research potential of HEIs and scholars located outside the current centers. As stated above, high-performing Indonesian HEIs are heavily concentrated on Java. Since the selection of CCR Leads was limited to the top 11 institutions in Indonesia (in terms of high-quality S&T research), it is not surprising that all five Leads are located on Java. Of the 25 distinct Affiliates, however, only 9 are on Java. As shown in Table 6 and Figure 2, CCRs were purposely constructed to bring HEIs in different regions and with different resource levels together with the Leads. All but four of the Affiliates had been identified in the White Paper as having potential to grow in S&T research, if given some support. The strong participation of most Affiliates in the CCRs shows that this identification was valid. To aid in this process, all CCRs designed their programs to move around participating HEIs, established websites, held virtual meetings, and shared the visits of U.S. scholars. Lead HEIs also welcomed scholars 4 In the SHERA ARF, KPI 3 of the SHERA Goal section tallies the instances in which CCR products have been adopted by outside entities. Only two are listed. This may represent a reporting or definitional issue, as various reports mention several more, as shown in Table 4. This KPI is intended to capture how much research is already being put to use for development purposes (replicated, applied or taken to market). The few numbers appearing for this KPI do not, however, do justice to what has actually happened, as revealed by the narrative CCR reports. 5 This application was developed just after ARI’s subaward ended in January 2020 but is a direct result of the CCR’s work. 14 from Affiliates to use their laboratories and other research facilities, while Affiliates opened up new field research locations to scholars from Lead HEIs. Research projects were collaborative, and publications were often co-written by scholars from across the CCR. Many of the projects planned for post-SHERA continue to invoke these connections. Table 5. Geographical Distribution of Indonesian CCR HEIs Province Lead Institutions Affiliate Institutions Jakarta 1 2 Yogyakarta 1 Central Java 2 East Java 2 West Java 3 3 Aceh 1 South Kalimantan 1 East Kalimantan 1 West Nusa Tengarra 2 East Nusa Tengarra 1 Maluku 1 South Sumatra 1 West Sumatra 2 Bangka Belitung 1 Gorontalo 2 North Sulawesi 1 West Papua 1 Bali 1 Figure 2. CCR Map 15 Target 8: directing special attention to tapping and developing the research capacities of Indonesian women interested in S&T While women constitute an unusually high 35% of Indonesian tertiary students in S&T fields, those who go on to become faculty often languish at lower faculty ranks and have little opportunity for research. As detailed in Section G, SHERA has opened the door for such female scholars. For example, 40% of those participating in CCR research and 35% of the authors of SHERA publications were women, as were two￾thirds of the Indonesian scholars who received on-going mentoring from U.S. scholars. Underscoring SHERA’s attention to female empowerment in S&T research, 8% of the knowledge products developed by the CCRs concerned these issues, as did two major conferences (which attracted 322 participants). Comparison of the CCRs on SHERA’s overarching goal As indicated in Table 6, which compares CCRs on the publications, and as becomes clear in the following sections of this evaluation, there is much variation among CCRs on particular KPIs. Each CCR had its own strengths and challenges. Much depended on academic field, amount of scaffolding needed to get started, and nature of the research projects undertaken. Also, one CCR was tasked with combining what had been two separate proposals into a single project. And due to SHERA financial limitations as well as some managerial difficulties, two CCRs (ANBIOCORE and ARI) closed their SHERA operations earlier than the others (January 2020). Table 6. KPIs: USAID SHERA Goal - CCR Comparison FY18-FY20 Performance Indicators CDSR ANBIO￾CORE NCSTT ARI SMART CITY 1. # of peer-reviewed scientific publications resulting from USG support to research and implementation program 93 26 174 4 237 Journal Articles 36 14 37 4 46 Proceeding Articles 57 12 137 0 186 Indexed in Scopus 79 13 174 4 237 Not indexed in Scopus 14 13 0 0 10 2. Ratio of citation to publication produced by Indonesia researchers 34% 0% 7% 0% 27% # of Male authors 72 0 18 0 102 # of Female authors 28 0 0 0 54 Despite these variations, all CCRs reported that the research capacity of both their individual scholars and their participating HEIs changed as a result of SHERA. And all rose to the top for at least one of the KPIs, as shown in Table 7. For example, while ARI was low on publications, it was high on training sessions and participating scholars (also, in its close-out report in January 2020, ARI mentioned that 7 more articles had been submitted for publication and 14 drafts were being polished for submission). ANBIOCORE scored highly on participation in gender-sensitive events and improved policies and systems. CDSR took the lead in number of scholars participating in collaborative research. NCSTT did the same for conference presentations and knowledge products created. And SMART CITY excelled at publications and scholars being mentored by U.S. partners. In short, particularly if SHERA’s overarching goal combines capacity-building with immediate products, all CCRs registered progress. 16 Table 7. Number of KPIs for which Each CCR Came in 1st, 2nd, 3rd, 4th, or 5th Ranking on a Particular KPI6 CDSR ANBIOCORE NCSTT ARI SMART CITY 1 st 3 3 2 2 8 2 nd 4 3 2 3 6 3 rd 3 5 5 6 2 4 th 2 6 6 2 0 5 th 6 1 3 5 2 (Numbers indicate # of KPIs where a CCR had this ranking; e.g., CDSR ranked 1st on 3 different KPIs.) D. Further Discussion of Target 4: providing on-going professional development and hands-on research opportunities for faculty, students, and postdoctoral researchers Target 4 is a close match with Outcome 1 of the SHERA MEL: increased research capacity of faculty, Ph.D. students and postdoctoral researchers (here collectively referred to as scholars). The CCRs approached this target by 1) enhancing scholar skills, knowledge, and understanding concerning research, 2) engaging scholars in high-level collaborative research, writing, and presentation, and 3) building their research networks, both nationally and internationally. In short, building for the future while already producing in the present. To such ends, SHERA provided a significant infusion of research training and one-on-one mentoring across the 30+ Indonesian institutions in the CCRs. Scholars from all HEIs were drawn into active research projects. Annual reports mention the enthusiasm that participants had for these efforts, the beneficial effect of pushing scholars toward publication in Scopus-indexed outlets, the expansion of their research networks, their increased English language capacity for general communication and writing, their enhanced ability to collect and analyze data, their broadened view of research, their first forays into virtual learning formats, and the heightened quality of the papers they wrote and presentations they gave (several, for the very first time). Table 8 summarizes some key elements of this strategy. The CCRs held 45 short-term research training courses, attended by over 800 scholars in the peak year of FY18. U.S. scholars individually mentored roughly 40 of their Indonesian counterparts each year, both in person and by distance means. Another 300+ Indonesian scholars participated in collaborative research projects in the peak of FY19. The 531 knowledge products that were created and 60 knowledge-sharing events listed in Section F also fit here. Furthermore, these activities occurred at both Lead and Affiliate institutions, and some were opened to additional HEIs and partner organizations. The 534 publications deriving from SHERA scholars (with 94% indexed by Scopus) give further testimony to the effectiveness of these efforts, as do the 250 presentations at scholarly conferences (see Table 2). 6 Annex V of this report provides a complete breakdown of these rankings, per KPI. 17 Table 8. KPI Outcome 1: Improved Capacity of Scholars - All CCRs FY17-FY20 Performance Indicators FY17 FY18 FY19 FY20 Total/ %age7 1.1 Percentage of scholars who presented at conferences 0 46% 78% 13% 46% # of scholars 0 76 149 25 250 Male 0 62% 54% 80% Female 0 38% 46% 20% 1.1.1 # short-term training courses held 3 29 13 0 45 # of participants 80 817 381 0 426/year Male participants 34% 50% 51% 0 Female participants 66% 50% 49% 0 1.1.3 # U.S. scholars visiting Indonesian HEIs to lead training courses at partner HEIs 4 16 22 1 43 13/year 1.2 % scholars who participated in collaborative research 40% 76% 113% 4% 64% Male participants 57% 60% 59% 75% Female participants 43% 40% 41% 25% 1.2.1 # Indonesian scholars who participated in in-person faculty exchanges held in U.S. NA 10 18 0 28 Male participants NA 90% 50% 0 Female participants NA 10% 50% 0 1.2.2 # Indonesian scholars who received ongoing mentoring from U.S. HEI partners 4 28 49 0 81 Male participants 50% 42% 33% 0 Female participants 50% 58% 67% 0 As Table 9 further demonstrates, these activities occurred across all CCRs, each focusing on what was most relevant and effective to its scholars. ARI, for example, emphasized clinical research and manuscript writing. NCSTT sent an Indonesian delegation to MIT, each of whom was charged with spreading what was learned to others in the CCR, in a cascading fashion. SMART CITY arranged for its U.S. mentors to review their Indonesian mentees’ draft manuscripts and provide feedback, using review sheets to track the changes through this online process. ANBIOCORE also emphasized scientific writing, providing the guidance that enable one of their scholars to publish in an international journal for the first time. Table 9. KPI Outcome 1. Improved Capacity of Scholars - CCR Comparison FY18- FY20 Performance Indicators CDSR ANBIO￾CORE NCSTT ARI SMART CITY 1.1 % of scholars who presented at conferences 32% 86% 43% 16% 84% 1.1.1 # short-term training courses held 11 10 6 14 4 7 Totals for the percentage-based indicators in Table 8 were calculated by dividing the cumulative number of SHERA scholars who participated in the activity divided by the cumulative number of SHERA scholars. 18 Performance Indicators CDSR ANBIO￾CORE NCSTT ARI SMART CITY 1.1.2 % scholars participating in training courses 53% 38% 30% 43% 64% 1.1.3 # U.S. scholars visiting Indonesian HEIs to lead training courses at partner HEIs 1 8 5 9 20 1.2 % scholars who participated in collaborative research 73% 63% 44% 49% 132% 1.2.1 # Indonesian scholars who participated in in-person faculty exchanges held in U.S. 10 0 3 8 7 1.2.2 # Indonesian scholars who received ongoing mentoring from U.S. university partners 9 21 3 14 34 E. Further Discussion of Target 3: elevating the research infrastructure and culture of Indonesian HEIs SHERA also aimed its capacity building efforts at the institutional environment for research at Indonesian HEIs. Outcome 2 of the ARF (strengthened, inclusive institutional environment for research and management in Indonesian HEIs) fits here. Toward such ends, SHERA directed much attention to moving CCR Lead institutions toward management systems that would effectively support the planning, collaboration, implementation, and results dissemination of their research operations. SHERA also directly asked CCRs to consider gender-inclusive procedures that would move away from the all-too￾frequent pattern of relying on small and already known research teams. CCR Reports uniformly state that the research environment at participating HEIs has changed as a result of the trainings, guidance, and modeling they received on these topics. It is for such reasons that SHERA took the bold step of having Lead institutions manage the finances and work of each CCR, thereby creating a learning laboratory that would develop skills and alter institutional processes for facilitating complex, externally funded research projects. To aid in this process, as discussed further in Section H, SHERA staff provided continual guidance and mentoring, including the 146 training interventions listed in Annex II, which ranged from financial management, record-keeping, and project planning to partnership, personnel and crisis management. Some of these meetings occurred at the SHERA office in Jakarta; others at CCR institutions. As shown in Table 10, 60 knowledge sharing events were also held for the CCRs to discuss best practices and lessons learned. And a good number of the 531 knowledge products created by the CCRs concerned research management topics. Finally, as discussed further in Section G, several gender-focused events occurred, attended by over 500 participants. Table 10. KPI Outcome 2: Strengthened, inclusive environment for research and management at Indonesian HEIs - All CCRs FY18-FY20 Performance Indicators FY18 FY19 FY20 Total 2.1. % Indonesian CCR HEIs with improved research policies and systems in place 67% 13% 17% 28%8 8 Cumulative number of CCR institutions which adopted new and/or improved and/or developed research policies and/or systems divided by the cumulative number of CCR Indonesian institutions 19 Performance Indicators FY18 FY19 FY20 Total 2.2. % of CCR members with improved administrative and management systems 25% 27% 24% 14%9 2.2.2 # of CCR knowledge products created 236 195 100 531 2.2.3 # of CCR knowledge sharing events held on best practices & lessons learned 31 24 5 60 2.2.4 # of participants in activities addressing gender equality or female empowerment in science and technology research 282 81 146 509 # of CCR knowledge products created on gender 17 18 6 41 In terms of results, all CCRs report that they learned a great deal about grant management according to international standards. 17 of the participating HEIs register improved research policies and 13 register improved administrative and management systems. SHERA staff also see progress, although they feel many of these HEIs will need more guidance as they move forward after SHERA. While there is some fanning out, improvements have also been concentrated at the Lead HEIs. Table 11 shows the progress each HEI made on this outcome. There has been wider spread of improved research administrative and management systems than improved research policies. Themes that run through the CCR reports highlight the value of adopting SHERA’s MEL system for tracking projects and the many lessons that have been learned about financial management and accountability. Some CCRs also set up a distributed management system, ARI designated a finance officer at each HEI as well as a research coordinator at each hospital; each SMART CITY HEI established its own research cluster, some of which took on management functions; CDSR spread some of its management procedures across all participants; and NCSTT set up a standard operational procedure across the CCR for stipend calculation on research work research and publication incentives. In terms of research systems and policies, several the HEIs of ARI established Ethics Committees and the CCR as a whole set up a data management system through the RedCap platform, accessible across the CCR. SMART CITY also developed a knowledge products library on research and communication media, such as video, posters, papers, guidebooks, website, and a kaleidoscope. Table 11. KPI Outcome 2: Strengthened, inclusive environment for research and management at Indonesian HEIs - CCR Comparison FY18- FY20 Performance Indicators CDSR ANBIO￾CORE NCSTT ARI SMART CITY 2.1. % Indonesian CCR HEIs with improved research policies and systems in place 0%10 200% 14% 57% 46% 2.2. % Indonesian CCR HEIs with improved administrative and management systems 7% 75% 14% 20% 33% 2.2.2 # of CCR knowledge products created 78 59 230 28 136 9 Cumulative number of CCR institutions which adopted new and/or improved and/or developed administrative and/or management system divided by the cumulative number of CCR Indonesian institutions 10 See KPI definition in Annex III. CDSR focused on building its administrative and management system by developing standard procedures in several focus areas, such as finance and grant monitoring, as required by the program. However, it did not develop research and management policies that covered all research topics. 20 2.2.3 # of CCR knowledge sharing events held on best practices & lessons learned 12 12 18 5 13 2.2.4 # of participants in activities addressing gender equality or female empowerment in science and technology research 0 146 52 117 194 # of CCR knowledge products created on gender 31 0 2 1 7 F. Further Discussion of Targets 5 and 6: building collaborative linkages with HEIs and public/private stakeholders Targets 5 and 6 identify the importance of collaborative linkages for enhancing research quality and generating new research. Such partnerships have the potential to stimulate thinking, share resources, open new opportunities, and exchange knowledge and best practices. They also draw participants out into broader worlds, both domestically and internationally. Working in partnership was a key part of SHERA’s strategy. As shown in Table 12, the KPIs of Outcome 3 of the ARF summarize the formal relationships that were developed. Table 12. KPI Outcome 3: Enhanced collaboration - All CCRs FY18-FY20 Performance Indicators FY18 FY19 FY20 3.1 Percentage of CCR HEIs engaged in joint research projects 100% 100% 100% # of joint research projects with public or private sector 4 7 7 3.2 % of partnerships created to support joint research projects 97% 63% 3% Lead 19 12 31 Affiliate 10 7 17 Annex VI gives a more complete list of the key relationships established by each CCR as part of SHERA. They fall into three main categories: Lead and Affiliate Indonesian HEIs; Indonesian business, industry, community agencies, and research centers; and U.S. HEIs. The first has yielded one of SHERA’s most important successes. The second has taken key steps. And the third has advanced Indonesian S&T capacity and laid the groundwork for – but not yet attained – more two-way collaborations in the future. Collaborations among Indonesian HEIs The CCRs are collaborative units, in and of themselves. Initial doubts about the possibility of forming meaningful connections between more and less prestigious Indonesian HEIs have been dispelled. The connections forged by the CCRs have been genuine and mutually beneficial. They have enabled scholars from less well-resourced HEIs to use the facilities and learn from those at more well-resourced ones. Conversely, scholars at the former have been able to offer access to new research sites to those at the latter. CCR Leads took an active role in building the research capacity of Affiliates. CCR reports also praise way these collaborations brought different disciplinary and institutional perspectives together, leading to innovative approaches. Despite the kind of management issues that can occur in multi-institutional arrangements, scholars from Lead and Affiliate HEIs collaborated frequently, wrote articles and presentations together, selected research sites across institutions, and rotated holding events. As shown in Table 12, all CCR HEIs were 21 involved in such collaborations. Still other Indonesian HEIs were invited to some SHERA activities as well. Table 13. KPI Outcome 3. Enhanced collaboration - CCR Comparison FY18-FY2011 Performance Indicators CDSR ANBIO￾CORE NCSTT ARI SMART CITY 3.1 # of CCR HEIs in joint research projects FY 2018 9 10 8 10 8 Indonesian based 8 8 7 9 5 U.S. based 1 2 1 1 3 With public or private sector12 0 0 0 4 0 3.1 # of CCR HEIs engaged in joint research projects FY 2019 9 10 9 13 9 Indonesian based 8 8 7 12 5 U.S. based 1 2 2 1 4 With public or private sector 0 0 0 7 0 3.1 # of CCR HEIs engaged in joint research projects FY 2020 11 10 9 13 9 Indonesian based 8 8 7 12 5 U.S. based 3 2 2 1 4 With public or private sector 0 0 0 7 0 Table 13 shows how consistent such inter-institutional collaboration was across CCRs, as well as how additional HEIs were included in the research over time (especially for SMART CITY). CCR reports give additional insight into how their collaborations worked. ARI dealt directly with what it called the “gap” among HEIs in research infrastructure and practices by developing an overarching research project conducted across the CCR and making active use of new information technology to communicate and manage data. CDSR expressed how beneficial the research collaboration was for Lead and Affiliates alike and noted that the local government of Gorontalo province had replicated the CCR partnership scheme on a smaller scale in working with Universitas Gorontalo. ANBIOCORE made active use of its geographically dispersed HEIs (and added new ones) to conduct research among different cattle populations and in different fisheries. SMART CITY accomplished the note-worthy feat of having 100% of CCR scholars participate in collaborative research, which – in turn – developed several smart city tools that are being adopted by government agencies and private partners. Collaborations with Indonesian partners outside academia As shown in Tables 3 and 4, SHERA’s CCRs also took major steps in forming linkages with government agencies (national, provincial, local), hospitals, public transportation, private industry and research centers. Because much S&T research at HEIs has been theoretical, creating such partnerships have been critical to refocusing on priority development issues and understanding how to fit university research programs to the needs of those who might use their results. 11 While KPI 3.1 is percentage based, this chart shows the numbers on which these percentages are based, in order to show the addition of new Affiliates and partners over time. 12 Number of Indonesian and U.S.-based institutions in the public or private sector, versus academic institutions. 22 In some cases, the public/private linkages developed by the CCRs have been first explorations of these issues and are still a work in progress. Training sessions provided by such partners have nevertheless had an impact on the scholars who participated. And public awareness of university research as a resource for innovative solutions to Indonesia’s challenges has grown. CCR reports comment on the ways CCR activities increased the capacity for S&T research by groups outside academia, from clinical healthcare professionals to animal breeders. In other cases, the linkages were able to move into collaborative research projects. ARI worked with 12 hospitals to enroll patients in clinical trials. SMART CITY’s Smart Land Surveillance System is being tested in Depok City. The United Nations Development Program (UNDP) has adopted CDSR’s Sustainability energy methodology to use in Bali, East Nusa Tenggara, and Southeast Sulawesi. NCSTT credits SHERA with helping it align its research with governmental and business strategies, enter into these arenas, create high-quality products, and be appointed to work on designing the transportation system for the new capital of East Kalimantan. Given the time it takes to develop the relationships and understandings that must undergird such endeavors, these developments are significant. Collaboration with U.S. HEIs The linkages with U.S. HEIs took on a somewhat different character from those in Indonesia. Here the focus was very much on strengthening research at Indonesian HEIs. Fully dispelling initial concerns that U.S. institutions might not embrace such capacity-building, U.S. scholars made 43 visits to partner CCRs (some visiting two or three times), gave training sessions, mentored Indonesian researchers, and inspected research projects. They U.S. visited Affiliates as well as Leads. And they received CCR delegations for brief visits to the U.S. CCR reports emphasize how beneficial the visits of these U.S. scholars were. The visitors provided expert advice on research protocols, project development, and academic writing, which several Indonesian HEIs even credit with advancing their accreditation efforts. The development of more comprehensive institutional partnerships between U.S. and Indonesian HEIs did not, however, occur. While several CCRs have had conversations with U.S. HEIs about building more robust partnerships that generate multiple projects across multiple fields, they are still in the fledgling stage. Such partnerships generally take several years to develop, and SHERA’s truncated time span has made itself felt here. Two CCRs had to cancel their visits to the U.S., and all worry that the early termination of SHERA has damaged the credibility of Indonesian partnerships in the eyes of U.S. institutions. Still the groundwork has been laid for some future possibilities. Even knowing SHERA was ending, conversations between CDSR and the University of Colorado-Boulder about a larger partnership move forward, and still other U.S. HEIs (Colorado State University, University of Texas-Austin, Auburn University, California Polytechnic, and University of California-Davis) have signed MOUs to explore possible collaborations. Based partially on its experience with two CCRs, in January 2020, MIT announced that MIRA (MIT Indonesia Research Alliance) would fund seed projects with 10 Indonesian universities (many of which were involved in SHERA, especially NCSTT and ANBIOCORE) in the fields of energy, biotechnology, economics, health, and transportation. 23 G. Further Discussion of Target 8: developing the research capacities of Indonesian women interested in S&T Promoting access to research and professional development opportunities for women in S&T is yet another key component of SHERA’s overarching goal. The untapped potential is great. While UNESCO 2016 data show that women constitute an unusually high 35% of Indonesian tertiary students in S&T fields, those who go on to become faculty often languish at lower faculty ranks and have little opportunity for research. The 2019 OECD PISA report (Programme for International Student Assessment) confirms that female interest in S&T is a distinguishing trait for Indonesia. This latest report, which covers 32 countries, lists Indonesia as one of the rare countries where girls scored higher on mathematics and science than boys (it has the 6th greatest difference in scores, ranking the same as Iceland). In short, there is a cohort of young Indonesian women who are interested in and pursuing careers in S&T. SHERA wished to make sure the doors were open for them in research. The CCRs embraced this idea. They created 41 Knowledge Products addressing female empowerment in S&T research (8% of the total Knowledge Products) and offered knowledge sharing events that attracted 509 participants (both male and female). ARI, for example, conducted an event on “The Role of Women in Medicine and Education” in which 35 female scholars presented their research, and after which all CCR HEIs signed a decree on Gender Mainstreaming by promoting women researchers in medical science, “Deklarasi Bandung untuk Wanita 2018.” NCSTT, by way of another example, held a seminar on “Women’s Empowerment in the Academic and Research Environment” aimed at engaging women in international research collaboration. The results of such activities are evident in the role that women played in SHERA, as shown in Table 14. With the exception of co-authorship (where more attention could be directed in the future), the percentages of female participation in SHERA are at or above their representation in S&T fields in Indonesia. Table 14. Female Participation in USAID SHERA H. Evaluation of USAID SHERA’s Distinctive Approach In both structure and management, SHERA broke with much past practice in international research partnerships. As CDSR stated in one of reports, SHERA is not a usual research grant. This is a grant to develop the institutional capability of the grantees. Very challenging process in executing [but] the grant increases CDSR capability in managing dynamic[s] of collaborative program[s]. Women were: • 27% of the authors on CCR publications • 38% of CCR conference presenters • 51% of participants in short-term training sessions • 40% of participants in collaborative research • 37% of those who traveled to U.S. partners • 62% of those who received on-going U.S. mentorship • 48% of participants in meetings with SHERA staff • 24 SHERA’s approach was indeed demanding. It awarded grants to Indonesian HEIs, not U.S. HEIs. It allocated many development and management functions to these HEIs, asked them to develop consortial relationships with others (including some they had never worked with before), and asked that they pay attention not only to current projects but also capacity-building for the future. The program was concerned with spreading such capacity beyond the usual HEIs and locations and was as interested in creating collaborative groups as doing specific projects. This approach asked much of everyone. It also meant that much scaffolding work had to be done before CCR research got under way. This long start-up time was a good investment, however. It was needed to yield the results already documented in this evaluation: high quality projects were done, Scopus-indexed publications came out, private and public sector organizations were engaged, and capacity was built. In other words, SHERA’s success was directly related to its approach. These outcomes were a direct result of the way SHERA was defined, the degree to which the CCRs embraced the challenge, and the many interventions of the SHERA staff. SHERA was, in fact, a learning laboratory for what works and what does not in building research capacity on a national scale. It did this by creating a feedback loop between training, infrastructural development, and network building, on the one hand, and real-time research and publication, on the other. These two elements played off each other in a process of continuous learning and improvement. All CCRs experienced significant immediate results that energized the research ambitions of participating scholars. All also built capacity for further such work that will tap the potential for nationally-relevant research across a range of Indonesian HEIs. In short, SHERA opened up new ways of doing things, with attendant twists and turns along the way. This was new territory for everyone. SHERA’s ultimate success was the result of three basic principles: local ownership, collaboration & engagement, and oversight & learning. Local ownership SHERA was premised on building Indonesian expertise in managing large, externally-funded grants and projects by having Indonesian HEIs assume many of these functions themselves. This strategy would only work, however, if the HEIs had some experience on which to build. Before selecting the CCRs, IIE did a comprehensive study to determine which of the 7,000 Indonesian HEIs might be capable of taking on responsibilities. It identified a pool of 10-15 top institutions with the greatest potential for further S&T research development in priority areas and another 25-100, spread across Indonesia, with possibilities for Affiliate status. Once the CCR Leads were selected, SHERA staff conducted a detailed risk assessment of the management challenges these institutions might encounter. Only one had been the direct recipient of USAID funding before, although several had been subawardees. Few women were involved in the initial CCR teams. SHERA realized that much guidance and mentorship would be needed. To reinforce local ownership, a team of a team of Indonesian professionals with experience managing similar USAID programs was assembled to work closely to take on this role. The path was not always smooth, but all sides persevered, thereby laying a better foundation for future Indonesian grant management than might have happened if U.S. HEIs had taken on this function. Collaboration and engagement SHERA was also premised on the necessity of collaboration in S&T research. World-class research no longer occurs solely within single institutions, but is nearly always the result of multiple collaborations, 25 both local and international. The CCRs were designed to serve as learning laboratories for research partnership building, while simultaneously spreading research capacity across many types of Indonesian HEIs. Adding U.S. HEIs to this mix brought international perspectives and standards into the conversation, while interacting with governmental, community, and private sector organizations enabled the research to respond to development needs. SHERA staff guided such collaboration throughout the project. They set such requirements as having all research teams contain scholars from multiple HEIs and having training sessions spread across the CCR. They also used IIE’s and USAID’s GOI and private sector relationships to assist CCRs in making connections within Indonesia, and IIE’s network of several thousand U.S. HEIs to build bi-national connections. SHERA staff also assisted CCRs in navigating the communication and commitment challenges that partnerships often raise, as well as the complexity of managing multiple partners, not all of whom share the same understandings and goals. Learning and oversight SHERA also operated as a learning laboratory for collaborative research program management. To this end, SHERA staff provided a robust process of mentoring and oversight to the CCRs. The Training Tracker in Annex II captures the formal events in this process. There were 146 training sessions and meetings on grant management; research protocols; reporting, monitoring & evaluation; creating knowledge products; connecting to national goals/needs; publicizing successes; financial management; invoicing and paying subawardees; closing the gap between U.S. and Indonesian practices; and providing updates and mentoring on current activities. Of equal importance was the continuous monitoring, evaluation, and learning (MEL) system that operated over the life of the program, containing Quarterly and Annual Reports. The KPIs of the Results Framework were used to measure progress towards activity results, identify shortcomings in activity interventions, inform decisions to adjust course, and facilitate communication of results to USAID, Counterparts, and other stakeholders. Working with IIE headquarters-based experts in the U.S., SHERA staff analyzed the CCR reports and provided feedback for continuous performance improvement. Funding was disbursed to CCRs in phases, after confirming that each had met predetermined targets (while maintaining flexibility to allow for mid-course corrections). As SHERA staff attest, most of the challenges working with the CCRs concerned finances and completing reports. The CCRs agree. There was much to be worked through. Accounting systems used at the research or faculty-level at Lead HEIs did not allow for tracking or reporting on program-specific funds or tracking different types of expenses. All Leads used GOI procurement thresholds. The difference between obligated funds and award ceiling proved to be a particular sticking point. SHERA staff thus developed program-specific procurement procedures and templates that fit USAID requirements. They also advised on procurement planning and activities and established mandates on setting up financial accounts and reaching out to HEIs in other areas. An area where more guidance might have been useful concerns what was mentioned in Section D: there was only nascent development of comprehensive institutional partnerships between U.S. and Indonesian HEIs. Building and sustaining effective academic partnerships is complex work for which there is a growing body of insightful literature. Guidance on how to move from individual relationships to more institutional ones might have spurred more in this direction. 26 Table 15. Key Lessons Learned from Managing USAID SHERA (as articulated by SHERA and CCR Staff) • Devote at least six months to CCR formation and mentoring, as essential part of the process, before proceeding to projects • Clarify expectations for role of Lead, Affiliate, and U.S. partners in designing research • Spend some time discussing and practicing financial accountability requirements, as well as basis for disbursing funds • Develop risk mitigation and crisis management strategies for early termination and other unexpected events • More mentoring on managing multi-institution consortia • More mentoring on HEIs working with public and private sector organizations • More mentoring on developing partnerships with U.S. institutions • Bring senior executive managers at Lead institutions into the process of improving research policy • More mentoring on how to manage different policies and procedures across participating institutions • More mentoring on using IT to enhance communications among members of the CCR • More mentoring on selecting which research projects to pursue, including how many can be managed and how to find connections among varying research interests • More and earlier mentoring on sustainability planning for when the original funding ends • Ask participants at program events to complete event evaluation forms • Develop lines of communication with U.S. partners, to discuss program goals and solicit their evaluation of program progress I. Sustainability and Long-term Impact As stated in the SHERA Sustainability Assessment Out-Brief completed in July 2019, it was time for the CCRs to pay more attention to the funding, administrative, and other logistical issues needed for their work to continue post-SHERA. Such efforts were hindered, however, by the funding limitations and pandemic disruptions that then occurred. Travel and communications with partners, both existing and potential, became more difficult; attention became focused on finishing existing projects. The situation was exacerbated by no articulated vision for what direction and support might be forthcoming from GOI. Even with this less than optimal situation, all CCRs have enabled and energized participating HEIs and scholars. The work that SHERA started will go on in various ways. Certainly, through the improved research capacity of participants. Also, through the application of the research accomplished under SHERA for various development initiatives. The networks built through the CCRs will also continue, sometimes through a continuation of the CCR itself, and sometimes through a transition to other collaborative formats. At present, this is what is being planned. CDSR (CCR for Development of a Sustainable Region) UGM and its Affiliates intend to continue their collaboration. They greatly value the “mutual trust [that has been developed] among the lead and affiliates as well as related supporting stakeholders.” UGM has secured from the Ministry of Research and Technology to continue the CCR’s research on microalgae and supply chain coverage, working with UBB and UNG in their regions. UBB will hold a conference with CDSR scholars as reviewers and speakers. The University of Colorado-Boulder has also received a grant from the National Science Foundation to continue its sustainable region work with UGM and ITB. 27 ANBIOCORE (CCR for Animal Biotechnology and Coral Reef Fisheries) ANBIOCORE will likely split its two research focuses into separate centers, as it faced challenges bridging the two during SHERA. It has identified several possible new sources of funding, as well as partners to extend their research activities (BUSKIPM, USAID SEA, WCS, BIBD Baturiti, and BPTU HPT Pulukan). In one way or another, it hopes to continue research on the creation of genomic markers to maximize livestock production and to identify key food web components for coral reef fisheries using an ecosystem approach to management. NCSTT (National Center for Sustainable Transportation Technology) ITB and its Affiliates plan to continue working together under the NCSTT name. It has already procured funding and/or GOI support for several projects and has also expanded its partnership with MIT (as part of the MIT Indonesian Research Alliance). Work on the e-Trike Research Prototype will continue, as will the CCR-established International Journal of Sustainable Transportation Technology. ARI (CCR for Acute Respiratory Infections) This CCR may not continue with the same name but some of the HEIs will continue their collaboration. A large 5-location epidemiological research project will continue. Patients have been enrolled, permissions received, procedures established, and significant analysis will begin summer 2020. UNPAD has obtained some internal funding for these activities. ARI will also continue its work in public awareness and risk assessment with respect to COVID-19 in Bandung and West Java. Agreements have already been reached to work on the following: • Indonesian Eclampsia and Pre-eclampsia Registry (UNPAD) • Genetic Association between the Pfk 13 Gene Mutation and the Artemisinin Resistance Phenotype in Plasmodium falciparum Isolates from Indonesia. (UNSYIAH) • Molecular Profiling of Neonatal Sepsis Using Saliva with Fourier Transformation Infrared Spectroscopy (FTIR) (ULM) • Control of Hypertension as a risk factor of stroke by building a monitoring system using mobile phone application in Ambon City, Indonesia (UNPATTI); and Stunting (UNRAM) SMART CITY (CCR for Scientific Modeling, Application, Research, and Training for City-Centered Innovation and Technology) SMART CITY will be transformed as a special work unit (UKK) for service and community service (PPM) under the name SMART CITY Research Center at UI. A branch is being developed at Udayana University, and UNPAD and UNSRI are expected to follow. SMART CITY received funding from ASEANAUN/SEED to hold the International Conference on SMART CITY Innovation (ICSCI) in Bali in August 2020 (postponed to October 2020 due to COVID-19). It has also joined the MIT Indonesia Research Alliance. Several current SMART CITY projects will continue: 1) partnering with PT Pangripta Geomatika (PT.PGI), a private company focusing on data acquisition and processing technologies, to develop the “Situation Room”, a command center for implementing the Smart Land Surveillance System and Adaptive Traffic Intelligence System; 2) developing a SMART CITY Assessment for ASEAN’s Sustainable Development Goal (SDG); and 3) collaborating with the Mappi Regency Government in Papua to develop smart transportation and IT infrastructures. As for SHERA itself and what it has learned about managing research programs, SHERA will introduce its MIS (Management Information System) to the GOI. This will highlight the program’s accomplishments and demonstrate its use to the government. After gauging the GOI’s interest, SHERA will propose that the MIS be housed in government offices once SHERA ends, thereby sustaining the utility of the system. 28 J. Final Thoughts SHERA has been an unusual program, aimed not at a single research project but at something more comprehensive. It pursued a goal with eight distinct and significant targets, asked over 30 Indonesian and 12 U.S. HEIs to work together, brought public/private entities into the mix, spanned regions and institutional types, allocated program management responsibilities to HEIs that had never had them before, and aimed at tapping the unused research potential of lesser known institutions, women scholars, and the country as a whole. Above all, SHERA was about building capacity for future research as much as conducting it in the present. It was intended to change the Indonesian landscape for S&T research, both now and in the long-run. This was a tall order. Not every initiative worked. But most did. SHERA’s impact is both broad and deep. It touched the research lives of hundreds of scholars and changed the research environment even at institutions where research had not previously been a priority. All this, while also pursuing innovative research on five priority issues for Indonesian development. SHERA cannot be assessed by a single measure but must be seen as the complex whole that it has been. And even with its shortened time frame, SHERA has established several important facts. First, with the right supports, a wide range of Indonesian HEIs succeeded in diving into significant, world-class research. Second, despite initial fears, Indonesian HEIs worked well together in these endeavors, sharing resources and finding mutual benefit. Third, building the scaffolding and relationships to undertake such research is a lengthy process, but provides a good return on this investment. Fourth, Indonesian HEIs have much to contribute to the development challenges facing the nation. Fifth, there is much to be gained by building even more comprehensive partnerships between U.S. and Indonesian HEIs. The time has come for Indonesia’s universities to be globally recognized for what they contribute to the country’s and the region’s development. SHERA has enabled over 1,200 scholars and 30 HEIs to see themselves in this endeavor. And, as discussed in Section D, the work they did during SHERA was a significant contributor to the exceptional showing Indonesia made in the 2020 THE Global Impact Rankings. Table 16. 2020 THE Global Impact Rankings – Top 100 Placements of Indonesian HEIs Overall SDG 8 Decent Work and Economic Growth #47 University of Indonesia #41 University of Gadjah Mada #72 University of Gadjah Mada #77 IPB SDG 1 Impact on Poverty SDG 10 Reduced Inequalities #9 University of Indonesia #49 University of Gadjah Mada #25 University of Gadjah Mada #35 IPB #58 Padjadjaran University #73 University of Brawijawa SDG 2 Zero Hunger SDG 11 Sustainable Cities & Communities #11 IPB No Indonesian HEIs in Top 100 #16 University of Gadjah Mada #32 Padjadjaran University 29 #34 University of Brawijawa SDG 3 Good Health and Well-Being SDG 12 Responsible Production & Consumption #87 University of Indonesia #49 Diponegoro University #93 Airlangga University #87 University of Indonesia SDG 4 Quality Education SDG 13 Climate Action #17 University of Indonesia #54 IPB #75 IPB SDG 5 Gender Equality SDG 14 Life below Water #19 IPB #34 IPB #57 University of Gadjah Mada #59 University of Gadjah Mada #73 University of Indonesia #70 University of Indonesia #98 Padjadjaran University SDG 6 Clean Water and Sanitation SDG 15 Life on Land #19 University of Indonesia #26 University of Gadjah Mada #34 University of Gadjah Mada #33 IPB #72 ITB #48 University of Brawijawa #90 IPB #74 University of Indonesia SDG 7 Affordable and Clean Energy SDG 16 Peace, Justice & Strong Institutions #80 IPB #53 University of Indonesia #95 University of Gadjah Mada #62 University of Gadjah Mada SDG 9 Industry, Innovation & Infrastructure SDG 17 Partnerships for the Goals #50 ITB #11 University of Indonesia #97 University of Gadjah Mada #24 University of Gadjah Mada #37 Padjadjaran University #73 IPB 30 Annex I. The Centers for Collaborative Research and Their Initial Goals ARI (CCR for Acute Respiratory Infections) Public Health and Infectious Disease, Universitas Padjadjaran (UNPAD), Clinical Infectious Disease Research Center. UNPAD proposes to create a CCR on respiratory infectious disease with the goal of enhancing capacity among Indonesian universities to perform high-impact clinical research that addresses the burden of disease, use of immunizations, and prevention of viral and bacterial respiratory illnesses in highly-impacted populations, including young children, pregnant women, and the elderly. Epidemiologic studies will establish the hospital-based burden of disease at five disparate sites; develop the knowledge base to implement vaccine studies and vaccine policy; and establish a training environment for faculty, PhD, and postdoctoral students, at five Indonesian universities. ANBIOCORE (CCR for Animal Biotechnology and Coral Reef Fisheries) Food Security and Self Sufficiency Institute, Pertanian Bogor (IPB), Department of Veterinary Clinic Reproduction and Pathology. IPB proposed to create a CCR focused on advancing animal biotechnology and graduate education in Indonesia through the development of functional genomic markers for bull fertility, determination of genomic markers for economically important traits for selection and production of cattle, and empowerment of the next generation of Indonesian researchers with advanced knowledge and skills in contemporary functional genomic research. CDSR (CCR for Development of a Sustainable Region) Environment, Energy and Maritime Sciences, Universitas Gadjah Mada (UGM), Center for Energy Studies. UGM proposes to create a multidisciplinary partnership with the goal of ensuring energy￾efficiency and independence within Indonesia. UGM proposes to do this by supporting activities that enable Indonesia to maximize its many potential sources of renewable energy, including developing a hybrid energy system and prototypes to support near zero-energy buildings in tropical urban areas, building the capacity of local industry, and creating a network of renewable energy stakeholders within Indonesia. SMART CITY (CCR for Scientific Modeling, Application, Research, and Training for City￾Centered Innovation and Technology) Urban Development and Planning, Universitas Indonesia (UI), Directorate of Research and Community Engagement. UI proposes to develop a CCR to address the challenges facing Indonesia’s urban centers, including the slow adoption of green technology; lacking and unreliable urban infrastructure; slow adoption of IT in civil services, administration and governance; and a rapidly declining quality-of-life. NCSTT (National Center for Sustainable Transportation Technology) Innovative Technologies Institut Teknologi Bandung (ITB). ITB proposes to create a CCR that focuses on developing advanced technologies for the next generation of electrical vehicles in Indonesia through the creation of integrated transportation solutions that address technology, policy, infrastructure and planning development. 31 Annex II. USAID SHERA Training Tracker (cumulative through Q3 FY20) No FY Date Activity Purpose of Activities Location/City Participants Total # of Participants Total Rector/ Vice Rector Faculty/ SHERA F M F M F M 1 FY 17/Q3 May 17-18, 2017 CCR UGM Final Negotiation To finalize the draft agreement between IIE and UGM UGM, Yogyakarta 2 2 2 2 4 2 FY 17/Q3 24- May-17 CCR IPB meeting with SHERA Meeting to discuss accelerating the Program Descriptions and other required documents. SHERA Office, Jakarta 4 4 4 4 8 3 FY 17/Q3 June 5 - 9, 2017 CCR UGM Kick off meeting Preparation on Partnership capacity assessment UGM, Yogyakarta 2 7 7 7 9 16 4 FY 17/Q4 July 4- 8, 2017 Kick-off Meeting for CCR ARI￾UNPAD To provide follow-up on program design, policies, procedures, finance, grants, and MEL UNPAD, Bandung 11 9 11 9 20 5 FY 17/Q4 Aug 15-16, 2017 Direct Mentoring to CCR ARI￾UNPAD To provide direct support to staff for completing their initial MEL and finance reporting UNPAD, Bandung 1 6 3 6 4 10 6 FY 17/Q4 Aug 21-24, 2017 Kick-off Meeting for SMART CITY￾UI To provide follow-up on program design, policies, procedures, finance, grants, and MEL UI, Depok 7 8 7 8 15 7 FY 17/Q4 Sept 14-16, 2017 Kick -off Meeting for NCSTT-ITB To provide follow-up on program design, policies, procedures, finance, grants, and MEL ITB, Bandung 2 7 7 7 9 16 8 FY 17/Q4 Sept 18-20, 2017 Kick-off Meeting for ANBIOCORE￾IPB To provide follow-up on program design, policies, procedures, finance, grants, and MEL IPB, Bogor 1 4 8 4 9 13 9 FY 18/Q1 Oct 18- 20, 2017 Direct Mentoring to CDSR-UGM Provide assistance on voucher preparation, QuickBooks, and financial reports UGM, Yogyakarta 5 1 5 1 6 10 FY 18/Q1 31-Oct￾17 Direct Mentoring to CCR-ARI Finalize the CCR’s MEL Plan and Implementation Plan SHERA Office, Jakarta 2 3 2 3 5 32 No FY Date Activity Purpose of Activities Location/City Participants Total # of Participants Total Rector/ Vice Rector Faculty/ SHERA F M F M F M 11 FY 18/Q1 Nov 8- 10, 2017 Direct Mentoring to ANBIOCORE￾IPB Train CCR on subaward, finalize the MEL Plan, IP, discuss quality benchmarking SHERA Office, Jakarta 4 7 4 7 11 12 FY 18/Q1 Nov 14 & 17,2017 Direct Mentoring to CCR ARI￾UNPAD Provide follow-up training on QuickBooks and financial transactions filing UNPAD, Bandung 2 2 2 2 4 13 FY 18/Q1 Nov 15-16, 2017 Direct Mentoring to NCSTT-ITB Provide training on QuickBooks, financial transactions, reconciliation of pre-financing ITB, Bandung 3 1 3 1 4 14 FY 18/Q1 Nov 22-23, 2017 Direct Mentoring to SMART CITY￾UI Provide follow-up training on QuickBooks and financial transactions filing UI, Depok 2 2 2 2 4 15 FY 18/Q1 27- Nov-17 UI Office of Research Administration, Departments of Treasury and Procurement with the Rectorate Discuss the challenges related to the internal flow of funds and procurement policies UI, Depok 1 1 3 4 4 5 9 16 FY 18/Q1 17-Oct￾17 Kemenristek meeting with USAID, SHERA and CCRs to discuss internal administrative challenges Discuss and document challenges in accessing funds and procuring goods and services Hotel Blue Sky, Jakarta 1 1 2 6 3 7 10 17 FY 18/Q2 15-Jan￾17 CCR Quarterly Meeting Share progress, successes, learning, and challenges in meeting key targets over the current quarter • Discuss and plan activities for the quarter period of January - March 2018 • Review CCRs’ workplan and budget Santika Hotel Bandung 15 20 15 20 35 33 No FY Date Activity Purpose of Activities Location/City Participants Total # of Participants Total Rector/ Vice Rector Faculty/ SHERA F M F M F M 18 FY 18/Q2 18-Jan￾18 Direct Mentoring to NCSTT (MEL) To mentor CCRs on: Routine data collection • Work-based structure and its relation to the program’s performance indicators • Key Performance Indicators (KPI) Data form To discuss with CCRs: • Research Log Sheet template and application to the CCR • Quality benchmarking process and correlation to data reporting • Possible additional indicator related to gender ITB, Bandung 2 4 2 4 6 19 FY 18/Q2 19-Jan￾18 Direct Mentoring to CCR ARI (MEL) UNPAD, Bandung 2 1 2 1 3 20 FY 18/Q2 22-Jan￾18 Direct Mentoring to ANBIOCORE (Grants and Finance) Review SHERA’s Grants Manual and Guidelines and coach on how to utilize these resources when preparing backup of supporting document for reporting purpose. Topics include travel by airplane/land, per-diem rules, procurement expectations, bank transactions, workshop delivery, and requesting PAR. SHERA Office, Jakarta 2 4 2 4 6 21 FY 18/Q2 1-Feb￾18 Monitoring to NCSTT (CCR workshop and field visit) Document event and interview participants to gather success stories from private sector collaboration and generate media coverage. Semarang 1 1 0 1 22 FY 18/Q2 5-Feb￾18 Direct Mentoring to CDSR (MEL) To mentor CCRs on: Routine data collection • Work-based structure and its relation to the program’s performance indicators • Key Performance Indicators (KPI) Data form • To discuss with CCRs: Research Log Sheet template and application to the CCR • Quality UGM, Yogyakarta 3 3 3 3 6 23 FY 18/Q2 1-Mar￾18 Direct Mentoring to ANBIOCORE (MEL) SHERA Office, Jakarta 1 3 1 3 4 34 No FY Date Activity Purpose of Activities Location/City Participants Total # of Participants Total Rector/ Vice Rector Faculty/ SHERA F M F M F M benchmarking process and correlation to data reporting • Possible additional indicator related to gender 24 FY 18/Q2 5-Mar￾18 Direct Monitoring (Research Methodologies training) Increase the SMART CITY scholars’ capacity in research design, methodologies Savero Hotel, Depok 3 2 3 2 5 25 FY 18/Q2 8-Mar￾18 CCR Baseline Study Presentations Disseminate the results and best practices of SHERA Baseline Study. USAID office, Jakarta 4 5 4 5 9 26 FY 18/Q2 14- Mar-18 Meeting and Coaching to ANBIOCORE Review financial transactions in QuickBooks and clarify any unclear transactions. IPB, Bogor 2 8 2 8 10 27 FY 18/Q2 19- Mar-18 Meeting and Coaching to ANBIOCORE Continue the verification that still not done from the time in IPB Bogor SHERA Office, Jakarta 1 4 1 4 5 28 FY 18/Q2 20- Mar-18 Website training How to manage the content management system/ technical terms of website administration. SHERA Office, Jakarta 2 6 2 6 8 29 FY 18/Q2 26- Mar-18 CCR MEL training • Disseminate the KPI Tracking Records Format template • Assist the CCRs’ results measurement and analyses • Increase the MEL staff capacity in performing their roles and responsibilities, and identify success as required by SHERA’s performance indicators Santika Hotel, Depok 5 5 5 5 10 30 FY 18/Q2 27- Mar-18 CCR Quarterly Meeting • Share progress, successes, learning, and challenges in meeting key targets over the current quarter • Discuss and plan activities for the quarter period of April-July 2018 • Review CCRs’ workplan and budget • Conduct targeted trainings and Santika Hotel and UI, Depok 16 23 16 23 39 35 No FY Date Activity Purpose of Activities Location/City Participants Total # of Participants Total Rector/ Vice Rector Faculty/ SHERA F M F M F M ono-on-one meetings on partnership agreements and invoice review 31 FY 18/Q3 5-Apr￾18 Research Contract Discussion – UI/SMART CITY Assist the SMART CITY team with research management UI, Depok 1 3 1 3 4 32 FY 18/Q3 19-Apr￾18 MEL Direct Mentoring – UI/ SMART CITY Assist SMART CITY MEL management and system establishment SHERA Office, Jakarta 3 2 3 2 5 33 FY 18/Q3 Apr 20- 21, 2018 Women in Medical Research and Education – UNPAD/ARI Assist the workshop on increasing the women researchers in medical science in CCR ARI; ensure the objective of activity meets the target of KPI UNPAD, Bandung 2 1 2 1 3 34 FY 18/Q3 24-Apr￾18 SHERA Talk Show: Women in Science Facilitate the sharing and learning of women’s participation science and technology research in Indonesia Jakarta 87 31 87 31 118 35 FY 18/Q3 26-Apr￾18 Meeting with IPB Discuss pending issues under financial report by former Finance Manager IPB, Bogor 2 6 2 6 8 36 FY 18/Q3 29-Apr￾18 Joint Monitoring Joint monitoring with UGM/USAID Mission for installation of solar panel usage Karimun 1 1 0 1 37 FY 18/Q3 3-May￾18 Finance & Grant Mentoring – UNPAD/ARI Facilitate finance & grant discussion regarding the CCR's management and its reports UNPAD, Bandung 1 2 1 2 3 38 FY 18/Q3 6-May￾18 Finance Refreshment Workshop Discuss and clarify barriers to supporting document preparation among five CCRs Yogyakarta 15 12 15 12 27 39 FY 18/Q3 14- May-18 Annual Planning – UNPAD/ARI Facilitate CCR annual planning based on budget year UNPAD, Bandung 5 4 5 4 9 40 FY 18/Q3 16- May-18 MEL Direct Mentoring Facilitate an intensive mentoring to ARI and ANBIOCORE Online 2 3 2 3 5 36 No FY Date Activity Purpose of Activities Location/City Participants Total # of Participants Total Rector/ Vice Rector Faculty/ SHERA F M F M F M 41 FY 18/Q3 22- May-18 MEL Direct Mentoring - NCSTT & CDSR Facilitate an intensive mentoring to the MEL focal points Online 2 4 2 4 6 42 FY 18/Q3 23- May-18 MEL Direct Mentoring – SMART CITY Online 2 2 2 2 4 43 FY 18/Q3 24- May-18 ANBIOCORE Financial Review Finalize review of ANBIOCORE October 2017 – April 2018 expenses IPB, Bogor 1 3 1 3 4 44 FY 18/Q3 4-Jun￾18 UI Audit from USAID Assist with the audit process on UI UI, Depok 1 2 1 2 3 45 FY 18/Q4 July 9 - 12, 2018 SHERA Annual Conference and CCR Quarterly Meeting Share lessons learned, finalize Q4 work plans, and encourage collaboration between CCRs and public private sectors in strengthening their research and activities into sustainable research center. IPB, Bogor 1 3 78 61 79 64 143 46 FY 18/Q4 July 16 - 20, 2018 Field monitoring to CCR ARI affiliate - UNSYIAH & ULM Capture best practices and lesson learned in CCR ARI program implementation, document impact and conduct data quality assurance. Aceh and Banjarmasin 2 9 5 9 7 16 47 FY 18/Q4 July 23 – 26, 2018 CDSR Year 1 Assessment Review CDSR's performance against its targets, analyze the development of its Year 2 implementation plan, and provide direct mentoring in finalizing finance report. UGM, Yogyakarta 6 4 6 4 10 48 FY 18/Q4 August 5 - 10, 2018 Field monitoring to CDSR Affiliates - UNG & UMG Capture best practices and lesson learned in CDSR program implementation, document impact and conduct data quality assurance. Gorontalo 2 10 5 10 7 17 49 FY 18/Q4 August 12 – 15, 2018 NCSTT Year 1 Assessment Review NCSTT’s performance against its targets, analyze the development of its Year 2 ITB, Bandung 5 6 5 6 11 37 No FY Date Activity Purpose of Activities Location/City Participants Total # of Participants Total Rector/ Vice Rector Faculty/ SHERA F M F M F M implementation plan, and provide direct mentoring in finalizing finance report. 50 FY 18/Q4 August 22 – 25, 2018 CCR ARI Year 1 Assessment Review CCR ARI’s performance against its targets, analyze the development of its Year 2 implementation plan, and provide direct mentoring in finalizing finance report. UNPAD, Bandung 4 5 4 5 9 51 FY 18/Q4 27- Aug-18 ANBIOCORE Year 1 Assessment Review ANBIOCORE’s performance against its targets, analyze the development of its Year 2 implementation plan, and provide direct mentoring in finalizing finance report. IPB, Bogor 4 7 4 7 11 52 FY 18/Q4 3-Sep￾18 SMART CITY Year 1 Assessment Review SMART CITY’s performance against its targets, analyze the development of its Year 2 implementation plan, and provide direct mentoring in finalizing finance report. Bogor 5 5 5 5 10 53 FY 18/Q4 19-Sep￾18 MEL mentoring & refreshment virtual meeting Preparing the annual reports and confirming data for data submission. Online 4 5 4 5 9 54 FY 19/Q1 29-Oct￾18 Finance & Grant Training Led by IIE HO's controller Increase the knowledge of internal financial controls & fraud prevention SHERA Office, Jakarta 6 7 6 7 13 55 FY 19/Q1 30-Oct￾18 IIE HO controller visit to SMART CITY Discuss lessons learned on Payment Advance Requests and financial reports UI, Depok 5 6 5 6 11 38 No FY Date Activity Purpose of Activities Location/City Participants Total # of Participants Total Rector/ Vice Rector Faculty/ SHERA F M F M F M 56 FY 19/Q1 5-Nov￾18 Finance & Grant Refreshment Training To increase the F&G Manager’ capacity in managing their role and responsibility as reviewer, budget manager and fund manager • To increase the Finance and Admin Assistants’ capacity on activity budget and additional budget preparation. • To produce more efficient process and better report quality • To achieve the same view and wavelength on any standard, procedure, guidelines and policy • To work on real case of Advance (PAR) and expense report, based on respective CCR needs Courtyard Hotel by Marriot, Bandung 11 9 11 9 20 57 FY 19/Q1 Nov 12-17, 2018 MEL Training To improve CCR MEL Specialists’ knowledge on program’s monitoring and evaluation concept (especially on theory of change and results￾based management) for better understanding of program development and identifying intended outcomes • To improve CCR MEL Specialist’ knowledge and skills on identifying impacts that would reflect closely to their respective programming • To have a hands-on experience on one of CCR’s research location, understanding how the learning occurred and recorded into a required report • To increase CCR MEL Specialist’ capacity in taking the learning for better management Pangkal Pinang & Semujur Island, Bangka Island 13 21 13 21 34 39 No FY Date Activity Purpose of Activities Location/City Participants Total # of Participants Total Rector/ Vice Rector Faculty/ SHERA F M F M F M decision •To strengthen CCR MEL Specialists’ knowledge and skill in producing the communication materials based on the learning evidence – how to select and document the communication materials properly (incl. photographs and report writing). 58 FY 19/Q1 Nov 27-28, 2018 Direct Mentoring MEL CCR-ARI Mentor the CCR in extracting program data and its analyses into the KPI database SHERA Office, Jakarta 2 2 0 2 59 FY 19/Q1 28- Nov-18 Direct Mentoring Finance Grant - CCR ANBIOCORE Mentor CCR ANBIOCORE's reconciliation in QuickBooks SHERA Office, Jakarta 3 3 3 3 6 60 FY 19/Q1 6-Dec￾18 CCR Program management meeting - ANBIOCORE Year 2 Program Management￾Implementation Plans IPB, Bogor 2 5 2 5 7 61 FY 19/Q1 10-Dec￾18 CCR Program management meeting - NCSTT ITB, Bandung 4 4 4 4 8 62 FY 19/Q1 11-Dec￾18 CCR Program management meeting - CCR ARI UNPAD, Bandung 2 4 2 4 6 63 FY 19/Q1 12-Dec￾18 CCR Program management meeting - CDSR UGM, Yogyakarta 3 3 3 3 6 64 FY 19/Q1 13-Dec￾18 CCR Program management meeting - SMART CITY UI, Depok 4 5 4 5 9 40 No FY Date Activity Purpose of Activities Location/City Participants Total # of Participants Total Rector/ Vice Rector Faculty/ SHERA F M F M F M 65 FY 19/Q2 10-Jan￾19 SHERA - MEL Virtual Mentoring Using and sharing Google calendars, Reconfirm the bibliography style and applied in SHERA system, Reconfirm the CCRs' research cluster and topics, Refresh the updated PIRS on 1.2 & 3.1.1, Evaluate the CCRs' routine data and report Online 3 7 3 7 10 66 FY 19/Q2 16-Jan￾19 SHERA - MEL Virtual Mentoring. CCR -Smart City & CCR Smart City Management Direct Mentoring Reconfirm the CCR's research cluster and topics, evaluate the CCR's routine data and report, reconfirm the CCR SMART CITY IP FY2 Online 3 8 3 8 11 67 FY 19/Q2 17-Jan￾19 SHERA - MEL Virtual Mentoring CCR -NCSTT Reconfirm the CCR's research cluster and topics, - Evaluate the CCR's routine data and report Online 1 4 1 4 5 68 FY 19/Q2 18-Jan￾19 SHERA - MEL Virtual Mentoring CCR - ANBIOCORE Online 1 3 1 3 4 69 FY 19/Q2 23-Jan￾19 SHERA - MEL Virtual Mentoring CCR -ARI Online 2 3 2 3 5 70 FY 19/Q2 24-Jan￾19 SHERA - MEL Virtual Mentoring CCR -CDSR Online 2 3 2 3 5 71 FY 19/Q2 7-Feb￾19 Meeting with URI Dean and Representative at IPB To discuss IPB-URI payments, discuss sub-award SHERA Office, Jakarta 3 0 3 3 41 No FY Date Activity Purpose of Activities Location/City Participants Total # of Participants Total Rector/ Vice Rector Faculty/ SHERA F M F M F M 72 FY 19/Q2 Feb 11 - 12, 2019 SHERA Strategic Meeting To update the CCRs on new USAID approach on Journey 2 Self￾Reliance, obtain the agreement on the shift of USAID different approach toward the program impact and sustainability plan collaboration in the U.S. , update and discuss challenges and agree on the solution for different CCRs, related to IPs and the approved budget Hotel Oria, Jakarta N/A N/A N/A 73 FY 19/Q2 Feb 13 - 15, 2019 MEL Monitoring to CCR CDSR - UGM Validate the CDSR's achievement in Goal 3, capture the current impact that occurred at CDSR's collaborative research, reconfirm the research report & publication report UGM, Yogyakarta 2 4 2 4 6 74 FY 19/Q2 Feb 13- 14, 2019 Mentoring CCR ANBIOCORE Field Monitoring with CCR ANBIOCORE regarding the financial report IPB, Bogor 2 3 2 3 5 75 FY 19/Q2 Feb 17- 18, 2019 Mentoring CCR ARI Field Monitoring with CCR ARI regarding the financial report UNPAD, Bandung 2 2 2 2 4 76 FY 19/Q2 Feb 25 - 28, 2019 MEL Monitoring to CCR ARI & ANBIOCORE - UNSYIAH Ensure the research works at ANBIOCORE in Cattle Cluster at Aceh with its highlighted evidence and impact, Ensure the research works at CCR ARI in BoD and Mortality study at Aceh with its highlighted evidence and impact, Meet the IIE management and find solution on various management challenges in SHERA Unsyiah, Kota Banda Aceh 2 5 8 5 10 15 77 FY 19/Q2 Feb 19- 20, 2019 Mentoring CCR NCSTT Field Monitoring with CCR NCSTT parallel with CCR SMART CITY regarding the financial report ITB, Bandung UI, Depok 2 3 2 3 5 42 No FY Date Activity Purpose of Activities Location/City Participants Total # of Participants Total Rector/ Vice Rector Faculty/ SHERA F M F M F M Mentoring CCR SMART CITY 78 FY 19/Q2 Feb 21- 22, 2019 Mentoring CCR CDSR Field Monitoring with CCR CDSR regarding the financial report UGM, Yogyakarta 5 1 5 1 6 79 FY 19/Q2 Feb 28 - Mar 2, 2019 Grant Monitoring to CCR ARI - UNSYIAH Create a strategy to leverage achievement and engagement on relation Affiliate Cost Reimbursement on CCR ARI; Follow up the reporting cycle in Affiliate UNSYIAH; Follow up supporting documents needed in the reporting of Affiliate; Have an agreement on the submission on monthly reports in timely manner; Feedback and discuss for the recommendation the Affiliate Agreement with Hospital sub￾contract. Unsyiah, Kota Banda Aceh 3 3 3 3 6 80 FY 19/Q2 March 10 - 14, 2019 MEL Monitoring to CCR ARI - UNPATTI Ensure the research works at CCR ARI in BoD and Mortality study at Ambon with its highlighted evidence and impact, collect the data of impact from the respective CCR’s scholars during their involvement in SHERA program, UNPATTI, Kota Ambon 1 6 7 7 7 14 81 FY 19/Q2 March 13 - 15, 2019 Grant Monitoring to CCR ARI - UNPATTI Create a strategy to leverage achievement and engagement on relation Affiliate Cost Reimbursement on CCR ARI; Follow up the reporting cycle in Affiliate UNPATTI; Follow up supporting documents needed in the reporting of Affiliate; Have an UNPATTI, Kota Ambon 2 2 2 3 4 5 9 43 No FY Date Activity Purpose of Activities Location/City Participants Total # of Participants Total Rector/ Vice Rector Faculty/ SHERA F M F M F M agreement on the submission on monthly reports in timely manner; Feedback and discuss for the recommendation the Affiliate Agreement with Hospital sub￾contract. 82 FY 19/Q2 26- Mar-19 SHERA MEL Virtual Mentoring Set the MSC mechanism in SHERA, refresh J1-Visa mechanism, mentor the CCRs' data utilization and its analyses, prepare the upcoming 3rd MEL training Online 5 5 5 5 10 83 FY 19/Q2 March 22 – 23, 2019 Grant Monitoring to CCR ARI - MATARAM Create a strategy to leverage achievement and engagement on relation Affiliate Cost Reimbursement on CCR ARI; Follow up the reporting cycle in Affiliate UNPATTI; Follow up supporting documents needed in the reporting of Affiliate; Have an agreement on the submission on monthly reports in timely manner; Feedback and discuss for the recommendation the Affiliate Agreement with Hospital sub￾contract. Mataram, NTB 1 4 2 5 2 7 84 FY 19/Q2 24-Jan￾19 Performance Management report -IPB Discuss low performance of IPB, options need to be taken by ANBIOCORE SHERA Office, Jakarta 3 6 3 6 9 85 FY 19/Q2 13- Mar-19 MESP introduction Introduction of MESP to SHERA, plan to be conducted at SHERA, CCR Lead and affiliates, SHERA Office, Jakarta 2 2 2 2 4 86 FY 19/Q2 14- Mar-19 External audit￾CCR SMART CITY Provide support to USIP on the SHERA program, provide support to CCR SMART CITY UI, Depok 6 7 6 7 13 44 No FY Date Activity Purpose of Activities Location/City Participants Total # of Participants Total Rector/ Vice Rector Faculty/ SHERA F M F M F M 87 FY 19/Q2 28- Mar-19 Support UI in having the external audit Provide support to CCR SMART CITY in getting audited by external auditor UI, Depok 1 1 1 1 2 88 FY 19/Q2 29- Mar-19 Meeting ANBIOCORE Management and Board of Advisor Notify IPB Vice Rector on ANBIOCORE low performance, steps to be taken by IPB IPB, Bogor 6 1 2 7 2 9 89 FY 19/Q3 Apr 1- 2, 2019 Grant Monitoring to Universitas Lambung Mangkurat To monitor the CCR ARI's affiliates (ULM) finance report Banjarmasin 4 3 4 3 7 90 FY 19/Q3 Apr 1- 3, 2019 MEL Field monitoring to ANBIOCORE Affiliates, Universitas Brawijaya To ensure the research works at CCR ANBIOCORE with its highlighted evidence and impact To collect the data of impact from the respective CCR’s scholars during their involvement in SHERA program Malang, East Java 4 4 4 4 8 91 FY 19/Q3 Apr 3 - 7, 2019 MEL Field monitoring to SMART CITY Affiliates, Universitas Udayana To ensure the research works at CCR ANBIOCORE and SMART CITY with its highlighted evidence and impact, to collect the data of impact from the respective CCR’s scholars during their involvement in SHERA program, Kampus Udayana I & II, Bali 1 6 11 6 12 18 92 FY 19/Q3 Apr 8- 11, 2019 Finance & Grant Refreshment for CCRs To increase the F&G focal point's capacity in managing their role and responsibility as reviewer, budget manager and fund manager, To increase the F&G in preparing the procurement plan, admin, back up support in completing the documents of procurement process, especially in restricted commodities Courtyard Hotel, Bandung 7 8 7 8 15 45 No FY Date Activity Purpose of Activities Location/City Participants Total # of Participants Total Rector/ Vice Rector Faculty/ SHERA F M F M F M and equipment, To work in real case of procurement proposed to SHEAR, expense report, based on respective CCR's needs, To produce more efficient process and better report quality, To create pipeline related with PAR and WBS, To introduce the fraud and action needs 93 FY 19/Q3 18-Apr￾19 Monitoring to CCR SMART CITY's activity on research grant evaluation To monitor the SMART CITY's Research Grant process and results Hotel Aston Tb. Simatupang, Jakarta 1 2 1 2 3 94 FY 19/Q3 24-Apr￾19 Meeting with IIE Home Office Grants and Contracts To share the grants & finance current practice in SHERA project SHERA Office, Jakarta 1 4 1 4 5 95 FY 19/Q3 26-Apr￾19 Meeting Coordination IIE with CCR SMART CITY -UI To manage the UI expectation on situation room UI, Depok 2 5 2 5 7 96 FY 19/Q3 30-Apr￾19 Talk show on Transportation Technology To promote SHERA's research and researchers through their research work and partnerships @America, Jakarta 82 88 82 88 170 97 FY 19/Q3 3-May￾19 SHERA & CCR ANBIOCORE - IPB Management Meeting To meet the Vice Rector to discuss the lack of management at IPB IPB, Bogor 1 1 6 1 6 7 98 FY 19/Q3 May 14-17, 2019 MEL Training To build CCR MEL Specialists’ knowledge and skills on MIS web application and able to perform data entry for CCRs’ events, mentorship logbook and research logbook; To ensure the completeness of database stored in the MIS web application – Aryaduta Hotel, Bandung 9 7 9 7 16 46 No FY Date Activity Purpose of Activities Location/City Participants Total # of Participants Total Rector/ Vice Rector Faculty/ SHERA F M F M F M where all participants will have time to review and correct the database; To improve CCR MEL Specialist’ knowledge and skills on identifying impacts that would reflect closely to their respective programming. 99 FY 19/Q3 22- May-19 Meeting Coordination IIE with CCR CDSR - UGM To map UGM achievement and possibility of partnership UGM, Yogyakarta 2 3 2 3 5 100 FY 19/Q3 19-Jun￾19 MEL Virtual Mentoring To get any feedback on MIS practice as May Reporting time, to present MEL SHERA survey, to update the existing KPI tracking records Online 4 5 4 5 9 101 FY 19/Q4 15-Apr￾19 Meeting IPB Update on current condition on activities and funding IPB, Bogor 2 2 2 2 4 102 FY 19/Q4 16-Apr￾19 Meeting UGM UGM, Yogyakarta 2 2 2 2 4 103 FY 19/Q4 23-Jul￾19 Meeting with ITB SHERA Office, Jakarta 3 2 3 2 5 104 FY 19/Q4 24-Jul￾19 Meeting with UNPAD UNPAD, Bandung 2 3 2 3 5 105 FY 19/Q4 31-Jul￾19 SHERA's Strategic Meeting & its sustainability To confirm the findings of SHERA's sustainability study with CCRs, USAID, and GOI/ To discuss the lack of funding with all key stakeholders Semarang 3 10 12 10 15 25 106 FY 19/Q4 Aug 9 - 13, 2019 Assistance to UNPAD for Finalization July 2019 Report Completion of submitting and preparing July 2019 reports UNPAD, Bandung 3 2 3 2 5 47 No FY Date Activity Purpose of Activities Location/City Participants Total # of Participants Total Rector/ Vice Rector Faculty/ SHERA F M F M F M 107 FY 19/Q4 16- Aug-19 SMART CITY with Affiliates Cancelation of the affiliates research, working with MoRTHE Dr. Imah on possibility of continuing the research using MoRTHE fund UI, Depok 3 4 3 4 7 108 FY 20/Q1 11-Oct￾19 Meeting UI Smart City To revise the IP, and customize on sustainability plan SHERA Office, Jakarta 2 5 2 5 7 109 FY 20/Q1 Oct 22- 25 2019 CCR Workshop Subrecipient closeout procedures and CCR sustainability workshop Bandung 1 15 16 16 16 32 110 FY 20/Q1 14- Nov-19 Meeting Pak Bambang IPB To expedite the KPI achievement TIS Square 3 0 3 3 111 FY 20/Q1 14- Nov-19 Meeting Dr. Yayan UNPAD To update the administrative burden and the solution Jalan Sabang 1 1 1 1 2 112 FY 20/Q1 15-Dec￾19 Meeting ARI To work closely with Finance team preparing the closing out UNPAD, Bandung 3 4 3 4 7 113 FY 20/Q1 16-Dec￾19 Meeting ARI To update data collection UNPAD, Bandung 2 1 2 1 3 114 FY 20/Q2 Jan 20- 22, 2020 Grants team visited CCR ANBIOCORE Direct mentoring, physical check of documents prior to end of IPB subagreement IPB, Bogor 3 1 3 1 4 115 FY 20/Q2 Feb 5- 7, 2020 Grants team visited CCR NCSTT-ITB Direct mentoring, physical check of documents at CCR NCSTT-ITB NCSTT, Bandung 3 1 3 1 4 116 FY 20/Q2 13-Feb￾20 SHERA Communications Visit To assist UBB's training and document successes Bangka Belitung 5 2 5 2 7 117 FY 20/Q2 Feb 18- 20, 2020 Grants team visited CDSR-UGM Direct mentoring, physical check of documents of CCR CDSR UGM, Yogyakarta 2 2 2 2 4 118 FY 20/Q2 Feb 25- 27, 2020 Grants team visited SMART CITY UI Direct mentoring, physical check of documents of SMART CITY UI, Depok 1 2 1 2 3 119 FY 20/Q3 4-Apr￾20 Grant Monitoring Coordination Meeting with CCR NCSTT Finance team to discuss advance reconciliation Online 3 1 3 1 4 48 No FY Date Activity Purpose of Activities Location/City Participants Total # of Participants Total Rector/ Vice Rector Faculty/ SHERA F M F M F M 120 FY 20/Q3 9-Apr￾20 Coordination meeting with NCSTT To prepare for program closeout, discuss final deliverables and impact of COVID-19 Online 3 8 3 8 11 121 FY 20/Q3 Coordination meeting with SMART CITY 4 7 4 7 11 122 FY 20/Q3 Coordination meeting with CDSR 5 7 5 7 12 123 FY 20/Q3 10-Apr￾20 Grant Monitoring Discussion with SMART CITY on remaining advance Online 3 2 3 2 5 124 FY 20/Q3 16-Apr￾20 Grant Monitoring - UI Audit Assistance and Mentoring Coordination meeting with external auditor to discuss audit logistics Online 1 1 1 1 2 125 FY 20/Q3 20-Apr￾20 Grant Monitoring - UI Audit Assistance and Mentoring Coordination meeting with SMART CITY and external auditor Online 1 1 1 1 2 126 FY 20/Q3 21-Apr￾20 Grant Monitoring - UI Audit Assistance and Mentoring Coordination meeting with SMART CITY and external auditor Online 1 1 1 1 2 127 FY 20/Q3 22-Apr￾20 Grant Monitoring - UI Audit Assistance and Mentoring Coordination meeting with SMART CITY and external auditor Online 2 1 2 1 3 128 FY 20/Q3 28-Apr￾20 Grant Monitoring - UI Audit Assistance and Mentoring Coordination meeting with SMART CITY and external auditor Online 1 1 1 1 2 129 FY 20/Q3 6-May￾20 Final Evaluation meeting with SMART CITY Follow up meeting with SMART CITY on Final Evaluation questionnaire Online 2 2 2 2 4 49 No FY Date Activity Purpose of Activities Location/City Participants Total # of Participants Total Rector/ Vice Rector Faculty/ SHERA F M F M F M 130 FY 20/Q3 8-May￾20 MEL Coordination with CCRs To provide SHERA updates and discuss new MIS MEL features with CCRs Online 5 6 5 6 11 131 FY 20/Q3 11- May-20 Grant Monitoring - UI Audit Assistance and Mentoring Coordination meeting with SMART CITY and external auditor Online 2 3 2 3 5 132 FY 20/Q3 11- May-20 Coordination SHERA with NCSTT team Preparation for June 30 subaward closure. Status of award modifications, asset disposition, final deliverables to IIE, final payments Online 4 4 4 4 8 133 FY 20/Q3 Coordination SHERA with UI team 5 4 5 4 9 134 FY 20/Q3 Coordination SHERA with CDSR team 6 4 6 4 10 135 FY 20/Q3 12- May-20 IIE - NCSTT meeting Discussion with NCSTT on advance, award modification, remaining funds, close out Online 2 3 2 3 5 136 FY 20/Q3 12- May-20 Grant Monitoring - UI Audit Assistance and Mentoring Coordination meeting with SMART CITY and external auditor Online 1 1 1 1 2 137 FY 20/Q3 13- May-20 Grant Monitoring - UI Audit Assistance and Mentoring Coordination meeting with SMART CITY and external auditor Online 2 4 2 4 6 138 FY 20/Q3 20- May-20 Grant Monitoring - UI Audit Assistance and Mentoring Coordination meeting with SMART CITY and external auditor Online 2 3 2 3 5 139 FY 20/Q3 28- May-20 Final Evaluation meeting with ANBIOCORE To discuss and revise findings of CCR desktop review as part of the SHERA Final Evaluation Online 1 2 1 2 3 50 No FY Date Activity Purpose of Activities Location/City Participants Total # of Participants Total Rector/ Vice Rector Faculty/ SHERA F M F M F M 140 FY 20/Q3 29- May-20 Final Evaluation meeting with CDSR To discuss and revise findings of CCR desktop review as part of the SHERA Final Evaluation Online 3 3 3 3 6 141 FY 20/Q3 3-Jun￾20 Final Evaluation meeting with SMART CITY To discuss and revise findings of CCR desktop review as part of the SHERA Final Evaluation Online 2 2 2 2 4 142 FY 20/Q3 3-Jun￾20 Final Evaluation meeting with NCSTT To discuss and revise findings of CCR desktop review as part of the SHERA Final Evaluation Online 2 3 2 3 5 143 FY 20/Q3 16-Jun￾20 Grant Monitoring - UI Audit Assistance and Mentoring Coordination meeting with external auditor, USAID, and UI. Online 7 7 7 7 14 144 FY 20/Q3 17-Jun￾20 Grant Monitoring - UI Audit Assistance and Mentoring Coordination meeting with external auditor Online 1 3 1 3 4 145 FY 20/Q3 24-Jun￾20 Grant Monitoring Coordination with SMART CITY Finance team to discuss financial projections, advance, final procurements, QuickBooks entries, and asset management disposition plan. Online 2 1 2 1 3 146 FY 20/Q3 30-Jun￾20 Grant Monitoring - UI Audit Assistance and Mentoring Coordination Meeting with external auditor regarding the 1st draft audit report Online 1 3 1 3 4 Total 14 23 757 796 771 819 1,590 51 Annex III. KPI Definitions Goal Indicator 1: Number of peer-reviewed articles resulting from USG support to research and implementation program Precise Definition(s): Peer-reviewed writings: defined as either 1) writings by SHERA scholars which are reviewed by several other experts in a Science and Technology (S&T) field before publication (either in national or international)in a specialized journal called a “peer-reviewed journal,” or 2) proceedings, selected papers presented by SHERA scholars at S&T conferences and reviewed by the respective external peer-reviewers for quality purposes (the external peer-reviewers may be the publisher itself). In this definition, the proceeding article has a value of 3 to everyone journal article (3:1). This process aims at ensuring the quality of the product (article, research) that are published. Articles Index Scopus: defined as published journals and/or proceedings that are labelled as “Science and Technology”. Goal Indicator 2: Ratio of citation to publications produced by Indonesian researchers Precise Definition(s): Citation: a case where a scholar across the world references a writing authored by SHERA funded scholars. The citation must be in the related research topic and the self-citation is not counted into this indicator measurement. Numerator: Number of times of SHERA funded scholars’ publication cited Denominator: Total number of publications produced by SHERA funded scholars throughout project lifetime This is a cumulative indicator and the data will be counted annually to be able to reflect the achievements against end of project’s target. Goal Indicator 3: Ratio of academic research initiatives whose findings are utilized to address development challenges Precise Definition(s): Academic research initiative is defined as research activities that carried out by the Center for Collaborative Research (CCRs). To be counted, the research must be funded by the U.S.G. assistance. Ongoing research may be counted in this indicator if there is any partial findings or methods that is being utilized to address development challenges. Addressing development challenges can be in several forms, i.e., whether there is any methodology, approaches or processes carried out by CCR’s scholars during his/her research activities being replicated by other scholars. Other form includes using the academic research initiatives to inform decision-making (such as being used by government, institutions, private sectors, bilateral donors, communities, etc.). This indicator is a stand-alone indicator and its counting does not necessarily refer to SHERA Goal Indicator #1 as the research findings do not need to be published before its utilization in addressing the development challenges. Indicator 1.1.: Percentage of scholars who present at conferences Precise Definition(s): This indicator is defined by the number of SHERA scholars who participate in such events. Participations can be in the form of presenting paper(s) and/or as panel members. Indicator 1.2.: Percentage of scholars who participate in collaborative research Precise Definition(s): This indicator is defined by the number of SHERA scholars who participate in collaborative research. Participations can be in the form of leading the team or having a specific role in the research activities, such as methodology designing, data analysis, reporting, etc. Output Indicator 1.1.1: Number of short-term training courses held Precise Definition(s): activities organized over a 1 to 48 hours’ period to share knowledge with SHERA scholars. Output Indicator 1.1.2: % of researchers participated in short-term training courses Precise Definition(s): Number of SHERA’s scholars who participated to training sessions. Output Indicator 1.1.3: Number of U.S. scholars visiting Indonesian institutions to train Indonesian partners Precise Definition(s): U.S. Scholars: scholars from the U.S.-based higher education institutions whose institutions are part of the CCR collaborative research consortium; to train Indonesian partners: the U.S. scholars provided capacity building on specific courses or mentoring for SHERA scholars. 52 Output Indicator 1.2.1: Number of Indonesian scholars who participate in in-person faculty exchanges held in the U.S. Precise Definition(s): Indonesian scholars: scholars eligible to participate in the exchange activities must be from SHERA’s CCR institutions members. Output Indicator 1.2.2: Number of Indonesian scholars who receive ongoing mentoring Precise Definition(s): Indonesian scholars: scholars eligible to participate in the mentoring activities must be from SHERA’s CCR institutions members. Mentoring: Mentoring activities are provided either by U.S. institution partners or by Lead CCR for their respective affiliate institutions. Output Indicator 1.2.3: Number of scholars who participate in virtual knowledge exchange Precise Definition(s): Scholars: those eligible to participate in the mentoring activities must be from SHERA’s CCR institutions members; virtual knowledge exchange: the exchange could be in the form of –but not limited to— webinars, Skype video/calls, or other online forms. To be able to be counted, such exchange must be arranged by the CCRs. Outcome Indicator 2.1: Percentage of Indonesian institutions with improved research policies and systems in place Precise Definition(s): Indonesian institutions: Lead Indonesian university (CCR) and all affiliate universities. Improved research policies and systems in place: Each lead Indonesian university and its affiliates updated or developed and implement policies and systems for how the research will be managed Outcome Indicator 2.2: % of CCR members with improved administrative and management systems in place Precise Definition(s): Indonesian Institutions: Lead Indonesian university (CCR) and all affiliate universities. Improved administrative and management system in place: Each lead Indonesian university and its Indonesian affiliates develop and implement the system relating to operate and manage the activities of the CCR Output Indicator 2.1.1: Number of institutions with improved research and management policies developed Precise Definition(s): Indonesian institutions: Lead Indonesian university CCR and all affiliate universities. Improved research policies and system developed: Each of lead Indonesian university and its affiliates developed new and better research and management policies Output Indicator 2.2.1: Number of CCR websites that are operational and being used regularly Precise Definition(s): CCR website: A set of related web pages located under a single domain name based on each CCR (either Lead or its Indonesian Affiliates) or as a part of the respective institution. Operational and being used regularly: The website must be regularly updated at least once a month. Output Indicator 2.2.2: Number of CCR knowledge products created Precise Definition(s): Knowledge products: A set of information as results from the research such as research reports, publications of articles, essays, etc., and/or materials produced in relation to the development or capacity building to conduct research, such as training modules, guidelines, and other information, education and communication materials. Output Indicator 2.2.3: Number of program-wide SHERA knowledge sharing events on best-practices & lessons-learned held Precise Definition(s): Knowledge sharing events: These could be in the form of seminars, workshop, partner meetings, webinars, etc. Best practices and lessons learned: These must come from SHERA CCR results. Indicator 2.2.4. Number of people participated in activities addressing gender equality or female empowerment in science and technology research Precise Definition(s): This indicator is defined by the number of SHERA CCR members and partners who participate in activities that addressing gender equality or female empowerment in science and technology research, including its administrative and management system policies and procedures. Activities can be in the form of trainings, meetings, focused group discussions or workshops. For training activities, to be counted, the activity needs to be held in a minimum of three hours. In the event of activities where gender is integrated within a broader sector, 53 the activity must include a substantial focus on gender issues (e.g., gender issues are addressed throughout the activity, there is a gender module that explores the relevant gender issues in depth, etc.). Outcome Indicator 3.1: Percentage of targeted institutions engaged in joint research projects Precise Definition(s): Targeted institutions: All Indonesian and U.S.-based SHERA CCRs. Joint research projects: Research project that involving at least two scholars from two different institutions. Institutions refers to tertiary education institutions and research institution, which is an organization that provides educational and/ or research opportunities that build on secondary education, providing learning activities in specialized fields. It aims at learning at a high level of complexity and specialization. Tertiary education includes what is commonly understood as academic education but also includes advanced vocational or professional education. This may include public or private universities, colleges, research institutes, training institutes, etc. These other institutions may include academic or non￾academic institutions, not for profit organizations (i.e., balai benih inseminasi buatan, research institutions, hospital’s research and development department, etc.) as long as they could show proof of research collaborative involvement – not just providing external resources to the research activities. Counting of this indicator is based on institution/organization, not individual or department. In the event that multiple departments or several researchers within one institution engaged in several joint research projects with other higher education institutions, they will be counted as one. However, the report will narrate the stories to which extend the respective departments and/or researchers’ involvement in the joint projects. Outcome Indicator 3.2: % of partnerships created to support joint research projects Precise Definition(s): Partnerships: Any agreement that would include funding, in-kind contribution, and technical assistance from other sources than SHERA project’s resources to conduct SHERA related joint research projects; the support may come from the Ministry of Higher Education and research (RISTEK/DIKTI), private sectors, foundations, other universities, other research institutions, or any other development partner/grant maker/donor other than USAID. Joint research projects: Research projects involving scholars from at least one CCR member. Numerator: Number of partnerships created by Indonesian CCR institutions Denominator: Total number of Indonesian CCR institutions Output Indicator 3.1.1: Number of selected affiliate institutions included in research consortium Precise Definition(s): Affiliate institutions: CCR’s Affiliate institutions; Research consortium: group of research institutions –which is in SHERA context—are those included in Collaborative Consortium Research (CCR) and involved in the research process and activities. This indicator also counts other institutions that involved in the research consortium at later stage of SHERA implementation. These other institutions may include academic or non￾academic institutions, not for profit organizations (i.e., balai benih inseminasi buatan, research institutions, hospital’s research and development department, etc.) as long as they could show proof of research collaborative involvement –not just providing external resources to the research activities. Output Indicator 3.1.2: Number of scholars at CCR institutions with increased access to external research resources and academic research engines Precise Definition(s): External research resources: resources not provided by SHERA program; Academic research engines: specialized search engines used by students and researchers; they locate relevant academic search results from web pages, books, encyclopedias, and journals. 54 Annex IV. USAID SHERA KPI Results (cumulative through Q3 FY20) Performance Indicators Achievement FY17 FY18 FY19 FY20 Total 1. Number of peer-reviewed scientific publications resulting from USG support to research and implementation program 0 51 127 95 273 Journal Articles 0 37 66 39 142 Proceeding Articles 0 43 182 167 392 Status of Scopus Indexed Scopus 0 71 230 206 507 Non-Indexed Scopus 0 9 18 27 2. Ratio of citation to publication produced by Indonesia researchers 0% 43% 21% 13% 21% Number of Cited Publication 0 34 52 24 110 Total Number of Publication 0 80 248 189 517 Journal Articles 0 21 25 13 59 Proceeding Articles 0 13 27 11 51 Number of Authors: 0 Male 0 69 99 24 192 Female 0 22 50 10 82 3. Ratio of academic research initiatives whose findings are utilized to address development challenges. 0% 0% 3% 0% 3% Number of Research whose Findings 0 0 2 0 2 Number of Topic of Research 0 77 78 79 79 Sectors: Governments 0 2 2 Private Sectors 0 0 0 NGO 0 0 0 Bilateral organization 0 0 0 Types: 0 Replicated 0 0 0 Taken to the Market 0 0 0 Applied 0 2 2 55 Performance Indicators Achievement FY17 FY18 FY19 FY20 Total Improved capacity of faculty, PhD students and postdoctoral researchers in target Indonesian universities 1.1 % of scholars who present at conference 0% 46% 78% 13% 46% Numerator 0 76 149 25 250 Denominator 0 166 191 191 548 Male 52 81 20 153 Female 24 68 5 97 Age under 25 2 3 2 7 Age 25 - 45 50 85 10 145 Age 46 - 65 22 50 11 83 Age over or equal 66 0 0 0 1.1.1. Number of short-term training courses held 3 29 13 45 Training 1 20 7 28 Workshop 2 9 6 17 Duration (Hrs.) 50 892 266 1208 1.1.2. % of researchers participating in short-term training courses 100% 63% 50% 60% numerator 80 817 381 1278 denominator 80 1294 769 2143 Male 27 407 193 627 Female 53 410 188 651 Academic Level: 0 Master 33 405 208 646 Ph.D. Students 18 297 123 438 Sp1 16 71 32 119 Sp2 8 8 8 24 Postdoctoral Students 5 36 10 51 1.1.3. Number of U.S. scholars visiting Indonesian institutions to lead short-term training courses for Indonesian partner institutions 4 16 22 1 43 Male 2 16 16 34 Female 2 6 1 9 Age 0 25 - 45 8 4 1 13 56 Performance Indicators Achievement FY17 FY18 FY19 FY20 Total 46 - 65 2 6 9 17 up to 66 1 2 3 Academic Level: 0 Master 1 2 2 5 Ph.D. Students 2 7 15 24 Postdoctoral Students 1 7 5 1 14 Number of visits 4 16 22 1 43 1.2 % of scholars who participate in collaborative research 40% 76% 113% 4% 64% Numerator 21 267 318 12 618 Denominator 52 350 281 281 964 Male 12 159 188 9 368 Female 9 108 130 3 250 Age 0 <25 5 9 6 20 25 - 45 13 154 178 11 356 46 - 65 3 79 110 1 193 up to 66 3 3 1.2.1. Number of Indonesian scholars who participate in in-person faculty exchanges held in the U.S. 0 10 18 0 28 Male 9 9 18 Female 1 9 10 Age 25 - 45 3 7 10 46 - 65 3 10 13 up to 66 1 1 Academic Level: Master 5 5 Ph.D. Students 6 12 18 Postdoctoral Students 4 1 5 1.2.2. Number of Indonesian scholars who receive ongoing mentoring 4 28 49 81 Male 2 12 16 30 Female 2 16 33 51 57 Performance Indicators Achievement FY17 FY18 FY19 FY20 Total Age under 25 1 2 1 4 25 - 45 3 19 36 58 46 - 65 6 11 17 up to 66 1 1 2 Academic Level: 0 Master 3 23 36 62 Ph.D. Students 1 3 11 15 Sp1 1 1 2 Sp2 0 Postdoctoral Students 1 1 2 Strengthened, inclusive institutional environment for research and management in target Indonesian universities 2.1. % of Indonesian institutions with improved research policies and systems in place. 67% 13% 17% 28% Numerator 10 3 4 17 Denominator 15 23 23 61 Lead 2 1 3 Affiliates 8 2 4 14 2.1.1. Number of institutions that develop improved research and management policies developed 9 2 4 15 Lead 1 1 2 Affiliates 8 1 4 13 2.2. % of CCR members with improved administrative and management systems in place. 25% 27% 24% 14% 21% Numerator 1 4 5 3 13 Denominator 4 15 21 21 61 Lead 1 3 2 6 Affiliates 1 3 3 7 Gender (Y/N) 2 1 3 2.2.1. Number of CCR websites that are developed, operational and used regularly 5 5 5 5 2.2.2. Number of CCR knowledge products created 236 195 100 531 Number of Products that have Gender Sensitive 17 18 6 41 58 Performance Indicators Achievement FY17 FY18 FY19 FY20 Total • Number of Patent 1 2 3 Number of Guide or SOP 3 3 6 Number of Book 86 2 1 89 Number of Modul 2 4 6 Number of Research Report 25 27 13 65 Number of Proceeding 38 56 67 161 Number of Paper 23 50 12 85 Number of Presentation Material 22 22 Number of Infographics 6 6 Number of Poster 17 12 29 Number of Factsheet or Flyer 1 1 Number of Bulletin 2 2 Number of Video 19 8 3 30 Number of Booklet 14 9 1 24 Number of Prototype 1 1 Number of Policy Briefs 1 Number of Others 0 2.2.3. Number of CCR knowledge sharing events held on best-practices & lessons learned 31 24 5 60 2.2.4. Number of people participated in activities addressing gender equality or female empowerment in science and technology research. 282 81 146 509 Male 74 52 32 158 Female 208 29 114 351 Enhanced collaboration in CCR/Consortium for collaborative research 3.1 % of institutions engaged in joint research projects 0% 110% 119% 124% 124% Numerator 4 45 50 52 52 Denominator 6 41 42 42 42 in-country based 3 37 40 40 120 US based 1 8 10 12 31 Third country based 0 Type of institution 0 Academic 3 41 43 45 132 59 Performance Indicators Achievement FY17 FY18 FY19 FY20 Total Private Sectors 0 Public Sectors 1 4 7 7 19 3.1.2. Number of scholars at CCR institutions with increased access to external research resources and academic research engines. 0 189 54 0 243 3.2. % of Partnerships Created to Support Joint Research Projects 97% 63% 3% 163% Numerator 29 19 1 49 Denominator 30 30 30 30 Lead 19 12 1 32 Affiliates 10 7 17 60 Annex V. CCR ranking Per Performance Indicator Performance Indicators CDSR ANBIO￾CORE NCSTT ARI SMART CITY 1. Number of peer-reviewed scientific publications resulting from USG support to research and implementation program 3 4 2 5 1 2. Ratio of citation to publication produced by Indonesia researchers 1 4 3 4 2 3. Ratio of academic research initiatives whose findings are utilized to address development challenges. 2 3 3 3 1 1.1 % of scholars who present at conference 4 1 3 5 2 1.1.1. Number of short-term training courses held 2 3 4 1 5 1.1.2. % of researchers participating in short-term training courses 1 4 5 3 2 1.1.3. Number of U.S. scholars visiting Indonesian institutions to lead short￾term training courses for Indonesian partner institutions 5 3 4 2 1 1.2 % of scholars who participate in collaborative research 2 3 5 4 1 1.2.1. Number of Indonesian scholars who participate in in-person faculty exchanges held in the U.S. 1 5 4 2 3 1.2.2. Number of Indonesian scholars who receive ongoing mentoring from U.S. university partners 4 2 5 3 1 1.2.3. Number of scholars who participate in virtual knowledge exchange 2.1. % of Indonesian institutions with improved research policies and systems in place. 5 1 4 2 3 2.1.1. Number of institutions that develop improved research and management policies developed 5 2 3 5 1 2.2. % of CCR members with improved administrative and management systems in place. 5 1 4 3 2 61 Performance Indicators CDSR ANBIO￾CORE NCSTT ARI SMART CITY 2.2.1. Number of CCR websites that are developed, operational and used regularly N/A N/A N/A N/A N/A 2.2.2.13 Number of CCR knowledge products created 3 4 1 5 2 2.2.3. Number of CCR knowledge sharing events held on best-practices & lessons learned 3 3 1 5 2 2.2.4. Number of people participated in activities addressing gender equality or female empowerment in science and technology research. 5 2 4 3 1 3.1 % of institutions engaged in joint research projects 2 4 3 1 5 3.2. % of Partnerships Created to Support Joint Research Projects 5 4 2 3 1 13 All CCRs equally achieved this indicator. 62 Annex VI. USAID SHERA CCR Lead and Affiliate Institutions CCR Name Lead HEI Partners (Affiliates) Industry and Agency Collaborations CCR on Acute Respiratory Infections (ARI) Universitas Padjadjaran (UNPAD) US partner: • University of Colorado, Denver Indonesian partners: • Universitas Syiah Kuala • Universitas Lambung Mangkurat • Universitas Mataram • Universitas Pattimura • 12 hospitals • 1 pharmaceutical company Scientific Modeling, Application, Research, and Training for City Centered Innovation and Technology (SMART CITY) Universitas Indonesia (UI) US partners: • University of Illinois, Champaign-Urban • University of Florida • Savannah State University • Alabama A&M University Indonesian partners: • Universitas Diponegoro • Universitas Padjadjaran • Universitas Sriwijaya • Universitas Udayana • Universitas Teknologi Sumbawa • Poltekes III Jakarta • Universitas Andalas • 19 Universitas PGRI • PT Telkom • PT Mass Rapid Transit Jakarta • Pemerintah Kota Depok • National Ministry of Internal Affairs (Kemendagri) • 4 Agencies for Planning & Develop for Depok, Serang, Sukabumi, & Bandung Center for the Development of a Sustainable Region (CDSR) Universitas Gadjah Mada (UGM) US partner: • University of Colorado, Boulder • University of Texas, Austin • California Polytechnic University Indonesian partners: • Universitas Indonesia • Institut Teknologi Bandung • Institut Pertanian Bogor • Universitas Bangka Belitung • Universitas Muhammadiyah Gorontalo • Universitas Negeri Gorontalo • Institute Teknik Surabaya • 6 local agencies in Province of Bangka Belitung • PT Korintiga Hutani • Shimizu Corporation • Government of Gorontalo Region 63 CCR Name Lead HEI Partners (Affiliates) Industry and Agency Collaborations National Center for Sustainable Transportation Technology (NCSTT) Institut Teknologi Bandung (ITB) US partner: • Massachusetts Institute of Technology • Temple University Indonesian partners: • Universitas Diponegoro • Universitas Sriwijaya • Universitas Lambung Mangkurat • Universitas Sam Ratulangi • Institut Teknologi Kalimantan • Universitas Negeri Sebelas Maret (UNS) • Ministry of Transportation of Indonesia • PT Bakrie & Brothers • BPPT (Agency for Assessment and Implementation of Technology) • PT PLN (State Electricity Company) • Palembang Department of Transportation • Jakarta Rapid Mass Transit Corporation CCR on Animal Biotechnology and Coral Reef Fisheries (ANBIOCORE) Institut Pertanian Bogor (IPB) US partners: • University of Rhode Island • Mississippi State University Indonesian partners: • Universitas Papua • Universitas Syiah Kuala • Universitas Padjadjaran • Universitas Brawijaya, • Universitas Udayana • Universitas Mataram • Universitas Nusa Cendana • Universitas Madura • BET (Livestock Embryo Center) Cipelang Bogor • BIB (Artificial Insemination Center) Lembang Bandung • BBIB (Artificial Insemination Center) Singosari Malang • Lolitsapi (Cattle Research Center) Grati-Pasuruan • BTPU HTP (Livestock Breeding Center) Indrapuri