2 Submitted July 22, 2008 A. Strategic Objectives and Intermediate Results 1. What progress has the project made toward accomplishing its results and expected end-of-project status? What progress has GHFP made toward achieving project intermediate results? (B1) The Strategic Objectives of GHFP are 1) to enhance the health sector cadre worldwide and 2) to strengthen staff capacity to lead and manage. During the course of the first two years of the program, GHFP has made significant progress toward both Strategic Objectives. SO #1 Enhancing health sector cadre worldwide GHFP focuses on three Intermediate Results (IR) in SO#1 to achieve a continuous enhancement of the health sector cadre throughout the world. These IRs are: 1) Health professionals are recruited, developed, and supported, 2) GHFP internships are implemented, and 3) Diversified workforce is improved. IR1.1 Health professionals recruited, developed, supported Since the program was inaugurated on July 3, 2006, a total of 125 Global Health Fellows have been recruited, including 62 transition fellows, 13 new fellows in Year One and 50 new fellows in Year Two. GHFP has placed fellows in 26 different countries and 34% currently work overseas. GHFP has played a key role in providing a large pool of technical experts who are committed to strengthening USAID’s health leadership throughout the world and implementing its vision. To increase the pool of qualified fellowship applicants and facilitate timely placements, GHFP implemented an intensive outreach strategy. Outreach is strategically targeted toward those venues that are most likely to generate fellowship applicants for GHFP, including schools of public health, professional conferences, and relevant employment fairs. For example, during Year Two GHFP conducted 53 outreach presentations including sites such as University of California at Berkeley and Emory University and career fairs at the Boston University School of Public Health and the Association for Schools of Public Health Career Services Council among others. In addition, GHFP has implemented outreach presentations at Historically Black Colleges and Universities and Hispanic-serving Universities, such as Florida International University and the University of South Florida. These face-to-face 3 outreach activities have been supplemented by GHFP’s re-designed website which announces all position openings and also enables online applications. The bi-annual fellows’ survey (IR1.1) results reflect the effectiveness of GHFP’s support system (IR 1.1). The survey results show that the percentage of fellows who agree that GHFP support services are satisfactory increased from 74.4% in 2007 to 82.7% in 2008. Similarly, the percentage of fellows who reported that the GHFP website is relevant to their needs increased from 25.6% in 2007 to 54.9% in 2008. Among those fellows affiliated with a University partner, the percentage who agreed or strongly agreed that this affiliation was beneficial increased from 57.1% in 2007 to 68.9% in 2008. 97.4% of the Fellows interviewed in 2007 and 100.0% of the fellows interviewed in 2008 reported that they would recommend GHFP to others. IR1.2 GHFP internships implemented GHFP took over the Global Health Bureau Washington, DC Summer Internship Program (IR 1.2), and supported six interns during summer 2007. In 2008, nine new interns were recruited; these interns are currently on location in Washington, DC. To promote the internship program, GHFP outreach activities now specifically incorporate information about the internship program and its activities. For example, during Year Two, GHFP conducted a presentation about the internship program at the Harvard School of Public Health that was attended by 41 participants. GHFP’s full-service website, which was implemented at the end of Year One, also promotes the internship program and enables online submission of applications. The appeal of the internship program is perhaps best illustrated by the fact that over 600 applications were received for the nine available internship positions in 2008. Undoubtedly, the extensive outreach and the easy online application system played key roles in this high demand for the internship program. The GHFP website already enables potential applicants for the 2009 GHFP internships to sign up for the internship listserv. As is the case for the fellows, GHFP develops and supports the interns. In line with these efforts, GHFP created professional development opportunities for the interns including a program of professional development activities consisting of technical presentations and participation in key events such as the USAID PHuNdamentals course. IR 1.3 Diversified workforce improved Rethinking the Diversity Initiative (IR 1.3) led to changes in approach and implementation. To increase the likelihood of a successful result within the program’s reasonable scope of work, GHFP re-visioned the program’s Diversity Initiative with three distinct areas of focus: Representation, Retention and Promotion. To implement this new vision, GHFP integrated its recruitment activities aimed at enhancing the diversity of the candidate pool into the 4 program’s overall outreach strategy. At the end of program Year Two, these efforts resulted in a diverse candidate pool for fellowships, with over 40% of fellowship applicants self-identifying as an ethnic or racial minority over a 12- month period (May 2007-2008, n = 1126). Yet from this same applicant pool, only 19% of the selected finalists self-identified as an ethnic or racial minority (n = 43), speaking to the need for developing influence with Agency decision￾makers regarding the desirability and importance of a diverse workforce. This challenge also carries forward into retention and promotion, which are directly linked to a strong commitment towards and action to create and sustain diversity in the workforce. To that end, GHFP will be strategically integrating information regarding the benefits of diversity in the workforce in its communications with Onsite Managers and fellows and finding opportunities to integrate diversity into its professional development activities. SO#2: To strengthen staff capacity to lead and manage GHFP has also made major progress toward the Strategic Objective of strengthening staff capacity to lead and manage (SO2). During the first two years, GHFP implemented 22 training activities aimed at developing and improving essential PHN skills and knowledge (IR 2.1). During the course of Year 2, GHFP implemented the following workshops: Working Smarter, not Harder (twice); Modern Business Writing (twice); PHuNdamentals for Success (twice); Annual Workplan/Individual Development Plan Sessions (four times); Working in the USAID Context, for the Avian Influenza Unit (twice), and the GHFP Intensive Leadership course. In addition, the following technical exchanges were held: Data Certainty for Performance Management, Social Marketing: Does it Work? and Child Health: The Next Generation. Approximately 300 people attended these events.1 Post-event evaluation surveys indicate that participants typically reported that the events were highly relevant to their work, and that both the course content and instruction were of high quality. In addition to the above, GHFP helped organize the LAC, ANE, and AFR State￾of-the Art conferences, as well as GHFP ANE and AFR Regional Fellows Meetings. Post-event surveys indicate that nearly all sessions were highly useful for the participants, and that the session content and speakers were very good. At this time, a post-event, on-the-job survey is being implemented. Data from the Fellows’ Survey further show that the percentage of fellows reporting that GHFP provided them with state-of-the-art information increased from 25.6% in 2007 to 59.6% in 2008. The data further reveal that the likelihood that fellows actually use the state-of-the-art information provided to them has increased substantially. In 2007, of the ten fellows who reported receiving such information, four reported using it (40.0%). By contrast, in 2008 this was the 1 People who participated in more than one event are double-counted. 5 case for 27 out of the 31 Fellows (87.1%). Regarding GHFP’s mandate to support organizational development, the program conducted a comprehensive analysis of the GH structure that assessed the overall GH staffing level, workload and cross-cutting functions being performed throughout the Bureau. For individual GH offices and divisions, GHFP designed and implemented ten planning/teambuilding retreats in Washington, DC and two in the field (USAID/Afghanistan and East Timor). The organizational retreats ranged from short half-day events with 15 participants to a complex two-day event for seventy. Each of these activities included pre-retreat interviews as well as post￾event follow-on work. The program also arranged for five consultancies to assist GH offices in Washington with a variety of organizational challenges ranging from developing new program areas to design efforts. A sixth consultancy – a family planning program design effort – was fielded for USAID/India 2. Is the underlying rationale for the project still relevant and valid? If not, should changes be made, and if so, how? The 2006 Request for Applications (RFA) stated: “In spite of increased demand, a dearth of professional development opportunities exists for individuals with limited field and work experience. The challenge is to provide opportunities for individuals at the junior-, mid- and senior-level to improve their skills and knowledge of the field while supporting the implementation of USAID programs…. By continuing to support fellowships and internships, USAID is assured of a “pipeline” of professionals who can fill critical positions as senior leaders begin to retire… The three fellows programs are now being combined to provide a “pathway” for health professionals across all technical areas in global health. This combined program of fellowships, internships, and professional and organizational development assistance will provide for greater efficiency and will also encourage new approaches.” This 5 - 10 year cooperative agreement was meant to improve Global Health program results by: (1) Increasing efficiency by combining three fellowship programs (2) Building a pipeline for the next generation of global health professionals (3) Supporting the performance of individuals and organizations doing health￾related work within USAID. When the RFA was written in 2005, USAID had forecast a significant human capacity shortfall due to retirements and previous staffing freezes. A 2008 Senate Foreign Relations Committee report indicated that for USAID, “Staffing is … an issue – more than half of USAID personnel are not specialists – …. and a third of the agency’s most seasoned staff are on the verge of retirement.” 6 Three years later, the rationale remains strong for one, combined, larger fellowship program with substantial outreach to all levels of potential interns and fellows. The program has evolved to include undergraduate, entry level, mid￾level, senior and uniquely skilled senior technical advisors. The professional development activities have increased in number of activities, participant numbers, and support to field-based fellows. Organizational development support remains a key intervention for the program. 3. Provide a summary of key issues/challenges and recommendations that have been resolved and/or implemented over the last two years? What issues are still outstanding? Key issues/challenges and recommendations that have been resolved and/or implemented over the last two years: ƒ Streamlined the GHFP partnership to enhance cost-effectiveness and implementation efficiency. ƒ Revised the fellows’ compensation package to address the GH Bureau’s desire for comparability with Agency compensation standards while maintaining a competitive, attractive package. ƒ Modified and tightened Letter of Agreement clauses to include non￾disclosure understanding and more flexibility in negotiations with Missions. ƒ Refined the coaching program to enhance cost-effectiveness while maintaining limited availability for GH staff on a demand-driven basis. ƒ Clarified distinction between the mentoring and coaching programs offered through GHFP. ƒ Improved the personalized service and support provided to Onsite Managers (OSMs) through increased one-on-one contact, a dedicated OSM Express and GHFP web portal focused on this constituents’ needs, and development and provision of reference materials and tools to inform OSM fellow-related decision-making and support. ƒ Dramatically decreased the GHFP allocable cost subsidy of Bureau-wide training events, while increasing the quality and quantity of delivery. ƒ Revised fellows’ individual professional development (PD) policies and enhanced management support for PD activities. Outstanding issues: • Keeping the balance between short and long term goals. It takes patience and an investment to keep “growing” the field but the current staffing crisis has fostered a focus on getting known people on board and a disinterest in international and/or NGO based internships which would support more mid-career switchers to enter the field. Reshuffling current global health professionals does meet urgent staffing needs but 7 may hinder the more strategic goal of new blood and greater diversity in the workforce. • Funding for and implementation of an overseas internship program aimed at enhancing the long-term diversity of global health professionals. ƒ Providing professional development support to overseas fellows at an equitable degree to domestic fellows. ƒ Providing mentoring specific to working in the USAID context. ƒ Increasing diversity of fellows and interns through activities targeted to increase the demand for and support of a more diverse GH workforce. ƒ Funding for and implementation of a Foreign Service National (FSN) professional development program based on the articulated needs of the population, health and nutrition FSNs. ƒ Approving GHFP’s revised cost share strategy. ƒ Obtaining clarity regarding the mutual and complementary roles of PDMS and GHFP in professional development implementation. ƒ Funding for diversity training aimed at creating a workplace environment conducive to supporting a diverse workforce. ƒ Balancing the short term need for staffing versus the long-term goals for a diversified, well-trained and appropriately experienced global health workforce. B. Intermediate Results 1. What progress has GHFP made towards achieving project intermediate results? Because the Strategic Objectives and Intermediate Results are closely related, questions A1 and B1 are being answered together. The response for both questions is posted under Question A1. C. Project Implementation 1. Are work plans realistic and consistent with the SO and IRs as related to the approved results framework? Are activities consistent with the work plan? Are there shortfalls or delays in the project implementation? If so, why? GHFP’s annual workplan is a succinct 14-page document that uses the approved Results Framework as its basic structure. The GHFP Results Framework is comprised of the PDMS Activity Objective, the GHFP Program Vision, two Strategic Objectives, 5 Intermediate Results and 16 Sub-Results. In the table below the numbers (in green) alongside each Intermediate Result and Sub-Result of the approved Results Framework represent the number of related activities taken directly from the GHFP Year Three Workplan. Many activities address multiple results. 8 In Year One, GHFP focused on addressing the challenges of transitioning fellows, hiring new fellows, managing senior staff turnover, right sizing the subcontractor team, and expanding the program systems. Year One workplan activities that were deferred were to “develop non-USAID site placements strategy and system,” “create and implement Diversity Initiative” and “research mission interest in the FSN exchanges.” Year Two had many accomplishments including recruiting and supporting 43 new fellows and implementing a robust professional development program. Deferred activities include implementing the international internship program, the FSN professional development program and moving forward with the Diversity Initiative. For the past two years, the few workplan activities that were delayed or deferred are a reflection of scarce staff resources and changing expectations and priorities which resulted in less USAID advocacy and funding. 2. Does the project have an internal monitoring and evaluation plan? If yes, does the plan include indicators and data sources at the input, process, output, and outcome levels? Are data being collected? How are data used and what trends have emerged under each result area? 9 To assess the progress of GHFP and to measure its impact, performance monitoring indicators were developed for the main program objectives, as well as for the intermediate results that will contribute to the achievement of those objectives. To that effect, GHFP developed a Performance Monitoring Plan (PMP), as well as a more detailed internal Monitoring and Evaluation (M&E) Plan. The goal of the Performance Monitoring Plan (PMP) is to clarify how GHFP plans to collect and use project performance data. The PMP identifies a select number of comparable indicators that can be collected regularly over the course of the project. The focus of the PMP is on those indicators that are within GHFP’s manageable interest, and that can be measured and communicated; are objective and credible. Consistent with the principles that have been used to select U.S. Foreign Assistance Performance Indicators, the PMP includes indicators that require only limited additional investment of time and resources and that can be easily collected as part of the established GHFP performance management system. The PMP was developed using a participatory process that involved all GHFP partner organizations. The final agreed upon version was submitted to USAID in June 2007. Because the PMP may not capture every single GHFP activity and its outcomes, the PMP is supplemented by a more extensive series of custom monitoring indicators that are used to evaluate specific activities such as individual professional development events. The M&E plan is a living document that will be updated as needed. For example, if GHFP is invited to implement a new type of activity, then indicators may be added to monitor progress on that activity. The data for indicators for our evaluations are derived from four main sources: 1) recruitment and performance records, 2) a bi-annual survey of fellows, 3) an annual survey of GHFP alumni, and 4) post-event evaluations of PD and OD activities. Recruitment and performance data are collected continuously. The development of the new GHFP website, which enabled a switch from paper applications to online applications, ensures that the recruitment manager can now readily access this information. A survey of fellows is scheduled to be conducted at intervals of roughly six months (allowing flexibility to accommodate the fellows’ work and travel schedules). Data collection for the first wave of the Fellows Survey started in Fall 2007, while data collection for the second survey wave started in Spring 2008. A total of 40 fellows were interviewed in Fall 2007, and thus far 52 fellows have been interviewed during the second survey wave. The Alumni Survey is scheduled to be conducted at one-year intervals. To ensure that the survey includes a sufficiently large number of alumni to enable analysis, the first wave of the Alumni Survey has been scheduled for Summer 2008 (and will be reported on in the Year Two Results Review). All professional development and organizational development (PD and OD) events are evaluated using a post￾event survey. 10 The data indicate that GHFP has made significant progress towards its Strategic Objectives to enhance the health sector cadre worldwide and to strengthen staff capacity to lead and manage, as well as toward its stated Intermediate Results. Illustrative data have been summarized in above sections. 3. Is GHFP demonstrating the expected ability to provide technical assistance? The senior management of GHFP have years of experience working with the Agency’s structure, its programmatic goals and its operating systems. This understanding informs the management practices of GHFP, resulting in the program demonstrating – and often exceeding – the ability to provide technical assistance to USAID across its three program result areas. Result Area One Under Result Area 1, the GHFP RFA anticipated up to 75 fellowships per year as a maximum capacity for the program. In the program’s first two years, GHFP’s experienced staff fielded 125 individual fellowships with its current peak of 78 concurrent, active fellowships with up to 105 on board, in recruitment or being planned. GHFP has demonstrated its significant facility to recruit and hire talented technical experts across the spectrum of public health disciplines to meet the needs of the Agency. Throughout its recruitment efforts, GHFP works with USAID hiring managers to find the most suitable candidates for its fellowships and employs efficient and effective processes to hire and place candidates quickly. GHFP’s ability to meet the needs of the Agency is evidenced by the number of requests the program has fielded for new fellowships and its ability to deliver quality candidates in a short period of time to take on critical technical work for the Agency. To support this ever growing cadre of fellows, GHFP developed support systems and processes that complement USAID’s own systems and are responsive to the Agency’s human resource management needs. These support systems include a transparent performance management process, individualized program support provided by GHFP’s program coordinators, an online management system accessible to USAID managers and fellows, and a user-friendly, fellows-specific Intranet with program resources and guides covering all aspects of administrative support. Result Area Two Within the very limited funding that GHFP has been provided for Result Area 2, the program has demonstrated its capacity and enthusiasm for creating and providing opportunities for individuals of diverse backgrounds to gain exposure to international heath programs and build their skills. In program Year One, without full funding for RA2, GHFP integrated activities 11 focused on increasing participation of historically underrepresented groups in global health into program-wide activities such as outreach, recruitment, internships and professional development opportunities. GHFP anticipates funding in program Year Three, plus the use of cost share, to more robustly implement activities under this result area to ensure that opportunities for individuals from these historically underrepresented groups are more widely available. These activities will include the much￾anticipated professional development for Foreign Service Nationals. Result Area Three GHFP’s ability to provide technical assistance under Result 3, “Organizational development assistance and professional development opportunities for current staff provided to USAID” has exceeded previous GH fellowship programs and the anticipated delivery under this new award. Resources include experienced GHFP staff and a partner, Management Systems International (MSI), who can make available a large cadre of full￾time and consultant experts with deep USAID operational experience. Evaluation data (to be reported in Year Two Results Review) indicate that these resources provide high-quality, cost-effective technical assistance capacity which has proven to be flexible, creative and responsive to the needs of the GH Bureau and its field colleagues. The number of training activities conducted and increasing number of participants who complete GHFP-sponsored training are both evidence of GHFP’s training capacity and quality. In program Year Two, GHFP conducted 16 training events at its training facility, attracting close to 300 participants. In post-event evaluations, these trainings each received ratings over four on a five point Likert scale with five being “excellent,” speaking to the quality of the instruction and the appropriateness of the topic to the audience. These results highlight GHFP’s familiarity with the GH staff’s professional development needs and the program’s familiarity with management training trends in the public sector marketplace. GHFP’s organizational development work started slowly but has been increasingly active in the last year. GHFP conducted two PHN teambuilding retreats in the field and were instrumental in the design, preparation and execution of three highly successful SOTA conferences. In Washington, GHFP conducted nine team-building or planning retreats including a two￾day office wide retreat for the Office of HIV/AIDS (OHA). Each of these events was considered effective in addressing the development needs of the recipient unit. GHFP’s flexibility and responsiveness have been proven repeatedly over the first two years of the program. While unanticipated in the RFA, GHFP supported five technical consultancies for the GH Bureau and one Mission. 12 Several of these consultancies came with very little notice and were delivered upon quickly and produced high-quality results. 4. Is GHFP responsive (to) the Missions’ needs regarding OD support relating to the PD requirement and Mission needs? If a process is in place that takes into account the field OSM perspective, please describe. Does the GHFP appropriately implement MOUs? Identifying Missions’ needs for organizational development support GHFP is very responsive to Mission requests for OD support, but identifying and being able to respond to true needs is more challenging. Missions that host fellows are more likely to know of our OD capabilities. We provide information in the OSM Express and on www.ghfp.net about our OD expertise but, due to the very contextual and confidential nature of this work, we have been conservative about advertising specific past performance. Presence at the SOTAs has been helpful. For example, we had three requests for OD assistance in Africa post-SOTA. Through the course of conducting a regional fellows’ meeting in Africa another OD request was generated. We believe that there is unmet need for organizational support in the PHN sector and that Missions would welcome the program’s services as well as those from PDMS. GHFP would like to collaborate with PDMS in the development of a new outreach strategy to expand GH OD and PD support to the field. A starting point for this would be to identify a short-list of focus Missions undergoing significant changes in PHN staffing or leadership. Ensuring equity in the provision of PD for field-based fellows In Program Year One, we recognized the disparity in access to PD events between GH and field fellows and have been looking for ways to address the gap. In Program Year Two, GH fellows had access to 11 different types of skills training or PD activities at GH; these are development opportunities unavailable to fellows in the field. To address this disparity we currently require field fellows to participate in one multi-day PD event in DC once every two-year fellowship. As we’ve been proactive about this in the field fellow’s Letter of Agreement (LOA), the issue of required PD attendance for field-based fellows has yet to be raised by any OSMs. For most new2 field fellows this simple requirement is met by coming to Washington to attend the GH orientation course “PHuNdamentals for Success.” There is anecdotal evidence that, while not skill training per se, this orientation course is critical to staff members’ performance within their first year of USAID work and OSMs in Washington, DC and in the field have been supportive of this time investment. The PHuNdamentals is a success story of how GHFP assisted 2 For fellows who have recent experience working inside USAID, we’ve waived this requirement. This has been the case for several fellows in PY1 and 2. 13 PDMS in taking ownership of a highly rated and comprehensive course that for years had been organized outside of GH. Currently the only other multi-day PD event sponsored by GHFP is the Annual Intensive Leadership course (conducted by the Center for Creative Leadership). The three-day course was attended by several field-based fellows who recommended the course enthusiastically. The Intensive Leadership course is an example of the type of PD experience that fellows simply could not access in the field. Another significant way to address the disparity of access to PD for field fellows is the short Regional Fellows’ Meetings that GHFP began implementing in Program Year Two. The model for the Regional Meeting is a two-day workshop that includes a full day of PD training (including two or more skill topics from the most highly-rated Washington workshops conducted at GHFP) and a full day on annual work planning and fellowship support (another fundamental PD support made readily available to Washington, DC-based fellows). Both the ANE and Africa Regional Meetings in 2008 were highly rated by the 16 fellows who participated in the meetings. The latter also provided a cost-effective opportunity for GHFP staff to conduct two Mission site visits. The venue was chosen to be within less than one day’s travel to/from post and no OSM objected to fellows’ participation. A post-meeting survey of attendees and their OSMs is under way. Assessing field OSM perspectives in providing GHFP services GHFP’s contact with an overseas onsite manager (OSM) begins with the request for a fellow. In response to a request, GHFP’s recruitment team provides the potential OSM with documentation regarding: ƒ The program’s fellowship levels, ƒ Sample budgets and letters of agreement, ƒ Recruitment and hiring processes, and ƒ Agency guidance regarding the role of the fellow. GHFP follows up on this initial information sharing with individualized contact (usually via phone) to answer any questions the OSM may have regarding GHFP. The OSM serves as the primary USAID contact during the recruitment and hiring phase of the fellowship cycle. OSM opinions, suggestions and criticisms are all critical information for the management of the program. Throughout the recruitment phase, GHFP remains in regular contact with the OSM to ensure he/she is informed of progress and is comfortable with that progression. During the subsequent hiring phase the OSM’s input is critical to ensuring that the fellowship is established with good will and faith on the part of all parties. Once the fellow is “on program” contact with the OSM tends to be less frequent, however GHFP has put in place procedures that maintain a proactive stance. GHFP settled on a system of six-month check ins with the OSMs to ensure that any potential issues identified in the fellows’ first six months are redressed immediately and that the OSM understands that GHFP stands by to 14 support them in their management and oversight of the fellow. The program looks forward to expanding these contacts in Program Year Three. Implementing letters of agreement (LOA) At the outset of the program, GHFP employed memorandums of understanding (MOUs) – modeled on a previous fellowship program’s MOUs – to document agreement between the program, the host organization, the funding unit (if different than the host organization), and the fellow regarding the fellow’s support at post. Based on feedback from participating host organizations and USAID’s legal counsel, GHFP modified its standard memorandum of understanding. Now referred to as a letter of agreement (LOA), the LOA is a tool used by GHFP to establish rapport with the host organization – specifically the Executive Officer’s team – while clarifying roles and responsibilities amongst the parties to the agreement. Using the LOA as a means to establish a communications pathway with the host organization has been useful in enhancing relationships between GHFP and the host organizations. GHFP will continue to refine the LOA in response to changing USAID and other host organization support policies and procedures. 5. Is GHFP responsive to the needs of the Bureau for Global Health? What is GHFP doing to work across the Bureau? Assessing needs through professional relationships with GH managers A priority of the program’s senior management has been to develop, maintain and strengthen excellent working relationships with key managers and staff across the Bureau for Global Health. These relationships foster a sense of trust and information sharing which is critical to GHFP’s ability to understand and respond quickly and effectively to managers’ requests. GHFP is working to have more regular contact with Division Chiefs and Deputy Office Directors on a range of issues from new recruitments to fellowship support issues to specialized organizational and professional development activities. The Bureau-wide workforce analysis conducted by GHFP for the Front Office in Program Year One was an excellent opportunity for GHFP staff to establish working relationships with over 60 senior GH managers of the Bureau senior staff. The acquired insight into the Bureau’s management issues was critical given that no comprehensive management or training needs assessment had been undertaken in GH since 2002. Maintaining professional contact with these senior managers is crucial for the program’s harmonizing fellowship and professional development support across all six GH offices. These contacts, and our constant evaluation of GHFP fellowship support, PD and OD activities will enable us to stay current with the Bureau’s staffing, strategic and management needs. 15 Percentage of PD participants by GH office HIDN 31% OHA 27% PDMS 11% PRH 26% RCS 4% SPBO 1% GHFP’s POD activities reach across the Bureau GHFP senior staff participates in every Training Advisory Group meeting that PDMS has convened since the inception of the program. GHFP also takes care to attend important public fora (such as the recent Global Health visioning series), and regularly seek feedback from Onsite Managers via semi-annual OSM meetings. GHFP’s ability to effectively work across the Bureau would be further enhanced by regular invitation to and attendance at open-Office meetings. GHFP’s acquired knowledge of staff interests and needs is reflected in the distribution of attendees among the GH technical offices in GHFP training activities. The pie chart on the left shows the percentage breakdown of participants from the six GH offices in GHFP PD events in Program Year Two. OD activities have also been distributed among the six offices. The Office of HIV/AIDS has been the most frequent user of these services and the Avian Influenza Unit in the Office of Health, Infectious Diseases and Nutrition has accessed GHFP’s technical assistance. GHFP’s OD approach is not limited to once-only workshops. GHFP is called upon frequently to help USAID managers or divisions work through staff, office or management issues. Managers trust that GHFP will be responsive, empathetic and most importantly, will provide that manager and his/her team the support they need to accomplish their work objectives. Recently, GHFP staff became aware of management challenges within an Office of HIV/AIDS division through fellows assigned there. POD staff, based on established relationships, were able to contact the team leader with an offer of organizational development assistance. This contact led to collaboration on a series of individual consultations and mini-retreats to address his team’s organizational issues. This is just one example where trusting relationships and sharing of needs between USAID and GHFP led to direct organizational improvement for the Agency. 6. What efforts are GHFP making to collaborate with GH and field? How proactive is GHFP in anticipating and supporting fellowship, professional development and organization support? Is GHFP’s outreach and support to OSMs and Offices adequate? 16 Awareness of GHFP as a high-quality program equipped to bring new talent to the Agency when needed is very high in the GH Bureau and the regional bureau’s health teams. In fact, many USAID staff are more aware of GHFP’s staffing capabilities then other aspects of its mandate. GHFP has the ability to provide more workforce planning assistance. Also, per request by the CTO, GHFP has not implemented an active outreach strategy to USAID Missions; awareness of GHFP within Missions is thus limited to where GHFP supports fellows, professional networks of GHFP staff, and those Missions whose staff have experienced fellowship programs elsewhere. GHFP welcomes the opportunity to conduct more active outreach to Missions through increased field contacts. GHFP pays careful attention to the GH Bureau’s and Mission needs regarding staffing, using its extensive networks within the Agency to uncover opportunities for placing fellows. The RFA stated that: “Placements may be located at USAID, host country governmental agencies, private voluntary organizations, academic institutions or other organizations consistent with the mandate of the program. Fellowship placements typically replace outgoing fellows and/or represent new requests. In both cases these requests are generated by the host organization or USAID.” GHFP is very responsive to these requests for new placements in USAID, evidenced by the program’s successful recruitment and placement of over 60 new fellows in the first two program years. GHFP’s reputation for recruiting excellent candidates and placing them quickly has resulted in GHFP being the ‘go to’ mechanism for high quality recruitment within the GH Bureau and in Mission PHN offices. The number of requests from USAID Offices and Missions has increased over time while attrition has remained steady. Yet while the program has enjoyed increased requests for fellowships within USAID, requests for partner organization placements (those at host country governmental agencies, private voluntary organizations, etc. as referenced in the RFA) have not materialized. This situation is mainly due to the decision made by GH Bureau management soon after award that partner organization placements in Washington were no longer desired, and that partner organization placements in the field would be much less frequent than under previous programs and would require that Missions’ co-fund these placements at least 50% with the sponsoring Washington-based technical office. As a result, GHFP has placed only two fellows in partner organizations, significantly less as a percentage of total fellows than under the previous fellowship programs. Both of the current fellowships are under the Population, Health and Environment (PHE) initiative that uses the fellowship mechanism as one of its primary vehicles for PHE program implementation. 17 GHFP takes a non-intrusive approach to managing and supporting its fellows. Each fellow is assigned a Program Coordinator who serves as the fellow’s primary point of contact. The Program Coordinator is tasked with orienting the fellow, supporting him/her through the fellowship for administrative support, professional development and performance management, and assisting the fellow with the transition off-program. This approach allows issues related to the fellow and his/her placement to be anticipated quickly or identified early and addressed. Through the Program Coordinators support to fellows’ professional development has been very proactive. Fellows’ access to PD is highlighted in the fellow’s orientation and when possible each fellow meets with the Deputy for Professional and Organizational Development. During the course of the fellowship, the Program Coordinator provides individualized support to fellows around PD resources and options. Fellows’ PD planning is also integral to the Annual Workplan and Individual Development Plan (AWP-IDP) that all fellows are required to prepare with their OSM and submit to GHFP. GHFP designed and piloted an AWP-IDP course for GHFP and collaborates with PDMS to facilitate this course quarterly. GHFP’s AWP-IDP forms were designed to mirror the form that PDMS is using with other non-direct hire personnel. To the best of our knowledge PDMS has been proactive in addressing the need for GH office OD during the annual Bureauwide budgeting process. Regardless, the program has developed a significant track record for conducting effective OD interventions. Support from the CTO to ‘get the word out’ would reflect well on PDMS’s ability to support its organizational development objectives. While GHFP’s communication with OSMs is necessarily frequent during the recruitment and placement phases, it tends to taper off after the fellowship begins. In recent months, GHFP has increased its focus on communicating directly with OSMs, recognizing that OSMs need continued support after the fellowship begins. GHFP intends to provide a light, non-obtrusive touch in these communications. Examples of GHFP’s on-program outreach to OSMs include: • GHFP’s USAID portal includes a range of program information and resources; • Periodic open meetings for Washington-based OSMs as fora for mutual information sharing and learning regarding the OSM’s challenges and sharing strategies to meet those challenges; and • Semiannual OSM Express to keep OSMs abreast of program developments and policies. Yet the challenge remains in keeping OSMs engaged with GHFP without overwhelming these busy managers with program information and requirements. GHFP will continue to work on its relationships with OSMs to ensure they are appropriately supported. 18 7. Are reporting requirements being met at the activity level (fellowship, POD, funding including program reports specified in the work plan)? Are the reporting documents informative? Are reports submitted in a timely fashion? The reporting requirements are being met at the activity level for each key result area, including reports specified in the work plan, as described in the table below. It identifies the report, the receiver of the report, frequency, due date, content and results. The reports are used by program staff, Public Health Institute (PHI) Central Office, and USAID in making programmatic, administrative, and funding decisions, thus making them useful tools. The reports are submitted on time per due dates noted below. REPORT TO FREQUENCY DUE DATES CONTENTS AND RESULTS 1. Annual Work Plan CTO, Project Director, PHI G/C Annually; form provided by GHFP approved by CTO June 30 Program activities by key result areas, status of fellowships, specify learning events and organizational development activities, funding needs for expected outcomes, staff travel plans, cost share plan, and timeline. 2. Annual Results Review (same as Performance Monitoring Report) CTO, Project Director, PHI G/C Annually; form provided by GHFP October 31 Establish goals/results framework, identify challenges and problems, delays which impair the ability to meet objectives of program, statement of action taken, assistance needed to resolve the situation, results thus far, analysis & explanation of cost. 3. Management Review (used for USAID/GH Portfolio Review) CTO, Project Director, PHI/GC Dates TBD, discretion of the CTO; questions provided by CTO July 2008 Questions selected by CTO. Typically, achievements, challenges and issues, and solutions to these issues. Reporting period determined by the CTO. 4.Financial Status Report (SF-269A) USAID Procurement, CTO, Project Director, PHI G/C Quarterly; Form SF 269- A provided by USAID; PHI Accounting submits an original and 2 copies No later than 30 days after the end of each quarter The status of funds including ceiling, obligations, accruals, and pipeline by budget category line items required in the cooperative agreement. The report is used as documentation to drawdown funds from the DHHS/Payment Management System. 5. Avian Influenza CTO, Project Quarterly; 25th day after The report includes voucher and 19 (AI) Financial Reports Director, PHI/GC, HIDN/AI Unit form provided by USAID/AI each quarter accrued expenditure data by country, funding source, and activity/focus areas. To meet Congressional reporting requirements associated with avian influenza funds. 6. Baseline Reports CTO, Project Director, PHI/GC Quarterly; form provided by USAID February 15, May 15, August 15, November 15 Monetary obligations per cooperative agreement modifications by Core Elements, OE, Field Support, and MAARD funds, reporting cumulative expenses to date, balance remaining, expenses for the past year, budget projections for the next year, actual/projected monthly expenses, and months of funding remaining. For USAID to confirm financial obligations and to track actual/projected expenses to date. 7. Core & Field Support Quarterly Financial Status Reports CTO, Project Director, PHI/GC, OSM by Country Quarterly; form/data provided by GHFP February 15, May 15, August 15, November 15 Financial data reported by Core/Elements and Field Support/Countries of funding source includes monetary obligations date, cumulative obligations to date, actual/projected expenses for the past twelve months, actual/projected monthly expenses; months of funding remaining; estimated funds remaining/required for current and next FY. Data is used to calculate future funding decisions. 8. Pipeline Analysis & Other Financial Information (GAC, HIV/AIDS, MORA, RHAP/South Africa Financial Reports) CTO, Project Director Ad Hoc; form/data provided by USAID and/or GHFP As requested by USAID Same financial data reported in the Core & FS Quarterly Financial Status Reports (7). Requested information varies by funding source and/or per fellow. Data is used for future recruitment and/or funding decisions. 9. Recruitment and Fellowship Update Report CTO, Project Director, Deputies Monthly; form provided by GHFP 15th of each month Provides job titles of open and pending positions including location, GSA level, funding source, SOW status, length of fellowship, and dates position 20 opened/closed. The fellowship update report lists all current, pending, and completed fellowships in DC and overseas. These reports are used to determine future recruitment and placement decisions. 10. GHFP Security Action and Status Report CTO, USAID Security Representative, Project Director, Recruitment & Fellowship Team Monthly; form provided by GHFP 1st of each month Provides security clearance status and updates for new candidates and fellows. The report includes a summary of security actions on Active Personnel Security Clearances (PCL), Pending Security Clearance Applications, USAID Employment Authorizations (EA), and Inactive Personnel Security Clearances (separated personnel). The report is used in hiring a fellow, scheduling security briefings, maintaining active security clearance status for staff/fellows, and transferring and terminating security clearances. 8. How effective has the cost share agreement been? What has worked and what hasn’t worked? In the past two years, the program has changed from what was envisioned in the original proposal. The changes in large part have been required or at the very least strongly encouraged by the program staff at USAID, significantly and adversely impacting PHI’s cost sharing commitments. The three major changes that effect PHI’s cost sharing commitment are discussed below: • Reduction in the number of partners: The proposal originally required cost share for “learning institutions” and it was strongly implied in the RFA that the type of cost sharing being sought was for academic course work and other types of related continuing education and professional development programs. PHI is not an academic institution and therefore sought out partners to meet this requirement. 21 PHI’s successful proposal included major partnerships with five organizations, three of which were academic institutions. USAID program staff strongly and consistently encouraged PHI to decrease its number of partners, indicating that this would be a performance expectation of PHI as the prime organization. PHI terminated its relationship with the University of Washington. During this time, the Diversity Initiative, the prime focus of United Negro College Fund-Special Programs (UNCF-SP) was dramatically diminished in its scope and emphasis. The Diversity Initiative has since been incorporated into PHI’s work, thus eliminating the partnership with UNCF-SP. The cumulative effect of these changes is to reduce the cost share PHI originally anticipated being generated by its partners and increasing the cost share of PHI. • NGO Contribution: The “junior” fellowship program formerly managed by the University of Michigan placed fellows at NGOs throughout the world. This type of fellowship was anticipated by USAID in the RFA and included as part of PHI’s proposal. PHI had anticipated capturing the non-governmental funds used by the NGOs to support these fellows as cost sharing. Again, this part of the program has significantly changed over the last two years thus greatly diminishing PHI’s ability to generate matching funds. • Expanding Program: As we enter the third year of the program, PHI anticipates the number of fellows supported by the cooperative agreement to be much larger than originally anticipated. Our initial proposal anticipated an on-going cadre of 75 fellows, and GHFP has supported over 100 fellows in Year Two. We anticipate it remaining at that level for the duration of the agreement. This is a 43% increase from our original proposal. While PHI is very pleased with the anticipated growth in the program, we are also concerned about the corresponding 43% increase in cost sharing obligations. These changes place a significantly higher financial burden on PHI than what was originally proposed. PHI partners will still provide a minimum of a 2% cost share calculated on the awarded funds that they receive. The types of cost share provided by the partners will not change from the original proposal. PHI is formally requesting a reduction in the level of shared costs required by the referenced agreement from 2% to 1% of the federal cost and has pending approval with USAID/Office of Acquisition and Assistance. D. Process 1. Project Organizational Structure 22 a) Is GHFP organizational structure efficient and cost effective? Have changes occurred that hindered and/or benefited project performance? If so, how? GHFP’s structure includes a bi-coastal organization with three partners. Work is grouped around three result areas as described in the RFA. This organization has proven to be efficient by placing people in locations that maximize communication and understanding: (1) finance and operational staff are next to the PHI central offices and can communicate and problem solve quickly; (2) Washington, DC based staff including the Project Director and two Deputies can interact easily with the CTO, other USAID staff, and DC-based fellows; and (3) partner staff based at the two major international health universities can access academic resources for the fellowship while the third, DC-based, partner provides technical talent and experience in coaching, professional and organizational development with little travel cost to the program. Towards the end of Year One, one significant change occurred. Subcontractor￾partnerships were revised reflecting USAID realities, resulting in the loss of United Negro College Fund Special Programs (UNCFSP) and the University of Washington. Their work was sorted among PHI and the partners without adding new staff. With decreased redundancy in the university partners and fewer resources going to Result Area Two which has been in flux, PHI was able to streamline systems and maximize use of current resources without loss of efficiency. b) Is there adequate collaboration, division of labor etc. among the different implementing partners? We believe that the partners provide crucial and significant contributions to the partnership. Each partner has specific tasks they lead and all partners participate together in bi-monthly conversations and bi-annual meetings to help plan and implement the program. Deeply familiar with USAID, they each provide creative solutions to challenges the program faces and take the lead in various areas. While PHI remains the sole fellows’ employer, the university partners, Harvard and Tulane, implement GHFP outreach activities for prospective fellows and interns, help with intern recruitment, maintain a faculty mentoring/community of interest moderator program, and provide technical experts to support professional development. Harvard has a special focus on organizing provocative presentations on various technical topics. Tulane leads both the alumni program and GHFP’s monitoring and evaluation system. Management Systems International supports both the professional and organizational development work. They provide organizational development experts for USAID office, division and Mission retreats, are leading the effort to update the CTO/TA modules, and provide training management support. 23 As GHFP continues to improve and streamline processes, the partnership will also evolve. To serve fellows more efficiently, GHFP is now reviewing the role of the Program Coordinator. One possible outcome could be a more balanced sharing of fellows’ support among the three Program Coordinators (based at PHI, Harvard, and Tulane). This would happen only after discussions with the partners and moving some current, more operational, responsibilities to other staff. 2. Project Management a) Are the current mechanisms for communication between CTO and GHFP appropriate and effective? Methods to access GHFP information and communicate about the program are abundant, varied, appropriate and generally effective. Communication channels include: (1) Bi-monthly, ninety-minute, agenda-driven CTO-Project Director meetings supported by phone calls and emails whenever necessary (2) Email and phone access directly to all three Deputy Directors. The Project Director is copied in on emails and briefed by the Deputy or CTO, as appropriate (3) Copies of bi-monthly Fellows Express and semi-annual OSM Express (4) Monthly PDMS-GHFP Professional Development meetings (5) As noted in “GHFP Program Reports” table included in workplan, all reports as required by the cooperative agreement (6) Special, 24/7 access to the GHFP virtual workspace which includes all financial and current workplan and reports; and the Fellows Management System containing details on each fellowship From GHFP’s perspective, it has been very helpful to have access to the USAID intranet and to receive copies of PDMS’ PHN sector-wide communications such as “Target.” b) What changes in the management relationship are needed or have been made to date to improve GHFP performance (e.g. communication between prime-and-sub-contractors, interactions between GHFP and USAID? Communication systems between the prime and subcontractor were set up in Year One. One change has been an increased partner presence in Washington, DC both to interact with fellows and work with PHI staff on team activities. No large changes are planned at this point though we remain vigilant that consistent contact and interaction remain a structured part of the program. Although data indicates that USAID staff has a generally positive view of the GHFP program and staff, our ultimate success with fellows/interns, professional 24 and organizational development, depends on GHFP having deep knowledge of USAID’s needs and challenges, strengths and weaknesses. It is important that program staff interact as much as possible to diminish perceptional differences and understand our client. GHFP will continue to seek this access including the OK to attend Office meetings. With the commencement of Year Three, GHFP, with active PDMS support, would like to make this happen. c) What aspect of the project should be closely monitored over the coming months/years? GHFP’s current monitoring and evaluation system sufficiently documents key aspects of the project’s results, based on GHFP’s Results Framework. But GHFP works within the ever-changing environment of a Federal agency preparing for changes to come from a new Administration. Environmental scanning of this situation and how GHFP will need to adapt to continue meeting PHN sector needs should be closely monitored over the next few years. As much as possible, GHFP would like to be proactive in responding to whatever its donor/client faces. d) What agreement modifications, if any, are required in order for the CA to act on identified recommendations? GHFP supports a modification to the agreement, as follows: • Pending approval from OAA/USAID on the request to amend the GHFP Marking Plan and Branding Strategy. The proposed changes are the use of the conventional PHI business cards by staff, partners, fellows, or interns (domestic or mission based) without the USAID branding, and changes to the Partners’ configuration. 3. Financial Management a) Given agreed upon budget, are funds allocated according to project priorities? The chart below reflects the total budget USAID approved Program Year One and Program Year Two of the work plan. The funds are allocated by key result area, and meets program activities and priorities except for the Diversity Initiative. The Diversity Initiative is one of the main program components, yet it did not receive funding. Per USAID’s request, PHI terminated its partnership agreement with UNCF-SP in Year Two. 25 Global Health Fellows Program PY 01 and PY 02 Budget by Key Result Areas July 3, 2006 - June 30, 2008 $32,876,549 Overseas Fellows 43% Washington, DC Fellows 48% Organizational Development 4% Diversity Initiative 1% Professional Development 4% b) Are project expenditures tracked and projected by line item and by country and element? If not, why? GHFP program obligations and expenditures are tracked and projected by budget line item, element, and by country as reflected in the Financial Status Report (SF0-269A), Baseline Report, and Quarterly Financial Status Report (refer to question C7). c) Is financial reporting useful for decision-making purposes, and submitted on time? GHFP extensively uses the financial reports described in question C7 in tracking and monitoring obligations, budgets, and expenditures by key result areas (recruitment and fellowship support, Diversity Initiative, professional and organizational development), by individual fellows, and by funding source (Core by elements, Field Support by countries). The reports are useful to GHFP and the client in financial analysis and forecasting to determine planned obligations for the next funding cycle. GHFP is incrementally funded and budgets are revised annually to reflect funds obligated to the program. As noted in the answer to question C7, GHFP submits the financial reports on time per scheduled due dates. d) Is the pipeline and ceiling appropriate (too large/too small) at this time in the project cycle? 26 As of July 1, 2008, the pipeline is $7,253,190. The estimated planned monthly burn rate of $2,000,000 is sufficient to support the program activities through September 30, 2008. e) What modifications to the budget are anticipated, if any and why? GHFP supports a modification to the budget, as follows: • Increase the ceiling amount of the program. The number of fellows has increased from 75 fellows, as originally proposed, to over 100 fellows in Year Three. • Pending approval from USAID/OAA on the request to reduce the cost share level required by the agreement from 2% to 1% of the federal cost (refer to question C8). 4. Level of Effort (LOE) Management a) Is the LOE tracked and projected in accordance with total project LOE? How well does the LOE track with financial expenditures? GHFP’s financial management system monitors and tracks the level of effort (LOE) by assigning an activity number to each key result area, fellow, and funding source. For the past two years, the percentage of LOE for fellowships (RA1) is higher; Diversity Initiative (RA2), professional and organizational development (RA3) is lower than originally projected in the GHFP RFA, as noted on the table. KEY RESULT AREA (% per RFA) % of LOE Fellowships (80%) 91% Diversity Initiative (5%) 1% Professional and Organizational Development (15%) 8% TOTAL 100% In fellowships, GHFP anticipated up to 75 fellows per year. In Year Two, the number of fellows increased and GHFP supported over 100 fellows. The Diversity Initiative, the prime focus of partner UNCF-SP work, did not receive funding. As a result, PHI terminated its partnership agreement with UNCF-SP in Year Two decreasing the LOE in this area. The professional and organizational development activities started gradually in Year One, 27 and expanded in Year Two with increasing requests for professional and organizational development from the GH offices and the field. b) Is the staffing appropriate for fulfilling project priorities? Is there an efficient and effective skill mix within the project to accomplish tasks? Previous fellowship programs were a fraction of the size of GHFP but the current program has actually fewer staff than several of the previous programs. The surprising number of requests for GHFP fellows has strained current service delivery systems which remain effective but at high personal cost and with more complexity than necessary. Also, a majority of Deputy Director time is spent on direct service or technical work (pipeline analysis and reporting, counseling fellows, leading workshops), leaving little time for management and leadership activities or special initiatives. Over the next year, GHFP intends to rethink its traditional structure to build capabilities required to support USAID effectively over the long term. Examples of potential changes include rebalancing the three Deputy roles, focusing several fellowship support activities, eg, travel reimbursements and allowances, closer to PHI’s HR and Accounting office but within GHFP; increasing our targeted recruitment capability and more fully tapping into university partners’ contributions to fellows’ career development.