Evaluation of the   Development Leadership Initiative (DLI):   Final Report Developed For: United States Agency for International Development (USAID) Under   OPM Contract No.: OPM‐07‐0004 Project Code: B80BS677 Submitted by: Federal Management Partners, Inc. 1500 North Beauregard Street, Suite 103 Alexandria, Virginia 22311 USAID DLI Evaluation 2 CONTENTS Executive Summary.......................................................................................................................................3 Introduction ..................................................................................................................................................5 Methodology.................................................................................................................................................7 Goal 1: The DLI program is meeting current and future staffing numbers and skills.................................11 Findings...................................................................................................................................................11 Recommendations..................................................................................................................................13 Goal 2: DLIs are provided with the training required to prepare them for a career with USAID...............13 Findings...................................................................................................................................................13 Recommendations..................................................................................................................................20 Goal 3: USAID is effectively using the Washington‐based time before the first assignment.....................21 Findings...................................................................................................................................................21 Recommendations..................................................................................................................................25 Goal 4: USAID is able to provide effective DLI assignments and has an effective process to make assignments. ...............................................................................................................................................26 Findings...................................................................................................................................................26 Recommendations..................................................................................................................................29 Goal 5: The DLI program has adequate agency capacity and infrastructure.............................................. 30 Findings...................................................................................................................................................30 Recommendations..................................................................................................................................34 Goal 6: DLIs are provided with adequate mentoring and supervision to position them for success within USAID ..........................................................................................................................................................35 Findings...................................................................................................................................................35 Recommendations..................................................................................................................................41 Goal 7: DLI retention is high........................................................................................................................ 42 Findings...................................................................................................................................................42 Recommendations..................................................................................................................................48 Conclusions.................................................................................................................................................50 Priority Recommendations..................................................................................................................... 50 Employee Teams.....................................................................................................................................50 USAID DLI Evaluation 3 EXECUTIVE SUMMARY   As of August 8, 2011, USAID has hired 809 of the 1200 DLIs anticipated at the start of the program in 2008. The agency has made a significant effort in this massive undertaking of doubling the Foreign Service workforce. While there are some areas for improvement, the program has seen many successes as well. DLIs generally seem pleased with the opportunity to work for USAID. Supervisors feel that the overall quality of DLI candidates is excellent, with many commenting that they are impressed by the DLIs’ technical and interpersonal skills.   In October 2010 USAID contracted with Federal Management Partners, Inc. to examine the factors that lead to program success as well as to identify any shortcomings and provide recommendations for improvement. USAID identified key program areas for study and evaluation. Therefore, the evaluation framework was structured to focus on seven program goals:    1. The DLI program is meeting current and future staffing numbers and skills.   2. DLIs are provided with the training required to prepare them for a career with USAID. 3. USAID is effectively using the Washington‐based time before the first assignment.   4. USAID is able to provide effective DLI assignments and has an effective process to make assignments. 5. The DLI program has adequate agency capacity and infrastructure. 6. DLIs are provided with adequate mentoring and supervision to position them for success within USAID. 7. DLI retention is high.   Each goal was assessed through focus groups, interview, and surveys. Additional documents were provided by the Agency where possible.   In the past two years, the DLI program has already made an impact on the culture of each mission it serves. DLIs are described as enthusiastic and hard‐working, and many missions have found these individuals to have a “good solid background in their technical backstops” and “superior interpersonal skills.”  Over 71% of field supervisors felt that DLIs had the right technical skills and more than 67% felt DLIs had the right interpersonal skills. Missions recognize that they will see the true benefit of the DLI program once the DLIs require less time to train and manage. Missions have been able to embed DLIs with the host government and with implementing partners, affording the mission connections with counterparts they wouldn’t otherwise have the staff or the time to foster. Many missions see the opportunity to bring some basic development work, especially related to monitoring and evaluation, back in‐house. Overall, the DLI program has offered missions an increased staff with the talent, energy and technical knowledge to carry out the USAID Forward Initiative. Many DLIs reported positive experiences in the program, with the majority reporting that they are satisfied with the work, training opportunities, and the organization as a whole. DLI survey respondents reported being most satisfied with coworkers (86%) and training opportunities (75%). Pay and benefits in the field were also rated relatively favorably, with 69% of respondents indicating satisfaction in this area.   89% of supervisors reported that DLIs have opportunities to perform meaningful work in their first‐tour assignment, and the majority of DLIs intend to stay with the organization for at least six years. USAID DLI Evaluation 4 An additional strength of the program has been tools and resources. Over half of DLIs and 70% of supervisors believe they have the tools and resources necessary to perform their jobs. Regarding Washington‐based assignments, 61% of DLIs were satisfied with office space, 74% were satisfied with training space, and 82% were satisfied with computer access.   Aside from these accomplishments, USAID faces some significant challenges for the remainder of the DLI as well as the agency as a whole as DLIs are mainstreamed into the workforce. The DLI was initiated without a sufficient level of strategic preparation and direction. There was, and in many cases, continues to be, very little in the way of formal structure, strategic direction, operational planning, and role clarity and guidance for DLIs, supervisors, and others related to the program. Where guidance and specified processes do exist, they are often poorly communicated. Although retention is currently high, increasing attrition could be an issue if these issues are not resolved.   Many evaluation participants reported problems related to communication, including unclear and conflicting information. The report identifies the following as key areas where communication can be improved:  Purpose of the program, roles and responsibilities, goals moving forward  Assignment process  Appropriate use of DLI training budgets and expectations for DLI training An additional recommendation is to take steps to ensure that supervisors have the experience and training to lead a group of very well qualified, very confident FSOs. This would include increasing the tools and resources available for supervisors, creating opportunities for sharing lessons‐learned across missions, as well as developing additional training and guidance in areas where information has been lacking to date (e.g., addressing performance problems).   Moving forward, training schedules should be better coordinated in order to minimize disruptions to mission operations. This includes front‐loading required training (while providing resources to ensure that information can still be accessed once new hires are in the missions), providing more advanced notice related to post‐Washington training schedules so that missions can plan accordingly, and focusing additional attention on effective on‐the‐job learning. Finally, in order to effectively execute the full range of improvements recommended in the report, a critical first step is to create employee teams and identify champions for each key action. USAID DLI Evaluation 5 INTRODUCTION Background The United States Agency for International Development (USAID) is an independent agency that provides economic, development, and humanitarian assistance around the world in support of the foreign policy goals of the United States. USAID is critical to emerging nations, working with countries recovering from disaster, trying to escape poverty, and engaging in democratic reforms. USAID plays a vital role in promoting U.S. national security through addressing poverty, one of the main causes of violence. For many years, the USAID Foreign Service Officer (FSO) workforce declined. Rather than focusing on technical leadership in the development community, FSOs, out of necessity, became internally focused program managers. With the President’s National Security Strategy placing a renewed emphasis on development and stating that it joins diplomacy and defense as one of the three key pieces of the national foreign policy, USAID needed to increase its overseas presence and its ability to engage host country leaders on the best policies, strategies, and programs to deliver U.S. foreign assistance. To increase its presence, USAID began to rapidly hire FSOs to rebuild the Agency’s technical base and provide stewardship of appropriated funds. In 2008, USAID embarked on the Development Leadership Initiative (DLI), a major effort with the goal of doubling the FSO workforce from 1200 to 2400 by FY12. USAID recognizes that a strong evaluation of its DLI program will help them shape the future and move forward effectively. Thus in October 2010 USAID contracted with Federal Management Partners, Inc. to examine the factors that lead to program success as well as to identify any shortcomings and provide recommendations for improvement. USAID identified key program areas for study and evaluation. Therefore, the evaluation framework was structured to focus on seven program goals:    1. The DLI program is meeting current and future staffing numbers and skills.   2. DLIs are provided with the training required to prepare them for a career with USAID. 3. USAID is effectively using the Washington‐based time before the first assignment.   4. USAID is able to provide effective DLI assignments and has an effective process to make assignments. 5. The DLI program has adequate agency capacity and infrastructure. 6. DLIs are provided with adequate mentoring and supervision to position them for success within USAID. 7. DLI retention is high.   Purpose of this Report Although the DLI program is near completion this evaluation and the resulting recommendations are applicable well beyond the DLI program. USAID is now facing a new workforce with nearly 60% of the workforce having been hired within the past three years.  As time goes on, and employees with more seniority begin to retire, this proportion of newer workers may increase. Acknowledging the long term applicability will allow USAID to realize a greater return for the investment in the evaluation and the DLI program as a whole.  The purpose of this document specifically is to 1) summarize high‐level findings USAID DLI Evaluation 6 regarding the DLI Program, and 2) provide an action plan for how USAID can make improvements moving forward. USAID DLI Evaluation 7 METHODOLOGY Evaluation Framework The initial stage of this study consisted of interviews with DLI stakeholders in order to gain background knowledge on the program as well as to identify relevant criteria for assessment. This information was used in the development of an evaluation framework, the data collection plan, the interview and focus group protocols as well as the surveys. In this initial information gathering stage, FMP talked to key contractors supporting the initiative (Encompass, MCT, Tai Pedro), seven coaches, two individuals in HR, five USAID senior leaders, and more than twenty DLIs. FMP also attended a DLI conference in Bangkok, and talked with many of the individuals enumerated above, as well as seven additional program stakeholders.   The purpose of the Evaluation Framework was to identify the appropriate scope of the DLI evaluation. In the development of the Evaluation Framework, it was determined that the evaluation would focus on the seven goals identified above. The Framework also identified several indicators upon which USAID’s progress towards those goals would be evaluated. This Framework was vetted by USAID DLI stakeholders before commencing data collection. The original framework included 3 goal areas with multiple sub‐goals under each. The final framework was developed in a stakeholder meeting on 4/15/11, during which it was decided that the evaluation would focus on those areas identified by stakeholders as high priority (shown below):     Specifically, the evaluation was based on seven program goals. 1.T h e D L I p r o g r a m is meeting current and    future staffing numbers and skills. 2.DLIs are provided with the training required to prepare them for a career with USAID. 3.USAID is effectively using the Washington‐ based time before the first assignment.   USAID DLI Evaluation 8 4.USAID is able to provide effective DLI assignments and has an effective process to make assignments. 5.The DLI program has adequate agency capacity and infrastructure. 6.DLIs are provided with adequate mentoring and supervision to position them for success within USAID. 7.DLI retention is high.   Data Collection Plan   The Data Collection Plan provided a table (based on the USAID DLI Evaluation Framework) illustrating all of the topics to be covered by the data evaluation and their corresponding data points. In relation to each of the seven evaluation goals, the Plan identified the relevant group or groups to provide input for each indicator along with target questions to be addressed during data collection. This plan served as the basis for the data collection, survey development, and interview and focus‐group protocol development. Surveys, Focus Groups, and Interviews Surveys were developed to gather information from DLIs, field and Washington‐based supervisors, backstop coordinators, HR personnel, Washington senior staff, coaches, and others associated with the DLI program. Separate online surveys were created for each group. Copies of each survey can be found in the Data Compilation Packet. Field focus groups were conducted via video conference in order to gather input from individuals working in various missions. Participating missions included: Pretoria, Cairo, Dakar, Lima, Nairobi, and Almaty. Focus groups were conducted with DLIs, supervisors, mentors, and Foreign Service Nationals (FSNs). Mission Directors, EXOs, and other senior mission staff joined the supervisor groups based on availability. The purpose of these focus groups was to collect perceptions and feedback related to the various evaluation goals. Fifteen focus groups were conducted, typically consisting of 8‐12 participants and lasting approximately two hours. Interviews were also conducted with DLI stakeholders in Washington, DC to supplement the focus group data. The evaluation team invited senior staff, bureau heads, backstop coordinators, supervisors, coaches, and DLIs based in Washington.   Survey, Focus Group, and Interview Participation Survey Focus Groups/Interviews DLIs 358 50 DLI Supervisors 150 50 Washington Personnel (backstop coordinators, coaches, Washington supervisors, bureau heads and senior staff) 18 31 FSNs 15 Mentors 15 USAID DLI Evaluation 9 Survey Demographics DLI Survey Participants by Backstop Percent Count 02/94 ‐ Program Management/Project Development 17.7% 63 03 – Executive Officer 4.8% 17 04 – Controller 7.3% 26 10 – Agriculture 7.6% 27 11 – Economist 4.8% 17 12 – General Development 0.0% 0 21 – Private Sector 8.5% 30 25 – Engineer 2.0% 7 40 – Environmental Officer 5.9% 21 50 – Health Officer 9.9% 35 60 – Education Officer 8.2% 29 76 – Crisis, Stabilization, and Governance Officer 12.7% 45 85 – Legal Officer/Advisor 0.8% 3 93 – Contracting Officer 9.9% 35 DLI Survey Participants by Class Percent Count DLI 1 3.5% 12 DLI 2 2.9% 10 DLI 3 3.7% 13 DLI 4 2.3% 8 DLI 5 6.9% 24 DLI 6 7.2% 25 DLI 7 6.6% 23 DLI 8 8.9% 31 DLI 9 5.2% 18 DLI 10 4.0% 14 DLI 11 6.3% 22 DLI 12 4.9% 17 DLI 13 4.9% 17 DLI 14 2.6% 9 DLI 15 6.3% 22 DLI 16 9.2% 32 DLI 17 8.9% 31 DLI 18 5.5% 19 USAID DLI Evaluation 10 *Note About The Data Included In This Report When not otherwise noted, findings presented in the report represent common themes observed across both focus groups/interviews and survey responses. Quotations used are representative of other comments and are used to illustrate the common opinions and themes of the data.   DLI Survey Participants by Current Location Percent Count Washington 42.5% 139 Other 57.5% 188    DLI Survey Participants by Level Percent Count Junior 86.6% 305 Mid‐Level 13.4% 47 USAID DLI Evaluation 11 GOAL 1: THE DLI PROGRAM IS MEETING CURRENT AND FUTURE STAFFING NUMBERS AND SKILLS   Findings In the past two years, the DLI program has already made an impact on the culture of each mission it serves. The resulting upsurge in young people has brought in “fresh perspectives, new technologies, and openness to doing things in new ways.” These “uniformly talented people” are enthusiastic and hard‐working, and many missions have found these individuals to have a “good solid background in their technical backstops” and “superior interpersonal skills.”  Over 71% of field supervisors felt that DLIs had the right technical skills more than 67% felt DLIs had the right interpersonal skills. Figure 1. Field Supervisor responses to DLI Skills (109 responses) The majority of field supervisors (63.3%) felt their mission currently had the right number of DLIs. Very few missions reported needing more than 4 DLIs in upcoming years. Figures 2 and 3. Field Supervisor responses to hiring needs (62‐98 responses) “We’re really impressed with the hiring on the DLI front; 98% are great recruits, top‐ notch, right skill set, right attitude; this is something HR has gotten right.” USAID DLI Evaluation 12 While training and management challenges around a rapid increase in staff have introduced obstacles, missions have experienced an increased field presence and improved coverage with their government counterparts and implementing partners, necessary Agency priorities that were previously stymied by a large workload and a smaller staff.    Missions recognize that they will see the true benefit of the DLI program once the DLIs require less time to train and manage. They have been able to embed DLIs with the host government and with implementing partners, affording the mission connections with counterparts they wouldn’t otherwise have the staff or the time to foster. Many missions see the opportunity to bring some basic development work, especially related to monitoring and evaluation, back in‐house. Overall, the DLI program has offered missions an increased staff with the talent, energy and technical knowledge to carry out the USAID Forward Initiative.   USAID’s Office of Human Resources (OHR) began the recruitment process for the DLI with the announcement of positions in virtually all technical and stewardship backstops. An open and continuous recruitment process was originally employed. However, in FY 2009, the Agency began to control the opening of requests for applications to be consistent with the need for various backstops to meet recruiting targets. One element in determining the functional categories against which these officers should be hired is the Consolidated Workforce Planning Model (Model), reflecting input from the 2012 Vision Statement and operating units (OUs) across the Agency. In August 2008, an Agency‐wide data call requested that each OU validate the Model’s assumptions, projecting personnel needs over the course of the next five fiscal years. The Model is now updated every year to reflect new data as well as new Agency initiatives and priorities that affect staffing allocations and priorities. While this workforce planning effort engages each mission, many field personnel report a lack of clarity on how hiring targets are set.        “Our new DLI… has given us some capability to do things we haven’t done before…a totally different skill set than what we’ve had historically in the mission.” “We have a much greater field presence now… Our DLIs come back and write blog entries, take pictures of the site. It helps us do a better job of monitoring and evaluation. At a smaller mission, DLIs are probably doing regular work. Ours are getting some time to do extra things.” “Staffing levels have allowed us to lead the way on almost every Agency initiative, as well as our own initiatives.” USAID DLI Evaluation 13 Recommendations       In an effort to mitigate confusion regarding how hiring targets and decisions are made, it is recommended that USAID continue to facilitate mission input into hiring needs to accurately reflect staffing needs and capacity issues while also better communicating why and how hiring plans are created.  Additionally, USAID should focus on building the mid‐level pipeline to prepare USAID for future retirements and other upper‐level attrition. Part of building this pipeline is most certainly providing the onward assignments and activities Junior Officers (JOs) need to develop their skills.    GOAL 2: DLIS ARE PROVIDED WITH THE TRAINING REQUIRED TO PREPARE THEM FOR A CAREER WITH USAID    Findings Individual Development Plans     Several DLI participants mentioned the Washington‐based training on how to write and utilize individual development plans (IDPs) as especially helpful. Comments about how the IDP process could be improved were somewhat variable, although a general theme was that they should more clearly link to the competencies needed to successfully launch into an FSO position and to gain tenure. Developing plans while still in Washington, before new hires fully understand what they will be doing in their new position or what is required for success, may drive this issue.   Language Training Many of the DLIs who participated in the evaluation raised language as an area they would like improved or where they would like additional training. The figure below presents the results of the DLI survey questions related to language training.   Figures 4a‐4d. Language Training Survey Responses:   All DLIs (346‐352 responses) vs. Most Recent 5 Classes (107‐111 responses) “We have to better evaluate individual training needs according to a person's background and their backstop.” USAID DLI Evaluation 14 The responses of the most recent five DLI classes (DLI 14‐18) were examined separately. While the majority of individuals in recent classes were not yet assigned to posts, the comments of those who had received training showed no appreciable difference from those of the DLIs in earlier classes.   Of note, 40% of DLIs surveyed reported that they did not have the language training necessary to be successful in their current assignment, whereas only 30% believed they had the necessary training. These results seem to be driven in part by DLIs in non‐language designated posts who indicated that they would feel better positioned for success in their posts if they had some basic skills in the local language, such as a fast course in conversational language. Those who had no ability to speak the local language with recipients and beneficiaries felt this disconnect reflects badly on USAID and the U.S.   Many argue that basic language skills are a valuable diplomatic tool. As USAID moves towards a model of working more closely with host country partners, this is an issue that should be examined further.   Other concerns raised were that the 3/3 designation is not sufficient in some languages. While not all respondents indicated the language in which they encountered this issue, French, Portuguese, Spanish, and Russian were specifically mentioned in relation to this concern. For posts where language skills are used every day, follow‐up training at post may be very valuable.   Many respondents were concerned that they were forgetting the language they took for tenure purposes, as they do not have the opportunity to practice these language skills at their new post. Moreover, a few respondents were not sure they would ever be assigned to a country where the language they were tenured in (and received training for) would be spoken. This raises questions of the value of training dollars spent on language training for the sake of fulfilling the tenure requirement. Many of these individuals felt that a better investment would be to hold off on the language training until the next assignment.   “Although I passed the exam, I do not feel confident in my ability to speak and comprehend at the level needed to carry out responsibilities at post. During a recent TDY, I was keenly aware that a 3/3 does not meet the needed level of skill for conducting business.” “I understand the budgetary constraints, but at the same time it seems wasteful for the DLI program to enroll JOs and mid‐ levels in 6‐9 months of Spanish or French in order for them to meet the language tenure requirement, only to send them to non‐ Spanish/French‐speaking countries.” “The way we are perceived in the field depends on whether we can communicate with people or not, and it makes a big difference when one can communicate in local language. People have more respect for you and the organization.” USAID DLI Evaluation 15 Finally, the content of the FSI language classes was raised as a concern by many who studied the language of their post country. Participants commented generally on the emphasis on State Department terminology, such as those related to consular or political duties, and the absence of USAID terms, such as those needed for discussions about contracts or with beneficiaries, and specific USAID and development vocabulary. The formality of the language taught was mentioned repeatedly in these comments, with DLIs noting that, typically, to perform the work of USAID, a more conversational, day‐ to‐day style of language is more useful. While many respondents did not mention specific languages, French, Russian, and Spanish were identified as problematic in this area.   Training Content and Quality As would be expected based on the sample (40% of whom are still in Washington), many of the DLIs’ comments about training content and quality were directed towards initial, Washington‐based training. These types of comments primarily fell in one of two categories: comments about level of information/detail and comments about the trainers.   Although most comments did not mention specific trainings by name, a few DLIs indicated that the Gender training is one that could improve by adding more details on how to incorporate gender into work activities and how it occurs in practice at USAID.   It may be worth noting from a few comments that Program Management training was also discussed as an area where classroom training should be better tied to real work. A few respondents pointed out that trainings in this area (e.g., Programming Foreign Assistance, Project Design and Management, Managing for Results) are repetitive and overly focused on soft skills such as teamwork. One respondent suggested that PDM should devote more time to issues such as how to effectively comply with adequate planning requirements and other pre‐obligation requirements. Note that AOTR/COTR is an exception in this area, and has developed a reputation as a very valuable course.   DLI and supervisor respondents also expressed preference for trainers who are current employees rather than contractors or retirees. This was discussed at length in the focus groups and by approximately twenty DLIs. FSOs who can provide better context and discuss more practical matters are strongly preferred. Trainers who understand the material from a practical perspective as well as a teaching perspective could add considerable value.   Although budgetary and workload concerns may make utilizing current employees as trainers prohibitive, missions should, at a minimum, have the opportunity to provide input in training content. This would enhance the quality of information, ensuring that skills and knowledge trained are applicable to work in the field and that content is based on relevant experience and reflects the newest innovations.   Cross training was another common concern among DLIs. Many DLIs indicated that they would like more “We don’t get as much from training provided by non‐USAID trainers; instructors really matter.  Some instructors are teaching from old policies (even when they are required trainings), while some really know their stuff; some trainers don’t know the answers.” “As a coordinator, not once was I asked to take a look at anything coming out of Washington for the junior officers… We need to have mission input on anything that Washington develops, especially the civil service in HR.” “Classroom trainings are not taught by people who actually work for USAID, and tend to be theoretical and starry eyed about how things work. Practical grounding in programming methodology is missing.” USAID DLI Evaluation 16 opportunities to take courses outside of their backstops. In general, this would enable new officers to develop a basic understanding of other program areas and allow better cross‐functional integration. Making technical training available to non‐technical backstops would allow them to better perform their duties. For example, contracting officers should understand the program descriptions and SOWs they read in order to provide sound business advice.  And, while backstop specific technical training is now mandatory for Program Officer DLIs, efforts should be made to ensure that Program Officers who entered before this requirement have opportunities to take this training.   In terms of access to training, the one week Financial Management course was discussed frequently. Many DLIs have had difficulty getting into this course due to infrequent offerings. Increasing the frequency with which this course is held and/or offering it outside of Washington would ensure that those who need it are able to receive this training. In addition, many respondents asked for more financial training, particularly related to the financial systems, the full budget cycle, project management, and accruals. Additional training should address financial management in complex environments, given the number of DLIs who are expected to go on to CPCs following the completion of their DLI assignment.   Figure 5. Mission Supervisor Responses on Training (87 responses) The figures below displays additional DLI feedback on training quality. Figures 6a and 6b. DLI Responses (329‐331 responses) USAID DLI Evaluation 17 When comparing these items by class, it was found that training to prepare for DLIs for supervising FSNs and USDHs shows a decline from DLI1‐6 (38% positive) to DLI7‐12 (24% positive) to DLI13‐18 (18% positive).   On a more positive note, the number of DLIs who believe the program is preparing and positioning them to gain tenure shows an increase from DLI1‐6 (58% positive) to DLI7‐12 (68% positive) and DLI13‐18 (74% positive). It is difficult to say whether this is a difference in perspective or a true difference in the experiences of early and later DLIs. Access to training has also generally improved over the course of the program, with 73% of DLIs in recent classes stating that they are satisfied in this area, compared with only 60% of the early classes. Again, this may be a result of different perspectives as many of those in more recent classes are still in Washington and therefore have relatively easy access to training.   Figure 7. Mission Supervisor Responses (87 responses) Disruptions Many supervisors expressed concerns about the considerable time DLIs spend in training and the disruption this causes. Several mentioned that excessive formal training and rotational requirements keep DLIs from becoming fully engaged in meaningful work.  In order to ensure that graduating DLIs are fully prepared for their onward assignment, it is important that a balance be struck between training and hands‐on experience with the work he or she will be expected to perform. Many supervisors would like to see as much training completed before coming to post as possible, in order to limit disruptions. While some individuals are concerned that front‐loading training would mean trainees would lack the appropriate context to fully benefit, there are things USAID could do to diminish this concern (discussed further in the Recommendations portion of this section). Supervisors and DLIs also note that, because the training schedule is not available 4 quarters into the future, planning for necessary disruptions (e.g., by scheduling training during slow periods) is currently very difficult.   USAID DLI Evaluation 18 Figure 8. DLI Responses (314‐328 responses) Figure 9. Mission Supervisor Responses (79‐83 responses) On‐the‐Job Learning Concerns raised about both the quality and content of formal training as well as the disruptive nature of these requirements (discussed above), both point to the importance of on‐the‐job learning opportunities. This evaluation addressed on‐the‐job learning directly. As can be seen in the figures below, both DLIs and their supervisors expressed a strong preference for on‐the‐job training, with 63% of supervisors indicating that the amount of on‐the‐job training should increase. Further, 83% of DLIs indicated that they find on‐the‐job learning opportunities to be more valuable than classroom learning, although some noted that this depends on quality of supervision.   USAID DLI Evaluation 19 Figure 10. Mission Supervisor Reponses (84 responses)                Figure 11. DLI Reponses (329 responses)     Figure 12. DLI Responses (330 responses) Some DLIs believe that being labeled a trainee comes with a ‘stigma’, and keeps them from being given substantive assignments that fully utilize their skills. Two of the four DLI focus groups discussed this viewpoint at great length.   DLI as well as Supervisors also commented on this concern through the surveys. DLIs, and all new hires, do best when given opportunities to apply the skills and experience they bring to the job, learning how to utilize these skills in the context of their new position. Most DLIs want more opportunities to learn on‐the‐job, through meaningful assignments and opportunities to learn from more seasoned FSOs. In some cases, DLIs feel they are given very menial tasks to complete, from USAID DLI Evaluation 20 which they learn very little. This is likely driven, in part, by time constraints on the part of supervisors. Although on‐the‐job learning can reduce time spent in formal training, it does require time on the part of supervisors and other mentors to ensure that new hires have the guidance they need to be successful in new assignments. One supervisor commented that supervisors have received mixed messages from HR regarding the work DLIs should be doing.  According to this supervisor, initially supervisors were told that “DLIs were trainees, could not be deputies, could not even be COTRs and [they could not] have any official responsibilities”.  Although the more recent messaging is that “DLIs are excellent, fully qualified professionals,” it has taken supervisors some time to adjust this new mindset in their assignments.   Recommendations Regarding language training, we recommend that particular attention is paid during the annual review of language requirements to the issues raised in this evaluation. Those involved in the review (i.e., Mission Directors, Principal AID Officers and Regional Inspectors General) should be encouraged to have conversations with the FSOs in the missions to assess the language requirements of the job as it exists currently. If the required level of language is truly not sufficient to perform job duties effectively, this should be addressed in the review. Another issue raised in relation to the language program was that many DLIs did not receive training in the language of their host country, and felt their diplomatic abilities were compromised as a result.  However, as specified in HB 28, Chapter 4, Foreign Language Program, “FS employees are expected to acquire a courtesy level knowledge of the host country language even if their particular assignments do not require a job‐related language proficiency.” The finding that many DLIs do not believe they’ve had opportunities to develop this courtesy level of knowledge suggests that USAID may benefit from a renewed look at how resources are used to support this requirement. The answer may simply be an issue of raising awareness of the resources already available to new officers in this area, and re‐emphasizing USAID’s commitment to ensuring FSOs have the skills required to be effective representatives of USAID and the U.S. (in this case, a minimum knowledge of the local language). Content and quality of formal training was also discussed in the evaluation. Recognizing that staffing shortages do not permit actual USAID employees from developing and delivering training, we recommend two strategies for increasing the quality and usefulness of training content, while maintaining the current model of relying on contractor support for training: 1) When developing new courses, assign working groups in the missions who will provide SME input to developers and review content for relevance and applicability.   2) When delivering courses, identify portions that can utilize FSOs. One strategy is to finish the course with a panel discussion, where trainees can ask questions about the content to FSO panel members, and panel members can share their experiences. These could be conducted entirely through VTC technology, and would not require advance preparation on the part of the panel members. A final area of concern was the disruption caused by, and difficulty with, planning for formal training requirements, once DLIs are in the missions. We agree with supervisors who suggest that as much training as possible be completed before going to the mission. However, for this to be effective, considerations are needed. Firstly, the time new officers spend in Washington before their first directed USAID DLI Evaluation 21 assignment is generally increasing. This means increased time to forget material learned because it is not applied. Further, much of the training is less meaningful without first‐hand awareness of the mission context. Thus, training should be interspersed with TDYs for new hires who will be spending more than two or three months in Washington. Once in the field, new hires should have access to all of the training materials and content in order to review information learned previously. Some organizations choose to record their trainings and post them on internal websites, in order to ensure that ‘refresher’ training is always available. Because some training necessarily requires more significant field experience, however, some amount of travel and formal training will likely always be required during the first directed assignment. Offering e‐training wherever possible is one way to minimize this burden (see Best Practices below). In planning for face‐to‐face sessions, the key will be to provide a training schedule well in advance. At the time of the evaluation, this was not being done for the DLI program. All DLI training courses that are required should be scheduled one calendar out (4 quarters in advance) for planning purposes. Backstop‐specific courses should be offered on a standard, yearly schedule.   Finally, Missions should continue to emphasize on‐the‐job training. Learning skills on the job prepares new hires to perform at full capacity and reduces the need for formal training. Rotations should be kept short and held on a specified schedule, so that supervisors are able to identify meaningful work that the new hire will be able to do with minimal disruptions.   GOAL 3: USAID IS EFFECTIVELY USING THE WASHINGTON‐BASED TIME BEFORE THE FIRST ASSIGNMENT    Findings Both DLIs and their Washington and mission based supervisors understand this reality. However, the comments from DLI’s and from the Washington and mission based supervisors indicates that the Washington assignment time would benefit substantially from better organization and planning, better definition of assignments and assurance that every minute a DLI spends in Washington assignment is preparing him or her to both understand and appreciate the headquarters functions and their contribution to USAID’s ability to accomplish its mission.  However, the execution of Washington based assignments requires substantial restructuring if the agency and its DLI hires are to realize maximum benefit “I think that the focus of first tour DLIs should be on on‐the‐ job training, actually doing the work they were hired to do.   Short rotations are helpful, but should be kept to short period or linked to short tasks (i.e., participate on a TEC or project review, etc.) Formal/classroom trainings should be conducted before they report to post.” “I feel like rotations in offices should be more concrete and defined.  In my rotation experience there hasn't been any structure which makes it harder to benefit from.” “Right now it's hit or miss, dependent on luck, timing, and personal ambition.   Opportunities for actual hands‐on learning in your backstop do exist, but they aren't inevitable and not always easy to find.” USAID DLI Evaluation 22 from this very substantial investment of time and effort. Only 11 percent of DLI’s “strongly agreed” that their Washington assignment was “a meaningful learning experience.   Another 41.1 percent “agreed.”   So while 52.1 percent had a positive experience, 47.9 percent did not.   When comparing classes, there has been a marked improvement in reports of meaningful Washington assignments, with the majority of recent DLIs responding positively in this area (69% of recent DLI classes), compared with only 44% of early DLIs. This suggests that USAID needs to continue to improve the way in which Washington assignment are identified and assure that the Washington assignment time provides the kind of learning experience that is essential to both the future success of the agency and to retention of DLIs. Figure 12. DLI Survey Reponses (326 responses) Even fewer DLI’s – 43.9 percent agreed or strongly agreed  ‐  felt that the Washington experience “provides valuable opportunities and exposure to on‐the‐job learning opportunities.”   32.9 percent “disagreed or strongly disagreed” that the Washington based experience was valuable.  Upon comparing classes, there are distinct differences of opinion between earlier and later classes. 64% of recent DLIs stated that their time in Washington provided valuable on‐the‐job learning opportunities, compared with less than a third of early DLIs (27%). This finding may be driven by a combination of intentional improvements to the Washington time and the relative availability of on‐the‐job learning experiences for the more recent classes who are stationed in Washington for longer periods of time than the earlier classes. While USAID has made some improvements here, given the importance of DLI’s understanding what important contributions are made by the Washington based staff and functions of USAID, this is an area that requires ongoing improvement. USAID DLI Evaluation 23 Figure 13. DLI Survey Reponses (328 responses) DLIs reported mixed feelings on whether the time spent in Washington prepared them for the field.  This is partly due to the mixed experiences of DLIs in Washington.  Another factor is the DLI expectations and assignments in the field. More recent DLIs described being somewhat more prepared for their field assignments than early classes did (38% and 21%, respectively) although this could also be a difference in perspective due to the fact that the majority of recent classes are not yet in the field. Figure 14. DLI Survey Reponses (322 responses) The good news is that USAID did an excellent job in providing office space, appropriate training facilities and computer access for DLI, thus assuring that these new employees had the basic tools needed to learning and achieve the work assigned.  Housing presented more of a challenge to the majority of DLI’s.   Of the 322 who responded to this portion of the survey, 72 percent were either neutral or dissatisfied with housing arrangements.  This suggests that USAID needs to reevaluate its policies and procedures for providing housing during Washington based TDY, particularly during longer term TDY’s. • 61% of DLIs were satisfied with office space   • 74% of DLIs were satisfied with training space   • 82% of DLIs were satisfied with computer access   USAID DLI Evaluation 24 • 28% of DLIs were satisfied with housing   • Satisfaction with housing has seen some decline in recent classes (only 20% positive compared to the 31% positive ratings of earlier classes), likely due to longer Washington tours. Figure 16. DLI Survey Reponses (322‐329 responses) Mission supervisors were queried on the appropriate length of Washington based assignments, the timing of these assignments in relationship to intensive orientation, the appropriateness of a desk officer assignment and whether mandatory HR training should be completed while in Washington.    Mission supervisors also strongly supported the idea of a Desk Officer assignment. 76.4 percent indicated that a “2‐3 months” would be a useful investment of DLI time in Washington.   Desk Officer assignments can be powerful training/learning experiences, but also this is a critical position for field success. Providing DLIs with such an assignment would enhance both the Washington assignment experience as well as provide substantial understanding of what life and work at the mission would entail. Figure 17. Mission Supervisor Survey Reponses (86 responses) “It's always helpful to understand the big picture. The Desk Officer has to relate to all parties, and it would provide a more well‐ rounded understanding of the issues of the region. We don't get area studies for where we will work, and this would be like a practical application of area studies that our State Department colleagues receive.” USAID DLI Evaluation 25 Figures 18 and 19. Mission Supervisor Reponses (86 and 89 responses)   Figure 20. Mission Supervisor Survey Reponses (91 responses) Recommendations Should DLIs move into a DC‐based assignment immediately instead of attending the intensive orientation program?   Would it be useful for DLIs to spend 2‐3 months as a Desk Officer to one of the countries in the region they will be going to? USAID DLI Evaluation 26 While the number of new hires will be limited going forward, it is important that USAID make some modifications to the time new hires spend in Washington.   With the backlog of DLIs currently in Washington, the time new hires spend in Washington is likely to increase and it is extremely important that new hires have a positive first impression. Because there was such variability in the Washington experience, it is recommended that USAID improve the coordination and structure of Washington time to ensure meaningful assignments and ownership of tasks.   Along with this, it is recommended that USAID better prepare Washington supervisors on mentoring, meaningful assignments and new hire expectations. DLIs would benefit from greater coordination between Washington and Field supervisors. This coordination would allow DLIs to better understand and prepare for their upcoming field assignment. Additional mission‐specific guidance for DLIs would also enable them to feel more prepared for their initial assignment prior to arriving at post. GOAL 4: USAID IS ABLE TO PROVIDE EFFECTIVE DLI ASSIGNMENTS AND HAS AN EFFECTIVE PROCESS TO MAKE ASSIGNMENTS.    Findings Since the assignment process is the method through which employees identify and are selected for specific jobs in USAID, it is a key factor in employee engagement, retention and productivity.  By its very nature the world wide assignment process is complex.   Factors to be considered include areas of expertise/backstops, nature of assignments available, language requirements, regular or CPC posts, as well as family considerations such as spousal employment, housing, schooling, medical facilities and a host of other issues.  These traditional factors are now further complicated by anticipated budget cuts and other factors which can potentially reduce assignment options.  This complexity requires clear and frequent communications to DLIs and others seeking assignments. From results of the surveys, interviews and focus group discussions, the initial assignment process generally worked well at the beginning of the DLI recruitment program.  However, as more classes were added, and additional assignments were required at post, a variety of issues began to surface, including the fact that securing NSDD38s for mission positions became increasingly difficult. The difficulties result from a variety of issues.  In some posts, the issues of available physical space, inability to persuade the mission of the need/importance of approval and a variety of other post specific reasons add to the complexity. However, the end result is that the wait time for initial assignment to post has risen from a few weeks after the completion of orientation to up to six to twelve months for assignment.  Those caught in these initial assignment difficulties are still assigned to Washington. Some have TDYs to posts or program evaluation assignments around the world.   Once assigned to post DLIs have had a wide variety of experiences.  Forty‐nine percent of DLIs responding to this question “strongly agreed or agreed” that their field assignments allowed them “to develop skills in my area of expertise.”   However, that means 51 “I liked that we learned 5 weeks in orientation where we were going.  Now DLI are languishing for up to a year without knowing their onward posting. It has contributed to unhappiness for some DLI and a hardship on them and their families.”    “Many missions did not know we were coming prior to the announcement going out (this has implications for mentorship and supervision as well as morale). Would have been more beneficial for all involved if list of DLI's background areas and skill needs would have been made available so that missions with specific needs could fill them. USAID DLI Evaluation 27 percent were neutral, had some disagreement with the statement or were not sure.   Just over 47 percent thought there was “sufficient opportunity to perform meaningful work in my field assignment.” Only 40 percent “strongly agreed or agreed” that “I am assigned projects that I can see through from beginning to end.”  This particular issue was the subject of substantial discussion in the focus groups by DLI as well as their supervisors and mentors.  For many training or other TDYs away from post disrupted their ability to receive meaningful assignments that could be completed while at post.  Given the general high quality of DLI hires, the responses to this series of survey questions by DLIs and their mission supervisors is troubling, and has potentially significant  impact on longer term retention of talent.    Figures 21. DLI Survey Reponses (228‐307 responses) Figures 22. DLI Survey Reponses (223‐236 responses) When comparing DLI classes, more recent DLIs responded less positively (58% positive compared to earlier classes at 76‐79% positive) when asked about whether their field assignment had helped them to develop skills, which likely reflects the relatively shorter time they have spent in their assignments. While the percentage of DLIs who received their assignments within four weeks of completing the orientation was consistently high in early classes, a noticeable drop off occurred in DLI 15 and continued through DLI 18 (the most recent class for which data was collected). The time to receive the initial USAID DLI Evaluation 28 assignment has increased significantly with the most recent five classes. Figure 23. DLI Survey Reponses (87‐118 responses) Mission supervisory responses are consistent with DLI responses regarding the initial assignment process.  They are also troubled by the experience to date with the onward assignment process and its implications for being prepared for a Foreign Service career in USAID and gaining tenure. Figure 24. Mission Supervisor Responses (97‐100 responses) Figure 25. Mission Supervisor Responses (100‐101 responses) Like the concerns surrounding the initial assignment process, the onward assignment bidding process is equally concerning. DLIs report that the process is not clear and guidance varies from person to person.    “I am concerned about my next bidding opportunity in 2012 (for 2013 transfer), when there will be a huge number of DLI bidders and possibly very few positions available.   Expecting that competition will be intense.” “The fall 2011 bidding assignment process was very dysfunctional, with DLIs bidding on positions that did not exist or that were knowingly going to be canceled. This is hugely demoralizing, builds distrust, and creates selfishness in individuals.” USAID DLI Evaluation 29 Figure 26. Mission Supervisor Responses (95‐96 responses) There are a high number of neutral responses for many of the questions in figures 24  ‐ 26. While the exact reason is unknown, it may be due to a lack of knowledge or insight from mission supervisors on some issues. For example, one might respond “neutral” to the question related to field locations receiving consistent and adequate guidance from State because some supervisors may not know what information other locations receive.  It may be that some supervisors felt that these questions required too much conjecture for them to answer appropriately.    Figure 27. Mission Supervisor Responses (100‐102 responses) Recommendations The majority of the issues identified appear to be based in a lack of foresight, planning and clear communications about the assignments process.  USAID has nearly doubled its Foreign Service Office core in approximately three years.  Some of the confusion and lack of understanding of the assignment “Listening to the DLIs who have been bidding for the first time, it is clear they are frustrated and don't understand the process.  At post we do our best to help guide them through the process, but even we are at a loss at times to explain the reasons for difficulties in obtaining assignments.   I believe the frustration for many is at such a level that we are losing the good DLIs.”   USAID DLI Evaluation 30 process arises from the sheer volume of individuals applying for assignment and the number of positions which must be available to accommodate all of the new hires. Without continued attention to the onward assignment process (including providing clear communications about the assignment process and assuring that assignments are substantive and meaningful), most of the effort invested in the hiring and training of DLIs could be dissipated if these individuals choose not to stay with USAID because of the variety of assignment issues. Therefore, it is recommended that USAID focus on improving the transparency and communication related to the assignment process (both initial and onward). Enhanced transition support would also help to alleviate concerns and ambiguity on upcoming assignments.   Additionally, 34 DLIs commented on the difficulties related to the tandem couple process. USAID would benefit from an improved tandem couple process, especially with the high number of new hires going to CPCs. GOAL 5: THE DLI PROGRAM HAS ADEQUATE AGENCY CAPACITY AND INFRASTRUCTURE    Findings The figures below summarize mission supervisor responses regarding capacity and infrastructure. As can be seen, the majority understand their role in the program (70%). However, all other areas show room for improvement, without only about 50% responding positively.   Figures 28a and 28b. Supervisor Responses (94‐100 responses) USAID DLI Evaluation 31 Communication and Guidance Respondents acknowledged that the DLI Reference Manual is a useful resource for clarifying expectations regarding IDPs, the typical two‐year tour, required trainings, and other program components. Missions also appreciate the flexibility in determining how the program will be administered, recognizing that each DLI and each Mission is different. In general, supervisors in the missions do not see a need for additional formal structure or guidance from HQ. Instead, they would appreciate HQ playing a more active role in information sharing, specifically regarding best practices that have been learned in the field. HQ could do some more facilitating of these exchanges, (e.g., distributing practices that they have received, coordinators meetings, etc.). Any additional guidance or structure should take into account the lessons learned in the missions, as this is currently perceived as a disconnect. One area where there was considerable ambiguity is whether DLIs should be viewed as full‐time trainees or an extra set of hands, and what the appropriate balance between the two should be. However, it is unlikely that a single best formula exists, as the balance will necessarily be influenced by the opportunities and constraints associated with the individual missions. Evaluation participants had clearly put considerable thought into determining the best solution for their particular situations.   Confusion around policies and practices is more prevalent in missions who do not have an in‐country USDH EXO. In some cases, when there is not an EXO, personnel duties are delegated to others who do not have the expertise or time required to perform these duties effectively. Both supervisors and DLIs describe the EXO role as critical. DLIs commented in particular on the support EXOs provided in preparing to go to post, as well as in interpreting policies and working through HR issues. As USAID moves to consolidate these services with the embassy, the agency should be careful to ensure that the current levels of support are maintained, clarifying who these roles and responsibilities will fall to. This will require clear policies that both State and USAID agree to at the DC level, which are clearly communicated to the field.   Strategic Planning Missions require additional guidance on the role of DLIs going forward. Specific   questions include:    What trade‐offs will be made to pay fixed staff costs in the face of static or decreasing budgets?    To what extent are DLIs meant to replace USPSCs?    What is the expected career path for DLIs following their first tour? What is expected of them during their second tour?   Although survey and interview respondents universally acknowledged the importance of addressing the first question, as can be seen below, most mission supervisors were unaware of any plans in this area. “We have a very detailed plan for roles and responsibilities within the mission that we created.” “[The EXO] was the only person at the Mission who bothered to reply to my questions before I arrived at post. I couldn't have made the move without him.” USAID DLI Evaluation 32 Figure 29. Supervisor Responses (92 responses) The latter two questions speak to the previous points regarding communication and guidance, as one supervisor points out, “All that might be needed for a supervisor is ‘at the end of 2 years, your DLI needs to know everything there is to know about being a foreign service officer in X backstop’ and then we could design a specific program that would ensure that there is a proper mix of specific trainings and activities so that they are well‐prepared when they leave.” A forward‐looking plan, including defined career paths, will not only ensure the successful transition of current DLIs into the organization, but would also ensure missions have the information required to prepare DLIs to be successful in their next roles.   Tools and Resources Across the survey respondents, 56% of DLIs and 70% of supervisors stated that they have the tools and resources necessary to perform their jobs. DLIs who felt they were missing resources largely mentioned intangibles, such as quality supervision, meaningful assignments, additional staff, and mentoring. Supervisors largely echoed these sentiments, stating that more staff resources and time are needed to provide effective supervision, and requesting training on topics such as leadership, supervision, and coaching.   Figures 30a and 30b. DLI (321 Responses) and Supervisor Responses (99 Responses) “At this stage, I would prefer to see the folks in Washington focus on figuring out a career path for the DLIs as they move forward.” USAID DLI Evaluation 33 Human Resources Although the survey did not contain any formal questions related to HR, HR was consistently mentioned as an area that negatively impacts DLIs’ experiences with the agency. Because of changes that have occurred in HR during the course of the program, we reviewed comments from the most recent five classes (i.e., DLI 14‐18) to determine whether there were systematic differences in these comments, compared to DLIs who had been with the agency longer. However, complaints were consistent across the DLIs. Processes are perceived as overly complicated, communications are unclear and inconsistent, and HR specialists are seen as unfriendly.   It should be noted that HR announced a new HR Helpdesk in April 2011, in response to customer concerns about responsiveness.  As part of this roll out, the following standards were announced:    Staff members in OHR will acknowledge requests within 2 business days;    90% of cases will be resolved within a week of receipt;    Any inquiry that cannot be resolved within this time frame will be elevated to a manager, and a “HR actions (or lack thereof) affect entire families, not just the employee.” “The support from Washington HR is a constant challenge that jeopardizes our ability to be successful.” I have the tools and resources I need to do my job. (DLI Response) USAID DLI Evaluation 34 timetable for resolution will be communicated to the client;    When a case ticket is closed, an e‐mail will be sent informing the customer of the action taken as well as an automated notice of case resolution from the HR‐HELPDESK. At that time, customers will receive a survey requesting rating and comment on the service provided. As FMP was not made aware of these changes until September 2011, we were not able to assess this effort directly. As a result, it is not clear whether the HR customer service issues communicated in the May‐July timeframe (i.e., through the focus groups and survey) were related to experiences with HR before or after the new standards were announced. No participants mentioned the new standards, although one respondent indicated that the new tracking system has been an improvement in this area. Travel Authorizations were a major area of concern, as well as the larger processes of getting assignments and getting to post. All three key groups included in the evaluation (i.e., DLIs, mission supervisors, and Washington personnel) expressed frustration with inefficiencies and lack of support from OHR/FSP to get Travel Authorizations done and funded in a timely manner, evidencing the wide‐ spread nature of this problem.   Regarding getting to post, one supervisor noted that DLIs are given the State Department’s "It's your Move" book to guide them through this process. However, because State and USAID have different processes for getting to post, this resource adds to the confusion.   Several DLIs, from both early and later classes, cited HR as a factor that would influence their decision to stay with the agency, indicating that the effects of difficulties in working with HR are having a profound effect on those individuals who entered through the DLI program. Many respondents indicated that they would like additional information on how to effectively work with HR.   Recommendations As increasing numbers of DLIs graduate from the program, adequate capacity and infrastructure will become increasingly critical to ensuring workforce effectiveness. USAID has an opportunity to respond to these challenges proactively, by thinking strategically about the various logistical challenges the changing workforce presents. We recommend USAID develop and communicate a Strategic Plan for the remainder of the DLI Program. This plan would also document lessons learned from the DLI, so that best practices developed in response to the DLI can be incorporated into future USAID practices. The strategic plan should:   • Include clear goals, roles, and intended outcomes going forward • Develop a communication strategy and plan for key messages • Incorporate into USAID Human Capital Plan In any strategic planning effort, effective communication is fundamental. Over the course of the DLI, missions have developed tremendous expertise related to the challenges and solutions, and should be consulted early in the effort. In addition, when the plan and messages are ready to be released to the organization, a communication strategy may be useful. Over the course of the evaluation, our team became increasingly aware of the confusion that exists in the agency as to what guidance and tools are currently available to support the DLI. In order to maximize the value of these efforts, the agency will be well‐served to ensure that a strategic and orderly plan is in place for raising awareness of the new plans and resources.   USAID DLI Evaluation 35 Throughout the evaluation, we also heard a need for Standard Operating Procedures (SOPs), which clearly delineate steps in key processes, as well as responsibilities for each step, time frames, key dates, etc. In some cases, it’s possible that these SOPs exist, but that DLIs, supervisors, or others with DLI program responsibilities are not aware of their existence or their location. As a result, we recommend that USAID create an online database for USAID SOPs (particularly around practices related to new hires), FAQs, sharing best practices, etc. If USAID is satisfied that such a resource already exists, attention should be turned to communicating about the availability of these resources, as well as soliciting ideas for their improvement. Such resources were not explicitly included in the evaluation, as our team was not made aware of them. It is possible that one reason users do not believe these resources exist is because the information currently contained on these sites is not the ‘right’ kind of information. A communication campaign that includes a forum providing feedback could help. In addition, information sharing is a two‐way street. In addition to HQ communications to the field, Missions could benefit from greater opportunities for sharing best practices (including DLI conferences) which would allow missions to leverage lessons learned and examine some of the inconsistencies. As noted in findings, many DLIs who participated in the evaluation commented on their positive experiences with the EXO at their location, in particular in the early stages of their career. Increased awareness and availability of online information would help ensure that missions without EXOs have clear understanding of how to access the necessarily information, seek support, and get prompt answers to questions about personnel issues. The evaluation findings also point to a need for improved delivery of HR services and communication, with a surprising number of individuals (over 45) stating that frustrations with HR will factor into retention decisions. While these are issues that HR has been working towards addressing, we recommend continued efforts in this area including further streamlining of processes, better delineation and communication of POCs for key processes, roles and responsibilities, and continued efforts related to standards for responsiveness. GOAL 6: DLIS ARE PROVIDED WITH ADEQUATE MENTORING AND SUPERVISION TO POSITION THEM FOR SUCCESS WITHIN USAID   Findings Due to the massive increase in the number of new hires, the need for supervision has also drastically increased in a short period of time.  As such, there is room for supervisory improvement. The quality of supervision varies from person to person where some DLIs report receiving adequate assignments, communication, and direction from their supervisors while others do not.  Some find that messages vary between supervisors, which causes more confusion among often tight‐knit classes.     DLI Perceptions of Supervisors Many of the DLIs who participated in the evaluation raised supervision as an area that could be improved.  In most areas, there were no appreciable differences between Washington and Field supervisors. The only exception was in responses to “You have to make the relationship with your supervisor or mentor work, but some supervisors don’t have a clear vision of what the DLI program is, or how to utilize a DLI.” USAID DLI Evaluation 36 the item, “My supervisor clearly and effectively communicates expectations regarding performance/work assignments.” While 61% of Washington‐based DLIs responded positively to this item, only half (51.7%) of DLIs at post indicated agreement. It is unclear whether this difference is due to differences on the supervisors’ behalf (e.g., time constraints, priorities, supervisory skills), or whether this is simply a reflection of a more ambiguous nature of work in the missions. The table below presents the results of the DLI survey questions related to supervision.      Table 31. DLI Responses to Supervision Survey Questions:   Washington (65‐111 responses) versus Post (170‐179 responses) Strongly Agree Agree Neutral Disagree Strongly Disagree My supervisor effectively facilitates working relationships among team members. In Washington 22 (23%) 39 (41%) 19 (20%) 12 (13%) 3 (3%) At Post 40 (22%) 69 (39%) 32 (18%) 23 (13%) 15 (8%) My supervisor assigns responsibilities and coordinates work effectively. In Washington 16 (17%) 37 (39%) 26 (27%) 12 (13%) 4 (4%) At Post 37 (21%) 62 (35%) 29 (16%) 31 (18%) 18 (10%) My supervisor clearly and effectively communicates expectations regarding performance/work assignments. In Washington 14 (14%) 46 (47%) 22 (23%) 13 (13%) 3 (3%) At Post 36 (20%) 56 (31%) 36 (20%) 31 (17%) 19 (11%) My supervisor is an effective mentor to me. In Washington 19 (17%) 36 (32%) 28 (25%) 21 (19%) 7 (6%) At Post 33 (19%) 54 (31%) 35 (20%) 26 (15%) 25 (14%) I am encouraged to participate in mentoring relationships with my supervisor or others. In Washington 18 (17%) 43 (39%) 28 (26%) 15 (14%) 5 (5%) At Post 26 (15%) 70 (41%) 34 (20%) 25 (15%) 15 (9%) It is clear how I can improve my performance, based on the feedback I receive from my supervisor. In Washington 7 (9%) 28 (36%) 20 (26%) 15 (19%) 8 (10%) At Post 21 (12%) 50 (29%) 38 (22%) 35 (21%) 26 (15%) I receive my performance review in a timely manner. In Washington 9 (14%) 29 (45%) 13 (20%) 9 (14%) 5 (8%) At Post 28 (16%) 78 (46%) 28 (16%) 21 (12%) 15 (9%) Mentoring Mentoring relationships often focus on topics like USAID, lifestyle, and culture. Missions vary in amount of formalization they provide to mentoring (e.g., offering mentor training, communicating expectations).  Some Missions provide both an FSO mentor as well as an FSN mentors.  Others provide only one or the other.  Less than half of the DLIs reported that their supervisor was an effective mentor. The most common complaint was that the supervisor did not have time for mentoring. In some cases, DLI mentors were Mission Directors or other senior level personnel who also did not have time for mentoring. DLIs did comment that FSO mentors outside of one’s technical area add value. 75% of “Mentoring is less about the technical fields and more about culture and USAID.” “Supervisors are simply too busy to effectively mentor a new officer” USAID DLI Evaluation 37 supervisors say they are actively encouraged to participate in mentoring relationships while only 54% of DLIs say they are encouraged to participate in mentoring. Figure 32. DLI Responses (317 responses) Performance Management Performance management is clearly an area for improvement. Only slightly more than 50% of DLIs reported understanding the performance management process and receiving the performance review in a timely manner. Understanding of the performance management process is better for earlier classes (66% positive compare to later classes 47% positive), which is not surprising as they would have more experience with this process.  Only 36.7 percent of DLIs were clear on how their performance can be improved based on supervisor feedback.  This highlights a definite need for stronger communication between supervisors and employees as well as greater communication around the performance management system in general. Figure 33. DLI Responses (307‐308 responses) “Supervisors don’t give assignments or correct mistakes. If they give assignments, it’s what the supervisor doesn’t want to do. You write objectives at the end of the year when you can match up some sort of objective with what you’ve done. It’s poorly managed on all levels.” USAID DLI Evaluation 38 Furthermore, performance management discussions are not occurring as frequently as they should be. Less than half of DLIs reported receiving feedback at least a few times a year and less than 30% are working with their supervisor to develop or revise the performance plan at least a few times a year. Figure 34. DLI Responses (298‐301 responses) Ideally, all supervisors would be held accountable for their performance as a supervisor. However, only 64.4 percent of supervisors reported that their performance appraisal includes a metric related to supervision and over 16% of supervisors did not even know.   Figure 35. Supervisor Responses (90 responses) USAID DLI Evaluation 39 Supervisor Training While 47%‐61% of supervisors report receiving adequate supervisory training on various topics (shown below), the responses are not overwhelmingly positive and indicate a need for improved supervisory training, specifically in the area of retention decision. The majority of supervisors do not feel they have received adequate training on how to make retention decisions.   Figure 36. Supervisor Responses (87‐90 responses) USAID DLI Evaluation 40 Less than half of DLI supervisors (48.4%) feel that supervisory training is readily available.  However, the majority of supervisors are encouraged to participate in mentoring relationships.   Likewise, many (66.7%) report understanding the performance management process. This still shows room for improvement however as ideally all supervisors should understand the process through which they are evaluating employees.    Figure 37. Supervisor Responses (91‐92 responses) Coaching The intended purpose of coaching lacks clarity for both DLIs and many supervisors. There are concerns about the type of people used as coaches as well.  Over 50 DLIs commented that their coach provided little to no assistance. Further, an The DLIs’ “understanding is that coaches basically answer questions as necessary and function as advocates during assignment process. Therefore, they perceive that the coaches are meant to have a fairly limited role.” USAID DLI Evaluation 41 additional 12 pointed to the use of retired FSOs as a limitation on the potential utility of coaches, emphasizing that USAID is a very different agency today than it was in past decades. For example, these respondents felt that “retired FSO’s who are not fully aware of current processes” may not be the best choice for coaches.  Some DLIs reported the coaches being proactive and helpful where others reported not knowing what questions to ask and therefore “they are only so useful.” Recommendations In general, effective supervisors often have many years of experience in supervising others. It is not a skill that comes naturally to most. Therefore, supervisory training is critical. With 50‐60% of DLIs reporting positive supervisory experiences, USAID should make an effort to leverage the experiences of what is working well to help others. With many supervisors reporting supervisory training is not readily available, there needs to be increased training opportunities as well as improved options. The importance of supervision must also be communicated throughout USAID and all supervisors should have a performance metric related to supervision.  If it is not clear that supervision is a priority, supervisory training becomes a “check‐the‐box” function or an activity that isn’t completed when there are competing priorities. Supervisory training can be made more readily available through e‐learning used in conjunction with communities of learning so that supervisors can share knowledge and best practices. E‐learning is a convenient way to reach many people at one time. It can be conducted synchronously (which can allow for interactive features), asynchronously (to allow supervisors to take the training on their own schedule), or a combination of both.  Supervisory training can also be improved to cover a broader range of topics.  Some examples are shown below. Additionally, USAID will need to develop and communicate a clear process and decision points related to retention. Performance management is a critical issue which will continue to be of great importance to USAID as time goes on. Supervisors could also benefit from additional training on dealing with poor performers and guidance in making such critical decisions.   Coaching and mentoring are both best practices and USAID is to be commended for utilizing these techniques. However, new hires would benefit from an improved clarity of the purpose of coaching. Many DLIs would like coaching to continue after they leave for post, but whether this occurs is inconsistent across DLIs. If coaching is intended to continue after new hired leave for post, more coaches will need to be added as coaches are reported to be overloaded as it is. Once the role of the coaches is clarified, additional consideration needs to be made to evaluate the particular coaches in order to determine if they are the best people for the role.   Mentoring should be communicated and formalized at each mission so that all new hires may benefit. Missions should be encouraged to share success stories and best practices so that other missions may learn from their experiences. “I met a supervisor in DC who laid out a strategic vision with regard to the office where he was working. He played a very important role in helping me accomplish success; helped structure daily goals, kept track of progress and thereby I got more and more out of him.” USAID DLI Evaluation 42 GOAL 7: DLI RETENTION IS HIGH   Findings Intentions to Stay The evaluation survey asked current DLIs how likely they were to stay with the organization for 1‐3, 4‐6, 7‐10, and 11+ years. The results can be seen in the figure below. Based on these responses, 89% are likely to remain for 1‐3 years. Somewhat fewer, 72%, are likely to remain for 4‐6 years. The most marked decreases are observed when DLIs are asked about their intentions to stay for 7‐10 years (37% are likely to stay) and 11 or more years (30% are likely to stay).   Figure 39. DLI Survey Responses (270‐301 responses) In order to further understand the factors driving these intentions, the survey asked about several factors identified by USAID stakeholders:    Student loans  Job market    Federal government environment  Spousal employment issues  Other family/spousal concerns  Assignment process  Assignment location  Availability of leave without pay Most commonly rated as extremely or moderately important is the assignment process (86%), followed closely by assignment location (82%). Also among the top drivers were the federal government environment (76%) and other family/spousal concerns (74%).   USAID DLI Evaluation 43 Figure 40. DLI Survey Responses (312‐318 responses) The chief complaint regarding the assignment process was a perceived lack of transparency, mentioned in open‐ ended responses over 50 times. DLIs believe that the assignment process is driven by “who you know and how you play the game,” and “hallway reputations” rather than merit or performance.   Married DLIs and DLIs with children, particularly children with special needs, place particular emphasis on assignments in their decisions to stay with the agency. Stories about children having terrible adjustment problems circulate amongst parents and cause considerable concern. For a parent of a special needs child, access to services is a necessity in order to continue with the agency. Parents place significant value on flexibility to negotiate for assignment conditions that take their families’ needs into account. Similarly, many DLIs discussed the importance of an effective tandem assignment process and being able to serve in the same country as their spouse. While many DLIs acknowledge that periods of separation are a part of the job, repeated and frequent separations put considerable strain on couples and will likely impact commitment to the organization. Similarly, couples who may have been willing to be apart before having children often put a premium on being collocated as they add to the family.   Although some participants commented that repeated assignments to unaccompanied posts would affect their decisions to stay with the organization, the majority of participants were willing to serve in CPCs, as long as expectations for CPC service are clearly and consistently communicated, and as long as these expectations include flexibilities for FSOs with young children or other personal considerations. As can be seen in the figure below, roughly half of survey respondents stated that assignment to a CPC would have either no impact or a positive impact on their commitment to a career at the agency. “The stress of applying for a new job every few years through an opaque, chummy, networking process is not what I expected when joining USAID.” USAID DLI Evaluation 44 Figure 41. DLI Survey Responses (311 Responses) Opportunities for assignments within one’s backstop was also raised as a concern. For DLIs who are more committed to the work itself and their area of expertise than to the organization, a lack of opportunities within their backstop would likely be a key factor in attrition.     Across the board, survey respondents would like to see the assignment process improved, such that it takes into account the individual FSO and his or her individual priorities and commitments, both personally and professionally. The process should be administered and communicated in a way that promotes transparency and eliminates ‘behind the scenes’ dealings.   Satisfaction In order to gauge current levels of job satisfaction, the survey asked respondents to evaluate their satisfaction regarding several key facets related to employment at USAID:    The USAID organization  Pay and benefits (in Washington and in the field)  The tour assignment process  The work itself  Training opportunities  Co‐workers  Supervisors  Senior leadership DLI survey respondents reported being most satisfied with coworkers (86%) and training opportunities (75%). Pay and benefits in the field were also rated relatively favorably, with 69% of respondents indicating satisfaction in this area. Conversely, the tour assignment process was, by far, the source of the most dissatisfaction, with only 21% of respondents reporting that they were satisfied in this area. Pay “There seem like there will be fewer and fewer assignment locations for my backstop (EDU) but they continued to hire dozens of new people... When I joined, I hoped to have opportunities to serve in countries all over.” USAID DLI Evaluation 45 and benefits in Washington also showed room for improvement, with only 38% of DLI respondents indicating satisfaction in this area.   Figure 42. DLI Survey Responses(303‐319 responses) As discussed above (see Intention to Stay), dissatisfaction about the tour assignment process stems from dissatisfaction with the transparency of the process, and the prevalence of ‘backroom deals’.   Regarding pay and benefits in Washington, a chief concern was the perceived lack of parity with State Department counterparts. While those who received it appreciated the relocation/signing bonus, many survey respondents described spending much of or all of their savings in order to make ends meet during Washington‐ based training. The problem is exacerbated as new USAID officers attend language training alongside, and in some cases are married to, State Department FSOs and, as such, are well aware of the disparity in housing and per diem benefits (see text box: State Department Benefits).   Recognizing that many new officers take salary cuts to accept the position and/or have families, student loans, or other financial obligations, the financial realities of finding housing in DC present a considerable hardship to new hires. Uncertainty about the length of time that will be spent in DC stifles new hires’ ability to plan for these hardships or to sign more long‐term leases that may reduce some of this burden. Learning that these challenges are USAID‐specific and not faced by new FSOs at Department of State does not create a positive first impression of the agency.    “The salary is so low and the cost of living in DC is awful. It was especially frustrating to find out how State Colleagues get a subsidy and we do not.” USAID DLI Evaluation 46 The moderate amount of dissatisfaction with the USAID organization is likely driven in part by dissatisfaction with the tour assignment process. Another factor frequently cited in this area was HR. HR is discussed in more depth under Goal 5 (Capacity and Infrastructure) and, as mentioned there, at this time, it is unclear whether the new Customer Service standards are having an impact in this area. Some amount of dissatisfaction with the work itself was also noted. Some DLIs feel they are treated as interns and are not receiving the substantive work assignments they were expecting when they took a position with USAID. Some supervisors note that some new hires do not understand the nature of development work, and expect to be doing more ‘hands on’ type of work than is reasonable, given the USAID business model. This suggests that this problem may be mitigated, at least in part, by better managing expectations.   Satisfaction and Attrition Recognizing that not all facets of job satisfaction will factor into decisions to remain equally, the DLI survey also asked respondents to rate how important each facet is in their decision to stay or leave the agency.   As can be seen below, although Pay and Benefits in Washington emerged as an area where dissatisfaction is high (see ‘Satisfaction’ above), this was one of the less important factors in decisions to stay or leave, with only 60% of respondents indicating that it is moderately or extremely important in these decisions. Conversely, 87% say that Pay and Benefits in the field will be important in their decision. This is one of the areas where survey respondents are most satisfied (see ‘Satisfaction’ section, above).   State Department Benefits Permanent Change of Station (PCS) Lodging Program: Employees who join the Department from outside the DC area are eligible for the PCS Lodging Program. Lodging costs are paid directly by the program, so employees do not have out‐of‐pocket expenses for lodging. Lodging and Meals and Incidentals Per Diem: Employees are eligible for per diem during initial training.  Employees participating in the PCS receive meals and incidentals per diem only. FSOs training at FSI for 60 days or less receive 100% of the DC rate. After 60 days, the per diem is paid out at 50%; after 120 days, it is reduced to 25%.   Reflecting on the program’s success, the average ratings that participants give it on evaluations are in the mid 90s on a scale of zero to 100. Program participants say it is “the most family‐friendly program the department has ever produced” … “This program was a godsend,” said one participant. “I was able to take nine months of language without impoverishing my family.” Source: Mirabel, R. (2010, February). Money‐saver: PSC Lodging Program aids with D.C.‐area stays. State Magazine, 16‐17.   “It is frustrating to watch the same mistakes be made repeatedly.   Sadly, if we were to leave AID, which I hope we don't, part of the reason would be HR.” “I am here at USAID this late in my career to do meaningful work overseas, to feel like I am making a difference in my work.  So far, this Agency has failed to provide me with meaningful work, except for the TDY assignment I took.” USAID DLI Evaluation 47 Figure 43. DLI Survey Responses (315‐317 responses) In order to understand how satisfaction with facets and the importance of facets in decisions to stay relate to one another, we plotted the two factors against each other to create the retention matrix shown below.   Note: Satisfaction is considered ‘high’ when at least 2/3 of respondents report being satisfied and ‘low’ when less than 1/3 are satisfied. Importance is considered ‘high’ when the majority of respondents rate a factor as ‘extremely important’ and ‘medium’ when the majority of respondents rate the factor as ‘moderately important’.   As can be seen in the matrix, ‘Tour Assignment Process’ is identified as a priority area in this matrix, as this is an area that is highly important in decisions to stay with the agency, but also an area where satisfaction is very low (only 21% are satisfied).   Other areas that are highly important in decisions to stay are:  Satisfaction with the USAID organization (59% satisfied)  Satisfaction with the work itself (61% satisfied)    Satisfaction with supervisors (61% satisfied)    Satisfaction with pay and benefits in the field (69% satisfied) USAID DLI Evaluation 48 Retention Matrix: Satisfaction vs. Importance Importance Satisfaction High Medium Low Low  Tour assignment process Med  USAID organization  Work  Supervisors  Pay and benefits in Washington  Senior leadership High  Pay and benefits in the field  Training opportunities  Co‐workers Recommendations The tour assignment process emerged as the priority improvement area for retention. While the assignment process is discussed elsewhere in this report (see Goal 4), there are changes that can be made to how the process is managed and communicated that could significantly improve retention.   To the extent possible, policy should be formalized before the bidding cycle. This will involve anticipating the upcoming bidding cycle, as well as the strategic and operational challenges and realities of the agency. While the evaluation team does not have insight into what obstacles to this type of planning have arisen in the past, DLIs commented on the frustration that such changes cause them.   Editing and error checking of postings should occur before they are published. DLIs would likely feel much more comfortable with the process if formal communications about the bidding process did not include statements such as, “There are some lingering policy issues to be ironed out, hopefully before bidding begins.”   Perceptions that the assignment process is fraught with ‘backroom deals’ are likely the result of a lack of information regarding how assignments are made, as well as the emphasis on the importance of ‘networking’ in agency communications about the bidding process. Networking may be useful for helping bidders better understand the requirements of certain positions and/or to identify positions that are a good match; however, selection decisions should be based solely on merit. While ADS 436 “Foreign Service Assignments and Tours of Duty” provides a general sequence of events related to the assignment process, we could not locate any criteria for how decisions are made during this process. This was frequently discussed as an area of concern in focus groups, with the decision process often being compared to a ‘black box.’ Because assignments have considerable impact for FSOs’ personal and professional lives, it is not surprising that they would like more insight into how assignments come about. Bidders want assurance that the process that landed them in their current assignment was a fair “It would be helpful if assignment processes did not involve changes in the process during the process and could be accomplished on time so people know early where they will be going.” USAID DLI Evaluation 49 one, and this is not possible without knowledge of how these decisions were made.   Recognizing that issues such as frequent assignments to CPCs are issues USAID can do anything about, that CPCs that need assistance are a fact of the current global reality, USAID should address dissatisfaction in this area by better communicating expectations for CPC service beginning during recruiting. These expectations should be communicated consistently to FSOs during their service. If changes in these policies or expectations are made, these changes should be communicated immediately, clearly, and consistently. Communications should include the rationale for these changes, and how current and future FSOs will be affected.   As an example of the issues in this area, one focus group spent considerable time discussing the difficulty of getting clear guidance from HR regarding expectations for bidding on CPCs. Although ADS 466.3.2.2 states, “All bidders must include at least one critical and one priority position in their EBFs, as long as at least one such position in the backstop on which they are bidding is included as critical or priority, respectively,” at least one DLI was told to disregard the ADS, and that he was required to bid on a CPC position outside of his backstop. In this case, the DLI in question had less than 24 hours to identify a CPC to bid on. If it is required that bidders bid on a CPC regardless of whether or not there is a position in their backstop, the ADS should be updated to communicate this, and a communication should be distributed to the workforce to this effect. HR should never be in a position to tell individual FSOs whether or not to follow agency policies.   In addition, messages during recruitment should include how long new hires can expect to spend in Washington and the type of work they can expect to be doing in the field. This will ensure that new hires are able to plan for the implications of extended Washington tours (as applicable), and also help ensure that candidates do not join USAID with unrealistic expectations for the type of work they will be doing in the field. A number of supervisors who participated in the evaluation reported that DLIs often enter with the unrealistic expectation that they will all become mission directors and/or that being a mission director is a goal that every new hire should strive for. Communicating the realities of the career ladder and promotion potential for FSOs from day 1 would help in this area.   Although issues like leave without pay and spousal employment ranked behind assignment concerns in DLIs’ importance ratings, these issues were nonetheless raised by many respondents in their open‐ ended survey responses and focus group responses. Specifically, most referenced a need for continued and improved flexibility. The majority of new hires’ spouses will also be working professionals and, as such, support for spousal employment and flexibility to take leave without pay will become increasingly common factors in decisions to stay with the organization. ADS 480 “Leave” states that, for FS Career Candidates “LWOP of 80 hours or more in a leave year will be granted only for compassionate reasons (e.g., illness, personal emergencies).” Recognizing that LWOP policies are specified in State Department policy (3 FAH‐1 H‐3500), it may not be possible to make adjustments to these restrictions (e.g., to allow for maternity leave). In this case, USAID should investigate other flexibilities to take work‐life considerations into account, such as telework and flexible schedules.   “On three occasions, I have been told things in writing by HR that have subsequently turned out to not be true and have all had a direct impact on our finances.” “USAID needs to figure out what the foreign service of the 21st century looks like, rather than feeling stuck in a foreign service where a the officer's spouse is always a woman and is excited to be the CLO or an admin assistant.” USAID DLI Evaluation 50 CONCLUSIONS Priority Recommendations Clear communication and continued comprehensive strategic planning are key to the success of agency initiatives such as the DLI program. The lack of strategic planning has led many individuals to comment that “the DLI program is like a boat that is being built as it is sailed” and other similar analogies. At this stage, it is crucial that top program stakeholders come together to agree on a vision and plan for the program going forward and define   the appropriate role for DLIs, supervisors, coaches, and other stakeholders based on that vision and plan.   Another recurring theme in the evaluation was the need for more clear and consistent communications. Many evaluation participants reported problems related to communication, including unclear and conflicting information. By developing a Communication Strategy and Plan for key messages, USAID will be able to clarify who is responsible for communicating information, ensuring greater consistency and timeliness of messaging. Suggested areas for inclusion in the plan include:    Purpose of the program, goals moving forward  Roles  Assignment process  Appropriate use of DLI training budgets  Expectations for DLI training An additional recommendation is to take steps to assure that supervisors have the experience and training to lead a group of very well qualified, very confident FSOs. This should not simply take the form of increased training requirements for already overburdened supervisors. Rather, this would include increasing the tools and resources available for supervisors, creating opportunities for sharing lessons‐ learned across missions, as well as developing additional training and guidance in areas where information has been lacking to date (e.g., addressing performance problems).   We also recommend that training schedules are better coordinated in order to minimize disruptions to mission operations. This includes front‐loading required training (while providing resources to ensure that information can still be accessed once new hires are in the missions), providing more advanced notice related to post‐Washington training schedules so that missions can plan accordingly, and finally focusing additional attention on effective on‐the‐job learning which is the preferred training method for both FSOs and their supervisors.   Finally, in order to effectively execute the full range of improvements recommended in the report we recommend that a critical first step is to create employee teams and identify champions for each key action. This concept is discussed further below.   Employee Teams Employee teams should be created to carry out the recommendations provided in this document. FMP recommends that each team have 3‐5 members with an advisor who is a member of the senior HR leadership team. A senior agency stakeholder should introduce the employee team concept and put out a call for volunteers. The communication should include an estimation of the time commitment. In the initial meetings, team members will craft a team goal that will be included in individual development plans. USAID DLI Evaluation 51 USAID would benefit from the use of employee teams in carrying out the action plan by:  Empowering and engaging employees on issues of importance to the organization and the DLI program. Engaging employees to help implement program improvements will send a clear message that employee opinions matter to USAID and USAID is committed to its workforce.  Serving as a communication channel. Employee teams will be useful in getting information out to the organization and gathering input from the organization.  Clearly identifying champions for key program improvements. DLI EVALUATION FINAL REPORT APPENDIX: ACTION PLAN Goal 1: The DLI Program is Meeting Current and Future Staffing Numbers and Skills Initiatives Action Plan Better integrate workforce planning and hiring efforts Offer additional training on workforce planning tools and processes (e.g., the consolidated workforce planning model, hiring plan, etc.). Communicate why and how hiring plans are created. Goal 2: DLIs are provided with the training required to prepare them for a career with USAID Initiative Action Plan Involve SMEs from the field in future training courses When identifying the need for and developing new courses:   1. Form a working group of 3‐5 subject matter experts (SMEs) from the field. 2. Conduct interviews with SMEs in advance of course development and vet design document with group. 3. Involve SMEs in review of training content and solicit relevant examples and case studies for inclusion. 4. Identify whether asking SMEs to facilitate parts or all of the training would increase impact. Incorporate panel discussions with SMEs where possible.   5. Test courses prior to finalizing curriculum content and process. Minimize mission disruptions caused by training   Increase availability of required trainings in Washington and communicate expectations for completion before departure from Washington to post location.   Utilize ‘refresher’ e‐learning and online references to ensure that DLIs can revisit concepts learned in DC once they are engaged with work in the field. When training cannot be conducted in Washington, or for FSOs already in the field, offer virtual trainings where possible, particularly for required trainings. Develop fixed training schedules (4 quarters in advance) for planning purposes. Continue to monitor the cost/benefit (the amount of money being spent versus the amount of learning and application achieved) of training offerings.  As on‐the‐job training improves and becomes more consistent, the need for formal training may decrease in some areas. Enhance on‐the‐job training efforts Communicate the importance of on‐the‐job training. 1. Distribute guidance related to on‐the job learning expectations. Include targets for relative time spent in on‐the‐job opportunities such as rotations, shadowing, and developmental assignments vs. Initiative Action Plan classroom and other formal learning. 2. Solicit ‘success stories’ about particularly valuable on‐the‐job learning opportunities and distribute throughout missions. 3. Develop guidance and/or training for supervisors on creating meaningful developmental assignments (e.g., characteristics of successful assignments, appropriate amount of guidance). 4. Develop effective practices and guidance for rotations (ideal activities, quantity, duration, etc.). Create methods and opportunities for sharing knowledge (e.g., message boards and forums, regional conference calls, virtual ‘Program Chats’, FSO conferences) so that missions can learn from each other’s successes and challenges. Goal 3: USAID is effectively using the Washington‐based time before the first assignment    Initiatives Action Plan Improve coordination/ structuring of Washington time to ensure meaningful assignments and ownership of tasks Create clearer and more structured opportunities for meaningful Washington rotations:   1. Create tip‐sheets for new hires on activities they should engage in, people they should meet with, etc. 2. Continue Rotation Fairs, and communicate these events to new hires well in advance so that they can plan their schedules accordingly.   3. Provide guidance/templates for writing effective rotation descriptions. 4. Consider supplementing fairs with a rotation ‘job board’ that can be accessed at any time.   Form a workgroup that can investigate the feasibility of increased TDYs to the field for DLIs in Washington. Better prepare Washington supervisors Develop training and guidance for Washington supervisors. This should include:   1. How to be an effective mentor. 2. How to develop/identify meaningful work assignments. Formalize and communicate expectations for new hires’ time in Washington. For new hires who are expected to spend an extended period of time in Washington, emphasis should be placed on completing meaningful activities including:   1. A rotation as an assistant to the Desk Officer for the region the DLI will be going to. 2. TDYs to the field. 3. Responsibility for a complete deliverable. 4. Exposure to interagency processes. 5. Opportunities to build awareness of the agency and its business processes (e.g., the strategy development process and how it Initiatives Action Plan links to country‐level programs, the legislative process). Improve coordination between Washington and Missions Design a formal handoff process for supervisors, beginning as soon as the first assignment is known. This would include: 1. A conversation at the time of assignment between Washington and Mission supervisor to discuss: – The new hire’s training accomplishments to date. – Objectives for the new hire’s remaining Washington time, in light of the needs and expectations of the receiving mission. 2. A close‐out conversation between supervisors at the time of the new hire’s departure.   3. A conversation between the new hire and the new supervisor prior to departure. Develop mission‐specific guidance for DLIs to review as soon as they receive their assignment. This could include:   1. Culture and regional awareness resources. 2. Mission‐specific FAQs. 3. POCs for relocation logistics. 4. POCs for personnel issues. Goal 4: USAID is able to provide effective DLI assignments and has an effective process to make assignments. Initiatives Action Plan Improve transparency and communication related to the assignment process Continue to develop SOPs for the bidding process, eliminating ‘backroom deals.’ SOP should include:   1. Key points in assignment process. 2. Guidelines for communication with bidders, intervals for status updates, and responsibilities for keeping bidders informed about where they stand in the process. 3. After final assignment is announced, explanation of how the decision was made. Communicate and apply guidance consistently. Formalize CPC requirements (number, frequency, flexibility of timing, exceptions, bidding requirements, etc.) and communicate them consistently to FSO workforce. Improve tandem couple process   Identify where critical breakdowns in the tandem couple assignment process are occurring. Refine the process with an emphasis on ensuring: 1. Tandem couple assignments are processed in a timely manner. 2. Tandem couple assignments are distributed in a manner that is fair and transparent. Goal 5: The DLI program has adequate agency capacity and infrastructure Initiatives Action Plan Develop a Strategic Plan to address the remainder of the DLI and the challenges related to mainstreaming the DLI workforce Conduct a strategic planning exercise that results in a formal document for distribution among all program stakeholders. Formalize:   1. Program goals moving forward. 2. Roles and responsibilities. 3. Intended outcomes.   4. Measures/indicators of success. Develop a communication strategy and plan Develop a communication strategy and plan to increase the consistency of key messages. Identify key messages that need clarification or further communication. These may include messages such as: – Purpose of the program, goals moving forward. – Roles and responsibilities. – Assignment process logistics. – Budget and training. Improve overall communication methods so that messages are consistent between Missions, DLIs, and Washington. Centralize information Continue to maintain online repositories of DLI and USAID resources and guidance (e.g., http://inside.usaid.gov/HR/fso/index.cfm). 1. Standard Operating Procedures (SOPs). 2. Answers to frequently asked questions (FAQs). 3. Boards/forums for sharing best practices. 4. Points of contact for all key processes. 5. Case studies of DLI experiences in each mission. Topics may include: – Adjusting to life at post. – Working with FSNs. – Working with other agencies. – Managing work demands. – Making time for family. – Preparing family for move. Improve capacity for addressing personnel/HR issues Ensure that missions without EXOs have clear channels for seeking support and getting prompt answers to questions about personnel issues.   6. Communicate these channels to DLIs.   Continue to improve/refine the HR Customer Service Model, to improve HR support of the DLI program. An effective model will include: 1. HR POCs for all key DLI processes. 2. SOPs for all HR processes relevant to the DLI program. 3. FAQs on key DLI processes. 4. Roles and responsibilities of HR, DLIs, Supervisors, Missions, etc. Initiatives Action Plan 5. Qualitative and quantitative measures of success. 6. A step by step plan to implement solutions to the problems identified. Goal 6: DLIs are provided with adequate mentoring and supervision to position them for success within USAID Initiatives Action Plan Improve supervisory training   Create additional training opportunities for (DC and Field) supervisors.   1. Topics can include:   – The special opportunities and challenges in supervising new FSOs. – How to hold difficult conversations and manage poor performers. – Effective mentoring/coaching techniques. – Creating meaningful work assignments and effective on‐the‐job learning opportunities.   – Managing across generations/generational differences. – Fostering a supportive working environment. Create communities of learning that could be in combination with e‐learning. 1. Utilize discussion sessions, exchange of ideas, lessons learned, etc. Communicate the value of quality supervision Communicate that the agency places real value on the work and role of a supervisor, and provide time for supervision to take place. 2. Consider supervisory duties in workload expectations; include elements of supervision in performance management plans. 3. Develop incentives for enhancing supervisory capabilities. – Establish a committee to receive and evaluate recommendations of excellent supervisors. – Determine criteria upon which to evaluate excellent supervisors for award. 4. Recognize excellent supervisors. Some options include: – Non cash awards (certificates, plaques, letters of appreciation). – Time off awards.   – Cash awards (spot awards, special achievement awards). Determine and formalize the purpose of coaching Identify the objectives and timeframe for coaching.   1. Note: If coaching is intended to continue after new hires leave for post, more coaches will need to be added. 2. Develop guidance for coaches in terms of their role and key messages. Based on the determined purpose, consider whether value can be added by: 3. Utilizing coaches in the new hires’ individual backstops. 4. Utilizing current employees as coaches.   Communicate role of coaches to new hires so that they can use them Initiatives Action Plan effectively. Continue encouraging missions to develop mentoring programs and provide guidance about how to ensure success Develop guidance explaining the benefits of mentoring.   1. Identify missions where mentoring is working well. Develop best practices for other missions. Identify best practices in mentoring outside of USAID as well. 1. Mentor and mentee skills.   2. Appropriate length of relationships. 3. Characteristics of successful relationships. 4. How to match mentors and mentees. 5. How to structure relationships. Develop a clear process related to performance‐ based retention decisions Define and communicate critical decision points for formal retention strategies and decisions. Distribute and communicate clear guidance related to: 1. Performance management and expectations for rewarding high performers and managing poor performers. 2. Retention process and decisions – roles and responsibilities, processes, timelines. Goal 7: DLI retention is high   Initiatives Action Plan Improve communication of expectations, beginning during recruiting. Identify points in the recruiting process during which a realistic job preview can be incorporated.   Consistently communicate messages such as:   1. Expectations for CPC service, how many times they will go to unaccompanied posts. 2. Challenges faced by FSOs and their families. 3. How long new hires can expect to spend in DC. 4. Realities of career ladder/promotion potential. 5. The nature of the work performed by USAID FSOs. Improve transparency of the bidding process and explain how assignment decisions are made.   Continue to improve and implement work‐life policies and programs.   Continue/develop/improve family‐friendly policies such as: 1. Support for finding spousal employment. 2. Leave without pay flexibilities. 3. Language training for spouses. 4. Telework and flexible schedules. Communicate the importance of work/life balance.   Provide training and tips on how to navigate in the FS environment. Adjust benefits during Washington tour to more closely match those of the State Department.