1 RUSSIA TRAINING FOR DEVELOPMENT EVALUATION REPORT Submitted to USAID / Russia Prepared by: Ethel Brooks Field Technical Advisor USAID/Global Bureau Center for Human Capacity Development and John Jessup Training Team Leader USAID/Global Bureau Center for Human Capacity Development October 2000 2 Evaluation of Training for Development Project USAID/Russia Outline I. Introduction II. Purpose and Background III. Executive Summary IV. Observations and Assessment of Program Implementation A. USAID Staff B. External Contacts 1. AED 2. Former RTD Participants 3. U.S. Training Providers 4. Russian NGOs and In-Country Training Providers V. Summary and Recommendations A. Summary B. Recommendations 1. USAID: Multi Year Strategic Training Plan 2. DIHR: Two-Stage Funding and Competitive Process 3. Activity Managers: Training Design 4. AED: Training Proposals and Communication VI. Appendix A. List of Interviewees and Contacts B. Interview Notes C. Evaluation Scope of Work 10/27/00 3 I. Introduction Since the inception of assistance to Russia in 1993 the mission has placed strong emphasis on the further development of Russian professionals, most of which is done through short-term training. Much of the training has been carried out under two back-to-back projects, NET (1993-1996) and the more recent Russia Training for Development Project (RTD) (January 1997-April 2001). Through creative administration of these projects, 9,600 Russian nationals have completed training in the U.S., in-country or in a third country, and with only one exception, all returned to Russia at the end of their program. Of this number, 1,882 were handled under the RTD contract that was awarded to the Academy for Educational Development (AED) via a buy-in to the G/HCD Global Training for Development IQC. The RTD contract, covering the period from January 31, 1997 to April 2001, is managed exclusively by the Mission. In the three+ years of operation, this AED contract has produced significant results as attested by participants and other stakeholders. Now as the April 2001 expiration date approaches, the Mission is directing attention to a next generation of training implementation that will be based on lessons learned in RTD. Thus, the Mission requested an external evaluation of the RTD to supplement its internal assessment. II. PURPOSE AND BACKGROUND The mission identified two broad tasks to be completed during this evaluation. 1. To prepare a menu of recommendations for USAID’s design of future training programs (out of the known options or a new one). These recommendations are expected to be grounded in the realities of the mission’s key strategic objectives and altered levels of resources. The recommendations will also draw upon successful approaches used by other missions, USG agencies and donor organizations. 2. To assess and recommend ways to strengthen the training capacity of local training institutions. These are strongly interrelated tasks that seek answers to who, what, and how questions. In approaching the tasks it was necessary to interview a sample of knowledgeable sources. The task also involved a review of relevant program documents, including: the RTD Project Paper, the R4 document for FY 2000, the AED Delivery Order, AED Quarterly Reports, and earlier evaluation reports. III. EXECUTIVE SUMMARY Training in general, and specifically the Russia Training for Development Project, is making an important and strategic contribution to achieving the mission's programs 4 objectives. A significant number of participants are being trained in-country. The strength and number of Russian NGOs providing in-country training have grown over the years. However, there is still a need for U.S. training of Russian decision-makers and technical experts, especially in areas that are comparatively new emphasis for Russia including selected aspects of social sector reforms, environmental policy reform, democratic initiatives and business development initiatives. U.S. training will be needed to support key beneficiaries of technical assistance in these areas at least for the short to intermediate term. Interviewees across the board find that "seeing is believing" in convincing and demonstrating to Russians how markets, local democratic practices, social services, etc really work. Having examined other options, we find distinct values of preserving the current system for implementing training in the U.S. and other venues, with a few modifications that will be discussed in another part of the report. Considering Alternatives Overall, the mission's training management procedures and implementation structure are responsive. Activity Managers and SO teams generally agree that AED is performing satisfactorily, and they value the support that AED provides to their technical assistance contractors.1 Current System (TFD/DIHR/AED) SOTs/ Multiple TA Contractors Public Affairs Section (PAS) Familiar to, supports, desired by Activity Managers (see suggested modifications) Duplicative management and infrastructures would be required Removes candidate selection & training design decisions farther from the responsible USAID Act. Mgr Facilitates cross cutting/cross sector collaboration More difficult to maintain uniformity and standards, including compliance with external requirements (visas, HAC, etc0 PAS works in broader, more general sectors (music, arts, etc); USAID's training needs are targeted to specific objectives Facilitates uniformity and quality control Requirement of multiple procurement actions to amend existing TA contracts More difficult for the Mission to maintain program accountability and controls Responsive management by competent in-house staff Unlikely that U.S. training would be continued without the necessary supports provided by a training contractor Rigorous front end needs assessment is usually not incorporated in PAS selection procedures Makes efficient use of staff time and resources Aggregated add'l costs for expanded TA contractors' services (to directly plan/implement trg) could equal or exceed ED costs USAID and PAS programs respond to different mandates Admin cost (approx 30%) is reasonable. Even greater cost efficiencies are envisioned under START (successor IQC) . OBSERVATIONS The Mission’s competitive process for training proposal selection was found to be an imaginative way of getting activity managers to think through their U.S. training 1 Note: Activity Managers suggested some modifications for inclusion in designing a successor contract. They would like an expanded pool of training providers rather than repeated use of the same ones. Suggested modifications are elaborated in the "Recommendations" section of this report. 5 needs. In preparing proposals they use a standard DIHR-developed template that ensures uniformity and equity in the review process, as well as uniform attention to issues of best practices in the training design. The time and thoughtfulness that training staff devoted to developing this product is just an example of their resourcefulness and dedication to their work. Strategic thinking occurs generally in reaction to the annual list of proposals that define the areas of choice, and when mission wide budget reviews force consideration of U.S. training. However, the difficult task of linking training to attainment of IRs and SOs still requires more rigorous planning and direct involvement at the SO level. Overall, information gained through internal and external contacts support the conclusions that training is effective and generally well carried out. a. Activity Managers - attach value to RTD but would like to see the competitive process as a more holistic complement to the technical assistance they provide, which includes in-country training under their direct control; they can’t count on U. S. training until a proposal is funded and approved; - are pleased that the AED contract spares them the burden of logistical support and processing participants for U.S. training; - in several cases wish for more interactions with AED or its training providers over course content and structure before the training provider is “locked in.” b. Trainees - are generally pleased with AED’s training arrangements and the course content; - are making effective use of the skills acquired during training; - some wish they had a greater understanding beforehand of the course content and learning objectives, to be sure the course is meant for them; - accept readily the responsibility of sharing course content with other Russians; - sometimes request more homogeneous trainee selection in courses; - frequently ask for follow-on activities: post-training seminars, internet networking in Russia (and NIS). c. In-country training providers - those interviewed appeared highly skilled as trainers and providers of follow￾on services; - desire greater involvement in needs assessment, course and follow-on design with activity managers.2 2 A Multi-Year Strategic Training Plan (MYSTP) can be used as a vehicle for involving local training providers in needs assessment, course design and follow-on. To address the concerns of equity, the implementing contractor (AED) can use the internet to post excerpts from the MYSTP that includes results desired from 6 d. Russian NGOs and other beneficiaries of training through AED - are consistently grateful for the USAID-sponsored training received; - are satisfied with AED’s job of administering training; - regret that follow-on is not universally given to reinforce course value; - would like to see courses designed in a sequence, not as stand-alone events. IV. INTERVIEWS AND ASSESSMENT OF PROGRAM IMPLEMENTATION This section provides more details of information collected from USAID Strategic Objective Team Leaders, Activity Managers and other technical staff, the Academy for Educational Development’s headquarters and local office staff, training providers, and Russian nationals who participated in the training activities. What did we learn… From USAID Activity Managers and other staff? Appropriately, participant training resides in the mission’s cross-cutting objective since it contributes to the overall successful implementation of other USAID-funded activities. Although there are no discrete SO indicators for training results, the Mission has identified some general outcomes that are expected from its investment in training.3 All trainees should: ·have relevant new skills and knowledge that directly relate to the achievement of the development objectives in the Mission's Strategic Objectives and that will enable them to enhance their leadership role and or advance in their work; ·have established potential long-term linkages with individuals or institutions in the U.S., third countries or Russia; ·understand and appreciate the value of democratic consensus-building and participatory decision-making; ·in case of U.S. training, complete their programs with a better understanding of the U.S. and be committed to sharing their new perceptions of the U.S. and their new skills and attitudes with their colleagues, friends and family upon returning to Russia. In responding to the question: "what do you consider a training success?" Activity Mangers generally stressed skill acquisition and attitudes that catalyze change within training. Broad accessibility to internet addresses the issue of equity among training providers. Those interested could propose ideas and activities for achieving desired results with the understanding that their input does not substitute for the competitive proposal process to select a training provider, and does not offer them priority consideration. 3 Russia Training for Development (RTD) Project Program Summary Report - Narrative Section for the period 2/1/99 - 3/30/00 7 the trainees' sphere of influence. They believed that AED-arranged programs embraced most of the above listed expectations. Most Activity Managers attach value to RTD, and generally agree that the contract with AED fills an important niche in supporting their program needs. Activity Managers appreciate the predictability of training support services provided by AED, which would be costly and duplicative if provided by their respective technical assistance contractors. All of the SOs have had at least one or two of their RTD training proposals approved over the two-year period of the internal competitive process. Even so, as noted by some of the Activity Managers, the process still leaves unmet needs in some areas. They would like more assurance that submitting training proposals will result in a pay off, and would like to see a higher budget level for training, thereby increasing the chances of having their proposals approved. They also voiced a desire for added flexibility in selected areas such as program costs and training providers. Several cited a need for more involvement in negotiations between AED and training providers. Some Activity Managers voiced expectations of AED that go beyond the current contract such as more assistance with recruitment. It may be useful to include recruitment among the menu of services requested in subsequent Delivery Orders. The overall sentiment of Activity Managers is that using the current central contract offers advantages that should be retained in future training implementation, with suggested modifications, including provisions for more structured coordination between AED and TA contractors. From U. S. Training Providers The consensus among training providers in the U. S. was that AED provides the basic information needed to design responsive training programs. Information is timely and accurate. In 4 out of 5 cases the actual training dates were consistent with the originally planned program beginning dates. AED was diligent in notifying the training providers in the few instances program start dates were delayed due to unavoidable circumstances. They found AED staff to be accessible and helpful throughout the training period. Most would like to have the training budgets increased in order to add enhancements to the programs including sites in more than one region of the U.S. to give the participants more exposure and broader basis for comparisons. From local training providers, NGOs and other external contacts Meetings with a sample of training providers provided information that should be useful as the mission updates its list of potential local training organizations. Most of the persons interviewed were attached to local, sector-specific USAID-sponsored NGOs including: · the environment (ECOLINE and the Institute of Natural Resource Management) · civil society/social services (Professionals for Civil Society and Project Harmony, or · business (the Academy for Management and Markets with its network of business training centers for Russian businessmen). 8 These NGOs use training as a part of the technical assistance that they provide. Based on our observations, they take seriously the task of disseminating new ideas to decision-makers across the country, especially in the fields of environment and Western business practices, which are comparatively new to Russia. Quite commonly the training workshops are linked with a follow-on activity through the medium of networking between trainees, thereby increasing income generating possibilities for the NGO. In conducting seminars the Russian training expertise is usually home-grown, i.e. provided by NGO staff members or sometimes former seminar participants, who are occasionally sent to trainer instruction courses and often work in conjunction with expatriate specialists in training methodologies or subject matter (SMEs). There is not yet a sufficient number of reliable Russian training specialists in some sectors whom one might hire as one does an interpreter from a public listing of professionals. However, this situation is rapidly changing in response to intense market demand in virtually all sectors, especially business. This growth matches the rise of professional consultant firms, who often provide training in their package of services at rates that are often exorbitant. Such firms are on the look for expert trainer talent knowledgeable in new areas such as training needs assessment or distance learning applications, and will hire quickly the best available onto their staff. One problem NGOs sometimes encounter when outsourcing training to Russian professional trainers is their credibility with trainee audiences, especially in regions outside Moscow. The challenge is twofold: first is the issue of finding an adequate supply of sector-specific professional trainers at the regional level. Then, the second challenge is overcoming the "outsider syndrome". Local trainee audiences at times resist training provided by an outsider not on the NGO staff. In some cases trainees of local stature will want to do the training themselves. “Russians are highly confident of their own abilities” according to one NGO, even though they may be innocent of effective training techniques. But allowing such trainee participation can have other benefits to the NGO, unconnected to training quality. Two organizations that we met specialize as training providers, serving clients directly with their own training staff in response to marketing efforts. A third group, InterTraining, was formed two years ago as a membership organization to offer technical support and information sharing to its members, comprised of 78 independent training companies or professionals in Russia and NIS countries, 56 of which are in Russia. InterTraining does not provide training directly, instead it functions as a match maker and refers requests to member organizations with capabilities in the requested area. This referral service comes free to paid-up members. Process Development Company (Alexei Kuzmin) is one training provider that appears to understand modern training processes well, and has worked with USAID￾sponsored NGOs designing and providing seminars. He is an example of the type of firm, along with others that InterTraining might recommend to USAID that could help strengthen training services offered by USAID-sponsored NGOs. 9 Other large corporate training firms such as the Center for Business Skills Development (CBSD, Dennis Hopple ) or MacKenzie Consulting are unlikely partners for USAID since they focus on serving Russia’s large corporations with business and executive management skills. These firms might consider Training of Trainer activities for NGO needs but only in cases where they were clearly not creating future competitors for their own business, and at rates which USAID might not be willing to afford. Still, the idea is worth exploring with Mr. Hopple at least, who was greatly assisted by USAID in earlier years, at such time when USAID/Russia seriously considers an activity to strengthen the quality of trainers used by its sponsored NGOs and other institutions. USAID/Russia’s interest is to see that NGOs it sponsors find reliable Russian trainers of professional quality to broadcast new skills, knowledge and attitudes (SKAs) to NGO clients as widely as possible. The current preferred practice of developing NGO project officers as in-house trainers seems to work well, enabling them to work effectively with U.S. expatriate trainers or other NGO staff and members in conducting local courses jointly. These newly minted trainers can do double duty for the NGO providing technical assistance to clients in coordination with the training. USAID/Russia may well decide to give further support to sector-specific NGOs by developing their local capabilities as more sophisticated training providers matching the rapidly advancing technical needs of their Russian clients. NGO clients’ technical knowledge and performance levels moved rapidly ahead in the past 5-7 years, requiring up to date technical and managerial information from NGOs as well as modern ways of delivering it in organizational contexts. NGOs now need access to skills such as applying Distance Learning variations, performance gap analysis and training needs assessment. Several NGOs have praised the Johns Hopkins University training program in NGO development, which includes presentation as well as marketing skills. Some program graduates made career shifts away from NGO work into the training profession, with the more successful ones setting up their own firms or finding jobs with the likes of CBSD. An easy first step would be to explore with Natalia Shakirova of InterTraining the sorts of workshop or other services her NGO could offer other USAID-sponsored NGOs who are working to develop their in-house trainer capacities. InterTraining’s current members could also be a source for outsourcing trainer expertise, although Russia does not have a credentialling or certification process for quality ranking and control any more than the U.S. does. U.S. universities have a universal accreditation process, but not training firms other than through Better Business Bureaus. InterTraining is reportedly working on such a set of standards for training firm assessments. Another source for information on the trainer market, unfortunately not interviewed, is GALUPKA, a large firm providing domestic training services. Its grant to InterTraining helped the latter get its start as a member-based activity. GALUPKA has evidently not worked with USAID-sponsored NGOs, and may have a discouraging rate structure for Mission NGOs. 10 If the Mission were to propose a set of trainer selection criteria to its NGOs for quality control purposes, it could start with some or all of these, in addition to currently used client reference checks. These standards prove to be useful for some Missions, however only experience will show whether they are too exclusionary or are applicable for USAID Moscow: - at least 3 years experience as a training provider for compensation; - prior work with at least 12 clients from all three sectors: non-profit, public sector and private sector; - experience providing training services outside the Moscow area; - fair pricing structure that does not differ significantly between services to USAID or its cooperating NGOs and the firm's other non-USG clients; - an organizational structure that includes: ·a board of advisors or directors in the chain of authority, ·an organizational vision or goal statement, ·a business marketing plan reflecting its organizational goals, and ·administrative-financial structures that assure prompt and accurate invoicing; ·a staff development plan for its trainers; ·training materials development capability; ·competence to provide follow-on services to its training program graduates; ·clearly identified area or areas of subject matter competence for training. Make Provisions for Follow-On as a Standard Component of Training In many circles follow on is considered to be of equal value as the primary training event. Many HR professionals in the private sector, as well as some donor organizations reserve 25% of the overall program cost for follow on activities. This is not to suggest that 25% is a magic formula that ensures a quality follow-on experience. Rather, it is mentioned to highlight the importance that industry and other donors attach to the post-training component. For most Mission staff and other persons interviewed the question was not whether follow on has value, but rather when and how best to provide for it. Follow-On versus Alumni Association: Although often used interchangeably, there is a subtle but important distinction between follow on and alumni associations. This distinction is embedded in the purpose for which they exist. In the USAID world, alumni associations traditionally served as organizations to further the interest of the Mission. They provided a ready source of individuals who, with a common experience of studying in the U.S., could be called upon to showcase projects, pitch in during official visits, and generally assist in representing US interests. Follow-on may have some of the same objectives, albeit to a lesser extent, but its broader purpose is continuing / reinforcing professional or technical development of the trainees through informal networking and organized program activities, thereby disseminating and sustaining the impact of training. Some approaches for follow-on appear below. It is clearly acknowledged that these activities would not be relevant for all training programs. 11 However, without some deliberate efforts for follow-on the impact of training may not survive within the trainee’s organizational setting, and may go no further than the limited number of persons actually trained. - Develop trainee action plans as part of the training program itself, to carry out back at work. These can be monitored later; - “Training of Trainer” components to the program itself, enabling trainees to share effectively what they learned with colleagues at work or in the profession; - Arranging in class future paired visits by trainees to the sites of professional colleague(s), to explain lessons learned from the class directly; - Providing internet/e-mail access to trainees, facilitating further exchanges with co-trainees, trainers or other professional cadres; - Structure trainee alumni associations providing newsletters and other services to help ex-trainees share new applications of training with each other and stay in touch professionally with the training provider or NGO; - Conduct seminars and workshops for ex-trainees but open to other stakeholders, preferably within 6 months or less of the program; - Provide access to professional publications that reinforce or move beyond training objectives; - Work through regional centers as follow-on seminar sites and sources of further technical advice or networking (internet access); - Select subsequent training for individual trainees who show special promise in furthering Mission or NGO objectives. From Former RTD Participants Input from former RTFD participants, as direct consumers of the training, was particularly useful in assessing and making recommendations about the implementation process. Through questionnaires and discussions they provided information that supported earlier surveys conducted or commissioned by USAID Moscow. The participants were pleased with their U. S. training program as well as the level and quality of pre-training support that was provided by AED. Eighty-eight (88%) knew precisely why they were nominated for the training and reported full congruence between their pre-training goals or expectations and what their program actually provided. With a very few exceptions, they reported that the course content was relevant and the material was clearly and thoughtfully presented. Participants particularly appreciated the patient and responsive manner that instructors listened to and answered their questions. Only two of the participants found their course too abstract or otherwise not relevant to Russian needs. An impressive 90% of the responders reported that they are making regular and effective use of the training on their jobs and have had positive post-training results, including changes in the quality of their work. Forty-two (42%) have been promoted to a higher position or have been assigned greater responsibility in the same job. 12 Others have started their own business as independent consultants, trainers and providers of technical assistance. Many of the participants are sharing their new skills with colleagues informally or through small group seminars. They shared examples of change, successes and personal gains that resulted from training, including: “I acquired a new quality approach to management methods, to the need for expediency of voluntary environment development and interaction of stakeholders. I have a clearer understanding of environmental management systems as a tool for sustainable development.” "After the training I opened my own consulting company "ECOSAUR" and find the knowledge useful in working with clients." "Since [completing] my U.S. training I have improved the services to tourists and conditions for ecotourism" "I would like to highlight flawless, high quality methods of training in the Russian part of the program. The acquired knowledge helped me to actively participate in a Moscow Government project related to development of environment education and introduction of ISO 14.000 [international standards in industry]. I acquired knowledge that helped me become the Director of the educational organization (Moscow International School "Business in Industry and Science." "A lot of the material was new to me and, what is important, it was very specific. The knowledge helped me to pass the state examination in eco-audit, receive a certificate and then teach the course of eco-audit in the program of State Environment Committee of the Russian Federation. I gained modern views in the area [how to prepare environment projects] and a broadened circle of business partners." "The training motivated me to work more productively." Ninety percent (90%) of the group would support post-training activities or follow on. From a list of exemplary post training activities, professional workshops / seminars, an internet listserve, a documentation center and informal gatherings were most favored. Participants suggested several other areas for improvement, some of which were mentioned in earlier evaluations: ·Include provisions for post-training activities ·Strengthen the practical sections of training courses ·Include internships with private firms as a training option ·Form groups that are more homogeneous with approximately the same rank, position and development level; ·Provide texts and printed resources as a standard part of the program ·Give the trainers more freedom in selecting participants 13 V. SUMMARY AND RECOMMENDATIONS: A. SUMMARY Our observations and conclusions support the Mission's earlier anecdotal and notational impressions about the Russian Training for Development Project (RTD). The results of our review suggest that the RTD has met objectives identified in the Project Plan. Former participants report that they have benefited from the training they received. There are compelling reasons not only for continuing the project, but also for continuing it under the existing administrative structure. The AED contract under management of DIHR is an efficient way to implement the training. We believe the Mission's needs will be well served through continued use of the central Global Training for Development (GTD) IQC mechanism of which AED is a part, and with the GTD successor. Although financial analysis was not a part of this review, we believe it is important to note that AED's FY 2000 administrative cost of approximately thirty percent (30%) 4 is reasonable and is in the lower range for USAID contractors. Under the GTD successor, which will be in place with overlap time before the August 18, 2001 expiration of GTD, the cost structure should be even more favorable to Missions. Another advantage accrues from a buy-in to the central mechanism: it is pre￾competed and saves times for the mission that otherwise would be required for the RFP process, while retaining full management and control remain at the mission level. As mentioned earlier, RTD objectives are being met; USAID's management is efficient and responsive; and, AED's performance in implementing the program is commendable. The recommendations and additions that follow mostly address procedural changes that can make training even more strategic. We hope they will be useful in designing and implementing the next iteration of the Russian Training for Development Project. B. RECOMMENDATIONS 1. for the Mission: - institute a multi-year strategic training plan integrating training needs with technical assistance and other inputs, using multiple funding sources, according to a set of Mission or SO-level priorities. The current training plan is a compilation of the annual approved list of training proposals submitted by activity managers as part of the competition process for funding. It covers a single fiscal year and is limited to only the funded activities, rather than the totality of training needs that are equally valid but did not make the 4 The FY 2000 training budget was funded at $1 million ; $300k was AED's administrative cost 14 funding cut. Since the current plan lacks the comprehensiveness of all activity inputs and all possible funding sources, it has limited use as a strategic document. 2. for DIHR: - institute a two-stage funding process: first, earmark a set funding amount to each SO (or office if more appropriate) for training, and establish SO level priorities through competition or other means; second, conduct competition based on the proposals as currently being done. - continue the training competition, but provide greater flexibility in scheduling and coordination, i.e.,: - try for semi-annual or rolling proposal screening and approval; - make sure there is input in course design from TA providers (through activity managers); - rank course proposals according to SO subsection and not in a single list; - attract more training providers into competitions by raising course budgets where competitive response has been poor; - consider greater use of Mission partners who can serve effectively as in-country or U.S. training providers; - adjust training proposal template to clarify the proposal’s role in supporting SO and IR attainment, and incorporating plans and budget for follow-on. 3. for activity managers: - apply standard performance gap and training needs analysis to the target NGO/organization where feasible before designing courses in order to determine if training is the appropriate input or is appropriately sequenced with other inputs; - specify how their training proposals reinforce the stream of technical assistance also being provided to the beneficiary, and how training results are themselves supported by TA or other inputs; - continue / increase efforts to show linkage between training and attainment of IR to the extent possible; - ensure representation at all of AED’s pre-departure sessions to review with trainees the training program content, objectives, anticipated action plans and follow-on; - broaden the recruitment of training candidates by drawing more consistently on NGO or in-country training provider networks; - supplement valid training candidate expenses not covered by RTD/AED, e.g. candidate travel for qualifying tests; - develop strategies for cost-sharing of training proposals jointly with other donors, USG Agencies, NGOs or trainees themselves, including follow-on components. 4. for AED: 15 - develop promptly a wider net of training providers responding to their RFPs; post course proposal abstracts on the internet to attract new bidders; request “Statement of Potential Interest” at pre-proposal stage; - share training proposals during their development not only with activity managers but also with TA providers and trainees; - follow up on mailings to trainees consistently to be sure they are received in timely fashion; - set more flexible deadlines to permit greater depth of interaction on course design between AED, activity manager and other stakeholders; - AED and USAID in pre-departure preparations should make sure that wherever possible, trainees are given names of other co-trainees and internet access for networking with them before the course takes place. Follow-on networking should be encouraged with co-trainees, other colleagues, and professional counterparts in other NIS countries.