FINAL EVALUATION REPORT U.S. DEPARTMENT OF COMMERCE COMMERCIAL LAW DEVELOPMENT PROGRAM IN EGYPT (CLDP) Submitted to: U.S. Agency for International Development Prepared by: Gary Goodpaster Team Leader Legal Expert William Thomas Training Expert Ahmed Abdel-Fattah (M.A.) Training Expert Assisted by: Mostafa Abou El Enein Egyptian Commercial Law Expert Submitted by: Results Reporting Support Activity Development Associates, Inc. 20 Aisha El-Taimoreya St., 1st Floor, Apt.2, Garden City, Cairo, Egypt August 4, 2003 TABLE OF CONTENTS Executive Summary...............................................................................................................iii Introduction............................................................................................................................1 Background and Context........................................................................................................2 CLDP as One Actor Among Others.................................................................................2 Local Conditions..............................................................................................................3 Technical Assistance “Hospitality” ...........................................................................3 Cultural Factor...........................................................................................................4 Open and Closed Doors.............................................................................................4 CLDP Workplans...................................................................................................................4 Overview..........................................................................................................................4 CLDP Workplans, Training, and Needs Assessment ......................................................5 Training Objectives and Evaluations.....................................................................................6 CLDP Activities.....................................................................................................................7 CLDP Efforts in Discrete Areas ......................................................................................7 Advising/Consultations....................................................................................................9 Study Tours......................................................................................................................11 CLDP Impact on USAID Strategic Objectives......................................................................15 Results Consistent with USAID Objectives ....................................................................15 Other Training Impacts and Skill Development ..............................................................16 GOE Participation in the CLDP Program..............................................................................17 Feasibility of Using CLDP Methods More Widely ...............................................................18 Sustainability of CLDP Technical Assistance Efforts...........................................................19 Conclusions............................................................................................................................20 Recommendations..................................................................................................................21 1. Relationship to USAID and USAID Projects............................................................21 2. Training......................................................................................................................22 3. Training Methods.......................................................................................................22 4. Technical Assistance Lead-Ins and Follow-ups ........................................................22 5. Evaluation ..................................................................................................................23 6. Future Activities.........................................................................................................23 U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report i Development Associates, Inc. APPENDICES Appendix I. LDP Technical Assistance Activities and Trainings ...............................I-1 Appendix II. List of Contacts and Interviewees............................................................II-1 Appendix III. List of Documents Reviewed...................................................................III-1 Appendix IV Overall Technical Review of CLDP Training Programs.........................IV-1 Appendix V. Data-Gathering Instruments.....................................................................V-1 Appendix VI. Evaluation Methodologies.......................................................................VI-1 Appendix VII. Glossary ...................................................................................................VII-1 U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report ii Development Associates, Inc. EXECUTIVE SUMMARY This is the first evaluation of a USAID funded PASA technical assistance activity, the Commercial Law Development Program (CLDP) that is being implemented through the United States’ Department of Commerce in Washington, D.C. The evaluation team was composed of an American lawyer, an American training expert and an Egyptian training expert. The team reviewed documents and memoranda related to the various activities conducted both in Egypt and the United States under the aegis of CLDP during the last five years. American and Egyptian stakeholders were interviewed during the period from July 7 to 31, 2003. The purpose was to determine the effectiveness and impact of the various components of the program and to explore possibilities for sustainability. Findings included the positive impact of CLDP on USAID strategic objectives, particularly in the policy and legislative arenas and moving Egypt towards a more globally competitive economy. In some areas, CLDP efforts, combined with those of other donor activities, had a positive impact and in other areas, they were a prime influence for change. One factor affecting the possibility of lasting change continues to be the receptiveness and proactivity of the various ministries of the GOE. They range widely on these dimensions and this is a major factor in terms of capacity building and sustainability. Nevertheless, it is clear that the CLDP staff has built excellent relations with many of the ministries and this will facilitate further development efforts. The range of Egyptian actors who have worked with CLDP in one way or another is extraordinary and reflects the range and potential impact of future CLDP activities. CLDP is a general utility player which is most useful and effective when it complements other technical assistance efforts, facilitates government to government official interaction and trainings, and is deployed opportunistically as a flexible technical assistance instrument. The findings lead to the following conclusions: 1. CLDP is an effective and flexible technical assistance program which in conjunction with other USAID technical assistance has contributed positively to achieving USAID’s SO 16 objectives which, if possible, USAID should continue. 2. CLDP’s combination of coordinated advising, consulting, and specific related hands-on training was its most effective means of producing positive results. 3. Egyptian authorities that have worked substantially with CLDP have cooperated fully and enthusiastically. 4. CLDP can help existing USAID projects deal with private sector issues and help Egyptian ministries educate the private sector. 5. CLDP has a potential role in helping Egypt to liberalize its economy and free at least some of its markets. 6. The whole of CLDP work on WTO related matters, including work on food standards, TRIPS, and IPR constituted a virtually ideal program of technical assistance. 7. Egyptian officials primarily want, and need, practical and hands-on training as part of study tours. 8. Conferences are mostly useful when held as part of a larger coordinated technical assistance effort having well-defined results or when designed to impart significant information to an audience not otherwise likely to receive it. U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report iii Development Associates, Inc. In terms of lessons learned, depending on CLDP’s own capacities, USAID could use CLDP in support of other technical assistance activities. One risk is that CLDP could be spread too thin, therefore it might be better to focus CLDP’s considerable capacities on a few high value targets, such as the proper role of government as a regulator of private markets. Recommendations are in the following six areas: • Link the CLDP more closely to the USAID portfolio in Egypt in order that the USAID projects and activities utilize more expeditiously the training opportunities of CLDP within their own work plans. • Insert realistic specific objectives containing measurable outcomes in all Training Plans. (see Appendix IV for a detailed analysis of training issues and recommendations) checking for the right match between a particular training agenda and the potential participants. • Focus Training Methods more on hands-on practical training. Study tours need to have time before final departure to assure the participants, in a pre-selection orientation session, that the program is appropriate to their expectations before departure. During the “observational study tours”, trainees should have the opportunity to discuss the relevance of what they have observed after each program. • Follow-up the study tours and training with workshops (seminars) in Cairo conducted by the participants from each training program. This practice should be included as an integral part of the CLDP and additional technical assistance provided in Cairo as needed. • Plan periodic evaluations of the total program and these reports kept in a central location within USAID/Egypt. • Future activities might include training for private sector organizations, as well as government staff. It might complement the training presently underway with other projects such as with the Ministry of Foreign Trade in reducing trading barriers by focusing on the private sector. The interviews yielded many suggestions for future activities for CLDP. In the private sector, for example, CLDP could complement the work of ATR, particularly in the trade area. There is still much to be done in terms of regulatory matters, especially in terms of authorities, procedures and tools US regulators commonly use. Egyptian officials also need substantial training in the fundamentals of competition economics and appropriate legal tools to insure free markets operate fairly and efficiently. IPR, patents and copyright are also areas needing more assistance, as is the custom reform process. Lastly, a critical area of conflict which CLDP might be able to facilitate is the relationship between the Egyptian WTO representatives in Geneva and the WTO unit in Cairo. U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report iv EVALUATION: U.S. DEPARTMENT OF COMMERCE COMMERCIAL LAW DEVELOPMENT DEVELOPMENT PROJECT IN EGYPT INTRODUCTION This report is an evaluation of a USAID funded PASA technical assistance activity, the Commercial Law Development Project (Egypt) (CLDP) of the U. S. Department of Commerce. USAID, seeking to advance its strategic objective of accelerating private-sector led, export￾oriented growth, selected CLDP because of "the intrinsic governmental nature of the issues involved and the lack of comparable expertise in the US private sector." This activity began in mid-September, 1998 and has continued to the present, having been extended several times. This is first USAID sponsored evaluation of CLDP. In the period of July 7 to 31, we met with numerous U.S. and GOE personnel (see Appendix II for a listing) and we reviewed many documents (see Appendix III for a list of documents reviewed.) During the evaluation team's interviews with US and GOE personnel, we were treated with the utmost respect and cordiality. The individuals we met were extremely forthcoming in their responses and invited us to explore issues further after the interviews, which in some cases we did, which enabled us to gain even more information about program impacts. Those supervisors who had not personally attended programs were very knowledgeable about what their employees had learned and were often able to articulate the benefit to their offices with specific examples. They were also not reluctant to state a reservation, if one existed and they all shared not just specific ideas for the future of CLDP but also a strong desire that it continue to play a role in strengthening economic growth and development in Egypt. The evaluation team's focus was on principal technical assistance activities, and we restricted our in depth interviews to persons involved in them. We did look at activities other than these, i.e., conferences and study tours, but primarily to determine how they complemented the main activities. The most recent technical assistance and trainings remain freshest in participants' minds, and we can have greater confidence in our assessment of them than we can have in the more remote trainings where, because of the passage of time or change of personnel, it is more difficult to make connections between activities and results. For example, all the participants in the Commercial attaché training except one have already been assigned to and taken up posts in embassies outside Egypt. Because many substantial and important activities took place during the early years of the program, we thought it important to attempt to assess their effectiveness and impacts. In general, we qualify our conclusions about these more remote activities simply by noting here that it is difficult to make strong inferences about impacts where assessments are not made either concurrently or within a reasonable time following the activity; for traditional training, ideally this should be within a year or shorter period. U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 1 Development Associates, Inc. In addition, as we will emphasize in this report, much of the CLDP activity is supplementary to that of other USAID projects, such as the Assistance for Trade Reform Project. In order to place CLDP work in the context of these other projects, we interviewed USAID project staff wherever possible. As the USAID IPR projects, SIPRE and TIPRE, and the MEIS insurance projects have ended, we were unable to interview their staff and were therefore somewhat limited in our ability to assess CLDP's intellectual property and insurance efforts within the context of USAID projects focused on these areas. BACKGROUND AND CONTEXT CLDP AS ONE ACTOR AMONG OTHERS The Economic Growth Office of USAID Egypt supports or has supported many substantial technical assistance projects in Egypt. Those of most relevance to this evaluation are those relating to trade, intellectual property, and insurance. Each of these projects has multiple goals and each of them has a training component, sometimes substantial in itself. In addition, USAID sometimes supports training through means other than these projects. Notwithstanding these training and technical assistance efforts, Egypt sometimes has training and technical assistance needs that these and other projects do not or cannot meet. This may be either because the needs are somewhat outside a project's focus; because a project is unable to locate the relevant private expertise, for example where that expertise is found in US government officials; because the need is immediate and the usual procurement times are too long; or perhaps because the need is an interdepartmental or interministerial one, and it is difficult for a project attached to a particular Ministry to respond. An example of this kind of training or technical assistance is that in the area of sanitary and phytosanitary standards for food. A country's food standards, if nonconforming with standards acceptable to the WTO, can comprise a barrier to trade, e.g., by restricting admission of certain food products that otherwise satisfy international standards. While USAID's Assistance for Trade Reform Project assists Egypt with regard to a WTO requirement of reducing technical barriers to trade, it has no staff experts on international food standards. Most of the likely experts in such matters are government officials dealing with such issues. Nonetheless, because of its participation in the WTO, Egypt must bring its food standards in compliance with WTO standards. While ATR does not have access to the necessary experts, CLDP has access to highly experienced U.S. government standards officials and U.S. standards organizations. CLDP, after consultations with ATR, provided them to assist Egyptian standards officials in their efforts to reconcile Egyptian standards with WTO requirements. The training and technical assistance CLDP provided regarding standards is a good example of the kind of work that CLDP does. It is important work that supplements and complements the work that other USAID projects undertake. In general, it appears that USAID officials, USAID project staff, and relevant US Embassy officials have realized that in the course of their technical assistance activities and training, they surface needs that must, or should, be met if there is to be further development progress. These needs, however, cannot be met by the usual aid structures and devices, and there is thus a need for a technical assistance and training instrument that is flexible, able to provide what is sometimes rather special or rarified expertise, and able to act quickly in response to an Egyptian realization of some immediate need. Thus CLDP is primarily, U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 2 Development Associates, Inc. although perhaps not solely, "gap-filling", in a technical assistance sense, a description offered by many persons interviewed during the course of this evaluation. USAID chose CLDP to assist it in its efforts in Egypt, and configured CLDP work in Egypt as a rapid response mechanism. As one interviewee stated, CLDP can “turn on a dime”, and it is to USAID’s considerable credit that CLDP has been effective in this regard. With substantial inputs from USAID and USAID funded projects, CLDP has been able to act quickly, appears to bring out the right people, and is particularly good at meeting immediate and short-term needs. CLDP is not a solitary technical assistance actor, and any assessment of its achievements or contributions to development in Egypt must recognize the important role played by other technical assistance projects and advisors in Egypt. In their work, they may succeed in moving matters to a point, and in uncovering emergent needs, such that CLDP's interventions are likely, or more likely, to be effective. While we characterize CLDP as a supplementary or complementary technical assistance actor, it is not clear whether this is completely the case. Although we are aware of the DEPRA Project that operated between 1995-2000, and understand that it had a trade and investment advisor, we have been unable, because the events are remote from this evaluation and parties are unavailable for interview, to determine what role DEPRA played in shaping CLDP’s early and substantial activities in WTO related matters, something we discuss below. CLDP certainly views itself as having brought some big issues to the attention of the Egyptian government, and it certainly seems to have done so recently, for example, with regard to Telecoms and frequency spectrum regulation matters. On the other hand, USAID and its projects have played an important role in orienting and guiding CLDP’s work, and on such evidence as we have, must assume this has always been so. Understanding CLDP's role as a complementary actor, however, clarifies its set of activities – otherwise a somewhat desultory list of technical assistance efforts – and provides a key to evaluating the effectiveness of its efforts. While other technical assistance players work on large parts of the technical assistance puzzle, filling in areas, CLDP is called on from time to time to bring certain missing puzzle pieces to the table. LOCAL CONDITIONS Technical Assistance “Hospitality” Any attempt to evaluate the efficacy of the CLDP must take into account the condition of the GOE and its various ministries. The X factor in efficacy is whether any given ministry will be willing to implement ideas and techniques that result from employees' participation in a particular training event, study tour, or consultation. People in the ministry, including higher-ups affect the outcome of a program. USAID officials noted, for example, that a problem with some study tours is that participants were not always free to apply what they learned. But there is always hope that they might after they move up in the organization. U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 3 Development Associates, Inc. Cultural Factors Cultural factors can also impede implementation of new ideas. Both Egyptian and US officials described Egypt as a society that is slow to move and said that the GOE bureaucracies are typical in inertia, internal and external conflicts, and the like. Some ministries have a view of their mission that may not fit the current realities. They may therefore not only just ignore issues falling under their purview, but also not be interested in equipping their personnel to meet the current needs. There are a number of interconnected problems such as these that slow down progress. While most people we interviewed see the GOE as honestly trying to grapple with issues, there are still strict hierarchical lines of control in the GOE. Therefore, there is no horizontal communication/cooperation between ministries unless the top orders it. This inhibits the "spread" of a positive training impact from one ministry to another. Open and Closed Doors The opportunity for providing technical assistance and training also depends on whether the “door is open or closed” for application, as one interviewee described it. Right now, for example, the government wants a mortgage market; therefore there is an open door. The same is true for capital markets. On the other hand, there is no e-commerce engagement between the US government and GOE at present, so even though ministries may identify that as a training need, steps need to be taken at higher levels to get more momentum going. Even open doors do not mean things are easy, however. For example, mortgage financing is now part of the Ministry of Housing, but they do not have experienced people to manage it effectively. The new anti-money laundering law and the fact that the GOE appears interested in a competition law may open other doors. The GOE will need help on anti-money laundering regulation and implementation, and assistance on the competition law and its regulations and their implementation. Whether CLDP is an appropriate instrument to address these matters is another question. CLDP WORKPLANS Overview USAID, of course, has many substantial projects in Egypt, and while it does not direct day to day operations of its projects, it does work with them to set goals, furthers their activities in various ways, makes them aware of new or developing opportunities, and reviews and evaluates their performance. In this sense, USAID steers rather than manages. Indeed, USAID gives its grantees large management responsibilities to ease the burden of its large and extensive development portfolio, and it expects its grantees to take appropriate initiatives, subject, of course, to USAID oversight. However, given the number of projects that USAID funds and oversees in Egypt, USAID is really the only actor that can fully see not only all of its development activities, but also assess the impact of various projects and their activities on one another and on USAID’s overall development goals for Egypt. USAID is thus positioned to see and develop promising U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 4 Development Associates, Inc. development openings and opportunities that may arise from interactions of its various projects, or otherwise, and to address them. It uses CLDP as one flexible instrument to do this, and it is essential to recognize this in assessing CLDP’s workplans. CLDP develops its annual workplans through consultations and negotiations with USAID, which orients CLDP and gives it points of focus, and then with U.S. Embassy officials, various USAID TA project staff, and with Egyptian officials. Once CLDP finishes it consultations and negotiations, it drafts a workplan and sends it to USAID for consideration, amendment, and approval. Once approved, CLDP carries through its workplan. For the most part, CLDP has performed in accordance with its approved workplan, and any deviations that arose seemed due to unforeseen circumstances and were at least informally approved. On a first reading, the workplans disclose a mixed bunch of activities in various scattered areas that seem not to follow an overall strategic plan, bur rather are opportunistic. When viewed in the context of other USAID technical assistance activities, however, the logic behind the workplans comes into view. CLDP is a supplemental player or actor in USAID's economic growth technical assistance efforts in Egypt. In order to appreciate the contribution that CLDP's planned activities will make, it is essential to understand what USAID and its projects have done and are doing as well as understand the jurisdictional or other limits that other USAID projects have. CLDP's workplans do not make this clear, but rather assume, probably because USAID will review the workplans, that this will be apparent to the reviewer. This evaluation team, however, takes the view that it would be well for CLDP workplans to articulate specifically how CLDP's proposed activities fit together with other USAID technical assistance. We do not suggest this because it will make evaluation easier – although it would - but because we believe this information would be useful to any USAID task manager affected by or interested in CLDP activities, and would be helpful to USAID personnel not previously involved with CLDP. In the five years of its work in Egypt, CLDP staff, in doing needs assessments and carrying out technical assistance activities, have developed good relationships with a number of Egyptian officials, something quite important in the Egyptian context. Given that USAID and Embassy people change, CLDP has thus far been a source of continuity and dependability for Egyptian officials who have remained in position, and this has value for future activities, and the reverse is true as well. They have also established relationships with other stakeholders on the ground in Egypt as well. CLDP Workplans, Training, and Needs Assessment A truly effective training program should benefit the individual participants as well as the organizations to which they belong. It should respond to an identified need, a gap between where the individual or organization is at present and where they would like or need to be in terms of performance and capability. Most formal training programs start with a structured assessment of these individual and/or organizational needs. It is not clear how systematic the CLDP’s needs assessments efforts have been. There was a Training Needs Assessment submitted to USAID in June 1998, but other than that, training needs assessments seem to have been implicit in workplans. U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 5 Development Associates, Inc. USAID officials stated that, in terms of needs assessment at the beginning of the program, the needs were so great, that almost anything would work. "In the beginning, the needs assessment was basically the GOE looking at PASA and saying we need help with WTO-related topics," said one USAID staff person. The genesis of the CLDP programs was perhaps more informal than we would have expected, but given the context and emergent needs, not at all inappropriate. The ongoing needs assessment seemed to be an iterative process. For example, GOE officials going to the States on a study tour lead to the identification of more needs, In other words, the process seems to be to assess each intervention and then carefully identify next steps and then implement those. Needs assessment is more at the sector than the individual level, and occasionally individuals attended programs which seemed less relevant to their personal needs. Training requests were also based on new GOE laws. Three years ago, for example, there was a push to strengthen the diplomatic corps in trade policy. That program seems to have petered out, but interviewees concerned with the work of Egypt’s commercial attaches believed there was still a need for such a program. The interaction between CLDP staff and both Egyptian and US officials in Cairo has facilitated this process. "The CLDP people have visited and held telephone conferences. They are very responsive and have good relations with our officials", declared one Egyptian official. Geneva Pham of CLDP, for example, was noted to have regular e-mail contact and have asked often for the next training need. CLDP staff has built strong relationships with not just US and GOE officials but also other stakeholders. The Assistance for Trade Reform Project, for example, has met with Geneva and other staff, and it is clear that the recently ended Insurance and IPR projects did so as well. The US Embassy is also involved in needs determination in the CLDP workplans. They feel the needs assessment is a 'building block' approach. There is a strong desire to not be redundant with USAID. To this end, USAID sees the draft workplan and tries to avoid duplication. Coordination between USAID and CLDP is important. Although CLDP itself may not have developed and administered a systematic needs assessment, this is not to say that the services they have provided have not addressed real needs. In fact, there are enough sources of training needs in Egypt that CLDP'S administering an independent survey or similar process would most likely be redundant. TRAINING OBJECTIVES AND EVALUATIONS One difficulty we faced in evaluating various CLDP programs was that the objectives were not specific or performance oriented, and did not include measurable results. For example, a judicial training conference on intellectual property had as its objectives “to further develop the quality of intellectual property protection in Egypt and to further improve the skill and knowledge level of Egyptian judges in the implementation, enforcement, and adjudication of intellectual property issues.” Neither the first nor second objectives describe end states in measurable terms, nor is any behavioral change indicated. Another example is a study tour on food safety for EOS officials in February and March, 2003. The stated objective was “to assist the GOE to develop the capacity to achieve and maintain national and international standards of food safety by training Egyptian officials on cutting-edge U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 6 Development Associates, Inc. innovations in food processing and preservation technologies and by familiarizing them with recent advances in food safety research and applications”. This lengthy objective statement does not address behavioral change and does not point to indicators for new and better trainee performance levels following training. The objectives in both examples focus on processes rather than end-states. In the case of the food safety study tour, this made it difficult for DT2 evaluators to identify changed behavior. For example, the DT2 report stated that one participant gave a lecture to others, but did not discuss any new skills that were acquired or procedures that were implemented. The participants identified that one challenge to the implementation of HACCP and nutritional labeling standards would be getting the government and private sector to work together, but there was no indication as to how this challenge might be met. Nevertheless, DT2’s assessment was that “there is a good probability of enhancing standards and quality control of Egyptian Food Products.” It would be hard to support this judgment from the data presented without making some large inferences. These criticisms relate primarily to the question of including within training plans or designs measurable skills, performance, or behavioral change indicators. That they were not included in CLDP training plans or designs does not mean that changes did not occur. Indeed, we did find examples of changed behavior and note them elsewhere. Nonetheless if training plans or designs do not focus on specific, measurable results, it is not clear whether trainings will achieve them, and their absence makes meaningful evaluation difficult. CLDP ACTIVITIES In its gap filling role, CLDP has undertaken many kinds of technical assistance activities in a number of different areas The activities include organizing study tours and trainings, organizing and putting on conferences, providing advisors to work directly with Egyptian officials, and advisors to draft advisory technical assistance memoranda for the use of Egyptian officials. CLDP has worked with Egypt in the following areas: WTO issues, such as trade remedies, services, agriculture, and related matters; intellectual property rights; standards; insurance; E￾commerce; finance; procurement/BOT; and miscellaneous other matters. (A matrix, listing all CLDP activities, organized by subject matter, type of activity, place and number of participants is attached as Appendix I.) CLDP Efforts in Discrete Areas Because of Egypt's participation in the WTO, it has a need to make itself WTO compliant with regard to intellectual property rights, food standards, and other WTO matters, and the larger part of CLDP's activities involved this "package" of technical assistance. Using days devoted to activities as a rough measure1 of CLDP effort, the following chart discloses the level of effort CLDP devoted to different activities. 1 Quite rough, indeed, as it does not measure intensity, type of activity, number of participants, or impact, but only refers to the amount of time devoted to the activity. U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 7 Development Associates, Inc. Subject Number of Days Percent of Total WTO Package WTO Related Activities 180 27.8 Intellectual Property 91 14.1 Standards 56 8.7 Subtotal 327 50.6 Other Insurance 89 13.8 Telecoms 85 13.2 Procurement/BOT 53 8.2 E-Commerce 45 7 Miscellaneous 26 4 Finance 15 2.3 Subtotal 319 49.4 Total 646 100 Quite rough, indeed, as it does not measure intensity, type of activity, number of participants, or impact, but only refers to the amount of time devoted to the activity. As the chart discloses, roughly half of CLDP activities directly involved foreign trade related matters, and approximately 85% of its efforts were devoted to the trade package, insurance, Telecoms, and procurement. The following chart, based on an approximation of days2 spent on different kinds of activity, discloses that CLDP's principal kind of technical assistance activity was advising/consultation, with approximately more than 60% of its effort devoted to this, followed by study tours (26%) and training (6% - either in the United States or Egypt). This categorization, however, is somewhat misleading, as in some cases advising/consultation included some training, assumedly whatever was necessary to make the advising useful and effective. CLDP Days Spent on Activities, by Kind Activity WTO Package Ins. Telecoms Pro￾cerement E-Com. Fin. Misc. Large seminar/ Conference 21 2 2 Advising/ Consultations 200 40 31 40 9 603 Training 33 5 Study Tours 65 28 19 5 23 Workshops 8 2 Estimation of days taken from those reported by CLDP, with no attempt to exclude weekends or holidays. 3 The specific activity was "ATM Network Advisors" for the period March - June, 2002 in the United States and Cairo. U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 8 Development Associates, Inc. Advising/Consultations The category of Advising/Consultations is not sharply defined, and includes advisors working with a small group of officials, drafting memoranda (1) diagnosing problems and suggesting remedies or plans of action; (2) responding to specific inquiries from Egyptian officials; or (3) providing analysis and reviews of draft laws. In some cases advisors worked with Egyptian officials in Egypt to provide some form of training: e.g., "hands-on" case study fashion, or classroom work followed by more individual consultations afterward. CLDP sent advisors to Egypt to work with officials on: ! WTO agriculture obligations, injury determinations, services negotiations, standards, creation of an Egyptian WTO unit; ! procurement procedures and law; ! patent document management and WIPO conformity; ! industrial designs office procedures and staffing and the industrial designs law; ! reforming Egypt's insurance sector and insurance regulation, conducting financial analyses, and designing a public awareness campaign for the Egyptian Supervisory Insurance Authority; ! drafting implementing regulations for the real estate law and devising a model for an Egyptian real estate finance market; ! launching a pilot project to electronically link Egyptian postal offices using an ATM network; and ! reviewing legislation hindering E-commerce and consulting on cybersecurity and legislation regarding on-line privacy and bank security. We have reviewed the memoranda that CLDP advisors produced on these matters and have found them excellent. In terms of CLDP combination of advising, consultation, and training, we found its efforts on the trade package particularly impressive. It worked effectively across different Ministries or departments to help Egypt remedy inadequacies in various areas that interfered with its ability to become WTO compliant. It worked with the Ministry of Foreign Trade (MOFT) on establishing a WTO Unit, on trade remedies, technical barriers to trade, dispute resolution, and trade in services and agriculture; worked with the Egyptian Organization for Standardization and Quality Control (EOS) to change sanitary and phytosanitary standards that amount to technical barriers to trade; and worked with the General Organization for Export and Import Control (GOEIC), the Patent Office, Academy of Scientific Research, and the Trademark and Industrial Designs Offices, Ministry of Supply and Domestic Trade, the Copyright Office, Ministry of Culture, and with judges on intellectual property issues to help Egypt become TRIPS compliant. These efforts appear to have been effective, as interviewees from MOFT, EOS, and the Patent Office were enthusiastic about the CLDP assistance provided and stated that it had definitely changed and improved the way they worked. As Egyptian Ministries and Departments appear to have some difficulty in coordinating activities and working together, CLDP's "all fronts" approach in its trade package effort was particularly helpful in circumstances in which USAID projects, working within individual Ministries or Departments, may have been unable, for good reasons, to have made similar cross-agency efforts. In this way, it appears that CLDP supported, supplemented, and advanced other substantial efforts of the USAID trade and intellectual property projects, and was synergetic with U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 9 Development Associates, Inc. them. Viewed from a USAID point of view, CLDP is a utility player that USAID and its projects can deploy to meet emergent, but unprogrammed needs. Considering other CLDP activities in addition to the trade package and focusing on the memoranda alone, it is not clear that Egypt in every case closely followed the advice or recommendations made. It is clear, however, that these documents, together with the associated activities undertaken to make them effective, contributed to Egyptian officials' understanding of problems, needs, and goals; approaches, techniques, procedures; and possible or desirable solutions either to immediate or longer term problems in upgrading or moving various sectors toward greater liberalization or bringing Egypt into conformity regarding its trade treaty obligations. Interviewees noted that CLDP brought out very experienced advisors, some of whom had experience in other developing countries as well. As government officials, these advisors had faced regulatory problems similar to those confronting Egyptian officials, and their experience was directly, and practically, relevant to Egyptian officials' needs. For example, a CLDP advisor prepared a series of detailed memoranda for the Ministry of Foreign Trade concerning the establishment of a WTO Unit within the Ministry. The memoranda covered placement of the Unit, functions, staffing, and training. The Minister of Foreign Trade issued two decrees establishing the Unit, and while the placement of the Unit was different from that recommended, the functions, staffing, and training plan became a baseline which was used, along with other ideas, to set up the Unit and make it operational. Another example is the CLDP effort on Telecoms. One problem facing Egypt as it considered a new Telecoms bill was the question of security. In particular, the Egyptian military and security agencies, which control certain parts of the spectrum, were concerned that the Telecoms law might compromise their ability to manage security issues. CLDP information and expertise showed concerned Egyptian parties how such issues were handled in the United States and helped persuade Egyptian parties that their concerns could be assuaged. Essentially, they are now managed through a Spectrum Consulting Committee, on which security agencies sit, that reviews frequency applications. CLDP also introduced the idea of spectrum auctions to Egypt. Egypt didn't, in the event, adopt an auction model for allocating frequencies, but it did adapt the model for the Egyptian environment and now uses a hybrid, a combination of a "beauty contest" and an auction. Egypt has, however, had auctions of telephone numbers of ISP providers and for universal numbers for Call Centers, e.g., for companies, such as McDonald’s, that work internationally. A final example shows how CLDP combined study tour training with follow-up advising to produce a significant result. Employees from the Egyptian Insurance Supervisory Authority went on an insurance study tour in the United States, and members of its PR group received training in how to deal with the media and how to manage public relations in a crisis, in part learning to appreciate the needs of the press. They reported that this training transformed their relationship with the Egyptian media – effectively from an adversarial to a cooperative relationship – and that now the Insurance PR group has become a highly trusted source of insurance information for the Egyptian media. In addition, after their return, they involved CLDP in helping them create commercials regarding insurance for the agency, the first time they had done so. U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 10 Development Associates, Inc. Overcoming Resistance to the Telecoms Draft Law; ISP Auctions One problem facing Egypt as it considered a new Telecoms bill was the question of security. In particular, the Egyptian security agencies, which own parts of the spectrum, were concerned that the Telecoms law might compromise its communications ability and its ability to manage security issues related to communications. CLDP information and expertise showed concerned Egyptian parties how such issues were handled in the United States and helped persuade Egyptian parties that their concerns could be assuaged. CLDP also introduced the idea of spectrum auctions to Egypt. Egypt eventually adopted a hybrid system combining a "beauty contest" and auction, and has had several of these. In addition, Egypt has conducted successful auctions of telephone numbers for ISP providers and Call Centers. A Public Relations Strategy for the Egyptian Insurance Supervisory Authority Employees from the Egyptian Insurance Supervisory Authority went on an insurance study tour in the United States, and members of its PR group received training in how to deal with the media and how to manage public relations in a crisis, in part learning to appreciate the needs of the press. They reported that this training transformed their relationship with the Egyptian media – effectively from an adversarial to a cooperative relationship – and that now the Insurance PR group has become a highly trusted source for the Egyptian media. CLDP also devised a public relations strategy for the Egyptian Insurance Supervisory Authority, which the Authority put in place. Finally, CLDP helped the IT group create and produce commercials regarding insurance for the Authority, the first time they had done so. Study Tours With some exceptions – a short study tour to the U.S. for two Egyptian business leaders and study tours for diplomatic attachés – all study tours were conducted to support some other activity, e.g., as a part of other CLDP WTO, standards, intellectual property, or insurance efforts. They should therefore be considered in the context of these other activities. Although the study tours were of different kinds – observational, consultative, or training — all interviewees reported that study tours were quite valuable. Interviewees said that training courses were excellent and that it was very important for them to have met and worked with U.S. government officials during these tours. As they perform similar functions and face similar problems, their advice was thought pertinent and was respected. Indeed this is an important aspect of the program, as Egyptian officials are exposed to American officials who do the same or similar work. CLDP staff feel they provide value by pairing Egyptian agency employees with their U.S. counterparts, and GOE personnel certainly agree. Working with these US government officials enabled training program and study tour participants to get to see their jobs in new ways. Those who visited the US on study tours cited time with the USTR, FCC, USTTI, ANSI, NIST, and the USPTO as very valuable. This kind of exposure, which often demonstrates new tools or techniques for managing governmental work, shows Egyptian officials just what it is they need to know to better carry out their duties. When U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 11 Development Associates, Inc. such study tours are followed by tailored additional training, as has been the case with CLDP, they are most effective. Sending GOE officials to the States also helps teach them what they further need to know and leads to a much more discriminating identification of needs, moving from "I don't know I don't know" to "I know I don't know, but now know what it is that I don't know." This becomes an iterative process that leads to an identification of new needs and new opportunities for useful technical assistance. Although one USAID person felt that what's done in Cairo seems more beneficial than those programs conducted in the US, the Egyptians spoke highly of the opportunities to participate in various learning opportunities in the US. Interviewees also spoke of the value of "informal" learning during study tours, or what might be called the demonstration effect: that is, just the experience of how American systems operate and how Americans work in teams, and the efficiency of American institutions. This was repeated so often by different and unrelated parties that it is hard to avoid the conclusion that the study tours do have a substantial and positive impact: they really do influence how the Egyptian government employees think and what they do afterward, and they generate greater familiarity with, and a demand for, U.S. regulatory ideas and products.4 While observational study tours may have this effect, training experts assert that such tours are most valuable when the trainees know exactly what it is they want to observe. Essentially this means that observational study tours achieve the best results when trainees receive a pre-training orientation explaining what they are to see and why and when there is an opportunity for them to refine and prime their expectations. This appears to have happened to a certain degree with DT2 – IIE affiliated tours, but otherwise we do not think is a standard part of CLDP training methodologies. There were some constructive criticisms of a few study tours. Some participants said they were too short. When the study tour group was composed of subgroups with different kinds of expertise, some thought that general sessions provided to the entire group were not useful to all members. In other words, the topics of the general sessions were not of interest to all subgroups. This would suggest that CLDP in such situations in the future should attempt to tailor study tours by giving more attention to subgroup composition and divide the larger group into specialized study sections where warranted. There was also a complaint by an IT participant, who went on an insurance study tour, that where she hoped to get practice in using software, there was instead a demonstration of software. A single complaint normally should not bear much weight, but the reason assigned is important. The phrase, "hands-on" occurred in a number of discussions. All the Egyptian interviewees emphasized they wanted practical training rather than presentations: that they wanted to learn how to do things, not learn about things. This theme was echoed in more than one Ministry, including specific suggestions, e.g., to have an American expert come in and walk Egyptians through applying the HACCP system to one plant in Egypt (food safety). The expressed wish is thoroughly consistent with most models of adult education that say that adults learn best by 4 One supervisor said he wanted his personnel to appreciate "the whole package", from the minute they walked off the airplane into an American airport from the plane. Where interviewees found work procedures similar to those with which they were familiar, they reported that it gave them confidence that they were doing things right. U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 12 Development Associates, Inc. experiential activities and having the opportunity to reflect on and apply what they learn from those experiences. Apart from the example given above regarding the Egyptian insurance agency and greatly improved public relations, it is difficult to attribute results achievements – at least in terms of USAID's strategic objectives or intermediate results indicators – to study tours. Without question, they change participants' perceptions and understandings, do result in some change in working habits or operations, and sometimes result in recommendations for change made to Departments and Ministries. This useful, but at least for the time being limited, impact is understandable. Participants in study tours, except on the infrequent occasions, are usually at most middle managers that often do not have determinative authority within a department. Study tour participants are not always free to apply what they learn. Ministries are hierarchically organized and the heads of departments and higher-ups determine whether the ideas and skills that participants gain in study tours will be applied. It is their receptivity to participants' new ideas and efforts that is critical. At some point, participants may move up in the hierarchy and have more influence and power: one would hope that within a period of time their study tour learning remained useful. But this is only a hope and does not claim much for technical assistance effectiveness in terms of immediate results. Study tours appear most effective when the right people are sent and when they are undertaken in support of other ongoing technical assistance efforts, as was and is the case with CLDP standards work with EOS. Improving Food Safety One notable example of receptivity to new ides occurred in the Egyptian Organization for Standardization and Quality Control (EOS). One person from the chemical laboratory, after attending a study tour, sent memoranda to officials with suggestions for changes in inspection procedures. These suggestions were reviewed and implemented, thus improving food safety reviews in food processing plants. Workplan Based on Study Tour EOS middle managers went on an observation study tour for food safety officials. As a result of this tour, they developed a workplan for EOS to implement. The workplan focuses on encouraging national factories to apply the HACCP system in production; developing and introducing a new system for packaging and labeling; and establishing a coordination mechanism between EOS, academia, and companies in order to solve production related problems and issues of quality control. U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 13 Development Associates, Inc. Comparative Learning Technical examiners from the Egyptian Patent Office attended a U.S. Patent Office short course for two weeks in Washington, D.C., a course they raved about. As a byproduct, they also noted they had an opportunity to interact not only with U.S. instructors, but also with participants from India, Taiwan, Korea, Bulgaria, and Germany, some countries which, while more advanced in IPR matters than Egypt, were nonetheless well behind the United States. They realized that, while comparatively speaking, they were far from achieving U.S. standards of work and operation, they should not be discouraged because they were on a path that would lead to substantial improvements. Conferences and Large Seminars CLDP has organized and conducted several large conferences and large seminars with up to 300 participants. It is not clear from CLDP descriptions how seminars differ from conferences other than that conferences appear to be an invitational event involving more than one Ministry, other public institutions, and perhaps the private sector. Seminars by contrast appear to be events focused within a particular Ministry and may be supposed to offer greater elements of training. All conferences and seminars were part of a larger technical assistance program. In that sense, they were a form of groundwork to make other interventions more fruitful. They provided information and were a way to initiate and further discussions among influential parties, decision-makers, and those affected by potential policy changes, e.g., private sector parties. We were told that Egyptian Ministries sometimes find it difficult to work together, that there are few venues where issues affecting multiple Ministries and parties can be worked out, and that information transfer from the government to all stakeholders or affected parties was sometimes problematic. In this context, conferences and large seminars certainly are valuable because they provide participants with a common information base, present new ideas, allow stakeholders to perceive the differential impact of policy changes on their own and others' interests, and provide the basis for discussions about impacts and tradeoffs. The conferences and seminars that CLDP organized and held were reputed to be excellent and certainly appear to have served the functions mentioned. One USAID interviewee said that she had urged CLDP to have conferences on reforming the insurance sector in Egypt because it would be a way of insuring that all parties affected by insurance privatization efforts could find out about proposed changes and effects. The conferences and seminars that CLDP held on WTO matters served both information provision, discussion, and consensus building functions. The intellectual property rights enforcement conferences for judges were an effort to develop interest among and provide expert legal information to, judges who would have to enforce the new intellectual property law, a particularly important effort since IPR enforcement appears to be a problem in Egypt. U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 14 Development Associates, Inc. A Possible Conference Result CLDP, in conjunction with the Ministry of Justice and the National Center for Judicial Studies held a national Judicial Conference on the adjudication of intellectual property issues in Egypt, and there was an unprecedented involvement of judges in the conference. Although it would too strong to assert a causal relation, one month after the conference, an Egyptian judge upheld an exclusive license for Eli Lilly, an action which is an encouraging step in the enforcement of intellectual property rights in Egypt. CLDP IMPACT ON USAID STRATEGIC OBJECTIVES Results Consistent with USAID Objectives USAID's objectives relevant to the work of the CLDP project are: ! improving the policy environment to increase trade and investment; ! developing sound laws, regulations, procedures and policies; ! moving Egypt toward a more globally competitive economy; and ! and strengthening the GOE's institutional ability to function more effectively in the World Trade Organization (WTO). During CLDP's existence, the Egyptian government has created a WTO unit within the Ministry of Foreign Trade, has become WTO compliant in some areas, has adopted a comprehensive intellectual property rights law, has moved to privatize insurance, and has moved toward a free market in the provision of Telecoms services and toward a regulatory role in spectrum management. All of these developments are in accord with the USAID objectives listed above – although it must also be said, at least in some cases, e.g., the intellectual property rights law, that it is too early to determine effectiveness and that major enforcement issues remain. These are also all developments to which CLDP contributed - sometimes substantially, as appears to be the case with regard to WTO matters, including standards, and in spectrum management issues – in various ways. Given the rather unique nature of CLDP assistance – U. S. government officials and employees charged with regulatory activities working with Egyptian government employees and officials transitioning to regulation in a market, as opposed to a state-controlled, economy – CLDP has been particularly useful. A number of interviewees remarked that Egypt needs to learn how to manage a market economy, how to regulate appropriately for such an economy, and that CLDP interventions in this regard have been most valuable. All the developments noted may have many fathers and midwives, and it is presumptuous to claim that but for CLDP, they would not have occurred. Nonetheless, interviewees did believe that CLDP played a significant role in these development gains. As one U.S. Embassy staffer put it, in his view, CLDP was likely to be brought in as a rapid response player when the Embassy saw an opportunity and a need, and when there was a likelihood of success. We took this to mean that the Embassy sometimes sought to use CLDP to bring other technical assistance efforts to fruition or to capitalize opportunely on possibilities opened by other such efforts. U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 15 Development Associates, Inc. Given the state of Egypt's economy and the legal and institutional infrastructure in Egypt that supports a market economy, it is impossible to claim that CLDP's technical assistance efforts, including training, had a measurable on impact on trade, investment, and sustainable development. We do believe that it has had a positive impact, and it is best to explain that through an example. Egypt is clearly moving to make the telecommunications sector competitive. As a part of this effort, the Ministry of Communications and Information Technology (MCIT) has adopted market supporting policies of spectrum frequency regulation, modeled on FCC type regulation yet adapted to the Egyptian context. There is increasing private investment in Telecoms services in Egypt, and there will undoubtedly be more as parties buy frequencies and also when Egyptian Telephone loses its monopoly on international connections in 2005. If the United States and Europe are any examples, given its low rate of teledensity, Telecoms is, and will continue to be a growth industry in Egypt and would seem to be a case of sustainable development. CLDP’s provision of spectrum advisors and trainings on spectrum issues was very useful to the MCIT and the Egyptian Telecommunications Regulatory Authority on spectrum issues, enabling them in their respective roles as Telecoms policy-maker and Telecoms policy implementer. Much more is needed, of course, but the point is that CLDP's technical assistance moved them in the right direction at the right time. Other Training Impacts and Skill Development In the evaluation team's interviews with participants we learned of other impacts the various training and information sharing opportunities had on Egyptian participants in addition to those cited above. As mentioned earlier, effective training programs close the gap between where people are and where they need to be in terms of changing requirements in the workplace. In terms of CLDP, these changing requirements are dictated by Egypt's movement into the arena of World Trade and the requirements that brings as well as other initiatives which have implications for commercial law. The visit to the World Bank during BOT/BOOT Consultations in Washington, DC, for example, gave examples of BOT projects and possible barriers such as the investment environment. International regulations about BOT were explained. E-commerce practices, especially in the US, were also explained. This information will assist in the review decision about BOT projects, since Egyptian personnel need to know the purchase and procurement regulations. Participants from the Patent Office feel they are now in conformity with TRIPS. They said that they learned more about trademarks, copyrights, and industrial design, and they have shortened the time for issuing patents from 6 years to 3 years, 9 months. They also learned more about searching for patents in a more effective way, such as researching biochemistry applications. It was their judgment that the training has helped not just technical examiners, but all staff. Another important impact of CLDP program activities, although it may not be unique to CLDP, has been the enhancement of Egypt’s role as a regional source of new ideas, procedures, and training. What Egypt does influences the entire Arabian area. EOS, for example is the Arabic region leader in food standards and the Near East representative in international standards organizations. For other countries in the region, it operates as an intermediate source organization, does training in other Arabian countries, or trains individuals from other Arab U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 16 Development Associates, Inc. countries in Egypt. The Egyptian Patent Office also does training for other Arab countries, and its patent examiners feel that their CLDP training helped them in this regard. GOE PARTICIPATION IN THE CLDP PROGRAM The following Egyptian Ministries, Departments, and organizations have participated in CLDP technical assistance activities. ! Ministry of Foreign Trade (MOFT) ! Ministry of Foreign Affairs ! Egyptian Organization for Standardization and Quality Control (EOS), in the Ministry of Industry and Technological Development ! Ministry of Agriculture and Land Reclamation ! Ministry of Health and Population ! Ministry of Finance ! Government Procurement Agency, Ministry of Finance ! State Council ! Ministry of Justice ! Ministry of Electricity and Energy ! Ministry of Transportation ! Egyptian Patent Office, Academy of Scientific Research and Technology ! General Organization for Export and Import Control (GOEIC) ! Ministry of Culture ! Cabinet Information Decision Support Center ! National Centre for Judicial Studies ! Industrial Designs Office, Ministry of Supply and Internal Trade (MOSIT) ! Trademark Office, (MOSDT) ! Egyptian Insurance Supervisory Authority (EISA) ! Ministry of Planning ! Ministry of Communications and Information Technology (MCIT) ! Telecoms Regulatory Authority ! Central Bank ! Institute for Diplomatic Studies ! Capital Market Authority ! Ministry of Tourism ! Ministry of Industry and Mineral Wealth ! Egyptian Center for Economic Studies Of all these entities, CLDP worked most with MOFT, EOS, EISA, and MCIT. Representatives of all of these entities who had worked with CLDP or had participated in its activities, whether advising, study tours, or training, were highly enthusiastic about CLDP and prized their relationships with it. As far as we can discern, they are fully cooperative with CLDP – at least to the extent of their abilities, given their positions, bureaucratic issues, governmental policies, and willingness to change. These are good CLDP partners, and all wished CLDP to increase its activities with them in the future. The extraordinary range of Egyptian actors that CLDP worked with in one way or another reflects the range and potential impact of CLDP activities. We should note that some of these U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 17 Development Associates, Inc. involvements were not substantial, in the sense that some of these entities were but participants in conferences or large seminars. Notwithstanding, because CLDP often dealt with problems or issues that involved more than one Ministry, Department, or organization, it is clear that CLDP brought together parties to provide and share important information, acquire knowledge, and to facilitate discussions between different stakeholders that would not otherwise have occurred. FEASIBILITY OF USING CLDP METHODS MORE WIDELY We were asked to evaluate whether CLDP's methods could be extended to wider target groups and used in other GOE Ministries and Departments. One of our major conclusions has been that CLDP is a general utility player most useful and effective as a complement to other technical assistance efforts, in government official to government official interactions and trainings, and as a flexible technical assistance instrument deployable opportunistically. Given this conclusion, our answer to USAID's question is a qualified yes. As CLDP works quite effectively when other parties open and identify opportunities for it, extending use of CLDP depends principally on other actors' awareness of its capabilities and on their development of working relationships with CLDP. Viewing USAID's technical assistance efforts strategically, e.g., like a general deploying forces, USAID could act proactively and insert – more likely, offer CLDP services to technical assistance projects that do not now use them. In other words, depending on CLDP's own capacities, USAID could use CLDP in support of various other technical assistance activities. We cannot judge the appropriateness of using CLDP in support of other activities – such as the Partnerships for Competitiveness or Economic Reform, or the Project for Technical Assistance to Support Economic Reform, or the Project for Assistance for Customs and Trade Facilitation – because we have not developed familiarity with these projects. Perhaps USAID should explore the use of CLDP in them. Alternatively, and perhaps more desirably given USAID's workload, it could invite CLDP to attempt to work with a larger number of existing and projected USAID projects and to explore possibilities with other GOE Ministries and Departments. In this way, it would depend on CLDP to devise an expanded or extended workplan subject to USAID approval, as has been customary. This could, of course, present a problem as well, for any party that focuses on too many tasks does well at none. While CLDP has been something of a jack of all trades, it might be better to focus CLDP’s considerable capacities on a few high value targets, such as the proper role of government as a regulator of private markets, something we discuss in our recommendations for future activities. This, of course, is a USAID judgment call. One practical example of how this could happen would be for USAID to take a cross-cutting issue such as IPR and identify how and in what way various different Ministries and Departments are affected by it. This could be further refined by cross-referencing USAID projects already in place with those agencies. USAID project managers could then examine with their ministry counterparts what training and development needs exist with regard to understanding the laws, regulations, and procedures. They could then have a roundtable discussion with CLDP about how its resources could be brought to bear and which methodologies would be most appropriate for each target group. As part of this, opportunities to bring people from different ministries together in the training setting should be identified. By U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 18 Development Associates, Inc. taking the same issue area and reaching out to other ministries, they might even identify synergistic ways of implementing policies and procedures which are not in conflict with each other. SUSTAINABILITY OF CLDP TECHNICAL ASSISTANCE EFFORTS Some of the technical assistance work CLDP has done, in conjunction with other USAID technical assistance projects, is naturally or inherently sustainable. This is because of clear Egyptian commitments and because the technical assistance contributed to structural changes in Egyptian Ministries or Departments. Egypt’s commitment to the WTO and Telecoms and insurance privatization; and the consequent creation of a WTO unit within the Ministry of Foreign Trade, passage of an IPR law, and the creation of a market in frequencies are all examples of this. The WTO related work on patents and food standards also appears to have aided permanent changes in the workings of the Patent Office and of EOS. There also have been notable changes in the way that EISA manages public and media relations, an important aspect of insurance regulation. Whether CLDP technical assistance activities in other areas, where commitments such as those described above were lacking, have had sustainable effects is more questionable. Some respondents, when questioned about whether they could maintain the momentum which CLDP has created in their Ministries without USAID's continuing funding, were noncommittal. This suggests that one obvious key to sustainability is Egyptian commitment, either a clear governmental policy commitment such as that regarding Telecoms, or a more open-ended commitment such as the WTO treaty commitment, where it is more likely that Egypt did not initially completely understand its obligations, but has progressively come to do so. The other plain key to sustainability seems to be technical assistance that results in structural or operational changes within a Ministry or Department. These appear to create a kind of path dependency that is hard to reverse. The creation of new functions that actually deal with real and emerging problems creates a field of work and constituencies that didn’t, bureaucratically speaking, exist before. That in turn creates its own kind of learning and new needs and problems and leads to progressive developments. One interviewee suggested that one way to sustain technical assistance gains would be to establish units, such as the WTO unit, in various ministries where CLDP proposes to do substantial work. While this certainly worked with MOFT, the circumstances there were special, perhaps unique, and it is unclear whether other ministries or departments perceive such a need. The closest candidate for such an intervention appears to be the Frequency Management and Wireless Communication Administration in the Ministry of Communications, but even here the need appears to be less for a so-called unit than simply the presence of an advisor for several months. Another way to create sustainability for the program and insure the spread of information would to provide a training of trainers course for CLDP participants who have a responsibility for disseminating information learned to others. Personnel in both the Standards office (EOS) and the Patents group both spoke of their responsibility to share new techniques with colleagues within their ministries and throughout the Middle East. They expressed a need, and we concur in its importance, that they learn techniques for more effective transferal of the information they received. Inasmuch as many interviewees spoke of the need for hands-on training, a training U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 19 Development Associates, Inc. course that demonstrates adult education design and facilitation models and techniques and that offers practice in those techniques through using relevant case material seems highly appropriate. This would strengthen participants not only in their grasp of ideas originally learned in the training events, but also ensure that their transmission to others would be accurate and effective. CONCLUSIONS While the conclusions below state important findings from this evaluation, they should also be considered in conjunction with the recommendations that follow this section, for the recommendations comprise a form of “actionable conclusions”. 1. CLDP, in conjunction with other USAID technical assistance projects, has contributed positively to achieving USAID’s SO 16 objectives. In particular, CLDP made a contribution to positive trade and market-enhancing structural and operational changes in MOFT, EOS, EISA, the Egyptian Patent Office, and MCIT. 2. CLDP is an effective and flexible technical assistance program capable of meeting specialized and emergent needs, and, if possible, USAID should continue it. 3. CLDP is most effective when working together with other major USAID technical assistance projects. 4. USAID should not view CLDP primarily as a trainer, but rather as a general utility technical assistance player that it can deploy for emergent needs and special projects that other USAID projects, for whatever reason, cannot manage. 5. CLDP’s combination of coordinated advising, consulting, and specific related hand-on training was its most effective means of producing positive results. 6. Egyptian authorities that have worked substantially with CLDP have cooperated fully and enthusiastically. They value working with CLDP highly and wish to continue to do so. 7. All substantial SO 16 projects should become aware of CLDP’s capabilities and capacities. 8. All substantial SO 16 projects should be asked to consider carefully whether they can make effective, particularized use of CLDP 9. CLDP can help existing USAID projects deal with private sector issues and help Egyptian ministries educate the private sector and involve them in desirable policy changes. 10. To a certain degree, Egypt is moving to liberalize its economy and free at least some of its markets. Many Egyptian regulators do not understand how to regulate appropriately to provide a regulatory framework in which markets can operate fairly, efficiently, and for the benefit of the consumer. CLDP has a potential role here. 11. The whole of CLDP work on WTO related matters, including work on food standards, TRIPS, and IPR constituted a virtually ideal program of technical assistance. CLDP U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 20 Development Associates, Inc. worked together with existing USAID technical assistance projects to define needs they could not readily meet, devised a clear and detailed plan involving a number of different ministries and departments, provided technical assistance and training to meet those needs, and provided follow-up interventions, to secure or further gains. 12. While many respondents reported that study tours were quite useful, as a kind of comparative learning where participants gained merely by seeing how things were done differently in the United States, some participants did not find observation tours particularly useful, wanting instead to get practical training. The larger conclusion is that Egyptian officials primarily want, and need, practical and hands-on training. 13. Conferences are mostly useful when held as part of a larger coordinated technical assistance effort having well-defined aims in terms of specific results or when designed to impart significant information to an important audience not otherwise likely to receive it. RECOMMENDATIONS 1. Relationship to USAID and USAID Projects ! CLDP appears to be a flexible technical assistance tool that USAID can use to help it achieve its development goals for Egypt. In many areas of CLDP activity, it appears that CLDP initiated discussions with various actors in Egypt, including USAID, USAID projects, the U.S. Embassy, and Egyptian officials in order to devise a workplan. Afterwards, CLDP would present its workplan to USAID for approval. While this is appropriate, and a normal course of events, USAID may wish to consider whether it should be more proactive in its use of CLDP. If USAID thinks of its various projects, including CLDP, as a team, and CLDP as a general utility player, it can, like a coach, insert CLDP, strategically or tactically, where it thinks it is most useful. ! CLDP workplans should articulate specifically how CLDP's proposed activities fit together with other USAID technical assistance. ! CLDP workplans and activities do not seem as clearly integrated into the USAID SO 16 results framework as they might be. This integration might be better achieved through revisiting CLDP objectives and reframing them to reflect SO 16 objectives and through providing measurable indicators identifying what will be different as a result of CLDP activities. ! CLDP has helped some Egyptian officials and agencies to network with U.S. government organizations and officials. These contacts provide Egyptian officials and agencies continuing assistance. It could use the DT2 or IIE for continued and more structured follow-up to activities such as conferences and study tours. ! CLDP has developed excellent relationships with a good number of Egyptian officials and agencies. Despite working in a difficult environment, CLDP has leveraged its relationships with GOE ministries to increase the probability of success. It can and should use these relationships to assist with new initiatives in the economic growth arena. U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 21 Development Associates, Inc. 2. Training ! CLDP should employ the services of a training expert to assist it in devising training plans in all its training activities. In general, training plans should have specific objectives, have measurable outcomes, and should be realistic. To the degree that we could locate CLDP training objectives and infer training plans, we found them to be general and not contain measurable outcomes. For specific trainings, CLDP should consider using pre and post-training tests to assess skill and knowledge acquisition. (Cf. Appendix IV for a detailed analysis of training issues and recommendations for the future.) ! The conference, seminar and training materials that CLDP provides range from the quite good to the excellent, with the best materials providing background information, source material, e.g., WTO agreements, explanations, problems, simulations, and exercises. Given a recurrent refrain from Egyptian officials and other interviewees that Egyptians need practical training rather than theory, we recommend that CLDP always attempt in its trainings to provide problems, simulations, case studies, and practical exercises. Conferences, of course, do not always serve the same purposes as trainings, and are often intended primarily for information dissemination. To the degree that there is a training element in conferences, however, they should include these methods for getting participants to grapple with real problems they may face, such as CLDP’s conference on judicial enforcement of IPR did when it included a case study and a mock trial. ! While there was a quite positive response to most CLDP organized training tours, not all participants found them equally valuable. Egypt sometimes sends mixed groups on these training tours. Where there is a larger group of trainees composed of subgroups with different specializations, CLDP should seek to tailor training sessions according to the needs of the subgroups.\ 3. Training Methods ! While there is undoubted value in demonstrating new technologies, skills, or software and in showing how things are done in the United States, where possible, CLDP should always seek to incorporate some hands-on practical training. ! For observational study tours, CLDP should ensure that trainees receive a pre-training orientation explaining what they are to see and why it is important, and, following this, give trainees to discuss what they believe is most important for them to see. This will help refine study tours so that they will be most beneficial to the trainees. 4. Technical Assistance Lead-Ins and Follow-Ups ! Short-term training, etc .— Short-term training, conferences, study tours are most effective when integrated into a planned series of technical assistance activities and are appropriate lead-ins and follow-ups to them. ! Internships — One Ministry wondered if CLDP could use their in-US contacts for providing 2 or 3 months internships for GOE personnel to work in the States with US U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 22 Development Associates, Inc. organizations Alternatively, U.S. experts could come here for 2-3 weeks for intensive "hands-on" assistance. ! Seminars — A best practice which is prevalent in some ministries and which we would recommend be done more systematically is to follow the study tours with seminars in Cairo conducted by participants. This would be an opportunity for everyone to share their experience and would perhaps generate discussion leading to change. Perhaps the DT2 mechanism could be used to facilitate this process and disseminate the results. The key to the success of these follow-up activities will be the management and supervisory personnel in the Ministries. As CLDP staff continues to develop their relationships with these officials, they could recommend that these follow-up activities be formally structured as part of a total learning package. A side benefit of structuring this follow-up more formally is that it would provide both CLDP and USAID with more clearly identifiable results of the CLDP activities. 5. Evaluation ! If CLDP is extended for a substantial period of time, there should be periodic evaluations. ! CLDP should file a copy of its training needs assessments, training workplans and materials, and pre and post training evaluations with an appropriate entity in Egypt, e.g., the Results Reporting Support Activity. ! CLDP work and training plans should reflect the USAID SO 16 results framework and objectives. They should include measurable indicators identifying what will be different as a result of CLDP activities. ! It would be useful for CLDP or some other entity to conduct post-activity and post￾training evaluations, within an appropriate period of time, focusing on skills, behavioral, performance, and procedural changes. The aim here is not just to assist evaluations of CLDP, but to assist CLDP in determining its effectiveness. 6. Future Activities ! Private Sector Activities — Particularly in the trade area, CLDP could complement the work of ATR by focusing some of its activities on the private sector. While Egypt's Ministry of Foreign Trade appears committed to reducing trade barriers, certain parts of the private sector remain to be convinced that Egypt should remove various protections and high tariffs that remain barriers to trade. Except for leaders of very large private enterprises, many in the Egyptian private sector do not understand the benefits of trade barrier reductions, nor realize what opportunities might be created either for them or for Egypt. Unpersuaded, they comprise a strong interest group opposed to any change that they suspect may harm them or expose them to unwelcome competition. In other words, interventions are needed for the private sector to understand open market benefits and competition. In this respect, the prospect of a free trade agreement between the United States and Egypt, somewhat diminished recently, creates another potential large opportunity for CLDP, not only with respect to Egyptian ministries, but also with respect to the private sector where much resistance and many impediments lay. (As one person U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 23 Development Associates, Inc. noted, speaking of potential private interests battles over the prospect of free trade in a context of various protections for domestic industries, "If you could get the textile sector behind you, for example, you could win.") ! Regulatory Matters — It is important to recognize that even with the best of intentions regarding market liberalization, Egyptian officials, who worked and advanced under a regime of state control of the economy, simply do not know how to regulate for free, or freer, markets. In each sector where Egypt is moving from state control of the economy to appropriate state regulation of the market, Egypt needs substantial help in learning how to regulate. Essentially, it needs all the authorities, procedures, and tools that U.S. regulators use: legal authority, appropriate regulatory structures and procedures, data collection processes and data analysis, and administrative adjudicative procedures. This also includes a clear designation of functions and activities, job descriptions, analytic procedures and tools, review structures and even assistance in defining the nature of work products. ! Competition Matters — A necessary corollary to the foregoing is that appropriate Egyptian officials need substantial training in the fundamentals of competition economics and appropriate legal tools to insure that freed markets operate fairly, efficiently, and in the interests of the consumer. Egypt may yet be some significant distance away from enacting a general competition law. Nonetheless, to the degree that Egypt is creating free markets in any sector, it is important for regulators in that sector to create and adopt regulatory tools to insure that anticompetitive practices neither dominate nor even gain beachheads in these markets. ! IPR and Patents — Both judges and lawyers need more training in patents and other IPR issues. This is a very new area, and the IPR law has been passed but executive regulations and procedures have not been published. Technical assistance would be useful in drafting or reviewing these regulations, and more training will be needed after they are published. Personnel working with patent procedures also acknowledged that they need training on new techniques: e-filing, e-techniques, e-payments, equipment, and website security. ! Copyright — The copyright law, which will become effective in 2005, involves a particularly troublesome area. Some outside observers consider this to be the weakest part of the IPR law and are particularly concerned with enforcement. It is unclear whether the Ministry of Culture, which is responsible for enforcing the copyright provisions of the law, is ready for technical assistance and whether the preconditions for effective technical assistance are there. This should be investigated further, and presents a possible area for substantial technical assistance. ! Customs — USAID is providing substantial technical assistance in customs reform, a large Problematic area. This might be an area where CLDP can partner effectively with the USAID project. ! SMEs — Small and medium enterprises need: a) more information on production and quality issues; b) training in marketing and packaging; and c) training in how to import and export to the United States. We unclear whether CLDP would be an appropriate U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 24 Development Associates, Inc. vehicle for the former two items as these seem to be mostly private enterprise matters in the United States, but CLDP could certainly help with the last item. ! Trade Matters — Egyptian ministries involved in trade matters would like to see more training on subsidies and safeguards analysis, on how to prepare for complex trade negotiations, including negotiating with domestic interest groups, on inquiry point operations, trade remedies, trade dispute resolution, and on the operation of international financial institutions such as the IMF and World Bank. They would in particular welcome being led through, or assisted in conducting, complex case studies. On the other hand, USAID and the GOE have put a great deal of funding and effort into WTO and trade￾related issues, and USAID has a significant ongoing trade project, ATR, doing this work. We have been informed, as well, that the GOE would like more development focus on areas that have not been as well-funded as trade-related matters. Under the circumstances, this seems an appropriate role for ATR rather than CLDP. CLDP should focus on such other areas. ! Commercial Attachés — The Ministry of Foreign Trade believes it is still important for Commercial Attachés to go to the US. It believes that the attachés learned to be more astute and professional in dealing with foreign businessmen, how to work faster, to understand market systems, and how to be more effective representatives for Egypt. ! Miscellaneous — Other areas identified where technical assistance needed were continued work on the practical aspects of BOT projects and IT systems, with virtually all Egyptian respondents asking for the latter. ! Facilitation — Some other areas for future assistance may not exactly fall under the rubric of CLDP but they certainly impact the process of trade reform in Egypt. There appears to be some conflict regarding WTO issues between Egyptian WTO representatives in Geneva and the WTO unit in Cairo. Apparently the WTO Unit, located in the Ministry of Foreign Trade, desires to be Egypt’s primary source of WTO expertise, while Egypt’s Commercial Representation Section in Geneva, a part of the Ministry of Foreign Affairs, asserts a similar role. CLDP suggested that the WTO Unit be located in Geneva, with the Commercial Representatives. In the event, it was located in Cairo within MOFT, and the conflict remains. The best assistance here would most likely be facilitation focused on a roles and relationship clarification and on consensus building rather than training. U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report 25 APPENDICES Appendix I: CLDP Technical Assistance Activities and Training ..................................I-1 Appendix II. List of Contacts and Interviewers................................................................II-1 Appendix III. List of Documents Reviewed.......................................................................III-1 Appendix IV. Overall Technical Review of CLDP Training Programs.............................IV-1 Appendix V. Data-Gathering Instrument ..........................................................................V-1 Appendix VI. Evaluation Methodologies...........................................................................VI-1 Appendix VII. Glossary........................................................................................................VII-1 U.S. Department of Commerce Law August 4, 2003 Development Project in Egypt Final Evaluation Report APPENDIX I CLDP TECHNICAL ASSISTANCE ACTIVITIES AND TRAININGS WTO RELATED ACTIVITIES Activity # & Aim Client Dates & Location WTO Trade Remedies Seminars 200 implementing WTO: antidumping & subsidies Ministry of Economy and Foreign Trade (MOEFT) September 20 - 24, 1998 Cairo and Alexandria WTO Agriculture Agreement Advisor 20 WTO agriculture obligations Ministry of Agriculture February 7 - March 4, 1999 Cairo ITC Trade Remedies Advisors 50 training on injury determinations Foreign Trade Sector, Ministry of Economy and Foreign Trade (MOEFT) March 14 - 25, 1999 Cairo Trade in Services Advisor Services liberalization; preparing for 2000 negotiations Ministry of Foreign Affairs (MOFA), Ministry of Telecommunications and Transportation, the Central Bank, Egyptian Insurance Supervisory Authority, Capital Market (Securities) Authority, the Electricity Commission, Ministry of Tourism April 25 - May 4, 1999 Cairo and Alexandria WTO SPS and TBT Agreement Seminar and Consultations 20 WTO agreement on sanitary & phytosanitary measures: how to create a standards regime Ministry of Economy and Foreign Trade (MOEFT), Ministry of Industry and Mineral Wealth (MOI) Cairo WTO Interagency Training 16 supporting creation of a unit within MOEFT to facilitate Egypt’s implementation of WTO agreements Ministry of Economy and Foreign Trade (MOEFT) June 2 –9 , 1999 Wash. DC WTO Unit Advisor for MOTS 20 same as above Ministry of Economy and Foreign Trade (MOEFT) July 6 - 30, 1999 Geneva, and Cairo Trade Remedies Training in US 9 a study tour on implementing the WTO Antidumping and Subsidies Agreements and on the interagency process that supports the USG's efforts to bring and defend trade remedies cases. Foreign Trade Sector, became Ministry of Economy and Foreign Trade (MOEFT) July 7 - 17, 1999 Wash., DC MOFA Diplomatic Attaché Training in US 3 study the USG interagency process that supports active and effective participation in the WTO, Ministry of Foreign Affairs Aug. 2-6, 99, Wash. U.S. Department of Commerce Law I- August 4, 2003 Development Project in Egypt Final Evaluation Report 1 Development Associates, Inc. Activity #& Aim Client Dates & Location Conference on Trade in Services and Consultations ? trade in services to highlight WTO services activities, GATS 2000, the history of trade in services negotiations, upcoming electronic commerce issues, legal considerations for services agreements, and how to include more effectively the private sector in setting trade in services policies. Ministry of Economy and Foreign Trade (MOEFT), Egyptian Center for Economic Studies (ECES) September 19 -23, 1999 Cairo WTO Unit Advisor for MOTS 20 provided follow-on training to MOTS and help establish a WTO Unit within the Ministry Ministry of Economy and Foreign Trade (MOEFT October 4 - 29, 1999 Cairo, Egypt Trade Remedies Training for Judges 41 week-long workshop for twenty-five State Counsel judges and six MOEFT lawyers on substantive and procedural aspects of trade remedies cases. Ministry of Justice, Ministry of Economy and Foreign Trade (MOEFT Nov. 7-11, Cairo Program in Washington for MOEFT Officials 21 opportunity for commercial attaches to learn about promoting exports and increasing foreign investment in Egypt when serving at Egyptian embassies Ministry of Economy and Foreign Trade March 31 - April 12, 2000 Wash. , DC Special Trade and International Commercial Diplomacy Advisor on Services 20 conducted seminars and consultations on commercial diplomacy, trade-related capacity building and services Ministry of Foreign Affairs (MOFA) and Ministry of Economy and Foreign Trade (MOEFT) June 23 - 30, 2000 Cairo, Egypt WTO Unit Advisor 20 evaluative assessment of the WTO Unit and develop an Action Plan that identifies the necessary steps to make the WTO Unit fully operational. Commercial Representation Section, Ministry of Economy and Foreign Trade (MOEFT July 1 - 21, 2000 Cairo, Egypt Study Tour for Eight MOEFT Officials 8 consult on issues regarding Anti-Dumping and Countervailing Duty policy. Ministry of Economy and Foreign Trade July 13 - 28, 2000 Wash., DC U.S. Department of Commerce Law I- August 4, 2003 Development Project in Egypt Final Evaluation Report 2 Development Associates, Inc. Activity #& Aim Client Dates & Location Program in Washington for Eight MOFA Officials 8 training to diplomatic attaches on World Trade Organization agreements Ministry of Foreign Affairs September 1 - 15, 2000 Wash., DC WTO Dispute Resolution Conference and Consultations 200 conference for GOE trade officials on the rules and technical procedures of the WTO dispute settlement process Ministry of Economy and Foreign Trade (MOEFT) and Ministry of Foreign Affairs (MOFA) October 21 - 22, 2000 Cairo Consultations for Commercial Representation Officials 17 consultations for Egyptian Commercial Representation officials with US Government and private sector officials to learn about the American model of trade policy formulation and implementation Commercial Representation Section, Ministry of Economy and Foreign Trade (MOEFT) July 5 - 20, 2001 Cairo, Wash., DC U.S. Department of Commerce Law I- August 4, 2003 Development Project in Egypt Final Evaluation Report 3 Development Associates, Inc. Standards Activity # & Aim Client Dates & Location Training Workshop on the Hazard Analysis and Critical Control Point System for Foods 85 workshop at EOS to train representatives of the Government of Egypt and Egyptian food industry on the regulatory and industry roles required for a Hazard Analysis and Critical Control Point (HACCP) food safety system to be successful Egyptian Organization for Standardization and Quality Control (EOS), Ministry of Industry and Technological Development June 15 - 23, 2001 Cairo Standards Advisor 25 exchanging information regarding food safety and biosecurity. Egyptian Organization for Standardization and Quality Control (EOS), Ministry of Industry and Technological Development, Ministry of Agriculture, Ministry of Health July 8 - 19, 2001 Cairo Consultations for Standards Officials in Washington, DC 12 understanding how standards can create trade barriers; (2) harmonizing Egyptian standards with international standards; (3) assistance in establishing better policies and regulations; and (4) awareness of the need for private sector participation in adopting and formulating voluntary and mandatory standards. Egyptian Organization for Standardization and Quality Control (EOS), Ministry of Industry and Technological Development July 20 - August 4, 2001 Wash. D.C. Standards and Conformity Assessment Seminar 170 training to assist the Egyptian government in promoting a standards regulatory regime that encourages trade and investment and complies more fully with obligations set out under Egypt=s WTO agreements Egyptian Organization for Standardization & Quality Control (EOS), Ministry of Industry and Technological Development, Ministry of Foreign Trade (MOFT) January 21 - 23, 2002 Cairo Consultations for Standards Officials 10 consultation tour to evaluate the U.S. system of developing, regulating, and certifying standards Egyptian Organization for Standardization & Quality Control (EOS), Ministry of Industry and Technological Development June 6 - 22, 2002 Wash. U.S. Department of Commerce Law I- August 4, 2003 Development Project in Egypt Final Evaluation Report 4 Development Associates, Inc. Intellectual Property Activity # & Aim Client Date USPTO Patent Documentation Advisor 20 trained several officials on how to manage foreign patent documents, improve automated search and information resources, follow WIPO standards and documentation guidelines, and search patent information on the Interne Egyptian Patent Office, Academy of Scientific Research and Technology September 13 - October 1, 1998 Cairo TRIPS Implementation and Border Enforcement Training 40 conducted a comprehensive training program on TRIPS implementation and border enforcement General Organization for Export and Import Control (GOEIC), Ministry of Economy and Foreign Trade (MOEFT) April 11- 22, 1999 Cairo TRIPS Enquiry Point Training 4 training on how the USG implements the TRIPS Agreement, enforces intellectual property rights, and handles the technical aspects of enquiry point operations General Organization for Export and Import Control (GOEIC), e Ministry of Economy and Foreign Trade (MOEFT) May 3-7, 99, Wash. Protecting Computer Software in Egypt Conference and Consultations 245 Conference on the Protection of Computer Software in Egypt Ministry of Culture (MOC), MOTS, the Cabinet's Information Decision Support Center (IDSC) June 23-27, 99 Cairo and Alexandria Conference on Judicial Enforcement of IPR in Egypt 150 prepare Egyptian officials for their new responsibilities upon passage of a TRIPS-compliant IPR law, which is anticipated to set new standards in patents, copyright, trademarks, industrial designs, data exclusivity and undisclosed information. National Centre for Judicial Studies, Ministry of Justice :March 22 - 23 and March 25 - 26, 2000 Luxor and Cairo Industrial Designs Examination Advisor 20 reviewed current procedures in the Industrial Designs Office and made recommendations on staffing, databases, and documentation; also reviewed the draft Industrial Designs Law and provided parities with comments. Industrial Designs Office, Egyptian Patent Office, Academy of Scientific Research and Technology October 8 - 20, 2000 Cairo Automation Study Tour for Four GOE Officials 4 teaching officials how to effectively use Internet resources, and by reviewing successful strategies in automation that could be adapted for use in Egypt. Egyptian Patent and Trademark Offices, Academy of Scientific Research and Technology, Ministry of Supply and Internal Trade October 8-20, 2000, Wash. U.S. Department of Commerce Law I- August 4, 2003 Development Project in Egypt Final Evaluation Report 5 Development Associates, Inc. Activity # & Aim Client Date Visiting Scholars Program at USPTO 2 foster a better understanding of international intellectual property obligations and norms; to expose participants to at least one method of providing TRIPs-level protection for a variety of intellectual property disciplines Oct-Nov., 02 U.S. Department of Commerce Law I- August 4, 2003 Development Project in Egypt Final Evaluation Report 6 Development Associates, Inc. Procurement/BOT Activity # & Aim Client Date& Location International Conference on Successful BOT Projects in Egypt 100 conference designed to highlight the importance of this private funding mechanism, examine the benefits and challenges of BOT projects, and provide examples of policies and practices Ministry of Finance (MOF) November 14 - 16, 1999 Cairo Transparency in Government Procurement Training 14 studied how to create and promote a procurement regime with more transparency in making tenders, negotiating contracts, and resolving bid protests Government Procurement Agency, Ministry of Finance (MOF), Ministry of Trade and Supply (MOTS), Ministry of Foreign Affairs (MOFA) April 26 - 30, 1999 Washington, DC Public Procurement Advisor 5 reviewed bid protest law, regulations, practices and procedures; then made an in￾depth review of the Tenders Law (in conjunction with Decision No. 3549) to determine its transparency and practicality. Government Procurement Agency, Ministry of Finance April 14 - 28, 2000 Cairo BOT/BOOT Regulatory Advisors 20 worked with the Ministry of Finance to set up mechanisms that provide more consistency in the BOT/BOOT tender process between GOE ministries Government Procurement Agency, Ministry of Finance October 19 - 31, 2000 Cairo Public Procurement Officials to Washington for Consultations 4 training on transparency in government procurement; focused primarily on bid protest processes and contract disputes. Ministry of Finance and State Council November 3 - 11, 2000 Washington, DC BOT/BOOT Consultations 8 officials consulted with U.S. Government officials, project companies, suppliers, lenders and sponsors involved in BOT projects (domestic and foreign) for in-depth discussions on procedures, common problems and solutions Government Procurement Agency, Ministry of Finance, Ministry of Justice, Ministry of Electricity and Energy, Ministry of Transportation November 1 - 8, 2001 Washington, DC U.S. Department of Commerce Law I- August 4, 2003 Development Project in Egypt Final Evaluation Report 7 Development Associates, Inc. Insurance Activity # & Aim Days Effort Insurance Advisors in Cairo 20 discussions re establishing a program to assist the GOE in reforming Egypt's insurance sector Egyptian Insurance Supervisory Authority (EISA), Ministry of Economy and Foreign Trade (MOEFT September 19 - 23, 1999 Cairo Insurance Team in Cairo 20 a targeted assessment of the state of the Egyptian insurance sector, from both regulatory and business perspectives, followed by meeting with senior staff at EISA, the Barents Group, and senior GOE officials from MOEFT that oversee insurance regulation. Egyptian Insurance Supervisory Authority (EISA), Ministry of Economy and Foreign Trade (MOEFT) November 1 - 11, 1999 Cairo Insurance Rate and Form Filing Advisors 20 advice re how a regulatory agency monitors health and life rates; advice on the type of information EISA should seek from casualty and property insurers in a rate and form filing Egyptian Insurance Supervisory Authority (EISA), Ministry of Economy and Foreign Trade March 17 - 31, 2000 Cairo Insurance Study Tour in US for MOEFT and EISA Officials 6 opportunity for the Egyptian insurance officials to study the American model of a privatized insurance market. Egyptian Insurance Supervisory Authority (EISA), Ministry of Economy and Foreign Trade (MOEFT April 21 - May 4, 2000 Washington, DC, New York, NY, and Richmond, VA Conference on Reforming the Insurance Sector in Egypt 300 conference re the changes that will occur because of the country's ongoing privatization and how those changes will impact the market, including the need to create new products, marketing techniques, and distribution channels. Egyptian Insurance Supervisory Authority (EISA), Ministry of Economy and Foreign Trade (MOEFT) June 10 - 11, 2000 Cairo Insurance Study Tour in US for MOEFT and EISA Officials 18 program trained EISA officials on elements of insurance supervision, including solvency regulation, financial analysis, on-site inspections, and public relations Egyptian Insurance Supervisory Authority (EISA), Ministry of Economy and Foreign Trade (MOEFT) July 20 - August 5, 2000 Little Rock, Arkansas, Frankfort, Kentucky, New York, NY, Kansas City, MO Public Awareness Campaign ? design and implement a Public Awareness Campaign to explain the concept of insurance and its benefits Egyptian Insurance Supervisory Authority (EISA), Ministry of Economy and Foreign Trade (MOEFT) 2001 - 2002 Cairo U.S. Department of Commerce Law I- August 4, 2003 Development Project in Egypt Final Evaluation Report 8 Development Associates, Inc. Activity # & Aim Days Effort Data Collection Advisor 20 worked with EISA staff to review the status of automation and data collection at EISA, and to evaluate "next steps" in the process of ensuring successful implementation of various computer applications Egyptian Insurance Supervisory Authority (EISA), Ministry of Economy and Foreign Trade (MOEFT) June 19 - 28, 2001 Cairo Health Insurance Advisor 100 USAID-sponsored conference on the development of private health insurance and met with EISA officials to discuss the draft medical insurance law and the drafting of Executive Regulations for the law Egyptian Insurance Supervisory Authority (EISA), Ministry of Planning, Ministry of Health June 8 --14, 2002 Cairo Financial Analysis Advisor 20 trained EISA officials on the principles of financial analysis to ensure proper preparation of financial reports Egyptian Insurance Supervisory Authority (EISA), Ministry of Planning August 25 - 29, 2002 Cairo U.S. Department of Commerce Law I- August 4, 2003 Development Project in Egypt Final Evaluation Report 9 Development Associates, Inc. Finance Activity # & Aim Client Dates & Location Mortgage Advisors 20 assisted government in drafting the implementing regulations for the recently passed Real Estate Lending Law; also provided MOEFT with a comprehensive model for a real estate finance market structure that details the roles of commercial banks, real estate lending companies, securities companies Ministry of Economy and Foreign Trade (MOEFT) July 22 -31, 2001 Cairo, Egypt Egyptian officials to Fannie Mae International Housing Finance Symposium 5 Participants gained a broad understanding of all the integral elements of the U.S. housing finance system as well as the tools, techniques and business practices of a successful secondary market institution Ministry of Economy and Foreign Trade (MOEFT), Central Bank September 10 - 14, 2001 Washington, DC U.S. Department of Commerce Law I- August 4, 2003 Development Project in Egypt Final Evaluation Report 10 Development Associates, Inc. Telecoms Activity # & Aim Client Date & Location Spectrum Management Advisor 20 consulted with Egyptian officials in critiquing the current and proposed spectrum management organization and processes in the draft Telecom Law; also presented a one-week spectrum management seminar to improve knowledge of spectrum issues in anticipation of the passing of the new Telecom Law Telecom Regulatory Authority (TRA), Ministry of Communications and Information Technology (MCIT) January 25 - February 10, 2001 Cairo Legal Advisor on Spectrum 20 issues concerning the appropriate regulatory regime for efficient use of spectrum based on the draft Telecom Law " March 16 - 31, 2001 Cairo Officials to USTTI Courses 2 learn about issues facing national civilian telecom spectrum managers so as to enable them to make technically sound spectrum￾related decisions “ March 30 - April 14, 2001 Wash. Telecommunications Regulatory Authority Officials to US for Training 2 learn about radio monitoring methods as utilized within an administration’s communications regulatory department “ May 31 - June 15, 2001 Wash. U.S. Department of Commerce Law I- August 4, 2003 Development Project in Egypt Final Evaluation Report 11 Development Associates, Inc. Miscellaneous Activity # & Aim Client Dates & Location Building Capacity in Africa for Legal and Regulatory Reform to Promote Business Development 2 learn more about the relative strengths and weaknesses of African commercial systems, the concerns of potential investors and trading partners, and international commercial standards Egyptian Business leaders October 29 - November 2, 2001 Washington, DC, and Philadelphia, PA ATM Network Advisors 20 assisted the MCIT with the process of launching a pilot project designed to electronically link the Egyptian postal offices using an ATM network Ministry of Communications and Information Technology (MCIT March - June, 2002 US and Cairo Institute for Diplomatic Studies Attaché Training 34 enhance the technical skills of Egyptian diplomats in order to improve their effectiveness on issues related to foreign investment and trade promotion Institute for Diplomatic Studies (IDS), Ministry of Foreign Affairs August 4 - 8, 2002 Cairo U.S. Consultations for Egyptian Diplomatic Attaches 14 consultations for new diplomats with various US agencies, multi￾lateral organizations, and Fortune 500 companies on trade promotion and policy and foreign investment issues Institute for Diplomatic Studies (IDS), Ministry of Foreign Affairs September 13 - 28, 2002 Washington, DC and New York, NY U.S. Department of Commerce Law I- August 4, 2003 Development Project in Egypt Final Evaluation Report 12 APPENDIX II LIST OF CONTACTS AND INTERVIEWEES CONTACTS Organization: Egyptian Insurance Supervisory Authority Contact: Dr. Ragaii Hassan, IT Consultant Telephone: 2-02-574-1845 Organization: Egyptian Organization for Standardization and Quality Control Contact: Dr. Mahmoud Eisa, President Telephone: 2-02-259-3480 Organization: Ministry of Communications and Information Technology Contact: Dr. Tarek Kamel, Senior Advisor to the Minister Telephone: 2-02-344-4544 Organization: Ministry of Communications and Information Technology Contact: Dr. Sherif Hashem, Director, Information Society Development Office Telephone: 2-02-344-4546 Organization: National Center for Judicial Studies Contact: Judge Aly El Sadek, Director Telephone: 2-02-262-1052 Organization: Ministry of Foreign Trade Contact: Abdel Rahman Fawzy, Undersecretary Telephone: 2-02-203-7817 Organization: Ministry of Foreign Affairs Contact: Ambassador Magda Shahin Telephone: 301-361-8612/361-8613/362-1905 Organization: Government Procurement Agency, Ministry of Finance Contact: General Mohamed Shobokshy Telephone: 2-02-336-0150 Organization: Institute for Diplomatic Studies Contact: Sherif El Fathy, Deputy Director Telephone: 2-010-679-5450 Organizations: Egyptian Patent Office, Academy of Scientific Research and Technology, Ministry of Higher Education and Scientific Research; Egyptian Trademark Office, Ministry of Supply and Internal Trade Contact: Dr. Fawzi A. El Refaie Telephone: 2-02-792-1286 U.S. Department of Commerce Law II- August 4, 2003 Development Project in Egypt Final Evaluation Report 1 Development Associates, Inc. Organization: US Embassy Cairo Contacts: Steve Bondy, Second Secretary; Andy Starks, First Secretary Telephone: 2-02-797-3837; 2-02-797-2761 Intellectual Property Officials: Dr. Fawzi El Refaie Academy of Scientific Research & Technology 101 Kasr El Eini 792-1324, 1323, 1291 Study Tour for BOT Mr. Amin Sabry Ministry of Economy & Foreign Trade 96 Ahmed Orabi Street, Aziz Street Mohandessin 202-305-7169 305 7168 Commercial Service Training Mr. Mohamed El Far General Manager for General Authority of Governmental Services 7 Dr. Mohamed Hamed Fahmy St Dokki, Giza 010-144-0444 INTERVIEWEES AT CLDP IN WASHINGTON, D.C.: Stephen Gardiner, Deputy Chief Counsel Jannette Hasan, International Attorney Advisor Stuart Kerr, Senior Counsel Geneva Pham, International Program Specialist Abigail Dos Santos, International Program Specialist INTERVIEWEES IN EGYPT Jim Norris COP TAPR 202 792 2712 Mohamed Sherif Omran Senior Agricultural Economist USAID 522-6614 U.S. Department of Commerce Law II- August 4, 2003 Development Project in Egypt Final Evaluation Report 2 Development Associates, Inc. Dr. Mahmoud Eisa Egyptian Organization for Standardization and Quality Control 603-1351 Ali Elhariri Ekbal Commercial Counselor Director of European Union and Western Europe Department Ministry of Foreign Affairs 3468815 Yasser Korany Commercial Attaché European Union and Western Europe Department Ministry of Foreign Affairs 347 1890-6 Ragaai Hassan IT Consultant Egyptian Insurance Supervisory Authority 574 1845 Manala El Samadony Senior Economist USAID 522 6689 Andrea Starks First Secretary U.S. Embassy 797-2761 Abdel Rahman Fawzy Undersecretary, Ministry of Foreign Trade 203 7817 Frank A Szumilo COP Data Access & Transmission Activity 260 1687 Gamalat Talaat Mohamed Commercial service training participant Samir Abdel Rahman Commercial Service Training Participant U.S. Department of Commerce Law II- August 4, 2003 Development Project in Egypt Final Evaluation Report 3 Developmen t As s ociates, Inc. Dr. Hisham ElDeeb Egyptian Patent Office Intellec tual Property Rights training 331 0502 Rachid Benjelloun, International Trade Specialist Assistance for Trade Reform Project 204 1034 Abdel W ahab Heikal Senior Trade Specialist Assistance for Trade Reform Project 204 0995 Steven Bondy Fi r s t Secretary US Embassy 797-3837 Ha m ed Fahmy Managing D irector Allied Corporation Egypt 749 6144 Hatem Reda Economic Analyst Ministry of Foreign Trade Trade Agreements Sector 201 0284 Ms Amany El –Wassal Head of Technical Office Ministry of Foreign Trade Trade Agreements Sector 201 0284 Ism ail A. El-Ghetany Frequency Managem ent and W ireless Communication Adm inistration 308 8206 Ibtisam A. El Gharieb Training Consultant Institute of International Education 337 7684 U.S. Department of Commerce Law II- August 4, 2003 Developmen t Project in Egypt Final Evaluatio n Report 4 APPENDIX III LIST OF DOCUMENTS REVIEWED STUDY TOUR & CONFERENCE MATERIALS GATS 2000: Issues, Coordination and Private Sector Input for Trade in Services Policies (Conference Materials, Sept 21-22, 1999) Judicial Enforcement of Property Rights in Egypt (Conference Materials, March 25-26, 2000) Creating a Global Marketplace: The Egyptian Insurance Sector in Transition (Conference Materials, July 10-11, 2000 US Study Tour for Egyptian Diplomatic Attaches, Ministry of Foreign Affairs (Study Tour Materials, Sept. 1-15, 2000) ADVISING MEMORANDA Public Relations Strategy for the Egyptian Insurance Supervisory Authority (Loretta Worters and Carolyn Gorman, Insurance Information Institute), March 15, 2001 A Review of the Draft Medical Insurance Law of the Arab Republic of Egypt, (Gwendolyn L. Fuller, South Carolina Department of Insurance), Aug., 2002 Report on Technology and Automation at the Egyptian Insurance Supervisory Authority (EISA) (Denise Matthews, National Association of Insurance Commissioners) Analysis of the Draft Egyptian Anti-Trust and Anti-Monopoly Law (FTC and DOJ authors), Nov. 19, 2002 Report on the Egyptian Patent Office Patent Documentation (Christopher Kim, United States Patent & Trademark Office), Oct. 11, 1998 Action Plan for WTO Unit (Irving Williamson, CLDP), July 24, 2000 Answers to Inquiries from the Egyptian Ministry of the Economy and Foreign Trade Concerning the Real Estate Finance Law (Fannie Mae, International Housing Finance), July 30, 2001 An Analysis of Telecom in Egypt and the Draft Unified Telecommunications Law for the Arab Republic of Egypt (Roxanne M. McElvane, Senior Counselor, US Federal Communications Commission), Sept., 2001 Developments in Electronic Commerce in Egypt (Milton Brown, Deputy Chief Counsel, National Telecommunications and Information Administration, US Dept. of Commerce), Dec. 2002 U.S. Department of Commerce Law III- August 4, 2003 Development Project in Egypt Final Evaluation Report 1 Development Associates, Inc. Final Report – Egyptian Post ATM/POS Project (J.E. Leiponis) June 28, 2002 Implementation Plan for the WTO Unit (Irving Williamson, CLDP), Dec. 29, 1999 Establishing a WTO Unit to Strengthen Egypt's Participation in the WTO (Irving Williamson, CLDP), Aug. 23, 1999 Report of Advisor, Ministry of Communication and Information Technology (David Martin), July 15, 2003-07-17 WORKPLANS AND REPORTS Egypt Progress Report for the Quarter Ending June 30, 1998 Egypt Progress Report for the Quarter Ending September 30, 1998 Egypt Progress Report for the Quarter Ending March 31, 1999 Egypt Progress Report for the Quarter Ending June 30, 1999 Egypt Progress Report for the Quarter Ending September 30, 1999 Technical Assistance Status Report and Updated 1999 Workplan 2000 Annual Report for Egypt Revised Technical Assistance Workplan for January 2000 through September 2001 2001 Annual Report for Egypt Egypt Half-Year Progress Report for the Period Ending December 31, 2002 Program History in Egypt 1998-2002 Technical Assistance Workplan for March 2002 through December 2003 Egypt Progress Report for the Period Ending June 30, 2002 Egypt Half-Year Progress Report for the Period Ending December 31, 2002 Statement of Work, Commercial Law Development Program, CLDP, undated Monitoring and Evaluation End of Training Report, Standards Officials Study Tour to US, June 6-22, 2002, Institute of International Education Participant Information for Egypt Programs 1998-2002, CLDP U.S. Department of Commerce Law III- August 4, 2003 Development Project in Egypt Final Evaluation Report 2 Development Associates, Inc. End of Training Report, Consultative Visit to US on Food Safety for EOS Officials, Feb. 27 – March 13, 2003, Institute of International Education End of Training Report, Insurance Officials to State Insurance Departments, May 29-June 14, 2003, Institute of International Education U.S. Department of Commerce Law III- August 4, 2003 Development Project in Egypt Final Evaluation Report 3 APPENDIX IV OVERALL TECHNICAL REVIEW OF CLDP TRAINING PROGRAMS By Ahmed Abdel-Fattah (M.A.) Training Expert The CLDP - Training Evaluation Team reviewed available documents and interviewed participants who went through these programs and their supervisors, in order to collect qualitative and quantitative data required for a training evaluation. Reviewed documents compris included CLDP Work Plans, quarterly and semi-annual and annual progress reports; IIE/DT2 training specification forms, end-of-training reports, including participants’ reactions, levels of satisfactions, and learning. Data collection, processing and analysis resulted in the following twenty observations in terms of training planning, implementation, evaluation of direct results and impact: IN TERMS OF TRAINING PLANNING 1/1 IIE/DT2 developed a “Training Specification Form” for each training event. This form could be considered as a preliminary or skeletal design to be finalized or fleshed-out filled in by the training provider. The form specifies the training program ID, title, type, field of study; partner institution and its work units; participants’ number, categories, positions, and areas of expertise. It gives a relatively detailed s background information and justification for the training program particularly in terms of current work-performance in relation to desired performance. Also, the form explains expected results to occur within one year, measures to be used, and non-training factors that may hinder the achievement of results. Moreover, the form details training objectives to be achieved by the end of the training program. These objectives are trainee-based. The form may highlight the program contents, instructional strategies, location, projected start and end dates, duration, number of events, and finally the form defines logistical and administrative support requirements. 1/2 It was concluded from reviewing the available training specification forms and interviewing participants and their supervisors that this design was built upon a “Functional Approach” particularly in its justification and consequently its targeted objectives and expected results, and participants’ nomination and selection made by client institutions. 1/3 The Functional Approach to justify the need for training, or to identify training needs is accepted at the early stages of a program. Later on, as programs become more specific, training events should use a combined “Functional-Personal Approach” to identify training needs, and consequently develop specific, realistic, achievable, measurable and time-based objectives. Interviewed participants started that they were not asked about their specific personal training needs. One EISA Official stated that he shared ILE in developing one “Course Outline”. U.S. Department of Commerce Law IV- August 4, 2003 Development Project in Egypt Final Evaluation Report 1 Development Associates, Inc. 1/4 It was noticed that CLDP there was had no “Master Training Plan” with “Sectorial Training Programs” detailed in “Training Events”, well-coordinated vertically and horizontally. In this case it had such a plan, CDLP could regulate its training component, rationalize its cost, avoid potential overlap or droppageomissions- and coordination and promote training effectiveness and efficiency. IN TERMS OF TRAINING IMPLEMENTATION 2/1 In the absence of a “Master Training Plan”, it becomes very hard to assess the level, degree, or percentage of achievements; and consequently, the level of effectiveness and efficiency of the program. 2/2 Also, in the absence of “Final Reports” prepared by the “Training Provider” explaining different steps of preparatory activities and implementation compared to planned activities and time-table for each training event; it becomes very hard to assess the implementation of the given training event. However; 2/3 A group of the Egyptian Insurance Supervisory Authority (EISA) participants developed a detailed day-by-day record of all activities implemented in their Observation Study Tour (OST). This factual record was developed in order to inform their supervisor and EISA chairman all about their OST. This record indicates that the implementation of this OST was accomplished exactly as its in accordance with the published agenda. Other participants gave oral feedback in favor of training implementation. IN TERMS OF TRAINING EVALUATION 3/1 At the end of each training event, participants were requested to fill-in an “End-of-Training￾Questionnaire” developed by IIE/DT2. The purpose of this process is to assess the participants’ levels of satisfaction with the training they went through, and the level of learning they acquired. This end-of-training evaluation focuses on the training's direct results, namely, participants’ reactions and learned knowledge and skills. It has nothing to do with training impact particularly in terms of applying acquired knowledge and skills, or association of these gains, which could be measured in 6-12 months after the end of the training when participants go back to their original work place and environment. 3/2 In terms of satisfaction, interviewed participants endorsed their responses they gave at the end of the training. Generally, the training event was very useful and its topics were interesting, informative, and persuasive. 3/3 They were very satisfied with the training materials and handouts, rich in content and simple in presentation; facilities were almost complete, logistics and IIE/DT2 administrative support were almost all positively rated. However, one hotel choice (in Colombia, South Carolina) was very poor. 3/4 They were just satisfied with the training objectives, contents, and instructional strategies. They appreciated the objectives that are directly related to their field of work, and the contents that they would immediately apply in their work place. However; U.S. Department of Commerce Law IV- August 4, 2003 Development Project in Egypt Final Evaluation Report 2 Development Associates, Inc. 3/5 Saome participants complained that the objective was not clear to them, and no one asked them about their training needs or what they hopes to achieve from attending in the training event. 3/6 All groups of interviewed participants were less-satisfied with the training duration, very short and condensed training sessions, and excessive travel within some of the tours themselves. 3/7 Almost all participants and supervisors appraised positively the OST type of training. It exposes participants to the U.S. culture and life-style; shows them in real situations how Americans work seriously, faster, in teams and task forces, follow the rules; use advanced systems, technologies, and techniques. They felt that exposure means more than just education or specific subject. It enriches knowledge in its most recent developments. 3/8 In the mean-time, some participants and supervisors were not satisfied with OSTs because it is usually some were not related to all participants’ fields of work. Also, because they felt it is was a too general type of training particularly for those who required a more specific focus. Some participants complained from about too many short visits to institutions and states implying a waste of time. 3/9 Participants of in the IPR Short Course found that exchange of experience with participants from other countries enriched their knowledge and supported what they learnt from the off-the￾shelf program. Though, they felt that the duration was very short and (some training sessions were only one-hour timelong), only two full-day presentations proved to be the most effective in the whole program. 3/10 In terms of assessing the impact of training; supervisors noticed that participants returned back to their work having acquiring advanced knowledge in different fields, understanding two systems;, the Egyptian and American. not only one. They added that they learned modern operation techniques, procedures, and standards. They acquired advanced skills and they became faster in their daily routine work. Most importantly, almost all participants expressed serious appreciation to of the potential role and involvement of the private sector in their different sectors of trade and investment-related areas. 3/11 Supervisors and participants mentioned that the impact of training is guaranteed only when future CLDP-TA focuses on institutional capacity building, develops new-systems and sub￾systems, and includes continuous assessment of needs, and internal follow-up and monitoring. 3/12 They recommended that future training programs should be more focused, where participants and supervisors attend in order to be more tolerant to new ideas and suggestions. 3/13 Also, they emphasized the importance and critical needs for practical training, hands-on￾training, internships, TOT particularly with these institutions that play an intermediate source organization in the region such as EOS. U.S. Department of Commerce Law IV- August 4, 2003 Development Project in Egypt Final Evaluation Report 3 APPENDIX V DATA-GATHERING INSTRUMENT MINISTRY INTERVIEW GUIDE CLDP PROJECT ASSESSMENT 1. What were some of the strengths of the CLDP program you (or your employees) participated in? The weaknesses? Were the objectives clear? How could it have been improved? 2. How would you compare this training with other trainings in which you participated? How would you compare it to the best training you have ever had? The worst? 3. Did you (or your employees) acquire new knowledge or skills and apply lessons learned to your jobs? (For a supervisor) Did you see a change in the level of performance after the training? What, in particular? How specifically do you (or they) use the new knowledge or skills? 4. How has participation in the CLDP training improved the capacity of your organization to fulfill its mission? 5. Are there any specific changes you have implemented in your organization as a result of participation in the training? U.S. Department of Commerce Law V- August 4, 2003 Development Project in Egypt Final Evaluation Report 1 Development Associates, Inc. 6. CLDP feels one of its value-added qualities is that it brings GOE personnel together with their US government counterparts. How did you experience that? Was there enough interaction? Would you have liked more? 7. How were people nominated and/or selected for the training program(s)? Did the right people go? Do they remain in the Ministry in a role in which they can use their training? 8. Did your organization play any role in a training needs assessment? If so, how did that happen? 9. Do you, or does anyone in your organization, continue to have professional contacts with the trainers or officials you met? If so, what kind? 10. In what ways could the CLDP methodology be applied on a larger scale and to more organizations/ ministries? 11. What other areas would you like to see included in future CLDP training? U.S. Department of Commerce Law V- August 4, 2003 Development Project in Egypt Final Evaluation Report 2 APPENDIX VI EVALUATION METHODOLOGIES The evaluation team used the following training evaluation model as a basis for interview questions and document reviews. Evaluation Model I TRAINING PLANNING/DESIGN: Sources of Information: A specific training workplan document; and Training provider, trainer/s and trainee/s sample, and client. Questions and issues to be addressed: 1- Was the justification of the given training based upon: Realistic work performance analysis; identification of gaps between current and desired performance, and breakdown of problems causes and consequences? Aspired objectives to be achieved and activities to be implemented? Or Other reason/s 2- Was training needs assessment conducted through: 2-1 A functional approach? 2-2 A personal approach? 2-3 A combined approach? 2-4 In-house system or external TA? 3- What sources were used to determine training needs: 3-1 Job-descriptions? 3-2 Personal expressions? 3-3 Supervisor judgment? 3-4 Neighbouring group opinions? 3-5 Beneficiaries complaints/remarks? 4- What specific objectives were set to meet identified training needs: 4-1 General and specific knowledge: Build new/refresh/update? Reinforce/consolidate existing conceptions? Or Correct/adjust misinformation? 4-2 Personal and institutional skills: Build new? Improve existing? Modify/reform? U.S. Department of Commerce Law VI- August 4, 2003 Development Project in Egypt Final Evaluation Report 1 Development Associates, Inc. 4-3 Attitudes and value: Introduce new? Reinforce existing? or Change rooted values? 4-4 Mind-set: Tune, regulate, or adjust? 5- How were participants/trainees selected: 5-1 As nominated/obligatory? 5-2 As a result of a pre-test, an interview, or other criteria? 5-3 Randomly? 6- How were resource persons/trainers selected: 6-1 Due to their educational background? 6-2 Due to their professional career? 6-3 Due to their positions? 6-4 Due to their specific training provision qualifications? 7- How were training topics/subject matters developed: 7-1 Off-the-shelf? 7-2 Customized? 7-3Specifically tailored? 8- How were training contents and materials developed? 8-1 Composing theoretical and practical aspects? 8-2 Giving priority to certain points; that Must be taught? Should be taught? Nice to be taught? 8-3 Time-based? 8-4 Properly respond to/cover findings of TNA? 8-5 Logically presented in lesson plan with well-defined learning objectives? And main points? 8-6 Comprised in a course outline, trainers/trainees guides? 9- Were the instructional strategies adopted in the training: 9-1 Modern experiential training methodologies? 9-2 Traditional teaching techniques? 9-3 An integrated package of participatory strategies? 10- Were preparatory activities accomplished in due time? Including: 10-1 Housing arrangements? 10-2 Travel and transportation? 10-3 AVA facilities and equipment? 10-4 Handouts and guides? U.S. Department of Commerce Law VI- August 4, 2003 Development Project in Egypt Final Evaluation Report 2 Development Associates, Inc. 10-5 Invitation letters? 10-6 Orientation? 10-7 Training agenda? 10-8 Other logistical/administrative/financial activities? II TRAINING IMPLEMENTATION: Sources of information: A specific training final report; A follow-up report; (donor or client) Training provider, trainer/s and trainees. Questions and issues to be discussed: 11- How proper (appreciate and adequate) was the implementation of the training, particularly in terms of: 11-1 Location? 11-2 Duration? 11-3 Timing? 11-4 daily schedule? 11-5 Length of training sessions? 11-6 Number and length of breaks? 12- How effective and relevant was the training content, particularly in terms of: 12-1 Coverage and sequence of its agenda? 12-2 Topics to be covered according to lesson plans? Theoretical and practical aspects? 12-3 Interaction between trainers and trainees? 12-4 Use of instructional strategies package and AVAs? 13- How effective were the instructors, in terms of: Technical expertise? Training provision expertise? Personal qualities? 14- How effective was the trainees’ participation and exchange of experience? 15- How effective was the training facilitator, administrative and logistical support services in terms of: 15-1 Smooth implementation? 15-2 Problem Solving? III TRAINING RESULTS AND IMPACT: Sources of Information: Final reports; U.S. Department of Commerce Law VI- August 4, 2003 Development Project in Egypt Final Evaluation Report 3 Development Associates, Inc. Follow-up report; Trainees reactions, post-tests; Trainers opinions; Supervisors observations and comments; and Work performance review and comparative analysis Questions to be asked and issues to be addressed: 16 Have the participants been pre-tested at the beginning of the training? 17- Have the participants been post-tested at the end of the training? 18- Have the results of the pre and post tests been analyzed and compared? 18-1 At the individual level? 18-2 At the group level? 18-3 At each topic level? 18-4 At all topics level? 19- Have the levels of knowledge acquired been improved? 20- Have the level of skills learned enhanced? 21- Have the levels of participants' attitudes been tuned/changed? 22- Have these levels of knowledge, skills, and attitudes been needed to achieve the training expected results? 23- Have these levels of knowledge, skills, and attitudes been relevant to work performance? 24- To what extent have newly acquired knowledge, skills, and attitudes been applied in work situations? 25- To what extent have this application contributed to the institution capacity building? U.S. Department of Commerce Law VI- August 4, 2003 Development Project in Egypt Final Evaluation Report 4 Development Associates, Inc. 26- Have the participants been given chances to express their reactions to and satisfaction with the training they went through, particularly in terms of instructor, subject and methodology: 26-1 At the end of each lesson plan? 26-2 At the end of each training day? 26-3 At the end of each training unit? 26-4 Immediately at the end of the training? 27- Generally, what were the major strength points of the training? 28- What were the major weakness points of the training? 29- What should be changed in this training? 29-1 Excluded? 29-2 Included? 30- Should this training be replicated/sustained? 31 Overall assessment of the training planning, implementation, results and impact: 31-1 Excellent (5 degrees)? 31-2 Very good (4 degrees)? 31-3 Good (3 degrees)? 31-4 Fair (2 degrees)? 31-5 Poor (1 degree)? 32- What are the recommendations you suggest to improve your work performance? and your institution capacity building? 33- What training program/s you suggest for your trainees’ supervisor/s? U.S. Department of Commerce Law VI- August 4, 2003 Development Project in Egypt Final Evaluation Report 5 APPENDIX VII GLOSSARY ANSI Administration ATR Assistance for Trade Reform BOT Build, Operate, Transfer BOOT Build, Own, Operate, Transfer CD/WTO Central Department of the World Trade Organization CLDP Commercial Law Development Program (USG) CRS Commercial Representation Section (GOE) DEPRA Development Economic Policy Reform Analysis (USAID Project) DT 2 Development Training 2 Project (Institute of International Education) EISA Egyptian Insurance Supervisory Authority EOS Egyptian Organization for Standardization and Quality Control (GOE) FCC Federal Communications Commission (USG) FDA Food and Drug Administration (USG) FTS Foreign Trade Sector (GOE) GOE Government of Egypt GOEIC General Organization for Export and Import Control (GOE) GPA Government Procurement Agreement (WTO) HACCP Hazard Analysis and Critical Control Point IIE Institute of International Education (USAID Training Contractor) IPR Intellectual Property Rights ISO International Organization for Standardization IT Information Technology MHESR Ministry of Higher Education and Scientific Research (GOE) MOALR Ministry of Agriculture and Land Reclamation (GOE) MOC Ministry of Culture (GOE) MOFT Ministry of Foreign Trade (GOE) MOF Ministry of Finance (GOE) MOFA Ministry of Foreign Affairs (GOE) MCIT Ministry of Communications and Information Technology (GOE) MOSIT Ministry of Supply and Internal Trade (GOE) NAIC National Association of Insurance Commissioners NCJS National Center for Judicial Studies (GOE) NIST National Institute of Standards and Technology (USG) NTIA National Telecommunications Information Administration (NTIA) PASA Participating Agency Services Agreement SPS Sanitary and Phytosanitary Measures (WTO Agreement) TBT Technical Barriers to Trade (WTO Agreement) TIFA Egypt-US Trade and Investment Framework Agreement TNA Training Needs Analysis TRIPs Trade Related Aspects of Intellectual Property Rights (WTO) USAID United States Agency for International Development USDA United States Department of Agriculture USDOC United States Department of Commerce U.S. Department of Commerce Law VII- August 4, 2003 Development Project in Egypt Final Evaluation Report 1 Development Associates, Inc. USG United States Government USITC United States International Trade Commission USPTO United States Patent and Trademark Office USTR United States Trade Representative USTTI United States Telecommunications Training Institute WIPO World Intellectual Property Organization WTO World Trade Organization Egypt (CLDP) – Final Evaluation Report.doc/Mobis Egypt-5 U.S. Department of Commerce Law VII- August 4, 2003 Development Project in Egypt Final Evaluation Report 2