THE AERA PROGRAMS OF KIASIA AN EVALUATION (JULY-SEPTEMBER 2003) Submitted to: U.S. Agency for International Development/Thailand In response to: USAID Evaluation IQC No. AEP-I-00-00-00023-00 Assessment team: Michael Calavan Ronald Bielen Submitted by: Development Associates, Inc. 1730 North Lynn Street Arlington, VA 22209-2023 and Nathan & Associates 2101 Wilson Boulevard Arlington, VA 22201 December 31, 2003 The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) i TABLE OF CONTENTS Page No. List of Acronyms ...................................................................................................................iii Executive Summary...............................................................................................................v I. Introduction................................................................................................................1 A. Background of the Evaluation .......................................................................1 B. Scope and Purpose of the Evaluation.............................................................2 C. Organization and Design of the Evaluation...................................................4 II. The Bank Training Program (BTP.............................................................................6 A. Purpose, Organization, and Operation of the BTP ........................................6 B. Available Information on BTP Performance .................................................8 C. An Overview of BTP Case Studies................................................................11 D. Evaluation Team Findings — the BTP..........................................................12 III. The Business Advisory Center (BAC).......................................................................15 A. Purpose, Organization, and Operation of the BAC..............................................15 B. Available Information on BAC Performance.......................................................17 C. An Overview of SME Consultancy Case Studies ................................................18 D. Evaluation Team Findings — the BAC ...............................................................22 E. A Related Activity — Cluster Competitiveness...................................................24 IV. The Business Support Organizations Partnership (BSOP) ........................................25 A. Purpose, Organization, and Operation of the BSOP............................................25 B. Available Information on BSOP Performance....................................................26 C. An Overview of BSO Partnership Case Studies .................................................28 D. Evaluation Team Findings—the BSOP ..............................................................33 V. Benchmarking — Can KIAsia Implement AERA-Initiated Activities in a “Sustainable” Manner? ..............................................................................................36 A. Evaluation Team Observations on Benchmark Areas .........................................36 B. Are Current Organization and Policies Appropriate for Sustainability?..............40 C. What is the Potential for Funding from other Sources?......................................40 D. Evaluation Team Findings Regarding KIAsia Management ...............................41 VI. Recommendations to KIAsia .....................................................................................42 VII. Recommendations to USAID ....................................................................................47 Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) ii ANNEXES Annex A. Scope of Work.............................................................................................. A-1 Annex B. Interview Schedule....................................................................................... B-1 Annex C. Template for BAC Case Studies .................................................................. C-1 Annex D. Interview Guides .......................................................................................... D-1 Annex E. Case Studies ................................................................................................. E-1 The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) iii LIST OF ACRONYMS ABA American Bankers Association ABI American Bankruptcy Institute ADRO Alternative Dispute Resolution Office AERA Accelerating Economic Recovery Asia ASEM (World Bank Project) ATSME Association of Thai Small and Medium Entrepreneurs BAAC Bank for Agriculture and Agricultural Cooperatives BAC Business Advisory Center BAY Bank of Ayudhaya BOT Bank of Thailand BPCS Business Process Control School BSOP Business Support Organization Partnership BTP Bank Training Program CAMA Computer Assisted Mass Appraisal CDC Citizen Development Corporation CFG Corporate and Financial Governance DOT Department Of Transportation DTEC Department of Technical And Economical Cooperation GAO General Accounting Office GTZ German Development Agency HR Human Resources IC International Competitiveness IESC International Executives Service Corporation IG/DOT Inspector General's Office of the Department of Transportation IO Intermediate Objective IQC Indefinite Quantity Contract ITB Invigorating Thai Business KIA Kenan Institute Asia KMUTT King Mongkuts University of Technological – Thornbury LJR Legislative and Judicial Report MABEC M B A Enterprise Corps MBA Master of Business Administration NGO Non Governmental Organization NPL Non Performing Loans NTU National Technological University OAG Office of Auditor General OJ Office of Judiciary RFP/RFA Request For Proposal / Request For Application SCB Siam Commercial Bank SEC Securities and Exchange Commission SEGIR Sustainable Economic Growth and Institutional Reform SME Small and Medium Enterprises SOW Statement of Work TA Technical Assistance TIBFA Thai Institute of Banking and Finance Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) iv TOT Training of Trainers TVCS Thai Volunteer Consulting Services USAID United States Agency for International Development USDP United States Development Program USFDA United States Food and Drug Administration VE Volunteer Expert WTO World Trade Organization The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) v EXECUTIVE SUMMARY BACKGROUND In 1993, the Kenan Institute of Private Enterprise and two partners responded to a USAID Request for Applications and won support to establish the US-Thailand Development Partnership. As it became clear a few years later that USAID/Bangkok would complete its Thai programs with a few million dollars unexpended, the mission made arrangements to donate the funds to the Kenan Institute-Asia (KIAsia) for an endowment. The Kenan Trust and Fund of North Carolina and the Department of Technical and Economic Cooperation of the Thai Government agreed to make equal contributions, providing an endowed capital of $10.5 million. In 1996, KIAsia was registered as a development foundation under Thai law. In mid-1997, the Asian economic crisis struck, and in 1998 USAID began to plan a recovery program called Accelerating Economic Recovery in Asia (AERA). KIAsia was selected as the implementing organization. KIAsia’s AERA programs focused on broad-based economic reform, rather than recovery, using failures in the banking system, the courts, and equity and property markets as reference points for reform. Total AERA funding to KIAsia was $15.2 million, and three AERA programs were initiated — the Bank Training Program (BTP), Business Advisory Center (BAC), and Business Support Organization Partnership Program (BSOP). With USAID concurrence, the implementation period was extended three times, ultimately to the end of FY'05. PURPOSE This Evaluation comes at a crucial time for KIAsia and USAID/Bangkok. Circumstances that have precipitated the study include: A USAID regional program is under design; Two years of KIAsia/AERA funding remains; and KIAsia needs to re-affirm its own agenda. In response, USAID and KIAsia are seeking answers to classic questions of impact, efficiency, and sustainability. METHODOLOGY The Team spent two weeks in Washington, then traveled to Bangkok for five weeks of work. Four weeks were devoted to gathering information regarding BTP, BAC, and BSOP. About a fifth of the time was devoted to gaining a clearer understanding of how KIAsia operates. The final week was devoted to analyzing information gathered and discussing findings and recommendations with USAID and KIAsia colleagues. With the agreement of USAID colleagues, the study was conducted mainly through a series of case studies, 13 for BAC, eight for BSOP, and three for BTP. BTP KIAsia’s AERA program responded to the crisis in Thailand’s banking system with the bank training program. BTP provided training and TA to Thai banks to improve managerial standards, provide transparency, upgrade banking personnel, and strengthen staff training Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) vi capabilities. Over the first three years, BTP reached 22,000 banking officers and operational staff. Activities were outsourced to consultant teams and each bank's program was custom tailored to meet its specific needs. BTP operated with minimal staff, but was able to cooperate with all Thai-owned banks. Banks were required to share in training costs. Most BTP training has been completed. Considering the relatively small amount expended on BTP and that activities are virtually complete, three bank case studies were deemed sufficient. In examining the cases, the Evaluators noted significant impact on bank operations. Risk management departments have been established and banks are managing risk with knowledge they acquired through BTP training. This was most pronounced at the Bank of Ayudhaya, which has fully embraced risk management as part of its operations. The Siam Commercial Bank has also begun to embrace risk management. As a result of a large internal training program, SCB personnel are convinced of the need for comprehensive risk management. At the Bank for Agriculture and Agricultural Cooperatives, training on responsibility center accounting was provided for executives, middle management, and training-of-trainers. BAAC is also heavily involved with a BTP-funded distance learning program, providing employees in outlying areas the chance to upgrade skills. The Team examined the cost effectiveness of BTP in delivering training. The cost per bank is quite reasonable (about $184,000), and the total cost of BTP is less than one-hundredth of one percent of the estimated cost of non-performing loans to the Thai taxpayer. BAC The Business Advisory Center is KIAsia’s largest unit; its annual budget is about 46% of KIAsia’s budget. The mission of the BAC is to: “Provide training and consulting services to Thai SMEs.” The BAC began work in late 1999, and by the end of 2001, 165 consultancies had been undertaken. Where required services fall within the competence of recent MBA graduates, BAC staff often take on the work themselves. When more specialized expertise is needed BAC recruits an American “volunteer expert” through IESC, CDC, or ACDI/VOCA; or recruits a Thai volunteer. From the beginning, BAC has pursued a policy of cost-sharing, but until recently the client firm’s share was modest, on the order of 10% of direct costs. More recently, BAC informed clients the fee for VE consultancies will be B10,000 ($240) per day, and demand has dropped off dramatically. One promising activity associated with the BAC was establishment of the Thai Volunteer Consulting Services (TVCS). Senior Thai businessmen were invited to register as volunteers. Unfortunately, during the Team’s visit, use of Thai volunteers was at a low ebb, and TVCS appeared to be dormant. Among the first questions that must be applied to any service organization is the impact of services offered. Based on 14 SME consultancies, the Evaluators found that BAC’s record is quite good. In the case of five firms (seven separate consultancies), the Evaluation Team’s judgment was “highly effective.” For four more firms (five consultancies), the judgment was “effective.” In the case of one firm, “mixed results” were achieved, and finally three consultancies were found to be “not effective.” Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) vii Based on the case studies, the Evaluators’ estimate is that the majority of consultancies have been “highly effective” or “effective.” However, some case study consultancies judged to have “mixed results” or be “not effective” show no evidence of shortcomings in BAC files. The Team’s strongest criticism is that BAC staff are not learning sufficiently from past consultancies, whether they were “highly effective,” “not effective,” or in between. Lack of follow-up has several costs: BAC staff are denied management information: significant marketing opportunities are missed; opportunities to identify promising new products are missed; and significant training opportunities are lost. BAC was initially set up as a short-term project, focused on helping individual businesses succeed. The goal was for BAC to assist 100 firms annually, and for two years, this goal was met. More recently, services have clearly slowed down. Now that the economic recovery is well under way and BAC is in its fifth year, questions need to be raised about the benefits of a few hundred SME consultancies for “the Thai economy.” Clearly, it is hard to justify further donor support for a program of this sort. Several operational trends that will (in the absence of donor funding) have direct implications for long-term BAC financial sustainability were noted by the Evaluators: American MBA staff are being pruned as a cost cutting measure. BAC recently began informing potential clients they must pay more for consultancies. The number of International VE consultancies has dramatically declined. For the past year, the BAC has marketed services primarily through Thai government support programs for SMEs. These highly subsidized programs have undercut direct demand for BAC services, but have allowed BAC to earn significant income as a contractor for consulting services. KIAsia began experimenting with development of “clusters” of SMEs in related business sectors relatively early in implementation of AERA programs. The approach remains experimental and unproven for KIAsia and Thailand, but it is a “wholesale” approach, capable of working with dozens, even hundreds, of firms simultaneously. And, based on a brief comparison with BAC clients, it appears to reach firms that are smaller, less well connected, and geographically more remote. BSOP The Business Support Organization Partnership program establishes partnerships between Thai and U.S. “business support organizations.” By nurturing working partnerships between Thai and American groups, BSOP aims to improve and support Thailand’s: financial and corporate governance; legislative and judicial reforms; and international competitiveness. Building partnerships is a varied process, sometimes involving intensive efforts to locate and bring together appropriate partners, while in other cases, partnerships are formed independently, and those involved make their own plans before approaching KIAsia for funding. The number of partnerships has grown steadily, now reaching 25. The Evaluators examined eight partnership activities in detail, offering a reasonable cross-section of BSOP’s work. E.g., A Thai/US partnership to set up a Mediation Center for Financial Disputes has enjoyed solid success. Hundreds of mediators have been trained, and hundreds of disputes involving billions of baht have been settled. After three years of collaboration between Purdue University and King Mongkut’s University of Technology, a USFDA-approved Better Process Control School has Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) viii been established in Thailand. Funding of $35,000 has enabled Thai food processors to gain the technical and procedural knowledge needed to export canned goods to the U.S. A series of Performance Audit Workshops, and follow-on activities, are one focus of a partnership between the Thai Office of the Auditor General and the U.S. General Accounting Office. BSOP seeks organization-level impacts on Thai government units and private organizations. The Evaluators noted: Of partnership activities examined closely, none have failed. Most are already delivering useful results or seem likely to do so soon. But concrete evidence of the macro￾impacts of BSO partnerships is not yet available. Nevertheless, an examination of potential macro-impacts can be useful as a rough gauge of their importance. For example: Court￾sponsored mediation can reduce costly litigation, lower case loads, and speed up actual trials. This makes an important point. The partnerships deal with important issues, with the potential for truly significant macro-impacts. The cost effectiveness of BSOP is impressive. Supporting a partnership costs only about $70,000, compared to $21,000 per firm under BAC. Since partnerships can have impacts on entire professions or industries, or on the overall Thai economy, the Evaluators find this program to be highly cost effective. However, it is unlikely BSOP can achieve financial self-sufficiency in the near future. KIASIA MANAGEMENT ISSUES KIAsia was established in 1996 during the years of economic boom, and three short years later was chosen to implement AERA. KIAsia's previous growth course was deflected as the urgency of project implementation came to the forefront. Emphasis on meeting targets may have diverted attention from the need to focus on financial sustainability and expanding of non-AERA activities. The sudden arrival and unpredictable extensions of AERA have continued to shape KIAsia, as normal growth would never have done. Now, with the renewed presence of a USAID Mission, KIAsia’s status as a preferred implementer is gone. Employee turnover has been high, especially recently. Experience gained at KIAsia increases employee’s appeal to outside organizations and an improved Thai economy affords staff ever more outside opportunities. Despite rapid turnover, overall the Evaluators believe that KIAsia staff are well qualified and quite capable. Furthermore, managers and the KIAsia Board are well connected with the government and the business community. A review of current sources of funding and an inventory of endowment assets also raise causes for concern. The endowment has not contributed as expected toward building and maintaining KIAsia. The DTEC component is in a savings account and the Evaluators understand that the account has been generating only 0.25% in FY'03. Non-USAID donation income represents a very small portion of KIAsia's total donation income, and unless firm steps are taken to find other sources, programs will need to be reduced dramatically. The KIAsia financial system is designed primarily for reporting, and delivers the kind of information donors require. But, in its present form, the monthly accounts not a particularly useful management tool; revisions in format could enhance its management value. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) ix KIASIA RESPONSES Despite distracting everyday challenges, the Evaluators found that KIAsia has given increasing attention to sustainability over the past two years. Notably, in January 2002 the Board of Trustees approved strategies for: Sustainability, Partnership, and Regional Programs. The Sustainability Strategy recognizes KIAsia’s sustainability depends on: a good relationship with donors and customers; a capacity to achieve increased returns from grants; efficient operations; a steady transition away from subsidized services; generating overhead from project management; fund raising; and creation and commercialization of intellectual property. Managers are working to develop a strategy and budget for the final two years of AERA that will maximize impacts on clients and contribute to program sustainability. KIAsia has already identified collaboration with DTEC in the region as a potential means of leveraging present institutional strengths and obtaining funds to sustain regional programs. And, of course, the USAID Regional Mission is a possible future partner for regional programs. The World Bank’s financial sector team will be phasing out and there is a need to insure monitoring continues to take place. KIAsia would be a very good candidate for performing such work. Corporate contributions now come primarily from American companies. But the Evaluators believe it is also worthwhile to explore long-term opportunities to solicit collaboration and financial support from Thai companies. PROGRAM EMPHASES In the Evaluators’ judgment, prospects for obtaining sufficient income to sustain AERA follow￾on activities at current levels after 2005 are small. But, as discussed above, some programs may be capable of becoming self-financing, while others may be sufficiently innovative to attract additional donor support. An essential question for KIAsia managers is: Do all activities need to be sustained? These are the Evaluators’ impressions: BSOP is molded in the true spirit of KIAsia. Its approach to partnership has significant potential and should be supported in whatever fashion is required to meet demand from reform-oriented organizations and government units. The model should be energetically tested in the region as opportunities arise, and USAID is encouraged to consider support for a regional BSOP program. BTP has performed its main task admirably, and it is appropriate that the program is ending now. However, modest follow-on activities in distance education for bank employees and microfinance may be appropriate. Regarding BAC, the Team is encouraged that the unit has begun to generate significant revenue. Nonetheless, assisting only 217 firms out of tens of thousands of Thai SMEs, has made a limited contribution to economic recovery. Accordingly, the Team does not believe budgetary support for a retail approach is warranted beyond the end of FY' 04. On the other hand, managers can encourage BAC to identify strategies to generate income to maintain the unit and its services. RECOMMENDATIONS TO KIASIA Overall, the Evaluation Team offers 29 recommendations to KIAsia. Some of these are listed below to give readers the “flavor” of the Team’s approach: Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) x BTP — Strive to finalize ongoing BAC Research on microfinance soon, to open the possibility of programming AERA funds toward this activity. BAC — Convene a BAC retreat to initiate a “financial sustainability in 2004 plan,” and form a “new BAC products” group.  Provide operational funding for FY’04 at 80-85% of the FY’03 level, while allowing the BAC to retain income earned during the year for expenditure in FY’05.  Provide no AERA or other donor funds to the BAC in FY’05. Allow the use of funds from FY’04 earnings, but only on a matching basis.  Build incentive structures to reward BAC staff who remain and play a positive role in achieving sustainability. BSOP — Continue support, using AERA funding, to BSOP operations and programs at the FY’03 level in FY’04 and FY’05.  Engage a distinguished Thai research organization to undertake detailed research on impacts of the BSO partnerships and engage the same institution, or another as appropriate, to document the work of building and nurturing BSO partnerships.  Seek international and Thai corporate support to establish new partnerships and sustain ongoing ones.  Build human and administrative resources to become a respected “regional player,” by experimenting with support of partnerships in Cambodia, Vietnam, and China. MANAGEMENT — Develop concrete plans for KIAsia’s "life after AERA."  Revise the budgeting process and financial software to extract useful management information.  Consider bringing an international volunteer for a two-three week consultancy on fund raising.  Establish an annual campaign to solicit donations from Thai and international corporations for attractive programs such as BSOP and cluster competitiveness, and for direct contributions to the endowment.  Seriously explore the feasibility of establishing “corporate social responsibility services” as a major income source. RECOMMENDATIONS TO USAID — The team offers nine recommendations to USAID. All are listed below:  Microfinance best practice awareness appears to be an important area still needing attention. Some support for awareness building is important for policy makers. This Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) xi could be accomplished through in-country seminars as well as select visits to leading microfinance programs in the region. Consider facilitating seminars and study visits for key persons using BTP.  Consider engaging a Thai institution to undertake a detailed study of implementation and impacts in Thailand, if similar programming to be undertaken in region.  Inform KIAsia that USAID will not provide any post-AERA funding for BAC programs that support individual firms.  Encourage KIAsia managers to end operational funding for BAC services to individual firms by end of FY’04.  In six-eight months, examine the potential of cluster competitiveness work by attending meetings and making field visits.  Under the right circumstances—e.g., a creative response to a regional RFP/RFA, consider funding a follow-on KIAsia program on cluster competitiveness, so long as there is a convincing prospect of effective work in neighboring countries.  Consider providing modest additional support (say $500,000-800,000 annually) to BSOP for Thailand activities after FY’05. In all likelihood, this work would be carried out as part of a regional Partnership program that might be funded at twice or three times this level.  Consider an RFA/RFP for partnership building as a regional program in economic reform.  If KIAsia managers approach USAID for permission to use AERA funds for a variety of analytical and planning tasks recommended above, give their proposals serious attention. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 1 THE AERA PROGRAMS OF KIASIA AN EVALUATION (JULY-SEPTEMBER 2003) I. INTRODUCTION A. BACKGROUND OF THE EVALUATION In 1993, the Kenan Institute of Private Enterprise, working with Chulalongkorn University and the Brooker Group, responded to a USAID Request for Applications and won support to establish the US-Thailand Development Partnership. The competition had been undertaken to fill a void that would be created by USAID’s imminent departure from Thailand. The Partnership’s office was established at the Queen Sirikit Center in 1994, and the organization awarded its first partnership grants. In 1995, as planned, USAID/Bangkok was closed. As it became clear USAID/Bangkok would complete its Thai programs with a few million dollars unexpended, mission staff made arrangements to donate the funds (estimated at $3.5 million) to KIAsia as an initial contribution toward an endowment. After negotiation, the Kenan Trust and Fund of North Carolina and the Department of Technical and Economic Cooperation (DTEC) of the Thai Government agreed to make equal contributions. An agreement signed in 1996 provided KIAsia an endowed capital of $10.5 million, divided into three separate funds. USAID funds were invested in the US, and were turned over to direct control by KIAsia in 2001. Kenan Trust funds are managed in a US account, under the control of the “Friends of Kenan.” DTEC funds are in a baht account. Given economic conditions prevailing in 1996, there were optimistic assumptions that the endowment capital would continue to grow, and modest programs and operating costs could be supported by endowment income, In 1996, the Kenan Foundation Asia (later “KIAsia”) was registered as a development foundation under Thai law, and formed its first board of trustees. Some observers regarded KIAsia as a promising model for post-USAID economic cooperation—one emphasizing close partnership with Thai government departments, private firms, and NGOs. The early emphasis of KIAsia grant programs was on health and environment. Productive partnerships were soon established with the international business community. For example, in 1998, American Corporations for Thailand was established under co-chairmen Henry Kissinger and Anand Panyarachun. Corporate donations of $1.1 million supported Thai training institutions and programs for the unemployed. In mid-1997, the Asian economic crisis struck, first impacting on Thailand, then spreading rapidly to countries across the region. As the crisis widened and deepened, experts came to realize it was caused by shortcomings in regional economies extending far beyond the exchange rate problems that had precipitated the initial breakdown in Thailand. In 1998, USAID began to plan a multi-pronged recovery program called Accelerating Economic Recovery in Asia (AERA). Planned assistance focused on Thailand, the Philippines, and Indonesia. AERA was undertaken through USAID missions in the latter countries. But, in the absence of a mission in Thailand, KIAsia was selected the best available implementing organization. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 2 Initial estimates of overall AERA funding far exceeded the $60 million that eventually became available for the three countries. Furthermore, by the time the AERA was in place, signs of recovery were already apparent in Thailand. Thus, to a substantial degree KIAsia’s AERA programs focused on broad-based economic reform, rather than recovery, using failures in the banking system, the courts, and equity and property markets as reference points for reform. USAID and KIAsia signed a one year Cooperative Agreement in FY 99 for $5.1 million, and substantial additional obligations were made in FY’00, FY’01, FY’02, and FY'03. Total AERA funding to KIAsia was $15.2 million, and additional funds for malaria control and environmental activities brought the total to $17.6 million. KIAsia expanded its staff to more than 70, as three AERA programs were initiated--the Bank Training Program (BTP), Business Advisory Center (BAC), and Business Support Organization Partnership Program (BSOP).1 Activities have been substantial, but expenditures have been relatively low. Thus, with USAID concurrence, the implementation period was extended in FY’00 from the original one-year period for two additional years to the end of FY’02. In FY'02 the period was extended two years more, to the end of FY'05. At the end of FY’03, more than $5 million remains available to finance AERA programs and cover operating costs. KIAsia is well established in Bangkok, a respected partner to donor organizations such as the World Bank and UNDP, and with Thai government departments, NGOs, and the corporate sector. Several U.S. government actions--including USAID funding of the US-Thailand Development Partnership in 1993; USAID’s contribution to the KIAsia endowment in 1996; USAID’s award of the AERA Cooperative Agreement in 1999; and inclusion of KIAsia in the Thailand-US Economic Cooperation Framework of 2001—have created a clear impression that KIAsia is USAID’s “successor” in Thailand. Now, however, USAID’s re-entry into Thailand by establishing a regional mission has made KIAsia’s position rather ambiguous. It is timely and appropriate for USAID and KIAsia to assess the AERA program, and to re-assess their working relationship. B. SCOPE AND PURPOSE OF THE EVALUATION This Evaluation comes at a crucial juncture for KIAsia and USAID/Bangkok. Circumstances that have precipitated the study include: A USAID Regional Program Is Under Design The new USAID mission has been established to undertake regional programs. It is reasonable to assume funds will be available to implementing organizations almost exclusively for activities with a regional scope. KIAsia cannot count on Thailand-specific funds. Furthermore, USAID will be awarding regional funds through competitive procurements. While it is reasonable to assume that KIAsia can play a useful role in regional activities, funding will depend on preparation of excellent proposals that build effectively on experience gained while implementing AERA programs, Detailed evidence regarding KIAsia’s performance in implementing AERA programs will be valuable to USAID for making future funding decisions. It will also be useful for KIAsia staff to reflect carefully on current programs, identifying strengths that can be built on, and shortcomings that need to be corrected. It is noteworthy that KIAsia is acquiring regional expertise through small programs in Cambodia, Vietnam, and 1 The Team also briefly examined the Labor Standards, Education, and Cluster Competitiveness programs of KIAsia. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 3 China. But, KIAsia needs to demonstrate that these activities can be run effectively and sustainable from a Bangkok base. Two Years of KIAsia/AERA Funding Remains USAID mission staff regularly remind KIAsia managers that funding after FY’05 cannot be assured, and stress the importance of financial sustainability. Sufficient funds remain in the AERA Cooperative Agreement to support programs through FY’05, providing a solid platform on which KIAsia managers can build sustainability strategies. It would be unfortunate if KIAsia were to pass through these two years and emerge without workable systems to achieve financial sustainability and cohesive programs. Fortunately, recent efforts of KIAsia managers--defining “vision, mission, branding, and strategies;” an all-employee staff retreat; intensive program budget reviews; and careful attention to initial recommendations of this Evaluation Team— suggest they share USAID’s sense of urgency. KIAsia Needs to Re-affirm its Own Agenda KIAsia managers appreciate that AERA programs have provided an excellent opportunity for institutional learning and growth. But they are also aware the influx of funds in 1999 diverted attention from goal setting and strategic and financial planning that would otherwise have been undertaken. Managers and staff members need to evaluate the “AERA experience,” deciding which activities are essential, and which were interesting activities that can now be set aside. In the process, they must consider the changing circumstances within which KIAsia operates. For example, the Evaluation Team heard interesting hints about a possible three-cornered partnership involving USAID, the Thai Government’s Department of Technical and Economic Cooperation (DTEC), and KIAsia in development activities in neighboring countries. KIAsia managers are well advised to explore these options with USAID and DTEC colleagues, but should ensure any programs undertaken are consistent with KIAsia goals and strategy, and that KIAsia is staffed and organized to be an effective partner. In response to these circumstances, USAID and KIAsia are seeking answers to classic questions of impact, efficiency, and sustainability. (See Annex A—Scope of Work.) USAID and KIAsia seek information on program impacts, thus providing essential information on the past performance of an organization that is a potential future partner. Impact questions take the form: As a result of KIAsia/AERA programs:  Are Thai banks better managed, capable of making sounder lending decisions?  Have Thai SMEs increased their productivity or found more effective marketing strategies, and become more profitable in the process?  Have Thai government departments succeeded in implementing reforms?  Have private associations been effective in introducing new professional standards? Second, USAID and KIAsia seek answers to questions regarding efficiency. The most straightforward way to approach this issue is by measuring and pondering cost-effectiveness:  What is the average cost of delivering a service, e.g., a consultancy provided to a Thai SME, or a training program for a bank?  Similarly, what is the average cost of establishing and nurturing a Thai-American organizational partnership? Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 4  Are average costs changing over time? Why?  Are costs justified when compared to impacts?  Can costs be lowered without unduly affecting service quality? Finally, USAID and KIAsia are concerned with issues of sustainability. Relevant questions include:  Which AERA activities are already financially self-sufficient, or show promise of becoming self-sufficient?  Are current policies—e.g., for human resources management and internal communications—supportive of financial and institutional sustainability?  Are there ongoing activities so innovative and so important for Thailand’s continuing development that they are strong candidates for scarce USAID funds?  Are there Thailand-based activities that show particular promise for expansion to adjacent countries? C. ORGANIZATION AND DESIGN OF THE EVALUATION This Evaluation was undertaken by a two person team. Michael Calavan served as the Institutional Analyst and Ronald Bielen was the Private Sector Analyst. Both are familiar with aid projects and the systems and procedures of USAID. Calavan has a long-term professional interest in Thailand, some knowledge of the language, culture, politics, and economy, and experience in designing, managing, and evaluating USAID projects in several countries in South and Southeast Asia. Bielen brings extensive knowledge of financial analysis, professional experience in banking and micro-enterprise services, and wide ranging professional knowledge based on long term assignments in the Near East, and short term assignments in 30 countries. KIAsia and Kenan/Washington staff were generous and thorough in supporting the Evaluation. When the team arrived at the Development Associates office in Rosslyn, excellent notebooks that had been prepared in Bangkok were awaiting us, offering essential information on AERA programs and a cross-section of activities. Kenan/Washington staff provided prompt, resourceful assistance in scheduling interviews and teleconferences, and provided valuable insights on how the Washington and Bangkok offices collaborate on AERA activities. Managers and staff in Bangkok were unfailingly helpful in providing requested documents. All staff we interviewed responded patiently and thoroughly to our questions. ANE officials prepared the Scope of Work for this study and provided valuable guidance before the team left Washington. USAID/Bangkok colleagues provided information on the early stages of the Bank Training Project, and offered further guidance on carrying out the study. The team spent two weeks in the Washington DC area, reading documents at the Development Associates office and interviewing representatives of US organizations participating in the BSOP program, and current and former Kenan staff. They also held the first telephone interviews with international volunteers who assisted Thai SMEs under the BAC. The team then traveled to Bangkok for five weeks of work. (See our Interview Schedule in Annex B.) The first four weeks were devoted to gathering information regarding the BTP, BAC, and BSOP. This included examination of KIAsia working files, financial data, and internal reports. There were numerous interviews with SMEs, banks, government departments, and private organizations. Most interviews were carried out in Bangkok and adjacent provinces. Three working days were spent Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 5 conducting interviews in Chiangmai, and one in Maehongson, gaining valuable insights into all three major AERA programs. About one-fifth of the time was spent gaining a clearer understanding of how KIAsia currently operates. Interviews with senior managers, and generous sharing of financial reports, were particularly valuable in this regard.2 The final week in Bangkok was devoted to analyzing information gathered, preparing a PowerPoint presentation, and meeting USAID and KIAsia colleagues to discuss our findings and recommendations. With the agreement of USAID colleagues, the team chose to conduct the study mainly through a series of case studies. There are 13 BAC case studies, eight for the BSOP, and three for the BTP. (For the template for BAC case study reports, see Annex C.) This approach was adopted because the rich insights that case studies provide generally offer more useful guidance on management issues than data collected under extensive, but less detailed survey approaches. The KIAsia/AERA case studies provide: 1) rich results data not ordinarily available in quantitative tables; 2) a view of activities over an extended timeframe (up to three-and-a-half years); 3) triangulation, achieved by viewing activities through the eyes of multiple observers; 4) information on unanticipated results, often unrecognized during normal project implementation; and 5) information that may be useful in future management of similar activities. Results of case studies are summarized briefly in tables in sections III and IV of this Report. More detailed information from cases is provided to support our analysis of the BTP. BAC, and BSOP. The two page case study reports have been made available to KIAsia managers and USAID staff. However, since they contain some information that is proprietary or sensitive, firm names and circumstances will be disguised before the final report is submitted to USAID’s Development Information Center. Information for case studies was gathered through examination of KIAsia files, and through semi-structured interviews. The latter approach is initiated by encouraging the informant to “tell the story.” Interviewers then interject follow-on questions as appropriate. Lists of interview topics need not be followed step-by-step, but are used as a checklist before each interview is completed. (See Annex D for interview guides.) Dozens of interviews were undertaken, with international and Thai consultants to SMEs, owners and managing directors of those firms, government officials, and representatives of professional associations. About two-thirds of the interviews were undertaken jointly; the remainder were completed by one team member or the other. Three concepts—impact, efficiency, and sustainability—were fundamental for the Evaluation, and will be re-visited later in this report. At this point it is useful to provide an overview of how they were applied to KIAsia/AERA programs: The IMPACT of KIAsia activities can be observed at the micro level of firm, organization, or department, as well as at the macro level of sector, profession, or the entire economy. Impacts at the firm level can be detected in altered financial status, new production systems or marketing strategies, etc. In most cases, positive impacts on firms will include increased profits, but under 2 Throughout this period, we were assisted by Khun Patcharee, who showed great patience and creativity in meeting our requests for interviews, documents, and travel arrangements. (We are also grateful for the assistance of KC Choe and Arda Simsek of Kenan/Washington in scheduling Washington DC meetings, and telephone interviews after we reached Thailand.) Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 6 some circumstances, simple survival may be a significant achievement. Impacts at the macro level may be reflected in new laws, regulations, or administrative systems, and in establishment of professional codes of conduct, systems for self regulation, etc. The EFFICIENCY of KIAsia activities may be measured through financial or economic Benefit/Cost ratios, or through Cost Effectiveness calculations. Cost effectiveness may be measured at the level of entire programs (e.g., BAC or BSOP) or specific products (e.g., comparing costs of international volunteer consultants, to Thai volunteers or in-house consultants of the BAC.) SUSTAINABILITY for KIAsia as an institution may be achieved through mixed strategies, including effective entry into commercial markets, generating non-US government support from the private sector or other donors, designing innovative programs able to capture USAID regional funding, and careful husbanding of endowment income. II. THE BANK TRAINING PROGRAM (BTP) A. PURPOSE, ORGANIZATION, AND OPERATION OF THE BTP The multilateral and bilateral response to the Asian economic crisis focused heavily and immediately on the financial sector, particularly the banking system. Weaknesses in the system resulted in non-performing loans (NPLs) reaching a peak of 48% of total loans. The resulting cost to Thai taxpayers is estimated at over $33 billion.3 Most of the response was channeled through the Bank of Thailand. But, individual banks required direct and practical assistance in a number of areas, particularly risk management. The response of KIAsia's BTP activity, launched in February of 2000, was swift and focused. The primary goal was providing training and technical assistance to Thai banks to improve managerial standards, provide transparency, upgrade banking personnel, and strengthen staff training capabilities. Over the first three years of the program, the BTP assisted about 22,000 (18,000 via distance learning) banking officers and operational staff by providing practical knowledge and interactive training. The quick first year response was due to effective use of the SEGIR IQC mechanism facilitated by KIAsia.4 The BTP has primarily outsourced the training to specialized consultant teams from companies including Pricewaterhouse Coopers, Deloitte Touche Tohmatsu, and Development Alternatives Inc. and the Barents Group. Each bank's training program was custom tailored to meet its specific needs. Initially, BTP funds were envisioned to be handled directly by KIAsia. But prior to the Cooperative Agreement being signed, most of the first year funding was maintained with USAID. Over the first year and part of the second year of the Program, ten SEGIR task orders totaling $1.72 million were issued. Subsequently, KIAsia handled the bidding directly, instead of using SEGIR. At the time of this report, the bulk of BTP activity has been completed, and staff have left KIAsia. BTP has been placed within the responsibility of the Business Advisory Center. BTP objectives included: 3 Thailand - World Bank Group Partnership for Development document, p. 10. 4 SEGIR is a USAID contracting mechanism under which pre-qualified firms can be engaged to undertake economic policy activities such as technical advisory services, analysis, and training. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 7  Promote best practices and governance standards  Upgrade banking personnel to make better lending decisions  Train banking personnel to better manage risks  Improve banking staff training capabilities BTP operated with minimal staff. It did not offer training directly, but worked first through the SEGIR IQC, and then through Requests for Proposals (RFPs) circulated to SEGIR contractors and local organizations. The unit operated with one primary person as coordinator and with the KIAsia Associate Executive Director as a member of the selection committee. BAC's senior financial consultant was assigned the remaining BTP portfolio in August 2003. The BTP cooperated with all Thai-owned banks. They ranged from smaller private sector commercial banks like Kasikorn Bank, large private commercial banks such as the Bangkok Bank, large government-owned commercial banks like Krung Thai Bank, to large specialized financial institutions like the Bank for Agriculture and Agricultural Cooperatives. When applying for assistance, banks were required to describe in detail the types of training desired, along with a justification and description of expected benefits. Additionally, banks were required to share in the costs for the training program (20% the first year, 40% for subsequent training). KIAsia had no internal capacity in bank training, therefore in its own words: "The BTP has acted primarily as a coordinating body overlooking the training programs to ensure they fully meet the needs of the recipient bank." This important role should not be underestimated. From 1996, when KIAsia was created, through mid-2003 there was no USAID mission in Thailand. Therefore a mechanism such as the SEGIR IQC, which a resident mission could easily utilize, required a coordinator "on the ground" in Bangkok. During the first year of AERA, KIAsia performed this role, initially seeking out banks and assisting them to articulate their training needs. Once a training request was laid out in a format for USAID incorporation into a task order request for proposal, KIAsia passed the documentation to the Regional Contract Officer (RCO) in Phnom Penh. He then undertook the normal procurement process, traveling regularly to Bangkok, and including KIAsia’s Associate Executive Director on the selection committee. This process was used for the initial BTP assignments through the SEGIR mechanism. Subsequently, KIAsia utilized funds from the Cooperative Agreement, specifically from Modifications No. 4 (September 2000) and No. 7 (May 2002) to support BTP activities. KIAsia undertook the procurement directly, but in a manner similar to that used previously with SEGIR, first developing clear scopes of work, then requesting bids from SEGIR IQC holders and local institutions. During this phase of procurement, the selection committee included the Personal Services Contractor (PSC) who oversaw USAID interests in Thailand, as well as representatives of recipient banks. During the entire process (SEGIR and post-SEGIR), KIAsia ensured that the training was conducted in a manner most beneficial to recipient banks. BTP was a demand driven program. The banks proposed the training assistance they required. Additionally, by requiring cost sharing, a considerable degree of "ownership" was elicited from each bank. Most training under BTP has been completed. Only five activities remain open, two of which require only final reports and program evaluation. The remaining three focus on general finance and management topics, unlike the primary thrust of BTP, which concentrated heavily on risk management. The final activities with banks will be completed in the first months of FY2004, and remaining funds will focus on training programs related to micro-finance. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 8 Overall BTP has assisted: eight commercial banks, five of which received two separate consultancies; five specialized finance institutions: and three institutions on a “wholesale” basis, i.e. a specific consultancy was not performed for that institution on an intense training basis (Thai Institute of Banking and Finance, Cooperative League of Thailand, SME Bank). Overall 21 separate contracts were issued under BTP. B. AVAILABLE INFORMATION ON BTP PERFORMANCE At the beginning of the AERA program, results indicators and quantitative performance targets were established for each of KIAsia’s major programs. For BTP, it was decided that progress of ongoing activities would be measured by adoption of “new policies, systems, or procedures” (Table 1) and by the efforts of trainee banks to undertake new training programs on their own (Table 2). According to both indicators, the BTP has made satisfactory progress, a finding that has the Evaluators’ concurrence. TABLE 1 BTP Indicator 1: New Policies, Systems or Procedures Adopted by the Participating Banks Year Target # of Banks Cumulative Actual # of Banks Cumulative 1999 0 Baseline -0 2000 4 4 2001 5 7 2002 10 12 2003 12 16* * The table is extracted from AERA reports, however there are actually 14 banks and two related organizations TIBFA and the Cooperative League TABLE 2 BTP Indicator 2: Training Activities in Related Topics Undertaken Independently by Participating Banks Year Target # of Banks Cumulative Actual # of Banks Cumulative 1999 0 Baseline -0 2000 3 3 2001 6 8 2002 10 13 2003 12 16* 2004 12 In 2001, the Brooker Group, a Bangkok research firm, undertook a study of the early performance and impacts of KIAsia’s AERA programs. At the time, the first round of bank training sessions had been completed, and there was some opportunity for training participants to Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 9 assess whether training had brought significant reforms to their institutions. The findings of the Brooker survey regarding the BTP are summarized in Table 3. Since each participating bank had an opportunity to specify the type of training it required, Brooker researchers had to assess satisfaction with and impacts of several different types of training. As a result, it is somewhat difficult to extract straightforward findings from their work. Ratings of 100% for “moderate or better” leave unanswered the question of whether the preponderance of responses were “moderate,” “good,” or “excellent.” Nevertheless, a few observations can be made about the first round of training, which was contracted under the SEGIR mechanism, but supported and monitored by KIAsia staff:  Overall, the ratings for the first round of training were not particularly high. Across-the￾board the respondents rated their training “good” or “excellent” only 59% of the time. (See G.1.) The only apparent exceptions to relatively modest ratings were the Responsibility Accounting course, for which 100% of respondents gave ratings of good or excellent (D.1), and Internal Control, which 80% rated good or better (F.1).  On the other hand, rating of the work done by KIAsia’s BTP staff to support and monitor the training courses is quite high, with 88% of informants (see G.2) rating their performance good or excellent.  There is little apparent relationship between how well respondents liked their training and their perception of whether the training had positive impacts on their institutions. E.g., While only 57% of respondents rated their Risk Management training good or excellent (B.1), fully 100% perceived some level of impact (“moderate” or stronger) on developing new policies and actually improving risk management (B.3 and B.4). On the other hand, the Responsibility Accounting training was given high ratings (100% good or excellent), while few respondents (20%) perceived “improvement” in responsibility accounting practices (D.1 and D.3). The Brooker Report indicates that the BTP was off to a relatively strong start after 18 months of training, although there were complaints from a number of respondents about the quality of training. Weaknesses cited in the report included the level of the training course (in some trainees’ opinions too basic) and the trainers’ inability to relate the material to the Thai situation. The following are the main Brooker Report recommendations regarding management of the BTP:  Work closely with the banks in drafting guidelines, preferred credentials for trainers, and required background information/experience that would help the banks rank the proposals and curriculum vitae submitted by the training organization.  Provide trainers with briefing sessions and materials on the banking industry in Thailand, so they have a better understanding of the problems and issues faced by the participants.  Ensure the level of training corresponds to the participants’ knowledge base and experience. The banks should provide the prospective contractors with more information about the background of the participants, such as educational experience, subsequent training courses completed, level of work experience, and other information that enable the trainers to prepare a more customized and appropriate training curriculum. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 10 TABLE 3 Brooker Group Findings Regarding the BTP A. Credit management 1. Quality of credit management training (policy level and staff) 69% good or better Policy level respondents 2. Impact of the training on changing the credit process 83% impact occurred 3. Implementation of new credit policies or procedures 83% implementation occurred 4. Level of impact the training has had on helping with implementation 100% moderate or better 5. Level of impact of training on improving credit consideration process 100% moderate or better Staff level respondents 6. Level of impact the training has had on reaction of customers to improved process 60% greater satisfaction 7. Learning something new from the training contents 75% new knowledge 8. Level of impact the training has had on improving customers' daily work 100% moderate or better 9. The adaptation of the training to clients' daily work 100% adaptation to work B. Risk management 1. Quality of the risk management training 57% good or better 2. Impact on identifying key indicators of risk measurement 72% moderate or better 3. Level of impact the training has had on developing new risk management policies 100% moderate or better 4. Level of impact the training has had on improving the bank's risk management 100% moderate or better C. Debt restructuring 1. Quality of the training 40% good or better 2. Level of impact of the training on development of NPL guidelines and procedures 60% moderate or better 3. Level of impact of the training on implementation of NPL guidelines, procedures 40% moderate or better 4. Decline in the number of NPL clients due to training 20% decline 5. Decline in the number of restructuring loans due to training 20% decline D. Responsibility accounting 1. Quality of the training 100% good or better 2. Level of impact of training on improving customer understanding re: resp. acctg 100% moderate or better 3. Improvement in responsibility accounting system 20% improvement E. Training the trainer on business plan writing for SMEs 1. Quality of the training 67% good or better 2. Level of impact the training has had on business plans of clients 67% moderate or better 3. Bank's rate of lending to trained clients more than those not receiving training 33% increase F. Internal Control 1. Quality of the training 80% good or better 2. Level of impact the training has had on helping with internal audit 100% moderate or better G. Overall client satisfaction with BTP: 1. Overall rating for BTP training 59% good or better 2. Overall rating for Kenan's coordination and support 88% good or better  Experiment with the length of training in terms of hours per day and the number of days, and with the size of classes to see if participants receive greater benefits from the training. Further probing by KIAsia after the first round of training revealed that one major factor in the “quality” of BTP training was that it was delivered almost exclusively in English. Many trainees Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 11 who had to rely on in-classroom translation were apparently concerned that they were not receiving the full benefits of training. There was a broad sentiment for introducing Thai-speaking lecturer/trainers in the second round of training. To their credit, KIAsia and the firms that secured training contracts in the second round responded positively to this request. C. AN OVERVIEW OF BTP CASE STUDIES Considering the relatively small amount expended on BTP (12% of non-administrative AERA funds) and that activities are virtually complete, a representative cross-section of three banks for case studies (out of the 14 included in the program) was deemed sufficient. One bank was selected from each of three classes: large commercial banks with substantial government ownership, mid-size non-government commercial banks, and large government-owned, specialized finance institutions. The following sections describe the case study banks: Siam Commercial Bank (SCB): SCB is the 4th largest Thai commercial bank by asset size, with 10,710 employees and 483 domestic branches at the end of 2002. Throughout 2002 and into 2003 the Bank was in the process of reengineering its operations. BTP supported SCB in two separate training consultancies related to risk management, with content based on the specific requests and requirements of the Bank. The first training was conducted in mid-2000 by Barents Group. Barents support consisted of three segments: 1) Development of training materials for SCB's internal training course. (A teaching manual for the trainers, case studies, transparencies, handouts, and course evaluation materials.) 2) Delivery of a training of trainers course to select bank employees (12) who would follow through with the training. 3) Delivery of a Risk Management Workshop on a pilot basis (30 trainees) using resources developed. Areas covered included market risk, financial risk, and operational risk. The second consultancy, awarded to Pricewaterhouse Coopers, trained 80 senior executives in Risk Management. Three training courses were delivered: 1)Board of Directors (2 hour briefing session), 2)President and Executive Vice Presidents (2 days of lecture and case studies), and 3)Senior Vice Presidents and Vice Presidents (3 days of lecture and case studies). Topics covered included: Enterprise-Wide Risk Management, Corporate Governance, Credit Risk Management, Market Risk Management, and Operational Risk Management Bank of Ayudhaya (BAY): BAY is the 5th largest Thai commercial bank, with 7,670 employees and 388 domestic branches at the end of 2002. BTP supported BAY in two separate training consultancies related to risk management. The first, delivered in mid-2000 by Deloitte Touche Tohmatsu, had two parts: 1)training, and 2)assistance in developing a framework for risk management in the Bank. Training reached 391 persons and consisted of three distinct modules: top management session for the President and 15 most senior managers (2 days); Integrated Risk Management for senior managers (3 days); and middle management courses (3 days), in credit analysis or market and operational risk management. The second BAY consultancy was delivered by Pricewaterhouse Coopers in 2002. After initial analysis, training materials were developed for six courses delivered from March to September. There were 1,066 participants in the courses, which included: Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 12 1) Best Practices in Credit Risk Management (2 days) 2) Introduction and Implementation of Credit Manual (2 days) 3) Introduction to Market Risk Management (2 days) 4) Market Standard Techniques and Model (2 days) 5) Market Risk Exposure Management ( 2 days) 6) Train-the-Trainer on the Bank’s new Credit Manual (3 days) Bank for Agriculture and Agricultural Cooperatives (BAAC): BAAC is a specialized financial institution focused on rural development. Based on asset size, were it a commercial bank, it would be about the 8th largest Thai bank. BAAC had well over 12,000 employees, 588 branches, and 888 service units at the end of FY 2001. Branches are supervised by 72 provincial offices. BAAC was the subject of three training activities, one aimed specifically at it and two aimed generally at government banks. The first training was delivered in mid-2001 by Development Alternatives on Responsibility Center Accounting, and consisted of three sessions with 380 BAAC participants: 1)Executive Seminar on Responsibility Center Management (2 days); 2)Managing the Responsibility Center (BAAC Branch) (15 days); and 3) Train the Trainer Workshop on Profit Center Management, The second BAAC training activity, undertaken from July through November 2002, was entitled “New Bank Professionals Development Network” and conducted via distance learning for government banking staff throughout Thailand. Other participating banks were the Government Savings Bank, Government Housing Bank, Small Industrial Finance Corporation, and Krung Thai Bank. The Thai Institute of Banking and Finance (TIBFA) and the National Technological University (NTU) developed the courses with the banks, and delivered them via satellite broadcast to BAAC and other satellite centers across Thailand. Six courses were offered: 1) Risk Management, 2) Banking Law, 3) Internal Audit, 4) Sales and Customer Relations, 5) Asset and Liability Management and 6) Economics for Bankers. Expected participation was 12,000, with actual attendees exceeding 18,000. Due to the tremendous response to the first program, the five banks requested additional distance training. The third activity, "New Bank Professionals Development Network Phase II," runs from May through November 2003. The courses are: 1.) Good Governance, 2.) Credit Analysis of SMEs and Microfinance, 3.) Marketing and Customer Relationship Management, 4.) Knowledge of Cheques and Money Laundering Protection, 5.) Banking Account Law and 6.) NPL Management and Techniques. NTU is the contractor on Phase II and participation is expected to again reach 18,000. D. EVALUATION TEAM FINDINGS — THE BTP We have noted significant impact on bank operations. Risk management departments have been established and banks are managing risk with knowledge they acquired through BTP training. This was most pronounced at the Bank of Ayudhaya, and was evident at the Head Office, District Office, and Branch Office level. Prior to training, BAY had a cadre of individual managers with an appreciation for risk management, and an understanding of the importance of commitment at the highest level. Accordingly, the first training delivered was a two-day session for the BAY President and 15 of the most senior managers. This session concluded with broad agreement on 1) how the balance of the training would be delivered, and 2) an implementation Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 13 plan for a new risk management framework. This set the tone for all levels of the bank to embrace the framework. Bank officials indicated that after successfully engaging with the first training consultancy, they overcame a reluctance to use consultants. Previously they had feared they would not be able to control consultants’ work. Subsequently, the Bank, using its own resources, hired a consulting firm to assemble a consolidated credit manual from the numerous directives and circulars in use. Bank officials forcefully asserted to the Team that without the first BTP experience, they would not have further utilized consultants. Later they engaged in another BTP consultancy for risk management implementation, covering all levels of the Bank. This included: introduction to and training on the credit manual; risk management techniques; and training of trainer (TOT) courses for branch managers and senior credit related officers who would carry on the training. The Evaluation Team found that BAY had fully embraced risk management as part of its operations. The branch manager interviewed had participated in the TOT course and subsequently trained 40 persons. The credit manual was not found gathering dust on the manager's shelf, but with the branch credit officer, where it was being utilized daily. The branch manager asserted there had been big changes in the way loans were processed. Prior to BTP training, the prime determinants for loans were: collateral, who the prospective borrower was, and if they had a business. Little or no thought was given to avoiding non-performing loans (NPLs) by simply not making high-risk loans in the first place. Now, a careful analysis is made of the business to determine if it has the capacity to service debt. The manager asserted that the bank now helps the customer to: examine the business, decide on investment in the business, manage funds of the business and look at sales, and where appropriate, connect to other customers in a win-win situation. Siam Commercial Bank's first round of BTP support was undertaken to: 1) develop an internal training manual for risk management covering market, financial, and operational risk; and 2) train 30 bank officers from middle management to function as in-house trainers. The result was establishment of risk management awareness at the middle management level. SCB trainers then provided risk management courses to educate other staff on the concept and to raise awareness. Subsequently, a senior executive risk management training program was agreed with BTP. Creating awareness of the importance of risk management within the board and senior executives was vital. Additionally, the training was used to review SCB's risk management framework. SCB's comprehensive Change Program has modified some processes and caused some of the internal training planned in the second consultancy to be delayed. Including senior executive risk management exposure and a review of risk management framework as the primary step for introducing risk management to SCB may have averted some of the delays. Nonetheless, SCB personnel are convinced of the absolute need for comprehensive risk management. The Bank for Agriculture and Agricultural Cooperatives, like government owned development banks in many countries, has operated with large subsidies. A decision has been taken for a gradual reduction in government subsidy, and a requirement was recently set for the bank to operate as a professional, profit making institution. As is typical of such organizations, BAAC had been run as an amorphous structure, with little information on which units are generating profits or losses. Accordingly, there was great need for adoption of Responsibility Center Accounting, the subject of the BTP consultancy. The German technical assistance organization Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 14 GTZ had done some earlier work with BAAC, and the BTP consultancy was leveraged against this earlier work. Importantly, the training was provided on three levels: executive, middle management and training-of-trainers. The trainers carried forth the implementation of responsibility center accounting throughout the branches. A performance analysis sheet based on responsibility center accounting data was piloted in 30 branches near Bangkok for six months. Subsequently, it was implemented for all branches in March 2003. All provincial managers now receive a monthly Branch Performance Analysis Sheet for each of their branches, and a similar summary for consolidated branch performance of the province. Management can compare one branch with another and one province with another. Additionally, branches within a province each have access to other branches' sheets, enabling them to conduct a self-assessment of their standing in relationship to other provincial branches. Likewise, provincial managers have access to other provincial performance sheets, allowing self￾assessment at the provincial level. BAAC is also heavily involved in the distance learning program. This provides employees in outlying areas the chance to upgrade skills without having to travel long distances to attend courses. Participation by employees is heavy, and since they must come in on Saturdays on their own time, it is an indication that the courses are worthwhile. This is a low-cost approach to training, and BAAC is increasing the number of facilities available for this training. The evaluators also briefly considered the BTP’s overall impact on the Thai economy, but found that doing so is difficult. Certainly, the three banks selected as case studies have changed their operational practices as a result of the training, although some have succeeded more than others. Prudent risk management was not widely practiced in the banking sector previously. The crisis of 1997 forced banks to seriously consider the need for risk management. Ever more rigorous Bank of Thailand regulations have forced banks to recognize distressed loans at an earlier stage. As a result of the BTP, each Thai Bank has a core cadre of trained personnel who understand risk management, a substantial contribution to the Thai financial system. This is a valuable contribution to the economy, even if some of the banks are currently lending more aggressively than sound risk management would justify. However, were the financial crisis to re-emerge today, banks are much better equipped to handle the shock. In the opinion of the Evaluators, the impact of such a crisis on the Thai Economy would be far less than in 1997. However, based on interviews with BAC clients, it appears some government-controlled banks are supplying credit in liberal quantities, oriented more toward short-term growth than toward sound management of risk. This condition is unfortunate, but out of the scope of BTP assistance. Finally the evaluators considered the cost effectiveness of the BTP in delivering training programs, as shown in Table 4. The cost per bank is quite reasonable when impacts on the institution are considered and when the one considers the impact of banks on the Thai Economy. Although BTP was essentially an institutional program, designed to strengthen the banks, another measure of cost effectiveness could involve the number of persons trained. The latest annual report indicates 22,000 were trained, which would equate to $135 per trainee. This statistic is distorted by the distance learning results of 18,000 trainees. Without their inclusion, the cost would be just under $750 per person trained, still a reasonable figure. The total cost of BTP is less than one one-hundredth of one percent of the estimated cost of non￾performing loans to the Thai taxpayer. The Evaluation Team believes that BTP will pay for itself Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 15 many times over. Healthy financial institutions have a widespread positive impact on the economy. In contrast, weak banking institutions, which cost Thailand dearly in the 1997 crisis, place a tremendous burden on the economy. TABLE 4 Cost Effectiveness of the BTP (6/30/03) Bank Training Program Item (KIAsia Cost Analysis as of June 30, 2000) Amount KIAsia Costs ($) (1) 1,257,040 SEGIR Costs ($) 1,718,918 Total BTP Costs ($) 2,975,958 No. Institutions (14 are banks) 16 Cost per Institution 185,997 Source: KIAsia Financial Reports, AERA Records. (1) Includes prorate admin. cost share III. THE BUSINESS ADVISORY CENTER (BAC) A. PURPOSE, ORGANIZATION, AND OPERATION OF THE BAC The Business Advisory Center is KIAsia’s largest unit. Its staff of two dozen constitutes more than one-third of KIAsia’s full-time manpower, and the annual budget, about $1.1 million for FY2002, represents 32% of KIAsia’s annual AERA budget. (For FY2003, the figures are $1.03 million budgeted, for about 20% of the AERA budget.) BAC occupies a separate suite of offices in the Queen Sirikit Center, an appropriate arrangement for a unit that regularly serves a range of SME customers. The mission of the Center is clearly stated in promotional literature: “The primary aim of the Business Advisory Center is to provide training and consulting services to Thai SMEs…By drawing on the expertise of professionals experienced in both western management training and local business customs and culture, the BAC provides support to SMEs through direct consultative services in various industries and also through informative seminars on a range of relevant topics.” The BAC began its work in late 1999 with fairly aggressive marketing. SME managers were invited to workshops on “preparing business plans” and similar topics. BAC staff distributed promotional literature, and met business people informally at coffee breaks. The BAC and its services were discussed on the radio. Interested individuals were invited to visit BAC offices, and staff members were sent to meet SME managers in their offices and factories. (Training/knowledge sharing activities that were initiated at this time have benefited more than 2,000 sme business owners and managers.) By the end of 2001, 165 consultancies had been undertaken. Systems for managing consultancies to firms that approach the BAC directly were established at the time, and remain in place to the present. Although costs of advertising and recruiting clients have been rather high, BAC staff offer a convincing argument that these efforts have played a useful role in promoting the concept of paying for business consulting services among Thai SMEs. Furthermore, this may have created a more receptive client base for Thai government-sponsored SME consultancy programs originated under the current government. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 16 Each potential client firm is assigned to a BAC staff professional, designated as an in-house “consultant,” who confers with a representative (generally the owner or managing director) of the firm and drafts an initial scope of work. He/she also prepares standard documents, maintains office files, and recruits outside consultants as needed. Where required services fall within the competence of recent MBA graduates (e.g., preparing business plans, establishing HR services, or improving internal reporting systems), BAC staff often take on the consulting work themselves. When more specialized expertise is needed, there are two other options: 1) Recruit an American “volunteer expert” (VE) through the International Executive Service Corps (IESC), Citizens Development Corp (CDC), or ACDI/VOCA. 2) Recruit a Thai volunteer through a similar program, the Thai Volunteer Consulting Service (TVCS), established by KIAsia under AERA funding. The AERA Cooperative Agreement with USAID specified substantial use of international VEs, and encouraged KIAsia to work with all three programs. After agreeing with the organizations on fee structures and minimum numbers of volunteers, BAC was obliged to hire roughly 135 VEs over the period of AERA. Although the volunteers do not charge daily fees, their services come at relatively high cost. BAC expenses include a recruitment fee of $6,000, international airfare, incidental expenses, two days of lodging, and a modest daily allowance. VEs who work for three weeks or more are entitled to bring along their spouse. Costs per volunteer are in the range of $10,000-15,000. From the beginning, BAC has pursued a policy of cost sharing, but until recently the client firm’s share was modest. They were required to cover the costs of the VE’s hotel, a modest per diem (B1500), hotel-to-office transportation, in-country travel, and translator/interpreter fees. These costs are generally in the range of B30,000-50,000 ($750-1250) for a two-four week consultancy. Thus, VE consultancies have been subsidized on the order of 90% of direct costs. More recently, BAC has informed clients that the fee for VE consultancies is B10,000 ($240) per day. VEs have been used almost exclusively on a ‘retail” basis, working with a single firm. Only occasionally have they offered a brief seminar to a larger group. One promising activity initiated in conjunction with the BAC was establishment of the Thai Volunteer Consulting Services (TVCS). Senior Thai businessmen were invited to register as volunteers. These individuals offer several advantages: 1)They are already “on-site” and no international travel costs are incurred. 2)Their services can be offered on a more measured schedule, say one day per week, rather than the hurried, intense two or three week visits that are the norm for international VEs. 3)They are intimately familiar with local business practice, though many have also gained valuable knowledge while working for international firms. 4)They are in a favorable position to call on other in-country expertise from among their friends and former colleagues. 5)They speak Thai. Unfortunately, at the time of the Evaluation Team’s visit, use of Thai volunteers was at a low ebb, and the TVCS appeared to be dormant. The original plan for the BAC was to hire 10 Thai and 10 American recent graduates of MBA programs as core staff. The Thais were predominantly graduates of US programs, or of the Sasin Business School of Chulalongkorn University. Americans were drawn from prestigious US programs. In line with the “project” status of BAC at the beginning, turnover was expected to be high. Americans were given one year contracts, subject to renewal, and Thais received two year contracts. This approach did in fact lead to high turnover and to rather high recruitment costs. Americans were hired through the MBA Enterprise Corps (MBAEC), initially at a fee of $8,000 Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 17 per selected employee. Later this was raised to $10,000 and then to $11,000. Recently, the Manager of the BAC has taken on recruitment herself, relying heavily on graduates of KIAsia’s partner institution, the Kenan-Flagler Business School of the University of North Carolina. In response to relatively high costs of recruiting and maintaining international staff, the decision has been made to reduce their number, perhaps to only two or three individuals at any one time. B. AVAILABLE INFORMATION ON BAC PERFORMANCE When initiated, the BAC had the following stated objectives:  Aid recovery of the Thai economy through SME growth  Provide technical consulting and customized training to SMEs  Solve business problems and enhance long-term competitiveness of SMEs by strengthening management and technical capabilities Readers will note that, although one broadly-stated objective of the BAC is to “aid recovery of the Thai economy,” work has preceded one firm at a time. In 1999, the BAC was set up as a short-term project, focused on the short-term goal of helping individual businesses succeed. The goal set, reflected in Table 5, was for the BAC to assist 100 firms annually. For the first two years, this goal was met. More recently, services have clearly slowed down. For example, in the first nine months of FY’03, only two dozen firms have been assisted. Whether or not the 100 firm goal is met, now that the economic recovery is well under way and the BAC is in its fifth year, questions need to be raised about the benefits of a few hundred SME consultancies for “the Thai economy.” Clearly, it is hard to justify further donor support for a program of this sort. TABLE 5 Total Number of SMEs Receiving BAC Consulting Assistance Year Target # of Assisted SMEs Cumulative Actual # of Assisted SMEs Cumulative Annual # Assisted 1999 40 40 2000 20 116 76 2001 120 165 49 2002 220 193 28 2003 320 217 24 2004 410 2005 500 It is important to note however, based on evidence from Table 6, that services provided through the BAC have been well-received by SME clients.5 Clients’ satisfaction with the performance of BAC staffers (section I.A) in planning and supporting consultancies has been very high. For each aspect of this work, at least 88% of respondents to a 2001 survey by the Brooker Organization gave a rating of “good” or “excellent.” Particularly impressive is the finding that 100% of respondents found that the “professionalism” of the staff was good or excellent. 5 The number of respondents generally varies between 41 and 44. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 18 TABLE 6 Brooker Report Data on the BAC A. Client's satisfaction: 1. Response rate to client applications 97% quick or better 2. Average number of days to respond to application 14 days 3. Professionalism of consultants 100% good or better 4. BAC consultants' level of understanding clients' businesses 93% good or better 5. Scope of work and qualifications of the expert 88% good or better B. Expert/consultant performance and client service: 1. Match between consultants' expertise and clients' needs 76% good or better 2. Consultants' overall aptitude and understanding clients' business 87% good or better C. Impact of services: 1. Improvements resulting from BAC project 63% improvement 2. Financial changes related to the BAC assignment: a) Increase in sales 17% increase b) Decrease in costs 20% decrease c) Increase in profits 7% increase D. Overall client satisfaction with BAC: 1. Projects met companies' needs 71% satisfaction 2. Satisfaction in terms of clients' time, resources, money 84% satisfaction 3. Willingness to use BAC services in the future 93% willingness to use again Ratings for actual consultancies (section B.1) are somewhat lower, with only three-quarters of respondents satisfied that there was a good match between their needs and the consulting services provided. Furthermore, a still smaller portion of respondents reported (section C.1) that there had been positive financial impacts on their business. A relatively small number of firms reported increased sales (17%), decreased costs (20%), or higher profits (7%). These low numbers for consultancy results probably have at least two explanations: 1)At the time of the Brooker survey, the BAC had been in operation for less than two years; for many of the firms too little time had passed for changes in production systems, sales strategies, etc. to lead to higher profits. 2)There is a widespread reluctance to openly discuss business profits in Thailand, in case such information reaches tax officials. Perhaps the strongest endorsement of BAC services is the 93% of respondents who expressed their “willingness to use BAC services in the future.” C. AN OVERVIEW OF SME CONSULTANCY CASE STUDIES Among the first questions that must be applied to any service organization is the effectiveness and impact of services offered. Table 7 offers a summary of the Evaluation Team’s findings about BAC services to 13 selected client firms. These cases actually represent 16 separate consultancies, since three firms benefited from two consultancies. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 19 TABLE 7 Brief Overview of BAC Case Studies6 Firm Business Type(S) Judgment Firm A Cross-Stitch Kits Marketing--CDC Highly Effective Firm B IT Consulting Human Resources--CDC Highly Effective Firm C Property Valuation and Databases Valuation Skills and Computer Assisted Mass Appraisal--CDC, Client Selected Highly Effective (2) Firm D (2 Consultancies) Designing/Installing High Tech AV Systems General Management, Marketing -- CDC, IESC Highly Effective (2) Firm E Tourist Hotel and Restaurants Business Plan, General Management--In-House Highly Effective Firm G Restaurant, Entertainment Marketing, General Management--CDC Effective Firm H Orange Orchards Growing Techniques, Marketing--CDC Effective Firm I (2 Consultancies) Coffee House/Restaurant Chain Franchising, General Management--TVCS, ACDI/VOCA Effective (2) Firm J Tourist Magazine and Web Site Marketing--BAC In-House Effective Firm K “Ethical” CD Recording Software Programming-- IESC (“Geek Corps”) Mixed Results Firm L Auto Parts General Management--BAC In-House (ITB) Not Effective Firm M “Khan Tok” (N. Thai) Dinner Shows Marketing to “High End” Tour Operators--CDC Not Effective Overall, the BAC’s performance record is quite good. In the case of five firms (seven separate consultancies), the Evaluation Team’s judgment was “highly effective.” For five more firms (six consultancies), the judgment was “effective.” In the case of one firm, “mixed results” were achieved, and finally two consultancies were found to be “not effective.” A number of observations can be made from this simple evidence:  There is no obvious relationship between the effectiveness of a consultancy and the source of the consultant. International VEs are heavily represented throughout the Table. Nor are there obvious differences among the three organizations supplying international VEs. For example, CDC is well represented at the top, middle, and bottom of the chart, while IESC-supported consultancies range from “highly effective” to “mixed results. International VEs are probably over-represented overall, since we emphasized earlier consultancies in order to have a longer timeframe in which to judge effectiveness, and international VEs were used heavily at the outset of the program. One BAC in-house consultancy occurs in the “highly effective” category, while another was judged “effective,” and a third judged “not effective.” The single Thai volunteer found on the Table carried out an “effective” consultancy. 6 Firms selected represent a cross section of those assisted by: size [small/medium], region [Bangkok/nearby/up country], source of consultants [IESC/CDC/ACDI-VOCA/TVCS/BAC staff], and industry [crafts/automotive/services/etc.] Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 20  Location does not seem to have any obvious influence on effectiveness. Consultancies in Northern Thailand are over-represented among the cases, but two fall in the “highly effective” category, two in “effective,” and one in “not effective.” Consultancies in Bangkok and environs also fall across the range of judgments.  It is possible that the Evaluators’ assessment of BAC was subtly “steered” toward successes by KIAsia and Kenan/Washington who suggested companies for interviews. However, the influence of such steering appears to be minimal. Of the half dozen cases the Team was steered to, two (Firms B and C) were judged “highly effective,” three (Firms I, F, and H) were judged “effective,” and one was judged to have “mixed results.” In three cases, firms selected independently by the Team were in the “highly effective” category. And, when the Team selected firms ultimately assigned to the “not effective” group, there were no visible efforts to deflect us toward other cases.  Another important observation, not obvious from this simple Table, is that a consultancy judged “not effective” or with “mixed results” does not necessarily reflect on the quality of the consultant’s work. In one case, the consultant offered potentially useful advice, but the firm owner delayed in following it for almost three years. In another, market conditions probably ruled out an effective consultancy, since the firm offering Khan Tok dinners has opted for expensive facilities and high quality performances in a highly competitive market, where tour operators emphasize bargain pricing over the quality of the food and performance. In the case of “mixed success,” the international VE was frustrated in his efforts to achieve knowledge transfer to his client’s professional staff. Not every consultancy can be effective. However, it is important to learn from those that are not. Four BAC cases, one per judgment category, are summarized here, to give readers a better understanding of the information the Team collected, and how judgments were made: Firm A (Highly Effective): This well-managed firm already had strong national and regional markets and high quality designs for cross-stitch kits. But managers decided they wanted to enter the large U.S. market with their products. The firm sought a VE with detailed knowledge of the U.S. crafts market. The individual selected runs a similar business in Colorado, and carefully assessed the U.S. market through web searches and consultations with friends in the business before her arrival. At her own expense, the VE traveled to a crafts convention in North Carolina to meet Firm A’s Managing Director, and introduce the latter to a number of her friends in the business. During a two week consultancy, the VE offered detailed advice on appropriate product lines for the U.S. market (e.g., cross-stitch pattern books instead of full kits, and emphasis on “oriental” designs); strategies for approaching individual customers (through one portal on a company website) and retailers (through another portal on the same website, and hiring a “fulfillment warehouse” company to handle orders). As a result of this advice, the firm has achieved monthly sales of B200,000 (almost $5,000) to the U.S., dealing with 100 retailers and numerous Internet customers. The firm has not yet succeeded in placing its products with Michael’s, the largest U.S. crafts products chain with annual sales of $2 billion. But, as a result of continuing contacts with the VE, managers have a clear sense of the steps that must be taken to achieve this major breakthrough. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 21 Firm F (Effective): This firm manufactures solid rubber tires for export to Japan, Europe, and North America. The firm, established as a Thai-Japanese joint venture, gained access to valuable technical knowledge. But when the Japanese partner pulled out, Firm F was not equipped to assure adequate quality control, or develop new products for an expanding market. Managers contacted BAC to request an international VE to advise on improving quality control and developing new products. The retired rubber engineer selected undertook a three week consultancy, first carefully observing procedures at the plant, then offering valuable advice. Much of the advice was accepted and acted on. For example, new production procedures were adopted, the reject rate for finished tires has dropped markedly, and specialized rubbers were developed for use in new tire products. There has been continuing email contact between the VE and the firm’s managing director, much of it aimed at locating a second expert, who can advise Firm F on manufacturing processes for a major new product, rubber tracks. While the firm’s managers were pleased with the consultancy, and the VE was satisfied that most of his advice had been heeded, he noted two shortcomings in the overall process: 1)Waste rubber scraps are still buried behind the plant, despite the VE’s advice on selling it to a nearby cement manufacturer. This would have the dual benefit of increasing plant income, and averting a future environmental hazard. 2)The VE noted little interest on the part of managers in transferring skills for developing new rubber products to the laboratory director. Firm K (Mixed Results): This firm is an “ethical” producer of CDs. That is, they ensure intellectual property rights are protected. They receive production orders from many other firms for music or data CDs. Firm managers were concerned that their computer software did not effectively integrate systems for recording orders, billing, and accounting, thus requiring repeated, labor-intensive re-entry of the same information. The original SOW negotiated with BAC specified an expert VE to write a software program integrating the firm’s ordering, vouchering, and accounting software. When the VE arrived, it became apparent that this task was impossible. Neither BAC staff nor Firm K managers had possessed the specialized knowledge required to approach the Thai firm supporting the accounting software, and gain their agreement to release information required for the work of integration to proceed. On his arrival, the VE determined that the computer firm lacked technical knowledge to adequately support its proprietary accounting software. The firm declined to provide access codes based on the argument they had proprietary rights to the software. After this setback, the VE undertook the simpler task of writing a program to integrate Firm K’s order taking with billing, enabling employees to type orders directly on the computer, and later use the same data to produce bills. The managing director and plant manager were very satisfied with the VE’s work, and highly complimentary regarding his professional qualifications. In retrospect, they believe the main benefit was in lessons learned for the future about procuring major software systems. On the other hand the VE was relatively dissatisfied with the consultancy, expressing his dismay that Firm K’s managers discouraged him from including the IT systems supervisor and a part-time programming consultant in his work. As a result, he observed, there was no “knowledge transfer.” He noted that he had provided little more than “cheap labor” for a task that could easily have been performed by the firm’s part-time IT consultant. Firm L (Not Effective): Firm L is a family-owned firm that manufactures parts for auto assembly, as well as replacement parts. The firm’s sales were severely affected in the late 1990s by the economic crisis, and have begun to recover significantly only in the past year. In 2002, the Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 22 firm applied to the Ministry of Industry’s ITB program for assistance in developing a more appropriate marketing strategy. Unfortunately, services provided by BAC were significantly hampered by timing problems, some the result of outside forces, some caused by slow reaction time within BAC. Firm L’s ITB application was submitted in March 2002 and approved in April. After the consultancy was contracted to BAC, two in-house consultants were fielded in July. During assessment visits, the team recommended that the consultancy not focus on marketing issues, but on the broader challenge of converting Firm L from a conservative, family-managed firm to a modern, professional corporation. Soon after this recommendation was accepted by KSK, the consultant team leader resigned from BAC. A new team was selected and began work only in September. They faced two challenges: 1)Their time was severely limited. Under Ministry of Industry regulations, ITB consultancies must be completed within the fiscal year, and the BAC consultants were required to complete their field work by November. This allowed only 15-20 half-day visits by the team. 2)Their task was over-ambitious. Firm L is operated as a family business by the founder and his six sons. While there was agreement among the younger generation on the need for reforms, each brother had a different view of what form they should take. The consultants devoted their limited time to learning the details of firm management, with little time left for the difficult task of defining workable alternatives, and the even more daunting challenge of getting six brothers and their father to agree on a single reform strategy. In the end, according to the brother most concerned with reform, the BAC produced “a new organization chart,” nothing more. D. EVALUATION TEAM FINDINGS — THE BAC As noted above, BAC consultancy results have been mixed, but our estimate is that the majority have been “highly effective” or “effective.” However, some consultancies judged by us to have “mixed results” or be “not effective” show no evidence of shortcomings in BAC files. Those files include a series of post-consultancy forms, filled in by the consultant, a representative of the firm, and a BAC staffer as they evaluate each other. Respondents check boxes for “excellent,” “good,” etc. and have the option of offering brief comments. As is often the case, such comments tend to be politely supportive, rather than useful for management purposes. At the same time, more useful follow-up documentation — e.g., notes from discussions with managing directors on the effectiveness of new production systems or marketing strategies — are lacking. Thus, our strongest criticism is that BAC staff are not learning sufficiently from past consultancies, whether they were “highly effective,” “not effective,” or in between. For example, BAC files do not reflect the decision of the manager of Firm J to ignore useful recommendations by a BAC in-house consultant on gaining greater income from the firm’s popular website. Nor is there any record of a successful “offshoot” of a VE consultancy at Firm B that led to establishing an effective new Systems Analysis Department that regularly uses an analytical tool provided by the consultant on departure. Nor is there any knowledge that the owner of Firm G reviews the VE consultant’s report weekly for inspiration on expanding and improving his business. This apparent lack of follow-up has a number of costs:  BAC staff are denied management information that may be useful in reducing failures and building on successes. For example, the analytical tool introduced at Firm B might be applied by BAC In-House consultants undertaking future consultancies focused on production systems. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 23  Significant marketing opportunities are missed. BAC follow-ups a year or two after each consultancy would also offer opportunities to identify next-generation problems that might benefit from BAC consulting services. For example, an hour of discussion with the owner of Firm E would have revealed she is a potential customer for a number of “micro-consultancies” regarding issues that have cropped up after the initial BAC consultancy.  Opportunities to identify promising new products are also missed. The notion of “micro￾consultancies” on such issues as evaluation of complex bank loans and software systems came to the Evaluators as they interviewed the owner of Firm E. The notion of offering a workshop and checklist on procuring major software systems occurred to the evaluators during the interview with Firm K managers. The notion of offering a workshop on establishing HR systems in small firms emerged from an interview with an international VE, and was also suggest by managers of firm B.  Significant training opportunities are also lost. New staff could learn a great deal by undertaking follow-on interviews with former client firms, and could then be asked to share their observations with colleagues through emailed memos or staff seminars. Another missed opportunity for staff training lies with the international VE consultancies. BAC professionals have not routinely joined the volunteers in carrying out consultancies, though they could have learned a great deal by doing so. Instead, BAC staff have taken a more limited support role, writing initial scopes of work, ensuring smooth logistics, and preparing required documentation. While the latter activities are necessary functions, in most cases they could have been accomplished satisfactorily while the BAC staffer joined the volunteer at the office or factory full-time. An important, and essentially free, training opportunity has been foregone. The Evaluation Team noted an apparent selection bias toward firms in Bangkok and environs and toward the “larger” of the “small and medium” firms. We are aware that Thai businesses are focused disproportionately in Bangkok and nearby provinces, but also noted the virtual absence of consultancies in the Northeast and South, and even in Central Thai provinces located more than an hour or two from Bangkok. We note that these biases can probably be attributed to circumstances largely beyond BAC control. For example, Bangkok firms have easier access to the BAC office, and larger firms are more likely to have the confidence and English skills needed to deal successfully with international VEs. However, we note that acceptance of contracts under the ITB program seems to have broadened the BAC’s geographic coverage, since a substantial portion of those consultancies are with firms outside Bangkok. Several operational trends that will have direct implications for long-term BAC financial sustainability were noted by the Evaluation Team:  American MBA staff are apparently being pruned. With the departure of two Americans during this Evaluation, only one remains. We understand that a few more Americans may be hired within the next few months, but limiting BAC American employees to three or four may be an appropriate strategy for lowering operating costs.  BAC recently began informing potential clients they must pay a daily fee of B10,000 for consultancies. While this appears to be a reasonable first step toward financial self￾sufficiency, it should be noted that, on one hand, the figure is sufficient to cover only 50- Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 24 70% of international VE costs, while on the other hand it has virtually dried up demand for direct BAC services.  Coincidentally, the number of International VE consultancies has dramatically declined. This may also be a necessary step toward controlling operating costs, but for some potential clients it may significantly reduce the attraction of BAC services.  B10,000 per day may be sufficient to cover costs of in-house consultancies, but only if the BAC can significantly reduce operating costs and bill for roughly 2000-2500 person￾days per year,  In any case, private clients have been unwilling to pay this higher fee. Even satisfied former BAC clients interviewed by the Evaluators stated clearly they would not pay fees at this level. There is little likelihood the BAC can achieve self-sufficiency at this time by providing services in the private consulting market.  Thus, another trend noted by the evaluators takes on greater significance. For the past year, the BAC has marketed services primarily through Thai government support programs for SMEs. These highly subsidized programs (client firms pay only 10% of costs) have undercut direct demand for BAC services (particularly at the higher daily rate of B10,000), but have allowed BAC to earn significant income as a contractor for consulting services. Under the Invigorating Thai Business (ITB) program of the Ministry of Industry, BAC consultants earn B10,000 daily and consultancies average 30 man-days. (BAC staff have won awards from the Thai government for the quality of consulting services provided.) At the time of this Evaluation, virtually all BAC consultancies are being undertaken under government programs, and are earning noteworthy levels of income, e.g., B6.7 (28% of BAC direct costs) million for the first nine months of FY’03. E. A RELATED ACTIVITY — CLUSTER COMPETITIVENESS KIAsia began experimenting with development of “clusters” of SMEs in related business sectors relatively early in implementation of AERA programs. This early work was done under the umbrella of the BAC. More recently it has been carried out by a separate unit within KIAsia, in conjunction with J.E. Austin, a consulting firm experienced in converting the “theory” of cluster competitiveness into practical programs. The Evaluators interviewed the leaders of the KIAsia and J.E. Austin cluster competitiveness teams and Austin’s Senior Competitiveness Consultant posted in Chiangmia, attended one meeting of a cluster in Nakhon Pathom Province, and examined materials provided by KIAsia. Based on these rather limited observations, our findings are: The cluster approach remains experimental and unproven for KIAsia and Thailand. No cluster can be labeled a success so far. It is too soon to estimate what success rates are likely to be achieved, and unclear how those successes might be measured. But, the approach shows considerable promise. It appears several clusters are moving ahead, and show some likelihood of undertaking effective collective action — e.g., influencing government policies, improving production standards, establishing “brand” recognition, setting up research or design centers, etc. The cluster approach has several facets that offer potentially attractive alternatives to the BAC’s current emphasis on two-four week consultancies to individual SMEs. E.g., It is a “wholesale” Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 25 approach, capable of working with dozens, even hundreds, of firms simultaneously. Based on a comparison with BAC clients, it appears to reach firms that are smaller, less well connected, and geographically more remote. From brief observation, it appears some facets of approach — e.g., group dialogue and analysis and action planning — remain under-developed. There may be ready made, or readily adaptable solutions to these shortcomings.7 There seems to be little opportunity for the early stages of cluster development to become financially self-sustaining. SMEs will be unwilling to pay until they are convinced of clear financial benefits to cluster membership. Such evidence is unlikely to appear until after a year or two. Similarly, other participating organizations, such as universities, will need financial help to set up appropriate support services. Cluster development will remain a subsidized activity for the foreseeable future. KIAsia staff will be well advised to carefully document emerging successes in this field, to justify future support from bilateral donors, international or Thai corporations, or the Thai government. IV. THE BUSINESS SUPPORT ORGANIZATIONS PARTNERSHIP A. PURPOSE, ORGANIZATION, AND OPERATION OF THE BSOP KIAsia’s Business Support Organization Partnership (BSOP) Program strives to establish effective partnerships between Thai and U.S. “business support organizations.” These groups are defined in KIAsia promotional materials in this way: Business Support Organizations are government and non-profit organizations that help the private sector with: better business standards; good governance standards; professional training and certification; legal support; and access to capital. By nurturing working partnerships between Thai and American groups, the BSOP aims to improve and support Thailand’s: financial and corporate governance; legislative and judicial reforms; and international competitiveness. The partnerships operate through organization-to￾organization and peer-to-peer information sharing. The BSOP has a relatively small staff. Two full-time Thai professionals are supervised by an American who also has other management duties. In addition, an American employee of a US consulting firm works part-time with the BSOP, and two employees of Kenan/Washington devote substantial time to building and nurturing BSO partnerships from the U.S. side. In all, perhaps four and a half full-time professional positions are involved, plus two additional support staff positions. Overall, personnel costs absorb about 25% of funding for the program. Building partnerships is a varied process, at this point more an art than a science. Sometimes the work involves intensive efforts by BSOP and Kenan/Washington staff to locate and bring together appropriate partners, then collaborate with them closely to define detailed, feasible plans for achieving useful reforms. BSOP efforts to form and nurture a partnership between Thailand’s 7 KIAsia and J.E. Austin staff may wish to examine experience using the Technologies of Participation (ToP) in analogous situations in the Philippines. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 26 Office of the Auditor General and the U.S. General Accounting Office falls into this category. In other cases, partnerships are formed independently, with those involved making their own plans before approaching KIAsia for funding. Support for establishing a Better Process Control School exemplifies this approach. In many cases, BSOP staff rely on colleagues in Kenan/Washington to locate appropriate partners and organize activities such as teleconferences and study tours. Sometimes partnership activities require close collaboration with other donors. Over the past four years, the BSOP manager has attended donor coordination meetings coordinated by the World Bank. The purpose has been in part to ensure that donor activities are not duplicative, and where possible, are complementary. An example of the latter approach played itself out when BSOP-supported activities to introduce mediation of financial disputes within the Thai court system were later supported and expanded under a World Bank program. B. AVAILABLE INFORMATION ON PERFORMANCE OF THE BSOP BSOP staff have assembled considerable data on the activities and achievements of assisted partnerships. Table 8 shows that two dozen results--new policies, systems, and procedures--have been introduced. Examples include:  A Better Process Control School established to train Thai technicians on food processing techniques to enable their firms to export canned goods to the U.S.  New skills or professional standards established among Thai property appraisers, investment advisors, and internal auditors.  New accounting curricula introduced in universities across Thailand. TABLE 8 Results Achieved by BSOP — Policies, Systems, or Procedures Adopted by Partner Organizations Unit of Measure Number of participating organizations implementing policies, systems, or procedures introduced or strengthened through BSOP partnership Year Expected Measures Cumulative Actual 1999 0 Baseline - 0 2000 2 7 2001 7 17 2002 10 23 2003* 15 25 2004 20 2005 25 • Q1 of Year 2003 (October – December): additional 2 policies/system changed Table 9 shows that the number of partnerships has grown steadily. By reducing complex processes to simple numbers, the Tables over-simplify the work and achievements of BSOP partnerships. For example, assessing whether or not an organization has “adopted” a policy is often debatable. Some observers are satisfied that adoption has taken place when a policy is Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 27 formally included in the bylaws of an organization, while others insist on clear evidence that new rules are being enforced. Similarly, different observers seek different kinds of evidence to determine if a partnership is being “maintained.” However, for purposes of this Evaluation, these are minor quibbles. The essential point is that the numbers in Tables 8 and 9 are broadly consistent with our strong impression that a substantial number of BSO partnerships are proceeding well, with positive results. TABLE 9 Results Achieved by BSOP — Number of Long-Term Partnerships Initiated and Maintained Unit of measure Number of partnerships Year Expected Partnerships Cumulative Actual 1999 0 Baseline - 0 2000 2 7 2001 4 16 2002 8 22 2003* 12 23 2004 10 2005 8 * Q1 of Year 2003 (October – December 02): one additional partnership with The Institute of Internal Auditors. In 2001, the Brooker Group interviewed representatives of Thai and U.S. organizations participating in partnerships about the performance of the BSOP. Table 10 summarizes their findings. In general, satisfaction levels were high. These results tally with opinions the Evaluators heard on several occasions — KIAsia staff in general, and those of the BSOP in particular, are highly professional and collegial in their work. The only exceptions to these consistently high ratings were reflected in the difficulty some partners had in understanding BSOP’s role and purpose (items A.1, A.2a, and B.2a) and in some partners’ dissatisfaction with procedures to reimburse expenditures (items A.3f and B.2e). Difficulties in understanding BSOP’s role and purpose are understandable. The program is innovative, and methods used and responsibilities undertaken by BSOP staff vary widely with circumstances. Nevertheless, it may be possible to explain “the BSOP approach” to potential partners more effectively by making brief, jargon-free explanatory materials on a cross-section of partnerships available to potential partners. In fact, materials of this type have already been drafted, But, perhaps greater effort needs be made to ensure the materials are readily available and reader-friendly. In any case, the Evaluators believe the inability of potential partners to fully understand “the BSOP approach” is a relatively minor problem: Results of partnership activities are more important than full comprehension, at the outset, of how partnership activities will proceed. Slowness or unduly complex procedures in release of public funds constitute a classic problem, and sometimes steps can be taken to alleviate the more extreme examples of under performance. But it should be noted that often slowness and complexities are created as much by the Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 28 unwillingness of receiving organizations to learn and abide by established procedures as it is by shortcomings of the paying organization. TABLE 10 Thai and US Partners’ Assessment of BSOP Performance* A. Thai organizations: 1. Clarity about the BSOP program 60% clear understanding 2. KIAsia staff's coordination role 100% good or better 3. Components of the BSOP program: a) BSOP program materials 54% good or better b) Developing the proposal 87% good or better c) Identifying an appropriate US counterpart organization 86% good or better d) Facilitating the relationship between the organizations 100% good or better e) Monitoring the project 93% good or better f) Reimbursement of expenses for project 54% good or better B. U.S. organizations: 1. Perceptions of BSOP and Kenan Institute 100% clear understanding 2. Respondents' perceptions about the amount of support by Kenan and BSOP: a) BSOP program materials brochure 76% good or better b) Identification of the assignment/scope of work 88% good or better c) Facilitating the relationship between the organizations 88% good or better d) Monitoring the project 88% good or better e) Reimbursement of expenses for project 50% good or better * From Brooker Report—2001 C. AN OVERVIEW OF BSO PARTNERSHIP CASE STUDIES The Evaluators examined eight partnership activities in detail. At our request, BSOP staff prepared a list of 14 activities they felt represented a reasonable cross-section of work performed in the unit. Activities that were really one-time TA consultancies or led to a single product, such as a manual or curriculum, were excluded from the BSOP list. The Evaluators chose eight partnership activities from the list that offered a reasonable cross-section. Our findings are summarized in Table 11. Two cases involved collaboration of the same Thai partners — the Office of the Judiciary and the Alternative Dispute Resolution Office — with a range of U.S. partners, including the American Bankruptcy Institute, Federal Judicial Center, and American Bar Association. Two other cases involved the Thai Office of the Auditor General with the U.S. General Accounting Office. But in one case there was an additional involvement with the Inspector General’s Office of U.S. Department of Transportation. Until now, the other four activities have been limited to one phase, and the same set of partner organizations. The Evaluators gathered as much information as possible--generally through multiple or group interviews with Thai participants, lengthy teleconferences with American participants, and examination of BSOP files and materials provided by Thai partner organizations. At the end of this process, we made a joint Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 29 evaluation of how well the partnership was moving forward (fifth column), and whether it was achieving significant results (sixth column). TABLE 11 A Cross-Section of Partnerships8 Subject9 Thai Partner Us Partner Budget Judgment: Partnership Judgment: Impact Mediation Center for Financial Disputes (LJR) Office of the Judiciary, Dispute Resolution Office American Bankruptcy Institute Ongoing-Rep. of US org. has strong commitment to Thailand Success￾Mediators trained, serving provincial courts. Some funding by World Bank Better Process Control School (IC) King Mongkut’s University of Technology, BIOTEC Purdue University, US Food and Drug Administration $35,000 Ongoing-Cont. contact with USFDA required, at least once annually. Thai FDA may be stand-in Near Success￾BPCS offered by Thai professors, cost recovery is within reach Capacity Building for Dispute Resolution (LJR) Office of the Judiciary, Dispute Resolution Office Federal Judicial Center, American Bar Association $121,677 Ongoing-Young Thai judges have clear reform agenda, strong ties in US Emerging Success-New methods piloted, strong interest Performance Audit Workshops— Office of Auditor General (CFG) Office of the Auditor General General Accounting Office $26,410 Ongoing-OAG will send interns and additional trainees to GAO Emerging Success￾Performance Audits used more often Strategic Framework and Operational Plan— Office of Aud. Gen. (CFG) Office of the Auditor General General Accounting Office, Office of IG/Dept. of Transportation Ongoing-Senior US auditors seem very committed. Trainees at GAO courses provide continuity Likely Success￾Framework to be completed soon Building Capacity and Professional Standards for Property Valuation (CFG) Securities and Exchange Commission; Two Valuation Associations The Appraisal Foundation $78,000 Ongoing-After draft standards completed, other activities feasible Likely Success￾Draft standards under review Upgrading Post & Telegraph Dept. Staff to Serve as National Telecom￾munications Comm. Secretariat (IC) Post and Telegraph Department (Note: PTD already has regulatory role) Federal Communication Commission $106,561 Ongoing-Three courses have facilitated personal ties between PTD staff and FCC Possible Success-Three training courses well attended. But PM’s office is eroding some authorities. Bond Market Development Thai Bond Dealing Centre Bond Market Association $30,334 Possibly Ongoing￾Future somewhat murky three crucial employees have quit TBDC Possible Success￾Brochures distributed; study tour effective 8 For more detail, see case studies in Annex E. 9 LJR = Legislative and Judicial Reform; IC = International Competitiveness; CFG = Corporate and Financial Governance. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 30 Although Table 11 offers simple summary information, several useful observations can be made:  A range of working relationships appear to be workable. Partnerships that appear to be proceeding satisfactorily include: public/public groupings such as the Office of the Auditor General and General Accounting Office; private/private groupings such as the Bond Dealing Centre and Bond Market Association; public/private groupings such as the Office of the Judiciary and American Bankruptcy Institute; and more complex groupings such as the Thai SEC (semi-public), two valuation associations (private), and The Appraisal Foundation (private).  There is no obvious evidence that high expenditure levels significantly affect success. Or, to make the point another way, it appears that partner organizations that are genuinely committed to an activity have little need to fund items that are marginal or merely aim to get hesitant parties involved: — The Better Process Control School has been launched with very modest BSOP, and partner contributions valued at $18,220. — The largest item on the list ($121,677 for Capacity Building for Dispute Resolution) includes support for action research and publication programs essential for convincing conservative judges that reform is possible. — In addition to modest budgets provided by BSOP, both Thai and U.S. partners generally make substantial in-kind contributions. — In any case, funds provided for each activity are quite modest in comparison with more typical donor-supported economic reform activities.  All partnership activities described in Table 11 are ongoing. This is not surprising, since the Evaluators originally requested a list of ongoing partnerships. More significant is the Evaluators’ sense that at least seven of these partnerships are likely to continue for at least another few years. Only the last partnership (focused on Bond Market Development) seems under threat of an early demise. In that case, problems internal to the Bond Dealing Centre have led to resignations by three key participants in partnership activities. Even so, one former TBDC official seems well placed in her new position to promote wider use of bonds as an investment instrument. A more detailed examination of three partnership cases will throw additional light on the work of the BSOP program: Mediation Center for Financial Disputes (Success): KIAsia support for judicial reform in Thailand is a complex, long-term, and heartening process. Two distinct circumstances that shaped Thailand in 1997—the financial crisis and approval of the new constitution—led to concerted efforts within the judiciary to undertake wide reaching reforms. A cadre of young judges have campaigned with diligence, using their contacts with senior judges, carefully designed pilot studies, well-drafted professional publications, and support from U.S. partner organizations to begin shifting minds and administrative systems. While there is widespread recognition that reforms will take decades, not months or years, promising steps have been taken, and KIAsia has played a crucial role by providing modest, but timely, support for several of the activities. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 31 The court reform campaign has several separate strands, two included in Table 11.10 The strand briefly reviewed here is listed first, the “mediation center for financial disputes.” In 1999, Thai court reformers in the Office of the Judiciary (OJ) and members of the American Bankruptcy Institute (ABI) with an interest in Thailand established contact. They were seeking an appropriate starting point for reforms to speed up processing of the enormous backlog of financial cases created by the 1997 crisis. They arrived at the concept of a mediation training program, to supply non-judge mediators to work with the Alternative Dispute Resolution Office in Bangkok. Eventually, this would also lead to training a cadre of mediators to work with provincial courts across the country. With an OJ/ABI partnership already established, the organizations approached KIAsia for financial and administrative support. A program of three components was agreed to and implemented: 1) Training of 100 “mediators and financial advisors” (with varying backgrounds in law and finance) was accomplished during two three-day workshops in September 2001. Many of those trainees have served the Mediation Center in Bangkok well. In 2002, settlements from 500 cases involving B505 million ($ 12.3 million) were concluded, In the first six months of 2003, 184 cases involving B674 million ($16.4 million) were concluded. This represents a significant reduction in case load for the formal court system, and a significant flow of funds into the economy. 2) A follow-on, three-day, train-the-trainers workshop was carried out in January 2002. This session was for 20 participants in the previous round who were deemed promising as trainers. Later those individuals took on the task of training hundreds of mediators who are now associated with provincial courts, in most cases as unpaid volunteers. Later rounds of training were financed under the World Bank’s ASEM project. In addition, also under ASEM auspices, a senior expert on mediation from the Federal Judicial Center in Washington took a leave of absence to join the Office of the Judiciary for a few months. He prepared a mediation handbook and organized additional training. As a result, mediation skills have been spread widely across Thailand. 3) ”Mediation mentorships” provided four employees of the Mediation Center two weeks of on-the-job training in U.S. mediation centers. Thus, while KIAsia-funded mediation training ended with these activities, as a result of excellent donor collaboration, neither the partnership nor the program has ended. Additional training sessions, using many of the same trainers, have continued under ASEM. In a particularly significant pilot program, ASEM has provided modest (B15,000) monthly honoraria to four recently-trained mediators associated with the Chiangmai Provincial Court. Preliminary data suggest this approach can have significant impact on case loads for the courts and provide more timely justice to plaintiffs and defendants willing to try a new approach. In the brief period of four months that the mediators have taken on cases, 125 cases have been successfully mediated, nearly a quarter of the financial disputes that reached the court during the period. Furthermore, there are ample indications that the Office of the Judiciary will continue their working 10 Another strand (third item in Table 11) includes support for in-court reforms including introduction of digital recording systems to record testimony, use of pre-trial conferences, introduction of continuous trials, and creation of a “fast track” for simpler cases. Still other strands include drafting of a secured transaction law and a class action law, and introduction of new procedures in administrative and bankruptcy courts. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 32 partnership with the ABI, and other U.S. institutions such as the Federal Judicial Center and American Bar Association. Better Process Control School (Near Success): FDA-approved Better Process Control Schools have long been offered by land grant universities in the U.S. to train food processors on safe methods to prepare canned goods. In recent years, overseas officials have begun attending BPCSs to gain information needed by canned food exporters in their home countries to successfully enter the U.S. market. The courses are sometimes offered overseas, for example in the West Indies and Sweden. But, overseas courses are generally offered by U.S. institutions. The idea of establishing a BPCS in Thailand, in a Thai university, grew out of the attendance of three Thai food science professors at the BPCS at Purdue University in Indiana. They discussed the idea with a Professor Emeritus at Purdue. The concept was that Purdue would offer the course the first year, and transfer needed teaching and administrative skills in the second and third year. It was decided King Mongkut’s University of Technology, Thonburi (KMUTT) would take the lead. The professors asked an official of the National Center for Genetic Engineering and Biotechnology, to approach KIAsia for funding of the second and third year courses, to ensure sufficient time for participating Thai professors and institutions to master their roles. BSOP staff recognized the value of this straightforward program, and agreed to modest funding to cover costs of visits by Purdue University and USFDA staff. KIAsia funding totaled $35,000. In addition to provision of matching in-kind resources by KMUTT and the other academic institutions, tuition payments by course participants (B19,500; $460) covered a large portion of costs. After three years, the BPCS has a firm footing in Thailand. Participants hear lectures on 16 topics, all offered by experts from Thai universities or industry, take exams, and in most cases, receive certification from the USFDA as “supervisors of thermal processing systems, acidification, and container closure evaluation programs for low￾acid and canned goods.” The presence of a certified technician on the assembly line at all times is a condition for export of low acid canned goods to the U.S. In addition, knowledge gained during the course minimizes the likelihood exporters will make processing or paperwork errors that cause consignments to be rejected or delayed. Note: Generally, a USFDA official must attend the course to write an assessment and ensure teaching content and testing procedures are sound. Bringing an FDA official each year is a major budget item, and there are explorations of an alternative approach in which officials of the Thai FDA would play the certifying role. The fourth annual BPCS in Thailand is scheduled for May, 2004. Performance Audit Workshop — Office of the Auditor General (Emerging Success): The Thai Office of the Auditor General (OAG) was given substantially expanded powers and responsibilities under the Thai Constitution of 1997 and the State Audit Act of 1999. Among the most important new responsibilities is undertaking performance audits (in addition to more traditional financial audits.) Upon the OAG’s request, KIAsia located the U.S. General Accounting Office (GAO) as an appropriate partner to provide training on performance auditing and render other assistance. After training, OAG auditors were expected to use the performance audit approach to improve financial governance and transparency in government agencies, state￾owned enterprises, and government-funded NGOs. Utilizing materials available from the International Organization of Supreme Audit Institutions, OAG and GAO collaborated in designing a two-week performance audit workshop, that was Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 33 then delivered by a retired GAO professional. The course, designed as a training-of-trainers session for 36 seasoned professionals, included exercises and casework appropriate to participants already experienced as financial auditors. A special three-day module focused on building instructional skills, so that participants could pass on their newly acquired knowledge and skills to OAG auditors across Thailand. With separate funding, the initial course was refined and offered to 40 other auditor-trainers. The training program was then delivered at OAG regional offices to 200 auditors, using the modified materials. Also with KIAsia support, the auditing standards of INTOSAI were translated into Thai for use as an OAG reference publication. OAG officials reported that since this round of training was completed in 2001, performance auditing has been well established within their organization. Performance audits are performed regularly, and their results and recommendations are frequently published on the OAG web page. They also noted that they have expanded their overseas networks, improved public relations, and facilitated information exchange. Note: The current phase of the OAG-GAO partnership includes preparation of a Strategic Framework and Operation Plan for the OAG. With assistance from BSOP and Kenan/Washington staff, a second U.S. organization, the Inspector General’s Office of the Department of Transportation (IG/DOT) has been brought into the partnership. D. EVALUATION TEAM FINDINGS — THE BSOP The BSOP program seeks organization-level impacts on Thai business support organizations, including government units and private trade and professional organizations. The expectation is that Thai partners will persist, and succeed, in proposing, supporting, and monitoring acceptance of appropriate public policies — e.g., fairer laws, more explicit regulations, more transparent procedures — or play a useful role in changing practices in crucial professions and industries. With these standards in mind, we assessed the eight BSOP cases, and found:  Of the partnership activities examined closely, none have failed. We do not rule out this possibility, but believe an occasional failure is readily justified in a program that is innovative, low-cost, and must regularly deal with unknown factors as minor as personality clashes, and as momentous as the next round of WTO negotiations.  Most of the observed partnerships are already delivering useful results or seem likely to do so in the not-too-distant future: — Hundreds of judges have been exposed through seminars and professional publications to systems for managing trials (e.g., pre-trial hearings to set parameters for timing and issues; and continuous trials to reach a verdict in days rather than months) and recording testimony that promise to make trials speedier and less expensive. — Mediators associated with the Alternative Dispute Resolution Office and Chiangmai Provincial Court are already successfully concluding scores of cases out of court. Statistics from ADRO indicate 500 cases were successfully mediated in 2002, and an additional 184 cases were settled in the first six months of 2003. In Chiangmai, 125 cases were settled out of court by\\within four months after mediators were appointed in April 2003. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 34 — The Better Process Control School is offering annual sessions, and training enough technicians to enable dozens of firms to export canned goods for the U.S. market. Furthermore, Participants are paying tuition at a level sufficient to cover the bulk of costs for the course. Not all BSOP activities are true partnerships; some are studies, single training events, TA consultancies, or deliverables — a manual or curriculum. Examples include:  Thai Bankers Handbook ($26,094),  Standards of Practice Handbook for Thai Security Analysts ,  Technical Assistance for Thailand Rating Information Services (Services of an international VE, and a US NGO),  Comparative Study on Thailand’s Savings and Investment Tax Policy, and public seminar ($56,410), and  New curriculum for three accounting courses--including CDs and seminar for instructors ($116,303). Most of these activities are potentially useful. Our main quibble is that it is misleading to wrap them in the rhetoric of “partnership.” It may be useful to give activities of this type a different label, such as “BSO Support Activities,” and consciously manage and document them in a different way. They can be readily justified if they fall within the range of “pre-partnership,” “post-partnership,” or “partnership-complementary” activities, but should be managed as one time events or short TA or training programs. It may be useful to explore the use of international VEs in this context, thus finding a “wholesale” application for their professional and managerial skills. As a major component of the AERA program, the BSO partnerships are expected to have significant, positive impacts on the Thai economy. Thus, it is useful, but not sufficient that partnerships have been established, laws passed, procedures reformed, and professional standards introduced. Once laws are passed and implemented, it is expected that they will have broad, positive impacts on business, bring economic growth, or shape the economy in other positive ways. Similarly, it should be possible to trace the macro-impacts of court reforms or a new regulatory regime for radio, TV, and IT. But concrete evidence on the macro-impacts of BSO partnerships cannot be offered here. There are at least two reasons: 1) Positive changes take considerable time. In most cases, the results of partnership-induced reforms will emerge only over several years, even decades. 2) A modern economy like Thailand’s is bewilderingly complex. Attributing an economic trend to a particular cause or set of causes requires careful research design and extensive data analysis. Thus, explicit statements regarding macro-impacts are beyond the scope of this Evaluation. Nevertheless, an examination of the potential macro-impacts of some partnership activities can be useful as a rough gauge of their importance. For example, we can state:  Court-sponsored mediation can reduce costly litigation, lower case loads, and speed up actual trials. This, in turn, will lower the cost of doing business, and increase the attractiveness of investing in Thailand. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 35  Regular Better Process Control Schools will enable canned food producers to enter the U.S. market successfully. Market expansion will increase demand for high value fruits and vegetables and create new jobs in horticulture, food processing, transportation, etc.  More effective Thai government audit services will reduce waste, fraud, and abuse, thus lowering the cost of government. Funds saved in this manner can potentially be “reinvested” in tax cuts, improved technical education, better health services, additional programs to stimulate SMEs, block grants to local governments, etc. Furthermore, regular audits of state owned enterprises may lead to public demand for further rounds of privatization.  Wider use of bonds will lessen volatility in securities markets. In addition, more extensive use of private bonds offers an important alternative source of investment funds, making investment in Thailand easier and more attractive. While these statements are hypothetical, with supporting evidence beginning to emerge in only a few cases, they make a convincing point: The BSO partnerships deal with important issues, with the potential for truly significant macro-impacts. Another way of assessing a program is to consider its cost effectiveness in delivering services. Table 12 shows that costs of the BSOP program are relatively low, about $70,000 per partnership (compared to $21,000 per firm under BAC). Since partnership activities seem likely to have impacts on entire professions or industries, or on the overall functioning of the Thai economy, the evaluators find this program to be highly cost effective. TABLE 12 Cost Effectiveness Compared — BAC and BSOP Item BAC BSOP Cumulative Costs (*) to 6-30-03 ($) 4,567,921 1,740,910 Number of Clients or Partnerships 217 25 Cost Per Client Firm or Partnership ($) 21,050 69,636 Source: KIAsia Financial Reports, AERA Records * Includes prorate administrative cost share The probability that the BSOP program can achieve financial self-sufficiency in the next several years is small. The process of actually building partnerships in the early months will almost certainly require subsidies. It may be possible over time to prevail on individuals or firms associated with the partnerships to cover “sustaining” costs through membership dues or some similar arrangement, but this stage will invariably follow one of subsidization. Nevertheless, the Evaluators believe this is a promising, low cost, non-intrusive approach to macroeconomic reform, one that deserves donor support. KIAsia managers should prepare themselves to argue convincingly for future outside support Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 36 V. BENCHMARKING — CAN KIASIA IMPLEMENT AERA-INITIATED ACTIVITIES IN A “SUSTAINABLE” MANNER? A. EVALUATION TEAM OBSERVATIONS ON BENCHMARK AREAS Organization KIAsia was established in 1996, during the golden years of economic boom in Asia and particularly Thailand. USAID had exited Thailand and used remaining resources to assist in creating and endowing the organization to carry on US-Thai cooperation. During 1997, the economic crisis hit Thailand and spread throughout the region. As USAID reacted to the Asian crisis and began designing the AERA program for Thailand, the Philippines and Indonesia in 1998, only Thailand was without a USAID mission. Subsequently, in 1999, three short years after KIAsia's creation, it was chosen to implement AERA in Thailand. KIAsia's previously normal institutional growth course was deflected as the urgency of project implementation came to the forefront. With rapid growth, the number of employees expanded to over 70. However, planning was difficult, as the term of the original September 1999 Cooperative Agreement for $5.1 million was only one year. A year later the term was extended to October 2002 and $5 million was added. Then in May 2002, as the additional two year term neared completion, a third major funding tranche added another $4.5 million, and the program was extended another three years to October 2005. Now as AERA is about to begin its final two years, USAID has returned physically to Thailand in the form of regional Mission. The sudden arrival and unpredictable extensions of the AERA program have certainly shaped KIAsia, as normal growth would never have done. The return of USAID has compounded the element of uncertainty. In the absence of a Mission there was a reasonable expectation KIAsia would be the preferred implementer of USAID-funded activities in Thailand. With the renewed presence of a Mission, the rationale for preferring KIAsia is gone. Additionally, the Mission is in the process of determining its priorities, so there are no assurances about how KIAsia might fit into the picture. The organization experienced explosive growth, both financially (See Table 15 below) and in human resources. Turnover has been high, especially recently. Experience gained at KIAsia increases employees appeal to outside organizations. The improved Thai economy affords staff ever more outside opportunities. Despite rapid turnover, overall the Evaluators believe that KIAsia staff are well qualified and quite capable. Furthermore, key persons in management and on the KIAsia Board are well connected with the governments and the business community in Thailand. AERA Program Emphasis Although there have been several activities under AERA, the principal ones are BTP, BSOP, and BAC. Activities under the first two programs operate on a wholesale rather than a retail (one￾firm-at-a-time) basis. BAC’s retail approach is a function of the initial program design, to which all parties agreed, but is a costly approach to assisting SMEs. These activities, mostly two to four week consultancies to individual firms, have consumed more than any other AERA program, about 46% of cumulative expenditure (including overhead allocation). Serious efforts Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 37 to address issues of sustainability have been initiated in FY'03 through increases in fees and actively seeking Thai Government contracts. This has resulted in BAC generating enough income in 2003 to cover about 28% of its direct costs. KIAsia managers sometimes justify the existence of the BAC, and heavy expenditures on its activities, by pointing out that the organization pioneered provision of business consulting services to SMEs in Thailand. This appears to be true. They then go on to assert that the Invigorating Thai Business (ITB) program, and similar programs of the Thai Government, are patterned on BAC. While the Team did not confirm this assertion, it recognizes that both BAC and ITB are set up to provide highly subsidized services to SMEs. While it is beyond the scope of this evaluation to comment on the appropriateness of Thai government programs, it seems clear to the evaluators that a program that provides assistance at a per-firm cost well in excess of $20,000 is no longer a sound investment for the U.S. government. Endowment Benchmarking the current financial situation, as well as the recent past, will assist in determining KIAsia's financial capacity to undertake future activities. A review of current sources of funding and an inventory of endowment assets is in order. The Endowment was created by contributions from DTEC, the Kenan Trust and Kenan Institute of Private Enterprise, and USAID to establish KIAsia as a financially viable institution for promoting US-Thai development linkages. The Endowment consists of three separate funds as illustrated in the Table 13. The essential point is that the endowment has not contributed as expected toward building and maintaining KIAsia, due to the financial crisis in Thailand and the severe drop in US financial markets. Each component of the Endowment is controlled separately, with the USAID contribution having fared most successfully. Although the Friends of Kenan and USAID components were both professionally managed by J P Morgan and invested in cash equivalents, equities, and fixed income, performance results were dramatically different. KIAsia assumed full ownership of the USAID component on October 1, 2001. This account has grown from the original $3.5 million to $4.7 million. The Friends of Kenan Endowment is maintained in the US by Friends of Kenan and legally must remain in their control. This account suffered most from the market decline and is now worth $2.7 million. Friends of Kenan pays a portion of the salaries of the Executive Director and Associate Executive Director from this Endowment. The DTEC Endowment component is in a savings account and fully in the control of DTEC. The DTEC contribution of 129 million Baht was originally worth $3.5 million. DTEC periodically transfers interest earnings from the account to KIAsia. DTEC's conservative investment philosophy has preserved the nominal Baht value of the account, but recently generated very modest returns. Due to exchange rate changes the value is now $3.1 million. In FY'00, FY'01 and FY'02 respectively, the annual returns generated on this account were 3.07%, 1.66% and 0.25%. The Evaluation Team understands that in FY'03 the account has been generating 0.25%. As can be seen from Table 13, the recent value of the KIAsia Endowment is approximately equal to its original value, considerably less if we consider the impact of inflation. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 38 TABLE 13 KIAsia Endowment Status Component Original Contribution ($) Value 6/30/03 DTEC 3,500,000 3,146,341 Friends Of Kenan 3,500,000 2,708,269 USAID 3,500,000 4,764,036 Total 10,500,000 10,618,646 Donation Income: Table 14 indicates sources of donation income. (As FY'03 is not closed, the '03 column is provided for input of data by KIAsia when the information becomes available). Donation income might not always be expended in the year of contribution; therefore; there can be large differences between income and expenditure, primarily with non-USAID funds. (See discussion on AERA and non-AERA expenditures below.) Despite the differences in income and expenditure, it should be recognized that non-USAID funds represents a very small portion of KIAsia's total donation income. TABLE 14 KIAsia Donation Income by Fiscal Year Donation Source 2003 2002 2001 2000 ACT 232 140 409 DTEC 7 48 101 FTTC 13 48 121 Other 31 0 30 Total Non￾USAID 283 236 661 USAID 2,847 * 3,204 2695 1991 Total 3487 2931 2652 Non-USAID % 8% 8% 25% Earned Income There are two AERA programs that generate earned income — BAC and the Labor Standards Advisory Service (LSAS). During the current fiscal year, BAC has generated $163,000 through consultancies to the public sector, or 28% of the unit’s direct costs through June 30. Opportunities to increase earnings through selling services to the private sector seem remote at this time. BAC customers interviewed by the Team were unwilling to pay full costs of consultations they received. Even in cases where an earlier consultancy had produced dramatic and valuable results, there was no apparent willingness to pay full costs of future consultancies. Business owners appear to be willing to consider only the highly subsidized initiatives such as ITB, where they pay only 10% of costs. BAC is not a donor and does not have the resources to subsidize consultancies to individual businesses. Thus, KIAsia managers have taken the expedient decision of seeking contracts under the ITB program, allowing the Thai government to pay the high subsidies and accepting consulting fees of B10,000 per day as a source of BAC revenue. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 39 KIAsia management has decided that LSAS no longer needs grant resources for operations. This is not a problem for the program at its present scale, as it has essentially reached a cost recovery basis. In fact, this situation prompted the management action. LSAS provides a useful and valuable service to companies on a demand driven basis. Training and TA allow client firms to adjust operations to bring them into compliance with internationally accepted labor standards. As a result they can be more competitive internationally and are better equipped to provide products for US markets. LSAS FY'03 expenditures through June were about 10% of BAC's and are estimated at $80,000 for the year. While LSAS is the only component of AERA presently capable of full cost recovery, at $80,000, LSAS income would cover only about 3% of AERA programs’ annual expenditure. Expenditures and Major Programs Table 15 illustrates the impact of AERA on KIAsia. AERA activity represented 70-80% of KIAsia annual expenditure over the first three years of the program. Expenditure is only one means of measuring program emphasis. However, it is objective and easily verifiable. If FY'03 expenditures are projected based on levels achieved through June 30, AERA programs will be about $3.2 million (and non-AERA activities about $770,000). The established pattern is for annual AERA expenditures to be three or four times the non-AERA expenditure. With approximately $5 million remaining for AERA programs, AERA funding for FY’04 and FY’05 will be at approximately the 2001 level. With the exception of 2002, non-AERA expenditure activity has largely remained at the same level over the past four years. There have been heavy demands on KIAsia time and resources from the AERA programs. There has been consistent pressure to produce results, and certainly in the first two years results were dramatic: BAC, BTP and BSOP substantially exceeded established targets. But, emphasis on this very commendable performance may have diverted attention from focusing on sustainability and expansion of non-AERA activities. TABLE 15 KIAsia Expenditures by Fiscal Year 2003 (as of June 30) 2002 2001 2000 Expenditure $'000 % $ ' 000 % $ ' 000 % $ ' 000 % AERA 2,353 80% 3,547 79% 2,455 77% 1,738 69% Non-AERA 580 20% 933 21% 734 23% 775 31% Total 2,933 100% 4,480 100% 3,189 100% 2,513 100% Source: KIAsia Financial Reports, year-end accounting adjustments are excluded. Note: KIAsia fiscal year ends on September 30. Finance and Budget The Team was able to access financial information readily and had unfettered access to any and all information requested of KIAsia. The financial system is designed primarily for reporting and delivers just the kind of information donors require. The AERA monthly financial statements provide balance sheet, income statement, detailed expenditures for each program by month and year-to-date, and budget-to-actual reports by program on a monthly and year-to-date basis. It is Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 40 noted that for FY'03 through June 30, (when expenditures might reasonably be expected to approximate 75%) overall expenditures are far below budget at 46%. The budgeting template has each month's budget equal to 1/12 of the expected annual expenditure. The Evaluators anticipate that expenditures for this FY will be about 40% below budget. This indicates that the budget in its present form is not a useful management tool. While the system does well in reporting, some revisions in format could enhance its value for management use. For example, the Team believes that our calculations of cost-per-partnership for BSOP and cost￾per-client in BAC were particularly revealing for KIAsia management were particularly revealing, and that regular tracking and discussion of these figures might be a valuable management tool. B. ARE CURRENT ORGANIZATION AND POLICIES APPROPRIATE FOR SUSTAINABILITY? SUCCESS? The Evaluators found that KIAsia is urgently looking for means to ensure sustainability. As we have suggested previously, the timing and scale of AERA, the Cooperative Agreement, and subsequent modifications as described in section V.A. explain why there had not been a focus on sustainability from the beginning of AERA. But this concern has been given increasing attention over the past two years. Notably, The Board of Trustees approved three strategies in January 2002 to guide KIAsia, focused on: Sustainability, Partnership Strategy, and Regional programs. These decisions apparently came at the urging of the present KIAsia President, then newly appointed. The Sustainability Strategy recognizes that sustainability of KIAsia and the services it provides depends on several factors: a good relationship with donors and customers; demonstrating a capacity to achieve increased returns from grants and provide higher levels of matching funds; efficient operations; a steady transition from subsidies toward full fee-for-service for some programs; generating overhead from project management; fund raising for specific projects; and creation, ownership, and commercialization of intellectual property. Within the past year, additional attention has been focused on sustainability. KIAsia is increasingly assuming a regional role, which positions it as a potential partner with DTEC and USAID, as each of those organizations looks toward supporting development in the region. KIAsia is presently working to develop a strategy and budget for the final two years of AERA that will maximize the impact of activities on clients and contribute to program sustainability. The present focus on sustainability is an encouraging sign. Presently, key KIAsia employees from the President on down, understand the importance of sustainability and are committed to pursuing it. C. WHAT IS THE POTENTIAL FOR FUNDING FROM OTHER SOURCES? Before looking to other sources, it is worthwhile for KIAsia to work with DTEC on investing its endowment component in higher return accounts. Twelve-month commercial bank time deposit rates during '00, '01 and '02 averaged 3.93%, 3.17% and 2.81% respectively. The Team estimates that an additional $150,000 could have been generated over the three-year period at those prevailing rates. An additional three Thai-US BSO Partnerships could easily be in place with that amount of funding. It is easy, of course, to make this analysis with hindsight, but the comparison is offered to indicate one avenue that can be explored to enhance KIAsia income. The Evaluators Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 41 understand KIAsia and DTEC have already discussed placing the funds in accounts that will provide higher returns. DTEC has asked KIAsia to draft a letter to facilitate the change. DTEC's willingness to proceed with this change, or at least to consider it, highlights their close partnership with KIAsia. Over the past several years there has been serious discussion, and some pilot programs, focused on providing economic development assistance to other countries in the region. DTEC will apparently be the main implementing organization. KIAsia has already identified collaboration with DTEC in the region as a potential means of leveraging present institutional strengths and obtaining funds to sustain regional programs. And, of course, the USAID Regional Mission is a possible future partner for regional development efforts. As USAID develops its plans, KIAsia should be ready to assist. Combining these two approaches, there is the possibility of an innovative tripartite partnership among DTEC, USAID and KIAsia in providing development assistance to the region. The World Bank has indicated that its financial sector team will be phasing out and there is a need to insure monitoring continues to take place. KIAsia would be a very good candidate for performing this work, and a monitoring contract could provide a significant income source to continue the more promising AERA programs. Corporate contributions are now received primarily from American companies. If approached in a strategic manner, these firms should continue to be a steadily increasing source of funds. But the Evaluators also believe it is worthwhile to explore long-term opportunities to solicit collaboration and financial support from Thai companies (and those of other nationalities) in the region. KIAsia should be prepared for a long-term campaign to build up this source of funding, and need not be modest in undertaking it. AERA has produced ample success stories, and KIAsia should make the point that they cannot be repeated without adequate funding. D. EVALUATION TEAM FINDINGS REGARDING KIASIA MANAGEMENT The challenge is clear. If KIAsia is to keep donation income at current levels after 2005, it must re-source approximately 80-90% of its donation income. In the judgment of the Evaluators, prospects for obtaining sufficient income to sustain AERA activities beyond 2005 at current levels are relatively small. Thus, an essential question is: Do all activities need to be sustained? The following paragraphs offer our initial thoughts on this question, while the discussion is continued in sections VI and VII, where we offer our recommendations to KIAsia and USAID: BSOP is molded in the true spirit of KIAsia. Its approach to partnership has great potential. It should be supported in whatever fashion is required to continue to meet demand from reform￾oriented organizations and government units on Thai side, and the model should be energetically tested elsewhere in the region should the opportunity arise. Accordingly, KIAsia is encouraged to make BSOP its flagship economic development program, and USAID is encouraged to consider support for a regional BSOP program. Regarding BAC, the Team is encouraged that the unit is generating significant revenue, covering 28% of direct costs in FY'03. Nonetheless we believe a wholesale approach to SME assistance is more cost effective. Overall, assisting 217 firms out of tens of thousands of SMEs in Thailand, has made a limited contribution to economic recovery. From an investment viewpoint for both USAID and KIAsia, a wholesale approach is more appropriate at this time. Accordingly, from a Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 42 program support perspective, the Team does not believe budgetary support for a retail approach is warranted beyond the end of FY' 04. On the other hand, there is no reason why KIAsia shouldn’t encourage the BAC to identify strategies for generating income sufficient to maintain the unit and its SME services. In 2000, a study of micro and small enterprises in Thailand found that: "While a quarter of MSEs in Thailand purchase accountancy services and almost a fifth purchase exhibition services, most MSEs do not purchase anything but essential or emergency services such as communication or equipment repair.11" In short, there appears to be insufficient demand, on a cost-recovery basis, for the type of consulting BAC has done. Assuming the Government did pattern ITB after BAC, we can question if it is an appropriate model. Regardless of potential impacts on individual clients, heavy subsidization by BAC in the past, and by ITB at present, distorts the market for sustainable consultancy services. On the other hand, selling consulting services via the Thai government’s subsidized programs seems to be the most promising opportunity BAC currently has to achieve financial self-sufficiency. Thus, the Team would encourage them to test this approach in the medium term, while seeking other, market-based approaches in the long term. BTP has performed its main task admirably, and it is appropriate that the program is ending now. However, modest follow-on activities might be appropriate in distance education for bank employees, possibly strengthening TIBFA, and in the area of microfinance. (Some of these activities might best be accomplished as BSOP-type partnerships.) LSAP, although relatively small, should be continued. It is providing a vital service on a cost recovery basis, and may be able to generate additional income for KIAsia. Importantly it is improving opportunities for trade by Thailand and other countries in the region with the U.S. LSAP is a real success story for KIAsia. Finally, in the opinion of the Evaluators, the cluster competitiveness work, although still in an experimental stage, shows considerable promise. In the coming year, KIAsia and USAID may wish to undertake a detailed assessment of its likely effectiveness in Thailand and in the region. VI. RECOMMENDATIONS TO KIASIA BTP The Evaluation Team recognizes that the BTP is in its final months, but encourages KIAsia managers to cautiously consider a few follow-on activities, to be managed in other units:  Consider further strengthening of capacity to deliver multi-bank training programs, perhaps by extending the current TIBFA/NTU/BAAC program in the form of a BSO partnership. This activity might also involve U.S. bank training institutes.  Strive to finalize ongoing BAC Research on microfinance soon, to open the possibility of programming AERA funds toward this activity. 11 The Hidden MSE Service Sector: Research Into Commercial Business Development Services Provision to MSEs in Viet Nam and Thailand, Working Paper No. 5, International Labour Organization, April, 2000. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 43  Consider follow-on collaboration on microfinance policy, possibly in the form of a BSO partnership. This might involve a three-way partnership, including regional microfinance organizations, as well as those in the U.S. BAC First, we recommend that KIAsia managers coach the professional staff of the BAC in developing an explicit Financial Sustainability Framework. This effort may usefully include the following elements, among others:  Request BAC staff to develop a “post-consultancy services plan” to directly address the need to achieve more effective follow-up of SME consultancies. The plan might mandate routine use of: semi-structured follow-up interviews, routine use of internal “lessons learned” memos and seminars to reflect findings from interviews, and developing explicit marketing and communication strategies for former clients.  Convene a BAC Retreat to initiate a “financial sustainability in 2004 plan.” This would include an explicit examination of recent trends in costs and income, and initial brainstorming on how to reduce costs and increase income.  Form a “new BAC products” group to brainstorm about new and previously used approaches such as “micro consultancies” and fee-based seminars.12  To build the “human sustainability” needed to support the BAC’s financial sustainability, prepare a training plan--including on-the-job-training, internships, peer training, etc.--for every employee who commits to staying at the BAC at least until the end of FY’05. KIAsia managers might also consider “strategic expansion” of the BAC. On one hand, adding other KIAsia units may be a useful contribution toward the BAC’s financial sustainability. On the other hand, adding units may facilitate significant “cross-fertilization” between programs. At least two units might usefully be added to the BAC:  The Labor Standards group might usefully be added to the BAC. They are on the verge of becoming a profit center, and could also add an important element of cross-fertilization. E.g.: — Current BAC staff could be trained and added to the roster of labor standards instructors/inspectors, serving this additional role when demand rises to an unusually high level. — BAC client firms (especially those that are export-oriented) may be candidates for labor standards training or inspections. Furthermore, this training might be added to “normal” BAC consultancies under ITB or similar programs.  Consider adding the Cluster Competitiveness unit to the BAC. While this group is unlikely to achieve financial self-sufficiency, they are a plausible candidate for attracting 12 “Incubator” services might be considered as a new product, as long as they promise reasonable assurance of cost recovery or profit making opportunities. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 44 outside funds, and could serve as an “innovation center” in the BAC. Cross-fertilization opportunities include: — Cluster competitiveness staff could offer general seminars to groups of former and current BAC client firms on such topics as “influencing government policies.” — The BAC may be able to offer services to members of “mature” clusters on a cost-recovery basis — e.g., group advisory services on “procuring appropriate computer software for handicraft exporters.” KIAsia managers need to focus on putting appropriate incentives in place as the institution moves from a “project-dominated” status to that of a “sustainable foundation.” It will be important to move current BAC activities toward financial self-sufficiency in a timely manner, while ensuring that needed staff remain with the unit. Recommended steps include:  Provide operational funding for FY’04 at 80-85% of the FY’03 level, while allowing the BAC to retain income earned during the year for expenditure in FY’05.  Provide no AERA or other donor funds to the BAC in FY’05. Allow the use of funds from FY’04 earnings, but only on a matching basis, so that at least 60% of operating costs are provided from current year income.  Build incentive structures to reward BAC staff who remain and play a positive role in achieving sustainability--e.g., innovation awards, performance bonuses, and longevity￾based severance packages. BSOP KIAsia managers should make the BSOP, KIAsia’s flagship program in economic growth:  Continue support, using AERA funding, to BSOP operations and programs at the FY’03 level in FY’04 and FY’05.  Seek international and Thai corporate support to establish new partnerships and sustain ongoing ones. In some cases, this approach may be sector specific — e.g., a Thai real estate developer sponsors part of the work of raising professional standards in property valuation. Alternatively, generic support can be useful — e.g., a “challenge fund” established by an American corporation to be used in three-way matches with donor and partner resources. While corporate funds will likely be modest in the beginning, they can be a convincing matching resource when seeking support from organizations like USAID. This funding source can be expected to grow over time, if corporate sponsors observe positive results from their contributions, and can find satisfactory ways to take public credit.  Build human and administrative resources to become a respected “regional player,” by experimenting with support of local/US partnerships in Cambodia, Vietnam, and China. In addition, experiment with a few local/Thai/US partnerships in the region — e.g., by Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 45 expanding the work on property valuation. Seek USAID concurrence for expansion to the region, as appropriate.  Explore opportunities in Thailand to develop “three-sided” partnerships that deliberately involve other organizations in the region. E.g. A regional association of jurists can promote new methods to improve courtroom efficiency, building on Thai and U.S. experience. As noted above, detailed data on impacts of BSO partnerships are often difficult to collect. Furthermore, the processes of creating and nurturing partnerships are elusive and hard to capture in a few sentences or summary statistics. Because BSOP activities are unconventional and somewhat difficult to understand, and concrete results have not yet been well documented, there is a danger funding organizations will hesitate to provide support. Thus, if KIAsia desires such support, managers will need to take aggressive steps to demonstrate impacts and document processes for selected partnerships. KIAsia should consider using AERA funds for this purpose, requesting USAID concurrence as appropriate. BSOP staff can:  Engage a distinguished Thai research organization to undertake detailed research and documentation on impacts of a sub-set of the BSO partnerships. E.g.: — Researchers can select recent graduates of the Better Process Control School and literally “follow them back to the factory,” using unstructured interviews and observation to record how production systems are changed and exports are expanded. — They can visit provincial courts, and borrowing the approach already used at the Alternative Dispute Resolution Office, record the number of cases settled and total size of financial agreements reached as a result of the work of newly trained and appointed mediators. They can also undertake longitudinal studies to determine if total cases are being reduced and the time from filing to verdict is being shortened. — They can observe the workload of units within the Post and Telegraph Department, seeking concrete examples of employees applying new skills learned from training courses offered by the FCC.  Engage the same institution, or another as appropriate, to document the work of building and nurturing BSO partnerships. E.g.: — Researchers can document, through observation and interviews, the processes used in building and nurturing partnerships. As appropriate, they may be able to construct a “typology of interventions.” — If possible, the data collected can be used to produce a “how to” manual. KIAsia Management Our recommendations regarding KIAsia management focus strongly on financial issues. We do not emphasize an over-simple “increase income at all costs” approach. But we do recommend Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 46 that KIAsia staff devote significant time and creative energy to making and implementing plans to ensure the long-term sustainability of KIAsia and those AERA programs with greatest potential for Thailand and the region. Thus, we recommend the following analytical and planning activities:  Make finalizing the plans and budget required for achieving maximum financial sustainability during the remaining two years of AERA KIAsia’s number one priority.  Develop concrete plans for KIAsia’s "life after AERA." This may involve scaling operations back significantly from levels established under the large AERA grant.  Revise budgeting process and software to extract useful management information. Some elements include: income information for each project, and unit cost information. A financial analyst is needed, and can perhaps be drawn from among current employees. KIAsia managers should focus on optimizing grant, donated, and endowed income:  Regularly monitor the return on DTEC Endowment in an effort to maximize income while maintaining safety. Maintain an open and cooperative partnership stance with DTEC. In the future, KIAsia might discuss placement of the account in Thai Government notes or bonds if the yield differential is attractive.  Consider bringing an international VE for a 2-3 week consultancy on fund raising. Utilize all possible opportunities to increase the size of present endowment.  Establish an annual campaign to solicit donations from Thai and international corporations for attractive programs such as BSOP and cluster competitiveness, and for direct contributions to the endowment. KIAsia managers also need to take concrete steps in the near future to secure specific income earning activities. These should, of course, be consistent with KIAsia’s institutional goals:  Explore ways to expand LSAS' success. The cutoff of grant support was proper to "tweak" the program toward sustainability in its present scale. However, there may be an opportunity (after careful analysis) to invest funds in "scaling up" this very useful program especially considering the strong potential for regional fee-based services. This is a clear example of the concept championed by KIAsia's President—grants leading the way toward earning fee income from clients.  Approach the World Bank about contracting to meet their monitoring needs after the financial sector team phases out.  Seriously explore the feasibility of establishing “corporate social responsibility services” as a major income source. (e.g., Establish for-profit affiliate, or encourage corporate clients to contribute to the endowment.) At the same time, KIAsia should continue to build on its favorable public image: Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 47  Consciously focus on retaining “good partner” image KIAsia has gained in Bangkok by undertaking frequent liaison meetings, joint programs, etc.  Consider leveraging the TVCS by arranging for volunteers to serve on a regional basis.  Target resources in remaining two years toward greater impact areas, with limits on retail approaches. Wholesale approaches are encouraged. VII. RECOMMENDATIONS TO USAID BTP In a few months, the Bank Training Program will be successfully concluded, and there are no compelling reasons for USAID to support this kind of program in Thailand. However, there may be some lessons learned that would be valuable in supporting bank reform programs elsewhere in the region, e.g., in Vietnam or China.  Microfinance best practice awareness appears to be an important area still needing attention. Some support for awareness building is important for policy makers. This could be accomplished through in-country seminars as well as select visits to leading microfinance programs in the region. Consider facilitating seminars and study visits for key persons using BTP.  Consider engaging a Thai institution to undertake a detailed study of implementation and impacts in Thailand, if similar programming to be undertaken in region. BAC Our major recommendation to USAID regarding the BAC is to provide KIAsia needed support in moving this unit steadily toward financial self-sufficiency. Appropriate actions may include:  Inform KIAsia that USAID will not provide any post-AERA funding for BAC programs that support individual firms.  Encourage KIAsia managers to end operational funding for BAC services to individual firms by end of FY’04. At the same time, USAID may wish to consider support to activities that are innovative and do not share the BAC’s current emphasis on one-firm-at-a-time services. For example:  In six-eight months, examine the potential of cluster competitiveness work by attending meetings and making field visits.  Under the right circumstances--e.g., a creative response to a regional RFP/RFA, consider funding a follow-on KIAsia program on cluster competitiveness, so long as there is a convincing prospect of effective work in neighboring countries. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) 48 BSOP KIAsia’s partnership program is low cost, non-intrusive, and seems to be enjoying real success in Thailand. It may prove to be a useful programming approach across the region. Thus:  Consider providing modest additional support (say $500,000-800,000 annually) to BSOP for Thailand activities after FY’05. In all likelihood, this work would be carried out as part of a regional Partnership program that might be funded at twice or three times this level.  Consider an RFA/RFP for partnership building as a regional program in economic reform. KIASIA MANAGEMENT  If KIAsia managers approach USAID for permission to use AERA funds for a variety of analytical and planning tasks recommended above, give their proposals serious attention. Aera Programs of KIAsia .doc/EvaIQC-36 The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) ANNEXES Annex A. Scope of Work.............................................................................................. A-1 Annex B. Interview Schedule....................................................................................... B-1 Annex C. Template for BAC Case Studies .................................................................. C-1 Annex D. Interview Guides .......................................................................................... D-1 Annex E. Case Studies ................................................................................................. E-1 The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) A-1 ANNEX A SCOPE OF WORK Accelerating Economic Recovery For Asia (AERA): Evaluation And Sustainability Assessment Of The Kenan Institute Asia’s AERA Programs DELIVERY ORDER STATEMENT OF WORK I. Background A. General The unexpected financial and social crisis in Asia in 1997 created an unprecedented focus on the weaknesses of the Asian economies and called into question the existing architecture of the international financial system. Addressing the root causes of the crisis became increasingly imperative after other countries, including the United States, felt the reverberations of the Asian crisis in their own economies. Analyses of the crisis, along with field inquiries, indicated that among the affected countries, Indonesia, Thailand and the Philippines demonstrated the highest levels of need for USAID assistance. In response, USAID designed the "Accelerating Economic Recovery in Asia” (AERA) initiative to help the region, and particularly these three countries, recover from the financial crisis. Launched in 1999, AERA helps these countries address key weaknesses in their economic, governance and social systems. These weaknesses must be addressed in order to restore investor and bank confidence, and help revive the credit and investment flows needed to revitalize business activity that will lead to economic recovery. Recognizing that an effective response must address both the underlying causes of the crisis and resultant social impacts, as well as take steps to ensure economic growth, the initiative has pursued its objective by:  Revitalizing and reforming bank and business activity;  Improving procedures, transparency and accountability in banks, businesses and governments; and  Establishing more effective mechanisms that alleviate the social tensions and adverse impacts associated with economic adjustments. AERA activities are closely linked and coordinated with World Bank and Asian Development Bank crisis response efforts. The AERA program brings technical assistance and training that complement ongoing structural adjustment loans being funded by the International Monetary Fund or the multilateral development banks. B. Project Funding Since 1999, USAID has provided $18 million in grant support to Kenan Institute Asia (KIAsia) to implement the AERA program in Thailand. USAID funding concluded with the FY2001 Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 bAn Evaluation (July – September 2003) A-2 obligation, although the life of the grant has been extended to 2005 to complete implementation of program activities. C. Project Implementation The AERA program initiated activities in Indonesia and the Philippines in July 1999 and in Thailand in October 1999. As this evaluation concerns only the AERA program activities implemented in Thailand by USAID’s principal partner, Kenan Institute Asia (KIAsia), this section limits its discussion to that component of the AERA program. In order to continue the successful business and institutional cooperation begun under the U.S.- Thai Development Partnership, USAID helped to establish a financially viable and self￾sustaining institution--the Kenan Institute Asia. Established in August 1996 through an endowment from three contributors, USAID, the Royal Thai Government and the Kenan Charitable Trust, KIAsia promotes U.S. and Thai linkages and expanded Thai and U.S. development cooperation. Under AERA, KIAsia has been USAID’s partner in promoting reform and accelerating economic recovery in three specific ways. Each of these approaches was designed to respond quickly to a specific weakness in the Thai economy:  Strengthening the enabling environment for small and medium enterprises by providing training to Thai banks in risk assessment and SME lending practices, among other topics;  Developing organizational partnerships between Thai and US business support organizations to strengthen the regulatory and competitive environment, business standards, and good governance practices of the Thai private sector; and  Improving the competitiveness of SMEs through firm-level assistance that enhances management capabilities and technical skills. Strengthening the Enabling Environment via Bank Training The Bank of Thailand began a reform program to strengthen the banking system and improve bank regulations following the financial crisis. Banks also recognized their weaknesses and accepted that they needed to adopt important changes in order to survive. USAID provided funding for advisory services to the Bank of Thailand and for the Bank Training Program in order to help these institutions adopt needed reforms in the areas of credit management, risk management, training for SME customers in the preparation of business plans, and improved internal control and governance. In 2001, the Bank Training Program added a new component that focused on improving the capacity of government banks to carry out community lending and micro-credit. Since 1999, the BTP has provided eighteen (18) customized training programs for Thai commercial and government-owned banks. Business Support Organizations The Business Support Organization Partnership (BSOP) program supports financial and business governance support organizations that play a role in ongoing reform and sustainable economic growth. The program matches those Thai organizations with US organizations that have similar operations and possess “assets” such as experience, technology, training materials, information Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 bAn Evaluation (July – September 2003) A-3 systems, and practices that could be of benefit to their Thai partners. Assisted Thai organizations include the Thai Bond Dealing Center and the Government Pension Fund. Organizations include several independent agencies tasked with promoting a more transparent and fair business environment such as the Mediation Center for Financial Disputes, the Secured Transactions Registration Office, the Credit Bureau Supervisory Agency, the Court of Justice, and the Administrative Court. Since 1999, the BSOP program has catalyzed over twenty-five (25) organizational partnerships. Improving SME Competitiveness through Firm-Level Assistance The USAID-funded Business Advisory Center (BAC) works with Thai SMEs to promote competitiveness by enhancing their management capabilities and technical skills, as well as to access sources of finance. US and Thai business consultants work with clients to conduct problem analyses and develop scopes of work to address issues faced by the SMEs. BAC staff often directly provides the expertise called for under the scope of work. However, BAC also taps the volunteer expertise of the Thai Volunteer Consulting Services, the International Executive Service Corps, the Citizens Democracy Corps, and ACDI/VOCA. Focus areas include international marketing, plant process improvement, computerization of production, environmental management, quality control, e-commerce, human resource management and labor standards, and business strategy development. During the first two years, the BAC completed 126 consulting projects and is managing another 110 on-going projects. Combined the BAC has reached 201 SMEs, providing training to 1,300 SME managers. Two smaller and more recently initiated projects have also been implemented: The IT and Education Project and the Cluster Competitiveness Project. II. Evaluation Scope Article I — Title Evaluation and sustainability assessment of the AERA projects implemented by the Kenan Institute Asia (KIAsia) under USAID’s Accelerating Economic Recovery in Asia Program (AERA). Article II — Objective This evaluation has two objectives: 1) to assess the impact to date of the AERA projects in Thailand implemented by the Kenan Institute Asia (KIAsia), in developing regulatory, banking and SME management capacity to support economic recovery and reform; 2) to assess the sustainability, given current market conditions in Thailand, of each of the AERA projects implemented by KIAsia once the USAID grant ends. These findings will assist USAID and KIAsia in planning future activities. It should be noted in regard to the question of the sustainability of the AERA projects, that the AERA program was designed to respond quickly and effectively to the economic and governance conditions wrought by the financial crisis in Thailand. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 bAn Evaluation (July – September 2003) A-4 Article III — Statement of Work The contractor shall provide a team consisting of one senior level Institutional Analyst and one senior level Private Sector Specialist to conduct a final evaluation of the AERA/Thailand projects implemented by KIAsia, and assess which of the AERA activities KIAsia might be able to implement in a sustainable manner beyond the term of the grant, ending in FY ’05, given current operating policies and the level of demand for current program services. The team shall develop and adopt a participatory evaluation approach that elicits and analyzes information, and provides conclusions, recommendations, and lessons learned on the issues below, with the participation of appropriate KIAsia management and staff. The evaluation team shall draw on project documentation and reports as well as structured interviews with project participants and partners. Impact: What has been the quantitative and qualitative impact of USAID-funded activities in each of the three areas where KIAsia is working? Is there evidence of any impact beyond those organizations directly supported? How has USAID-funded support to KIAsia’s partners in Thailand made a difference in the development of the SME and banking sectors, and in the regulatory and governance sectors? What effect did the approach adopted by KIAsia have on the impact achieved? What specific impact did KIAsia’s USAID-funded activities have on the ability of the Government of Thailand to strengthen the regulatory environment for economic recovery of SME’s, improved bank assessment of risk, and Thailand’s competitiveness? Has KIAsia direct assistance to SMEs made a difference in those enterprises’ abilities to compete? Assess the effectiveness of the projects in accordance with the success indicators set by USAID (see Appendix A). Efficiency: Are the results being obtained by KIAsia being produced at an acceptable cost compared with alternative approaches to accomplishing the same objectives? The evaluation team should have solid understanding of alternative AID approaches in mission and non-mission presence countries. Sustainability: As currently implemented, are (a) KIAsia’s AERA programs likely to engender sustainable development impacts after USAID funding have stopped? Will the organizations supported under AERA, whether business support organizations or independent agencies, have the capacity to promote economic recovery and reform when USAID funding has stopped? And, are (b) KIAsia’s AERA programs sustainable without the USAID grant support? Would market conditions support the consulting, training and partnering services that KIAsia has offered as part of the AERA program? If not, why not. If so, could KIAsia ensure that the AERA programs become and remain sustainable. The elements that are to be examined for the evaluation and assessment of programmatic sustainability must include, but are not limited to, the following tasks: a. Assess KIAsia’s organization structure and procedures related to the AERA programs’ past and future. This benchmarking exercise shall evaluate institutional issues such as administration, demand for services, finance and budgets, human resource capacities, general management and governance. The results of the appraisal stage are expected to provide a foundation of general information with sufficient levels of detail from which a Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 bAn Evaluation (July – September 2003) A-5 strategy for programmatic sustainability may be outlined. Illustratively the components to be reviewed during the benchmarking exercise include:  Administration: KIAsia’s organizational and administrative structures including chain￾of-command, personnel responsibilities, approval processes, promotional and marketing abilities, facilities appropriateness and administrative efficiencies.  Demand for Services: A key issue for developing a cogent strategy for sustainability involves understanding the basis of and demand for the services currently provided. Therefore, an assessment of the nature and scope of services offered shall be developed as well as analyses of KIAsia’s clients and donors.  Finance and Budgets: At the core of programmatic longevity is an organization’s financial management. KIAsia’s current budget/financial management procedures shall be examined to ensure procedures support existing programs’ growth.  Human capacity: An overview of the qualifications of the staff, trainers and their responsibilities should be compiled.  Management and Governance: Organization policies shall be reviewed to assess their contribution to encouraging programmatic self-sufficiency or to constraining it. In addition, KIAsia’s board structure’s impact on the implementation of the AERA program should be considered, with attention to contributions the board might be able to make to promote AERA project sustainability. b. Sustainability Strategy Development: This task shall use as its starting point the benchmarking appraisal that examined the essential factors relating to the capacity of the KIAsia to continue to implement self-sustaining AERA projects beyond the end of the AERA program. If the AERA programs can be sustained, designing a strategy for long￾term sustainability based on market conditions frequently requires altering organizational attitudes and behavior, perspectives, plans and long-term vision. Consequently, some of the elements of the strategic plan to be considered shall include:  Statement of Mission Goals and Vision: The strategy shall review the existing KIAsia mission, goals and a long-term vision.  Financial Self-sufficiency: A plan for continuing AERA programs and projects beyond the end of the AERA program shall be discussed in the context an overall organization strategy.  Governance: Changes to policies, regulations and administrative processes shall also be considered.  Priority Services: demand-driven services shall be identified and a strategy for prioritizing programs developed.  Constituencies: Strategies for identifying current and potential constituents shall help lay the foundation for generating resources. A systematic approach to profiling clients and constituents shall be outlined.  Pricing: The cost of providing services, the revenue generated and the competitive markets shall be discussed for the purpose of determining pricing policies for niche markets and services demanded.  Cost Recovery: There may be opportunities for recuperating costs generated from the sale of services. Recommendation for approaches to cost recovery shall be discussed in the context of an overall strategic direction. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 bAn Evaluation (July – September 2003) A-6  External Funding: Strategies shall be reviewed for identifying potential sources of external funding, i.e., non-USAID donors, collaboration with other NGOs, international organizations, etc.  Marketing and Delivery of Services: An examination of the methodology used to market KIAsia’s services and the vehicles for delivering them shall also be reviewed in the context of strategy development.  Administration: assessments of the process and systems of accounting and financial management, management information system design, physical infrastructure, management and organization structure and policy development shall be discussed. Article IV — Methodology On-site research and interviews for the study shall be carried out in Bangkok, Thailand for approximately 30 person-days. In addition, the evaluation team should schedule ten (10) days in the U.S. for trip preparation and meetings and/or interviews with U.S. partner organizations of the AERA program. The team shall interview KIAsia staff in Washington, DC and Bangkok, managers, board directors, participants, government officials, clients, potential donors and constituents. Written evaluations by and of KIAsia’s AERA programs, and its published materials shall be reviewed. Policies and administrative procedures shall be examined with management during this on-site phase of the study. In Thailand, the Kenan Institute Asia will make the following in-kind contributions to the participatory evaluation: 1. Involvement of senior level management and directors on an as-needed basis for individual consultations and interviews; 2. Participation of KIAsia personnel in meetings and interactions with AERA program partners and clients; 3. One KIAsia staff person assigned to work fulltime with the evaluation team. This person will help schedule meetings for evaluators with Thai government officials involved in AERA creation and implementation, US Embassy and AID officials, partners on BSOP program, Banks, SME's, as directed by the evaluation team; 4. Preparation of a AERA reading file for pre-departure study — primarily including official project documents such as the quarterly reports, the AID-KIAsia agreement, the Brooker report, internal program evaluations etc.; 5. Provision of any additional documents that may be requested once the evaluation team is on site; 6. Conduct preliminary analysis of data from questionnaires and research, under evaluation team's supervision; and 7. Respond to any additional data requests from evaluators as they write their report back in US. KIAsia will pay for any necessary translation services while in Thailand and for the roundtrip airfare trips between Thailand and the United States. An additional ten (10) person-days are required for the team to draft the findings of the benchmarking and strategy development research. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 bAn Evaluation (July – September 2003) A-7 Article V — Reports and Deliverables 1. Draft Work Plan. The evaluation team shall develop a draft work plan prior to departure from Washington, DC, and provide an electronic copy to the Cognizant Technical Officer (CTO) at rmaestri@usaid.gov. 2. Presentation to Executive Committee of KIAsia Board. The evaluation team shall give a presentation of preliminary findings to executive committee prior to its departure from Thailand. 3. Oral Presentation. The evaluation team shall provide an oral briefing of its findings and recommendations to the USAID Thailand program manager, Tom Elam; USAID regional mission representative, Scott Harding; the ANE Regional Program Manager USAID/W, Rebecca Maestri; and to the appropriate Kenan Institute Asia manager. 4. Draft Report. The evaluation team shall present a draft report in English of its findings and recommendations electronically to the CTO and KIAsia within ten business days from the time of return to the United States. The email address for KIAsia is: paulw@kiasia.org. 5. Final Report. Ten paper copies of the Final Report as well as an electronic version in Word 97 shall be submitted within 15 days following receipt of comments from the CTO. The Final Report must contain any CTO suggested changes/recommendations. Ten copies shall be provided to the CTO. The final report shall include an executive summary of not more than four pages, a copy of this scope of work, evaluation questionnaires, lists of persons and organizations contacted and contractor findings/recommendations. The Contractor shall send one electronic copy of the final report to the Development Experience Clearinghouse at the following email address: docsubmit@dec.cdie.org Article VI — Relationships and Responsibilities The evaluation team will receive technical direction from and report to the CTO or her designee. Article VII — Performance Period Approximately eight months. Article VIII — Work Days Ordered TBD Article IX — Special Provisions 1. Duty Post Bangkok, Thailand Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 bAn Evaluation (July – September 2003) A-8 2. Language Requirements and Other Required Qualifications English. Thai desired. 3. Access to Classified Information Not Applicable 4. Logistic Support All logistical support will be provided by the Kenan Institute Asia which will include providing assistance in setting up interviews with Thai government officials, USAID representatives, US embassy officials, Kenan staff, etc. 5. Work Week A six-day workweek is authorized while in Thailand. Article IIX — Key Personnel TBD Technical Qualifications and Experience Requirements for the Evaluation Team 1. Senior Institutional Analyst — The team leader shall be an evaluation specialist who has at least 10 years experience with developmental projects in emerging economies. The individual’s knowledge and familiarity of USAID process and procedures is desired. The individual shall have excellent written and oral communication skills, as well as exceptional organizational and analytical capabilities. The team leader shall have solid working knowledge of sustainable institutional characteristics and dynamics. 2. Senior Private Sector Specialist — The private sector specialist shall have a minimum of 10 years experience working on private sector development programs in emerging economies. Experience in the delivery of private sector consulting and training services is an advantage. The individual shall have analytical skills in the following technical areas: prudential commercial bank lending practices, SME capacity building; and institutional business support services. AERA THAILAND: INDICATORS AND TARGETS Please Note: These indicators and targets were developed for the AERA program prior to its launch in Thailand. They do not reflect subsequent AERA sub-programs programs created by KIAsia and approved by the AERA Steering Committee. The following narrative and tables define the indicators and associated targets for the Thailand component of the regional AERA Program. In some cases, activities clearly and directly contribute to one of the Intermediate Objectives (IO) for AERA, while contributing indirectly to another IO. In the indicator tables, activities are grouped under the IO to which they most Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 bAn Evaluation (July – September 2003) A-9 directly contribute. Indirect, but important contributions of other IOs are discussed in the narrative. I. INTERMEDIATE OBJECTIVE 1: BANK AND BUSINESS ACTIVITY RESTARTED The principal activity contributing to this IO is the Business Advisory Center (BAC) within the Kenan Institute. The BAC provides direct, targeted assistance to small and medium enterprises (SMEs). Small firms are defined as having less than Baht 20 million in fixed assets, not including land. Medium-sized enterprises are classified as those having fixed assets not including land that total between Baht 20-200 million. On occasion, larger firms may receive technical assistance from the BAC if important multiplier effects can be identified or if the firms pay for all costs of the assistance. However, the vast majority of assistance will be provided to small and medium-sized firms, as defined above. The BAC can provide assistance in a variety of areas important to restarting business activity and stimulating the growth of SMEs, including general management, quality control, production, accounting and financial control, marketing, and others. Through this assistance, SMEs will be better operated and more capable of performing the required range of management functions. For example, firms receiving assistance in accounting and finance will have more accurate and transparent accounting records and financial statements, facilitating the process of raising capital (short- or long-term), including bank loans. Assistance in the other areas above will likewise improve specific firm capabilities. Often this help will be in the form of business planning, which will vary in scope, depending on the size and type of business. In some cases the assistance will involve only one aspect of planning, such as marketing planning. Although several indicators along these lines were considered, the ability to use sound business planning is considered the most appropriate for capturing the entire range of expected improvements. Therefore, Indicator 1 is: Business plans (or parts of business plans) developed by SMEs assisted by the BAC. To measure this indicator, BAC will follow up with SMEs that have received assistance. It is projected that at least 50% of these SMEs will continue to develop and use some form of business planning in-house. The review of SMEs will be conducted six months and one year after the BAC assistance has been completed. The improved capabilities of SMEs in the areas of BAC assistance, absent uncontrollable external factors such as recession, should normally lead to increased sales, both domestically and overseas. Indicator 2 is: Total revenue of assisted SMEs. This indicator is a high level measure which will require improved firm-level capabilities in some of the key problem areas facing Thailand's SMEs. The target for this indicator is the average percentage of growth in total revenue achieved by firms receiving BAC assistance. The percentage increase in total revenue for each firm will be checked one year after the completion of BAC assistance and compared to the same quarter in the previous year. Another indicator will be the number of firms served. Since the firms must pay part of the costs of the service, the willingness of firms to pay for the service should be a good indicator of its perceived value to them. The targets for number of firms are cumulative and firms receiving assistance in the first year will continue to be monitored. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 bAn Evaluation (July – September 2003) A-10 II. INTERMEDIATE OBJECTIVE 2: CORPORATE AND FINANCIAL GOVERNANCE IMPROVED Three AERA activities contribute directly to this objective. The bank training program provides a range of training programs to Thai commercial banks, including risk management, credit analysis and management, internal control, and responsibility accounting. Through this training, banks will be better prepared to face the challenges of recovery in the financial sector. In each case, training materials will be developed for future use by banks’ training staff. In addition to the specific content and objectives of each individual training program, banks will be left with training staffs capable of developing and conducting training in other areas. Based on the training, banks will more effectively implement new policies and procedures to guide operations. For example, one bank has requested assistance to train a cadre of employees in all aspects of dealing with SMEs. When the program is completed, the bank will establish new policies and systems for loans and other assistance to SMEs. In several other cases, banks are receiving assistance in credit analysis and overall credit management. This training will result in more thorough credit reviews, including the establishment of new policies and procedures for credit analysis and approval. The bank training activity will have a significant impact on bank operations, resulting in better management of credit portfolios and improved risk assessment capabilities. This, in turn, will have an impact on the level of NPLs in the future for banks receiving training in related areas. BSOP partnerships include assistance providers; oversight and regulatory organizations; and organizations which set and monitor financial and business standards. As a result, new policies, standards, and systems and procedures will developed and in place which will improve governance and enhance investor and consumer confidence. For example, a partnership between the American Bankruptcy Institute and the Thai Central Bankruptcy Court is intended to improve the Thai court's capacity to administer bankruptcy cases by putting new procedures and administrative systems in place. Another partnership, between the Thai Securities and Exchange Commission (SEC) and the U.S. Treasury Department is intended to lead to the development and implementation of new Savings and Investment Tax Policies. The BSOP will also be an important activity to promote long-term sustainability of the programs started under AERA. Although every US-Thai partnership created through BSOP will focus on immediate problems and not necessarily continue, many will establish formal and informal relationships, which will endure after AERA funding is completed. The US and Thai entities involved in these partnerships will continue to cooperate to solve key problems and to promote good governance, as well as to stimulate economic growth. For example, the proposed partnership between the Thai Bankers Association and the American Bankers Association should continue to deliver a broad spectrum of raining to the Thai banking sector without USAID funding and long-term relationships will be developed between US and Thai organizations and associations. The third activity under IO2 is assistance to the Bank of Thailand (BOT) from the U.S. Department of Treasury. This activity is providing workshops and on-the-job training for BOT bank supervisors. In addition, a new supervision and examination manual is being developed for BOT staff. The result will be more focused examinations that concentrate on key commercial bank operations and financial indicators. BOT bank examiners will be better equipped to identify problems and recommend solutions to both commercial banks and the BOT. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 bAn Evaluation (July – September 2003) A-11 Three indicators have been set for IO2: Indictor 1: The number of banks that receive AERA bank training that implement new policies, systems, and procedures. The number banks taking such actions will serve as a simple measurement of achievement. However, achievement in this area cannot be reduced to a number. Narrative reports on the kinds of actions taken and the relative success of those actions will be helpful in assessing the success of the bank training program. Separate indicator tables have been developed for the bank training and BSOP activities. Each training program and partnership will focus on a specific outcome that will establish new policies, systems or procedures to improve performance and governance, thereby restoring customer and investor confidence. Indicator 2: Commercial bank training activities in related topics developed and conducted beyond AERA assistance. The number of such training courses, demonstrating an institutional impact emanating from the initial training activities will measure this indicator. It is expected that in some cases, training staff at each bank will continue to develop and conduct training courses, workshops, using the ideas, skills and materials provided through AERA assistance. In some cases, new courses may be conducted. In other cases, the course conducted under AERA may be repeated by bank staff for other audiences (e.g., at “up-country” branches). Indicator 3: Number of partnerships between U.S. and Thai entities still active at the end of the reporting period. One of the key features of the partnership approach is the development of long-term relationships between U.S. and Thai organizations. The continued existence of these partnerships will demonstrate the sustainability and efficiency of this approach. The number of partnerships implementing activities at the end of each reporting period will measure this indicator. INDICATOR TABLES FOR BUSINESS ADVISORY CENTER Intermediate Objective 1 Bank and Business Activity Restarted Indicators Total number of SMEs receiving AERA consulting assistance Increase in gross revenue Unit of Measure Number of firms assisted Average percentage change in gross revenue on the part of assisted firms. Source Company records and company reports to BAC Indicator Description Current GDP growth projections for Thailand are in the 2.5% range. SMEs receiving AERA assistance are expected to perform at least twice as well and a target of 5% growth in total revenue have been set. One year after assistance, firms’ revenue for the most recent quarter will be compared to the same quarter the previous year. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 bAn Evaluation (July – September 2003) A-12 Year Target # of Assisted SMEs Cumulative Actual # of Assisted SMEs Cumulative Actual Average % Increase 2000 20 2001 120 2002 220 2003 320 2004 410 2005 500 INDICATOR TABLES FOR BUSINESS ADVISORY CENTER Intermediate Objective 1 Bank and Business Activity Restarted Indicator Business plans or elements of business plans in use by SMEs receiving AERA assistance. Unit of Measure Percentage of SMEs with current business planning. Source Company follow-up reviews by BAC. Indicator Description Each SME receiving assistance will be encouraged where appropriate to use business planning methodologies. The percentage of firms that continue to maintain that planning one year after assistance is projected at 50%. Year Target Actual 1999 0 Baseline - 0 2000 50% 2001 50% 2002 50% 2003 50% 2004 50% 2005 50% INDICATOR TABLES FOR BANK TRAINING PROGRAM Intermediate Objective 2 Corporate and Financial Governance Improved Indicator Policies, systems, or procedures adopted by the participating banks Unit of Measure Number of participating banks implementing new policies, systems, or procedures Source Executive orders, and implementing rules and regulations of the participating banks Reports from the participating banks Indicator Description Implementation of new procedures and systems related to the training courses, i.e. new Credit Policies or Credit Policy Manuals, Risk Management Procedures, Internal Control Systems, Responsibility Accounting Centers or Systems Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 bAn Evaluation (July – September 2003) A-13 Year Target # of banks Cumulative Actual # of banks Cumulative 1999 0 Baseline - 0 2000 4 2001 5 2002 10 2003 12 Intermediate Objective 2 Corporate and Financial Governance Improved Indicator Training activities in related topics Unit of Measure Number of banks continuing training programs using concepts, skills and materials provided through AERA assistance Source Training units of the participating banks Indicator Description Number of banks conducting related courses, including development of the Manuals and Guidebooks developed to the Banks’ training units Year Target # of Banks Cumulative Actual # of Banks Cumulative 1999 0 Baseline - 0 2000 3 2001 6 2002 10 2003 12 2004 12 INDICATOR TABLES FOR BUSINESS SUPPORT ORGANIZATIONS PARTNERING PROGRAM Intermediate Objective 2 Corporate and Financial Governance Improved Indicator Policies, systems, or procedures adopted by participating partner organizations Unit of Measure Number of participating organizations implementing policies, systems, or procedures introduced or strengthened through the partnership fostered by the BSOP Source Participating US and Thai partner organizations Indicator Description Key organizations and measures targeted for reform include: 1. American Bankruptcy Institute providing technical assistance to the Thai Central Bankruptcy Court to improve court administration systems. 2. Thai Securities and Exchange Commission with assistance from the US Treasury Department to conduct Comparative Study on Thailand’s Savings and Investment Tax Policy in order to strengthen tax policies and administration. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 bAn Evaluation (July – September 2003) A-14 3. Thailand Rating Information Services to receive technical assistance to raise its bond rating capabilities thereby improving internal systems. 4. Thai Bankers Association and American Bankers Association (ABA) to partner and deliver new training programs and services to Thai bankers. 5. Institute of Certified Auditors and Accountants of Thailand is to produce multimedia education materials and establish new training methods for accounting professors with assistance from US university. 6. Securities Analyst Association to develop handbook covering new market instruments with technical assistance from American Institute for Management Research. The manual is to be used for training CFA students. 7. S&T Competitiveness partnership between Office of Economic & Technologies Competitiveness, and US Council for Competitiveness, is to increase the competitiveness of Thailand’s science and technology. 8. APICS certification to be established with Thailand Logistics and Production Society to improve operations management and logistics capabilities and systems in Thai industry. 9. Technical assistance to be provided to the Thai Credit Bureau by a US counterpart in order to established its operational procedures. 10. The Association of Thai Small and Medium Entrepreneurs (ATSME) to partner with the US Small Business Administration to establish better services and capabilities. Year Expected Measures Cumulative Actual 1999 0 Baseline - 0 2000 2 2001 7 2002 10 2003 15 2004 20 2005 25 Intermediate Objective 2 Financial and Corporate Governance Improved Indicator The number of long-term partnerships initiated and maintained Unit of Measure Number of partnerships Source Participating US and Thai partner organizations Indicator Description Partnerships between US and Thai organizations continuing to implement joint activities at the end of the reporting period Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 bAn Evaluation (July – September 2003) A-15 Year Expected Partnerships Cumulative Actual 1999 0 Baseline - 0 2000 2 2001 4 2002 8 2003 12 2004 10 2005 8 The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) B-1 ANNEX B INTERVIEW SCHEDULE Messrs. Michael Calavan & Ronald Bielen AERA Evaluation Bangkok, Thailand, August 2 – September 6, 2003 Note: Preparation period in Washington D.C. area (July 20-31) included: 1. Review of documents and preparation for field work in the office of Development Associates; 2. Consultation with USAID/ANE; 3. Interviews with Mr. George Irvine, Ms. K.C. Choe, and Mr. Arda Simsek of Kenan/Washington, and with Ambassador Willard Itoh (Department of State), Ms. Mira Gur-Arie (Federal Judicial Center), Mr. Brian Rohan (American Bar Association), Mr. Michael Decker (Bond Market Association); 4. Telephone interviews with Dr. James Chamber (Purdue University, Better Process Control School Partner), Mr. Chadwick Bidwell (Computer Programmer, Dyntec Disk Consultant), and Mr. Harlan Bentzinger (Citrus Expert, Chan Muang 39 Consultant), Mr. James Kelly (Rubber Engineer, Mould Mate Consultant), Mr. Joe Dougherty (DAI Trainers for BTP), Dr. Jennifer Bremer (Kenan/Washington), and Paul Wedel (KIAsia) August 2, 2003 (Sat) 10.55 PM Arrived Bangkok The Westin Hotel August 3, 2003 (Sun) August 4, 2003 (Mon) 8.30-9.30 AM Kenan Institute Asia Mr. Paul Wedel 10.00-11.00 AM USAID Representative Mr. Tom Elam Mr. Leon Waskin Ms. Pantita 1.30-2.30 PM US Embassy Mr. Joe Yun Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) B-2 3.00 PM The Securities and Exchange Commission (BSOP) Khun Supamas August 5, 2003 (Tue) 9.00-10.00 AM USAID Mr. Carey Gordon 10.00-11.00 AM The World Bank (BSOP) Mr. Mike Markels, mmarkels@worldbank.org 1.30-3.00 PM Kenan Institute Asia Meet with BSOP Program Mr. Richard Bernhard, richardb@kiasia.org Ms. Jiranya Ratchinda, bsop@kiasia.org Ms. Yada Kritsilpe, bsop@kiasia.org 3.00-3.30 PM Meet with Education Program Ms. Kamolmas Jaiyen, info@kiasia.org August 6, 2003 (Wed) 11.00-12.00 PM Thai Military Bank (BTP) Mr. Jiraphan August 7, 2003 (Thu) 10.00 AM Agency for Real Estate Affairs (BAC) Mr. Sopon Pornchokchai 02.30 PM Black Canyon (Thailand) Co., Ltd (BAC) Mr. Pravit Chitnarapong Mr. Michael August 8, 2003 (Fri) 2.30-3.30 PM Bank of Ayudhaya (BTP) Ms. Charlotte Donavanik and Dr. Kamol Boondiskulchok 4.30 PM Meet with Kenan Executive Committee (EC not available) August 11, 2003 (Mon) 9.00-9.30 AM Teleconference with Mr. Chai Jarungtanapiban VE (Thai), Black Canyon 02-374-9138 Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) B-3 9.30-10.30 AM Khun Indhira Wattanakasaem Bank Training Program AM Dr. Montri Chulavatnatol 2.00-3.00 PM Alternative Dispute Resolution Office Judge. Montri Sillapamahabundit 5.00 PM Meet with Mr. Alex Kaufman, Deputy Manager of Labor Standard Advisory Service Center August 12, 2003 (Tue) Teleconference Mr. Steve Dahl, VE of Black Canyon Company 7.30 AM (Thailand Time) (952) 935-9430 Mr. Peter Sullivan 3 PM (Thailand Time) (917) 981-9630 August 13, 2003 (Wed) 9.00 AM Mr. George Kelakos, Teleconference 9 AM (Thailand Time) (917) 981-9630 10.00-11.00 AM Office of the Auditor General (BSOP) Ms. Jaruwan Ruangsawadipong 02.30-03.30 PM King Mongkut University of Technology (KMUTT) Dr. Tipaporn Yoovidhya August 14, 2003 (Thu) 9.00-10.00 AM Bank for Agriculture and Agricultural Cooperatives Khun Apinya Poonyarit 12.00-2.00 PM Siam Commercial Bank (BTP) Mr. Wisit Shoowong 3.00-4.00 PM Meet with Khun Kesara Manchsree (Used to work with Bond Dealing Center Venue: Stock Exchange Building Floor 8 4.00-5.00 PM Kenan Institute Asia Dr. Saisawan Vadhanapanich Director of Better Business Practices Div and Manager-BAC August 15, 2003 (Fri) 10.00-11.00 AM Thai Bond Dealing Center Khun Nattapol Chavalitchevin Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) B-4 2.00-3.00 PM Office of the Judiciary Dr. Thammanoon Phitayaporn 7.00 PM Depart for Chiangmai, Flight TG 126 8.10 PM Arrive Chiang Mai August 16, 2003 (Sat) 9.30-10.30 AM Thai Lanna Food & Cultural Center Ltd Dr. Charnsiri Suetrong Email: vgroup@cmnet.co.th 1.30-2.30 PM Art & Technology Co., Ltd Dr. Busaya Techasen August 17, 2003 (Sun) August 18, 2003 (Mon) 10.00-11.00 AM Northern Adventure Co., Ltd (Chaing Mai Magazine), Ms. Margaret Bhudunzzhong 1.30-2.30 PM Chiang Mai Provincial Court Mr. Songwut, Director 4.10-4.45 PM Flight to Mae Hong Sorn Province Mae Hong Sorn Mountain Inn Khun Anchalee August 19, 2003 (Tue) 12.10-12.45 PM Return from Mae Hong Sorn Province 1.30 PM Chiang Mai Provincial District (BAAC) Khun Apirom Sukprasert August 20, 2003 (Wed) 9.00 AM Bank of Ayudhaya (Chiang Mai Branch) Khun Borirak, District Manager 10.00 AM Khun Suwan, Branch Manager 3.30 PM Chiangmai Laaw Restaurant, Co., Ltd Khun Songsak Wattanapol 7.15 PM Depart Chiang Mai by TG 117 8.25 PM Arrive Bangkok Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) B-5 August 21, 2003 (Thu) 3.00 PM Post and Telegraph Department Venue: 87 Phaholyotin 8 Rd, Bangkok Contact: Khun Rianchai Reowilaisuk Tel: 02-271-0151 August 22, 2003 (Fri) 2.00 PM Meeting with USAID 3.00 PM Meeting with US-AEP 8:30 PM Teleconference, Malcolm Peplow (Stream IT) August 23, 2003 (Sat) 6:30 PM Teleconference, David Layne (AREA) August 24, 2003 (Sun) August 25, 2003 (Mon) 10.00 AM Dyntec Disc Productions Co., Ltd. Khun Suthep 3.00 PM Meeting with Khun Suvicha Mingkwan. Cluster Competitiveness Program 4.00 PM Meeting with Dr. Frank Roman, Cluster Competitiveness Program August 26, 2003 (Tue) 7:30 AM Calavan to High Value Agriculture cluster 9.30 AM (Bielen) Mould Mate Co., Ltd Khun Supol 2.00 PM Stream IT Consulting Co., Ltd Khun Vichu August 27, 2003 (Wed) 10.00 AM A Group Consult and Holding Co., Ltd. Khun Sarawut 2.00 PM KSK Auto Part Co., Ltd. Khun Suthicha Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) B-6 August 28, 2003 (Thu) 9.30 AM (Bielen) Meeting with Khun Sorasak (BAC—Microlending Policy) 9:30 AM (Calavan) Attend Seminar on Testimony Recording System for Courts and Arbitration 2.30 PM Chan Muang 39 Agriculture Co. Ltd. Khun Anandchai/Khun Kusuma August 29, 2003 (Fri) 9.30 AM Meeting with Office of SME Promotion Khun Vivat Vinicchayakul, Director General Khun Wimonkan Kosumas, Director, Dept. of Intl. Cooperation 2.00 PM Meeting with US Ambassador and Joe Yun August 30, 2003 (Sat) Analysis, prepare presentation Sept 1, 2003 (Mon) Analysis, prepare presentation. Sept 2, 2003 (Tue) Analysis, prepare presentation Sept 3, 2003 (Wed) 9.30 AM Presentation to USAID Sept 4, 2003 (Thu) Preparation, analysis Sept 5, 2003 (Fri) 9.00 AM Debriefing with KIAsia management staff 3:00 PM Teleconference: Mr. Richard Moore, Pricewaterhouse Coopers Sept 6, 2003 (Sat) Dr. Calavan departs for Chiang Mai Mr. Bielen departs for US The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) C-1 ANNEX C BAC CASE STUDY TEMPLATE Firm Name: Line of Business: Volunteer(s)/Consultants: Services Provided by VE/Consultant: Timing: Cost to KIAsia: Cost to Firm: Major Elements of SOW: How the Consultancy Was Initiated: Performance of the VE/Consultant(s): Working Relationship Between Firm and VE/Consultant: Performance of KIAsia in Supporting the Consultancy: Major Recommendations of VE/Consultant: Client Efforts to Implement Recommendations: Impacts on Firm Management and Financial Status: Special Observations Drawn from this Case Study: Consultants’ Considered Judgment: The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) D-1 ANNEX D SAMPLE INTERVIEW GUIDES KIAsia Interview Guide BAC Client Firms (7/28/03 Version)  Tell me about yourself. What is your position in this firm, and how did you get involved with this business?  Tell me about the consultancy (ies) that was arranged for your firm through the business advisory center. (make sure we know the thai name for the center.)  What steps did you and your firm take to arrange this consultancy? (where did you first hear about the bac? Did you approach them? How?)  How exactly was it decided what work the consultant(s) would do with your firm? Was there a written description of what work would be done?  Did you express a preference for a thai, or international, advisor(s)? Who actually came to do the work?  How much did this consultancy cost you? (in baht and other resources?)  Do you think this was a good value for your business? Would you repeat this experience? How?  Has the consultancy been good for your business. Please take a few minutes to explain what has changed.  So far, have those changes had a financial impact on your firm? (e.g. lower production costs? Expanded sales? Increased profits? Greater access to credit?)  If there have been financial improvements, what do you think are the main reasons for those improvements?  Were there any shortcomings or failures in the consultancy(ies)?  Do you think consultancies like this can should continue in the future? Some of the money for your consultancy came from the us government. What would happen if there was little or no money from the usg. Would businesses be willing to cover more of the costs?  If you were advising another firm about requesting a consultancy similar to yours, what advice would you give? Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) D-2 KIAsia INTERVIEW GUIDE BSOP Thai Partner Organizations (7/28/03 Version)  Tell me about yourself. What is your position in this organization, and how did you get involved with this particular activity?  Tell me about the partnership arranged between your organization(s) and the American partner organization(s) through the business support organization partnership. (make sure we know the thai name for kiasia and for the bsop.)  What steps did you and your organization take to arrange this partnership? (where did you first hear about the bsop? Did you approach them? Or did they approach you? How?)  How exactly was it decided what activities the partnership would focus on at first? Was there a written description of planned activities? Who all had a role in finally deciding what the partners would do together?  Did you, or others in your organization, express a preference for what kind (public/private, academic or professional, etc.) Of american partner you preferred? Who actually came to do the work?  Was there a good match between the partners? How was it good (or not so good)?  How much has this partnership cost your organization so far? (in baht and other resources?)  Do you think the partnership activities have been a good value for your organization? Will you repeat this kind of experience with the same american partner(s)? How? With other partners?  Please remind me again, from your viewpoint, what the main activities have been under the partnership.  Would you rate any of these activities as more important, or more effective, than others?  Can you point to any specific improvements resulting from the partnership? E.g., in your organization? In the profession? In the legal/administrative framework? Etc.?  Do you think partnerships like this can should continue in the future? What would happen if there was little or no money from the use. Would your organization be willing to cover more of the costs?  If you were advising another organization about starting a partnership similar to yours, what advice would you give? The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-1 ANNEX E CASE STUDIES BAC CASE STUDY Firm Name: Firm A Line of Business: Production of high quality cross-stitch and beadwork kits. Volunteer(s)/Consultants: International VE from CDC. Services Provided by VE/Consultant: Provided detailed advice on marketing, particularly for the U.S. market. Timing: August 2001 Major Elements of SOW: The scope of work focused on providing the firm with advice on successful entry into the U.S. crafts market. How the Consultancy Was Initiated: A senior employee of the firm attended a seminar sponsored by BAC, and learned about the opportunity for consultancies, including international VEs. She in turn informed the MD, who in turn contacted BAC. The request was for an American business person who had an intimate knowledge of the U.S. crafts market. The fit was not quite perfect; the VE actually deals with imported “exotic/ethnic” apparel for women, but knows the general market, and has friends in the crafts marketing line. Performance of the VE/Consultant(s): The VE went out of her way to study up for the assignment, seeking out websites, contacting friends and others for information, and meeting the MD at a needle crafts show in North Carolina at her own expense. She wrote a useful report, that has led to success by the client in entering the U.S. market. Working Relationship Between Firm and VE/Consultant: The working relationship seems to have been excellent. The MD (and other staff, apparently) are in fairly regular email contact with the VE. The VE stayed in a hotel that is virtually across the street from the head office of the firm, and became intimately familiar with sales, production, design, etc. and with problems in entering the U.S. market. Performance of KIAsia in Supporting the Consultancy: BAC staff seem to have done an excellent job of discussing the requirements of the client and preparing an appropriate SOW, and to have located a highly appropriate VE. Three different VEs were offered. There seems to have been little need to intercede once the consultancy was under way. Major Recommendations of VE/Consultant: 1) To get around import quotas for the U.S. market (the cloth on which the patterns are cross-stitched is locally manufactured cotton cloth, and thus subject to import quotas), the firm should only sell single use pattern books through U.S. retailers. (Individual customers can order kits to be mailed from Thailand.) 2) Focus on the Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-2 “oriental” niche initially, selling patterns with Japanese/Chinese/Thai/Indian motifs. 3) Make extensive use of the website for ordering. Client Efforts to Implement Recommendations: Virtually all of the recommendations seem to have been implemented. The VE recommended “oriental” themes for the U.S. market, and the client firm has followed through. She recommended that they sell single use pattern books, and they have followed through. Etc. The firm is blessed with a highly educated, creative senior staff. One of the particular successes is setting up a website that can be used both by individual customers (one portal) and by retail dealers (other portal). This has been an important way of accessing the U.S. market, since about 90% of the users of the site are American. Impacts on Firm Management and Financial Status: This is a highly successful, well￾managed firm in any case, earning a good deal of money for the economy of northern Thailand, and employing 150 people in steady jobs. But, as a result of the VE’s recommendations, monthly U.S. sales have grown from virtually nothing to B200,000 per month. Special Observations Drawn from this Case Study: This firm has an extraordinarily well educated management, and they have enjoyed considerable success. However, this consultancy demonstrates how international VEs can be best used: In this case, the VE brought very specific advice on how to meet the demands of the U.S. market. In other cases, the American volunteers help to introduce cutting edge technologies. Evaluators’ Considered Judgment: This is a clear cut case of success. The firm has indeed expanded sales to the U.S. According to the MD, the current level of U.S. sales is a cool $60,000 per year, more than enough to justify the consultancy. Equally important, the firm has the knowledge about what must be done to “move to the next level,” namely to place their product with Michael’s, the titan of the crafts industry in the U.S. with annual sales of $2 billion. They must make a deal with one of the 5-6 dealers that supply Michael’s. They may or may not succeed in doing this, but now have a clear picture of how they must proceed if they want to reach the next level of penetration of the U.S. market. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-3 BAC CASE STUDY Firm Name: Firm B Line of Business: Information Technology Volunteer(s)/Consultants: International VE from CDC Services Provided by VE/Consultant: Establish HR Department and systems supporting it. Timing: July-August 2002 Major Elements of SOW: Assist firm with an HR master plan covering: work flow analysis, job description, job grading, performance appraisal and the use of HR scorecard for evaluation. How the Consultancy Was Initiated: Stock Exchange of Thailand was a client and BAC was housed in the Exchange building at the time. Firm B visited with BAC and learned of consultancy services. Firm B knew it's future depended on a well-run HR Department and subsequently requested assistance. Performance of the VE/Consultant(s): The client is extremely pleased with the services provided by the consultant. Based on discussions with the client and consultant, it is the evaluators' judgment that the consultant did a superb job. Working Relationship Between Firm and VE/Consultant: Client valued the consultant's input and had identified a well functioning HR department as essential for the firm's future. Cooperation was excellent as it is evident the consultant and client each worked to achieve maximum value from the consultancy. Client suggested that consultant's services should be offered to more Thai businesses using a wholesale approach. Performance of KIAsia in Supporting the Consultancy: Client was offered a number of VE consultants from which to choose. Firm B and the consultant both were satisfied with KIAsia support. Major Recommendations of VE/Consultant: 1) Establish HR Department with full functions (which was accomplished in concert with the client), 2) Develop performance appraisal system, 3) Create training and development plan for each employee. Client Efforts to Implement Recommendations: HR Department was established and a department head appointed. Job descriptions and performance appraisal are fully implemented. Training and development plans are in place. Evaluators' impressions are that the client will continue to follow through. Impacts on Firm Management and Financial Status: Firm has fully embraced the recommendations as they were developed jointly with employees of Stream. The Evaluators had no access to financial records and believe the initial impact will primarily be retention and increased productivity of well-trained employees. This should help the firm acquire and maintain business. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-4 Special Observations Drawn from this Case Study: The consultant often provides HR consulting assistance on a wholesale basis. Involving a number of companies in general sessions and then following up with guidance on proprietary aspects on an individual basis. This leverages the services of the consultant, thereby greatly reducing consultancy cost per firm. BAC should evaluate offering these services to more clients using the same VE consultant. Indications are that the VE would be pleased to participate in such exercise. Consultants’ Considered Judgment: The consultancy was an outstanding success. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-5 BAC CASE STUDY Firm Name: Firm C Line of Business: Property valuation, other services related to research, training and consulting related to real estate. Volunteer(s)/Consultants: International Volunteer identified by client; another provided by CDC. Services Provided by VE/Consultant: Specific and focused training related to real estate valuation. Timing: Independent: March-April 2002, CDC: March 2003 Major Elements of SOW: Independent: 1) Present real estate appraisal and property valuation course including: cost approach, sales comparison approach and income capitalization approach, uniform standards and professional appraisal practices, valuation of environmentally affected properties and emerging issues in the US and Thai real estate markets. CDC: 1) Consulting on Computer Assisted Mass Appraisal (CAMA) including: feasibility of developing CAMA in Thailand, considerations in developing CAMA services for marketing in Thailand, recommendations on marketing strategies. 2) Training on CAMA including demonstrations of use in other countries and applications to Thailand. How the Consultancy Was Initiated: Firm C's Managing Director an extremely proficient user of the Internet learned of BAC services through KIAsia's Website. He called and requested a volunteer to provide a course in property valuation. CDC, ACDI/VOCA and IESC were unable to locate a suitable volunteer so he advertised and received over 100 responses. The volunteer saw a notice in the American society of Appraisers Newsletter. After the first consultancy, the MD requested BAC to assist with CAMA training. A suitable volunteer was located through CDC. Performance of the VE/Consultant(s): Both consultants were highly specialized in the field of property valuation and were hand-picked by the MD. Because of this extremely close match and the stream of pre-consultancy communication both consultancies delivered the demanding results expected by the MD. The MD continues to communicate with them by Email. Working Relationship Between Firm and VE/Consultant: There was an extremely close working relationship with both consultants due to the high level of technical competency of the MD. In the case of the first engagement, the consultant developed a detailed syllabus for the course in close coordination with the MD prior to arriving in Thailand. Performance of KIAsia in Supporting the Consultancy: For the independent consultancy, KIAsia managed the recruiting roles normally accorded to the three volunteer recruitment organizations. Due to the hands-on approach of Firm C's MD, support demands on KIAsia were less that normal. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-6 Major Recommendations of VE/Consultant: Independent: The work product was principally the delivery of training however the MD asked a lot of questions about improving business. The consultant suggested that Firm C raise fees to the international market and retain an English speaking editor for Firm C documents. CDC: 1) Determine data needs for CAMA in Thailand relative to desired uses, 2) Develop database management system and quality controls for data collection, capture and analysis; valuation modeling and testing of results. 3) Review possible external sales of Firm C CAMA services to local government, mortgage agencies etc. Client Efforts to Implement Recommendations: Firm C's MD is a talented and savy manager who identified areas in which Firm C needed strengthening and secured consultants to strengthen those areas. As a result of the first consultancy, Firm C is now providing training courses on real estate valuation. As a result of both, the firm is moving ahead in developing its capacity to provide CAMA services. Impacts on Firm Management and Financial Status: The consultancies have assisted management in seeking new business. While the consultants did not have access to financial information of the firm, it is believed that the consultancies have had a significant impact on the Firm C's competitiveness and will add to sales. Special Observations Drawn from this Case Study: The three volunteer recruitment organizations failed to locate a suitable volunteer candidate for the first consultancy. The fact that the MD was able to secure over 100 applicants is testimony to the MD's capabilities. Both consultancies directly relate to the BSOP activity Building Capacity and Professional Standards for Property Valuation in Thailand between the Thai Securities and Exchange Commission and The Appraisal Foundation for the US. That partnership seeks to promulgate uniform valuation standards for Thailand. Firm C's MD agrees that standardization is important but believes that unless there is enforcement of those standards by SEC (or others) they will have little affect. Consultants’ Considered Judgment: This consultancy was an outstanding success. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-7 BAC CASE STUDY Firm Name: The Firm D constituent firms include: D-1 —Designs and installs sophisticated AV systems; D-2 —Provides repair and maintenance services for AV and lighting installations; and D-3—Manufactures and sells lighting dimmers and sound amplifiers. Line of Business: Design and installation of high quality audiovisual systems for conference rooms, auditoriums, hotel conference centers, etc. Also manufacture, repair, and maintenance of AV equipment. Volunteer(s)/Consultants: One international VE from CDC: Another provided by IESC. Services Provided by VE/Consultant: CDC: General management and production systems; IESC: Marketing strategy. Timing: CDC: July-August 2000; IESC: November-December 2001 Major Elements of SOW: CDC: 1) Improve use of IT in company management; 2) Improve application and implementation of ISO 9000 standards; 3) Increase efficiency of working procedures; 4) Establish management guidelines for future expansion. IESC: Suggest marketing strategies for each of the component firms, including a clear definition of who the clients are for each. How the Consultancies Were Initiated: Firm D does business with a number of media firms. Managers heard about KIAsia from some of their counterparts, and decided to take advantage of the “almost free” consulting services. Firm D contacted KIAsia to inquire about opportunities to bring in an international consultant. Performance of the VE/Consultant(s): Both consultants made useful contributions to Firm D. The CDC VE brought useful rigor into analysis of manufacturing and maintenance processes, and made stimulating suggestions for re-organizing sales and engineering functions. The IESC consultant helped Firm D managers to become far more analytical about potential customers for each company product, and suggested more aggressive sales strategies. Working Relationship Between Firm and VE/Consultant: The CDC VE had a particularly intense working relationship with the group’s CEO, spending one to two hours daily with him throughout a three week consultancy. On the other hand, the IESC VE had a hard time understanding the CEO’s English, and apparently spent little useful time with him. Both consultants also spent considerable time with managers of the constituent firms, and made stimulating suggestions that are still reflected in daily operations of the firms. Performance of KIAsia in Supporting the Consultancy: For the first consultancy, BAC offered three or four different VEs, but specifically recommended the individual that was ultimately selected. Major Recommendations of VE/Consultants: CDC VE: 1) The firm operates in a niche market, and needs to raise its overall operating margin from 15% to about 35-40%. 2) Serious consideration should be given to combining the sales and engineering departments of the three Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-8 constituent firms. 3) Establish a “Great Idea Group” to identify promising new products and service innovations. 4) Hire a Systems Analyst to undertake regular analysis of production processes, and use an analytical tool provided by the VE. IESC VE: 1) Narrow the potential market for AV system installation from “everyone” to clearly defined segments of the market, such as educational institutions, hotels, and conference centers. 2) Consider greater emphasis on private sector customers by carefully examining potential “sub￾segments”, and place less emphasis on the Thai government (which is slow in paying). 3) Avoid the “trap” of promising high quality at a low price. The firm’s customers are generally more interested in quality and reliability than they are in bargain pricing. 4) Make brochures less “technical,” more “customer-oriented.” 5) Use free technical seminars as a marketing tool. Client Efforts to Implement Recommendations: Many of the suggestions have been implemented. A Great Idea Group meets regularly, and has introduced a new product—an ingenious portable speaker’s platform/public address system that the group plans to market to commercial users, local governments, and political candidates. A more focused marketing strategy, based on educational institutions and government meeting rooms, has been pursued, and technical seminars are being used as part of the marketing strategy. The firms’s services are being marketed in technical publications, and regular sales calls are made on architecture and engineering firms. A Systems Analyst has been hired. In fact a new department has been established, and the employees of the unit are applying the analytical tool provided by the CDC consultant to good effect. Impacts on Firm Management and Financial Status: While it was not possible for the Evaluators to establish a clear causal relationship between specific recommendations and the “bottom line,” it is clear that many have been implemented, and that the Firm as a whole is thriving. For example, the Firm recently purchased and moved into a new building, purchased from cash flow, not from a bank loan. Special Observations Drawn from this Case Study: The CDC consultancy provided a by￾product that BAC employees were unaware of. As a result of the VEs recommendation, a new Systems Analysis unit has been set up for the Firm, and is now undertaking continuous analysis to ensure that all production and maintenance processes achieve optimal efficiency. Evaluators’ Considered Judgment: Both VE consultancies have been highly effective. The firm has entrepreneurial leadership, and high level technical skills. However, managers were inclined to make over-simple judgments regarding marketing (e.g., some segments of the private sector have defaulted on payments, so we will avoid doing business with all private firms). Although they work with cutting-edge technologies, the Firm was paying too little attention to innovation, and under pricing their products. They have responded in a positive, productive manner to advice they received on these matters. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-9 BAC CASE STUDY Firm Name: Firm E Line of Business: Hotel- restaurant Volunteer(s)/Consultants: Two BAC in-house consultants Services Provided by VE/Consultant: Business diagnostic, competitive analysis, Timing: May 2000 Major Elements of SOW: 1) Industry analysis, 2) Customer analysis, 3) Competitor analysis, 4) Financial analysis How the Consultancy Was Initiated: The Managing Director learned of BAC support at a booth in the Queen Sirikit Center during a visit to Bangkok. Performance of the VE/Consultant(s): The consultants provided high quality diagnosis and advice resulting in the client moving forward on recommendations. The Managing Director praised the consultant's performance. The Evaluation Team could readily identify significant positive changes in the operation resulting from the consultancy. Working Relationship Between Firm and VE/Consultant: The consultants spent the first week on site. The Managing Director credits this presence for the free and open exchange of information which provided added value. The MD had not previously used consultants and had limited opportunities for dialogue with peers, making this consultancy especially useful. Performance of KIAsia in Supporting the Consultancy: BAC provided the consulting services and support, much to the satisfaction of the MD. Major Recommendations of VE/Consultant: 1) Marketing efforts: revise registration form to collect marketing data from guests; segment marketing to major customer types; promote the Firm's differentiating features from competitors, 2) Products and Services: upgrade rooms and buildings to an international standard level of quality; increase revenue generated through price increases after renovation; differential pricing based on "view;" advertisements and incentives, 3) Human resources: hire a general manager to handle day-to-day operations; upgrade skills and responsiveness of staff; revise organizational structure for efficiency after general manager is hired including hiring a food & beverage manager, controlling staff levels for optimal high and low season levels, 4) Finance and Accounting: replace manual accounting system with computerized system; work with a professional accountant to develop a baseline system for recording and tracking accounting information; carefully review revenue and expense items which appear to be abnormal. Client Efforts to Implement Recommendations: The Managing Director valued highly the support from BAC and has readily implemented most of the recommendations. The Evaluators noted that the marketing recommendations had been fully implemented. The renovation of rooms and buildings was apparent, resulting in a warm and comfortable atmosphere keeping with Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-10 the cultural identity of the area. Seasonal pricing and differential pricing was in place. A general manager and subsequently, a food and drug manager were hired and served well. Recently, both managers moved on and the managing director is seeking candidates to fill those positions. Staffing levels are being controlled to avoid overstaffing during low season while maintaining service during high season. (Although a 70 room facility, the Inn faces demand for up to 300 rooms during high season) A computer was purchased for automating finance and accounting, but uses word processing and spreadsheets at present rather than integrated accounting. Overall, the implementation of recommendations is impressive. Impacts on Firm Management and Financial Status: It is clear the implementation of recommendations resulted in improved management. The Evaluators believe the financial status of the organization was improved. As a result of the consultancy, the business was able to obtain a 50 million Baht loan, largely to implement consultancy recommendations. Special Observations Drawn from this Case Study: Although the Managing Director was extremely satisfied with the consultancy, when faced with three substantive business issues which warranted additional consultancy, no thought was given to contacting BAC. (A two month old offer of 20 million Baht capital from SME Bank on which the MD needed an outside opinion, an offer for Internet booking and marketing, a high-priced integrated hotel management and accounting system.) This is an indication that there is little marketing followup by BAC on previous clients. Additional consultancy in this case could strengthen the status of the Firm and provide income for BAC. Evaluators’ Considered Judgment: The consultancy was an outstanding success and the business could be enhanced by followup assistance. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-11 BAC CASE STUDY Firm Name: Firm F Line of Business: Solid Rubber Tires Volunteer(s)/Consultants: International VE provided by IESC Services Provided by VE/Consultant: Improved quality control, assist in establishment of a research and development laboratory. Timing: January 2003, three weeks Major Elements of SOW: 1) Determine reasons for a higher than acceptable rate of rejects on certain products 2) Assist in establishing a research & development laboratory for compound formulation and mould production How the Consultancy Was Initiated: A National Science & Technology Development Agent visited the company and suggested the BAC. A BAC staff person visited and subsequently developed the SOW with the Managing Director. Performance of the VE/Consultant(s): The consultant provided the client with a range of services that were very well targeted to the SOW. Client's high satisfaction with the consultant was confirmed by followup contact between client and consultant requesting additional assistance. Working Relationship Between Firm and VE/Consultant: The Head Office is located in Bangkok and the plant about 100 Km away. The managing director located in the head office communicated well in English and his brother, the plant manager could speak English reasonably well. Communication was a constraint with others. A freer flow of information from the company would have made the consultancy more productive. Performance of KIAsia in Supporting the Consultancy: VE consultant felt that preparations were not particularly strong, indications of a 5 day week instead of actual 6, communications on site hampered due to little English language capacity, marginal hotel accommodation in field. Major Recommendations of VE/Consultant: 1) Reduce excessive time in compound mixing through better planning, 2) Carefully control components going into the formulation, 3) Secure training for laboratory supervisor, 4) Improve testing procedures to reduce need for destructive testing. 5) Sell waste trimmings to local cement factory Client Efforts to Implement Recommendations: Client reduced defect rates due to better knowledge imparted by the consultant for better selection and mixing of raw materials. Client was not motivated to seek additional training for laboratory supervisor. Client responded and acquired technical manuals for standard testing procedures on rubber, has implemented the procedures. Client was non-responsive to recycling product waste with the cement factory and continues to simply dispose of it at the factory site. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-12 Impacts on Firm Management and Financial Status: Consultant estimated that productivity was improved in the materials mixing area by 25%, thereby eliminating the need to purchase an additional mixer that would have cost about $70,000. This contributed to lower electricity costs, which the consultant learned post-consultancy had been a burdensome cost. The consultant identified reasons for defects in the products, but long-term solutions depended on better-trained staff, testing equipment and resource materials. Special Observations Drawn from this Case Study: This is an organization that could have easily paid the full cost of consultancy. It received economic benefits many times the market value of the consultant's time. It does not appear that full use was made of the consultant's very good recommendations. This may be a case where the client would be willing to engage BAC services on a full cost recovery basis as there are additional issues that could use consultancy. Consultants’ Considered Judgment: This was an effective consultancy that could have been leveraged even more with closer attention by KIAsia and the Managing Director. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-13 BAC CASE STUDY Firm Name: Firm G. Line of Business: Restaurant and entertainment venue. Volunteer(s)/Consultants: The IESC VE is a former restaurant owner, manager, and chef, and currently works as a restaurant consultant. Services Provided by VE/Consultant: Overall management issues for a newly opened restaurant. Timing: February-March 2000. Major Elements of SOW: The scope of work was broad-based. A very large restaurant was being opened by an inexperienced owner, who recognized he needed advice on almost every aspect of the business. Thus the VE was invited to comment on virtually every aspect of management and physical organization. He examined the menu, and recommended procedures for periodic assessment of customer preferences, and for adjustments in dishes and pricing. He examined issues of personnel management, and procurement policies, and did a critical review of the size, lay out, and equipment of the kitchen. He even commented on the quality of the draft beer and the house band (both received high marks). How the Consultancy Was Initiated: The owner/manager of Firm G knew he was entering a very challenging business with a unique concept, but no experience. His professional background is in engineering, and neither he nor his wife had ever run a restaurant/entertainment venue previously. So he contacted KIAsia about one year before the planned opening of the restaurant. He actively sought a VE with broad experience in conceiving, designing, managing, and owning restaurants. Performance of the VE/Consultant(s): The VE’s web site (www.foodserviceinsight.com) has some very useful tips for restaurant managers. And his CV demonstrates a rich educational and experiential background in the restaurant business. His report also shows that he is a warm and supportive person, one that knows how to make himself respected and listened to. The report is also rich in professional advice, not just the “details,” but broad strategic points regarding which parts of the business to emphasize (e.g. food service versus entertainment versus beverage service, and how the physical set up and the menu should be changed accordingly. He also has rich advice on personnel management. Unfortunately, it appears that he was not able to impart a lot of this wisdom in face-to-face sessions. Working Relationship Between Firm and VE/Consultant: An interview with the client and the consultant’s report suggest a high degree of mutual respect. The client noted that he regularly re-reads the consultant’s report, and the consultant makes it clear that the client has conceived and assembled an excellent “product.” But, as noted above, the extreme pressures on opening a large, unique new restaurant, and the to-be-expected emergencies of the first week of business ruled out the optimal face-to-face working relationship. Thus, the VE’s written report took on even greater significance than is usually the case. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-14 Performance of KIAsia in Supporting the Consultancy: The client gave the two BAC employees who assisted with the consultancy high marks for preparing an appropriate SOW, and for locating a highly qualified VE. Major Recommendations of VE/Consultant: 1) The VE stressed the importance of hiring highly qualified managers, and giving them adequate authority to undertake their jobs effectively. 2) He noted that beverages and entertainment appeared to be major draws for customers, and suggested that a simpler, snack-type menu be considered. 3) He recommended steps for regularly assessing and changing the menu. 4) He recommended an expansion of the kitchen. Client Efforts to Implement Recommendations: At the time of the consultancy, the client was overwhelmed with the daily challenges of establishing a new business, but over time he has adopted most of the suggestions made. The menu is revamped on a regular basis. However, the menu continues to be quite extensive; over time, as the clientele has become clearly defined (including about 50% Thai families, including children), it has become clear that an extensive menu, including dishes that are popular with children, is required. Professional managers are now in place, with sufficient authority to take much of the burden of day-to-day operations off the owner’s shoulders. This has left the owner sufficient time to establish a smaller branch in a Chiangmai shopping center, and to plan for as many as five new branches spread across northern Thailand. Impacts on Firm Management and Financial Status: Two-and-a-half years later, the restaurant is still in business. This is a strong track record compared to restaurants either in Thailand or in the U.S.; most fail in the first year. It seems probable that the consultant’s report had considerable value in helping the client to “keep his eye on the ball” and make sensible choices in: hiring and replacing staff, designing and re-designing the menu, deciding to make a firm commitment to retain the original house band, etc. Special Observations Drawn from this Case Study: An obvious point that can be drawn from this case is the importance of timing. The consultant had been identified, and was attractive both to the firm’s owner/manager and to the BAC project officer. However, due to other commitments, he (the consultant was not available before the opening of the restaurant. As a result, he arrived at a uniquely interesting and revealing time (during the restaurant’s opening week), but had considerable difficulty getting face-to-face time with the owner/manager. Clearly, a consultancy beginning about a month sooner would have been more useful in the sense that there would have been more leisure for discussions between the consultant and client. Evaluators’ Considered Judgment: This consultancy has been a qualified success, with the likelihood of long-term impact on a business that appears to be moving steadily toward success. It appears that some of the most crucial advice has been heeded, since the client has transferred more responsibility to his professional staff, and has more leisure to think about the future of the business. The client was eager to make the point that one of the most useful and frequent ways he “thinks about the business” is to re-read the consultant’s report on a weekly basis. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-15 BAC CASE STUDY Firm Name: Firm H Line of Business: Mandarin orange production Volunteer(s)/Consultants: International VE provided by CDC Services Provided by VE/Consultant: Advice on production and processing of citrus Timing: Three weeks in March-April 2002 Major Elements of SOW: 1) Improvement of growing techniques: a) Diagnose problems with the post-bloom drop of oranges and provide practical solutions, b) Provide practical solutions to improving coloration and reduction of scarring of mandarin peel, c) Evaluate operations for effectiveness. Suggest ways to reduce costs, improve efficiency. 2) Exploration of feasibility of fruit juice production. How the Consultancy Was Initiated: Firm was referred to KIAsia by Thai Farmers Bank. BAC personnel visited operation and developed work plan. Performance of the VE/Consultant(s): The consultant provided a quality product that met the elements of the SOW and fully satisfied the client. Working Relationship Between Firm and VE/Consultant: Consultant commented that the combination of receptivity by the client and a well-written SOW were keys to what he considered a successful assignment. The client confirmed that there was a very good working relationship. Performance of KIAsia in Supporting the Consultancy: Firm was provided 4-5 volunteers from which to select. Consultant was pleased with performance of the interpreter, who also assisted the Evaluators. The SOW was extremely well-written and fully addressed the client's requirements. Major Recommendations of VE/Consultant: 1) Change management on spraying of chemicals by purchasing more efficient equipment, minimizing use of chemicals, 2) Optimize irrigation by reducing frequency and increasing duration of watering, 3) Improve placement of fertilizer for maximum utilization, 4) consider long-term shift from mandarin oranges due to competition from China, 5) Addition of a juice plant is not feasible. Client Efforts to Implement Recommendations: Client fully followed the first three recommendations and concurred with the non-feasibility of adding juice production to the operation. Client has also adopted some recommendations to achieve a smoother skin and better coloration of the fruit. Client also is looking at alternatives to mandarin oranges and has some new varieties in the nursery. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-16 Impacts on Firm Management and Financial Status: Evaluators did not review financial records but were informed that the consultancy has helped with more effective marketing, reducing marketing costs and increasing market share; reduced production costs. Special Observations Drawn from this Case Study: The amount of investment in this operation suggests it is not a small business. It is a firm that clearly could hire its own consultants at full market value. The consultant indicated he received no feedback on the consultancy. This may largely be due to the fact that the client has no English language capacity. In instances where there is capacity, the Evaluators noted there often was followup. Followup by KIAsia may have facilitated feedback between the consultant and the client. Additionally, with followup, KIAsia may have learned that the client is experiencing some disease problems in the operation. The client has been unable to identify the cause using normal industry contacts. Thus, this could have been an opportunity to market additional consulting services to a client that could well afford the services at full cost. Consultants’ Considered Judgment: Overall, this consultancy was very effective. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-17 BAC CASE STUDY Firm Name: Firm I Line of Business: Coffee house/restaurant chain Volunteer(s)/Consultants: The ACDI/VOCA VE is a restaurateur and consultant who resides in Minneapolis; The TVS volunteer is retired from a large firm in the food and beverage business. Services Provided by VE/Consultant: The international VE provided professional advice on international franchising, while the Thai volunteer offered professional consulting on overall management of the firm’s Thai operations. Timing: ACDI/VOCA: January 2002; TVCS: October 2001 – February 2002 Major Elements of SOW: ACDI/VOCA: 1) Aid the firm in developing a “solid” franchising strategy for the short and long term. 2) Improve the current franchising system. 3) Improve the business strategy in general. TVCS: 1) Review the organizational structure to ensure it fits the industry; 2) Lay out an effective branch management system; 3) Establish effective recruitment techniques; 3) Set up a training program; 4) Lay out overall logistic arrangements; 5) Lay out a plan for a central kitchen and production facilities; 6) Review and develop more appropriate marketing strategies; 7) Standardize the franchise system to support expansion in Thailand. How the Consultancy Was Initiated: The owner of Firm I had “heard about” KIAsia and the BAC, and contacted BAC to inquire about both international and local consulting services. Managers wanted an international VE who was knowledgeable about franchising, but also about food and beverages services. Performance of the VE/Consultant(s): The international VE is warm, enthusiastic, creative and a businessman himself. He was able to learn a great deal through frequent informal interactions with the owner, and to give much of his advice in oral form. He did some invaluable work before his arrival, and was able to advise the client on adoption of the most useful features of industry standards in the U.S. for corporate/franchisee working arrangements. The TVCS consultant saw his role as a “professionalizing” task, helping senior managers to spend more time planning for the future, less on day-to-day challenges; giving less emphasis to the owner’s vision, more to customer preferences; and thinking realistically about cost and quality. Working Relationship Between Firm and VE/Consultant: The working relationship between the firm and the international VE has been excellent. They continue to be in touch frequently by email. The VE continues to advise the owner and his American in-house consultant on the feasibility of establishing a branch in Minneapolis. However, in retrospect, both the VE and Firm I senior managers perceive they made an error in where they placed the VE. He was placed in s “private” office suite where the owner and consultant usually work, rather than in the central corporate office. The latter would have assured far more frequent, more casual, and perhaps more productive interaction with Firm I’s entire senior staff. The TVCS consultant did work at Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-18 the corporate office, and had better opportunities to learn about and proffer advice about management of the Thai operation. He also worked one day a week over a five month period, offering more time for him to follow up on his ideas with managers. Performance of KIAsia in Supporting the Consultancy: The international VE noted that the BAC staff “did a nice job on the scope of work.” The Thai staff made “fabulous” arrangements, and overall the performance of BAC was “very professional.” The TVCS consultant noted the value of KIAsia in carrying out his work. He found it necessary to “re-prioritize” his SOW as he proceeded with the consultancy, but did not see this as a problem, since there was ample time to make adjustments. Major Recommendations of VE/Consultant: A rather simple SOW was used as a basis for offering a wide range of management recommendations. Among them: 1) Revise the mission statement, to better reflect the corporate culture. 2) Introduce bonuses for senior managers. 3) Initiate training in financial analysis for managers. 4) Introduce monthly staff meetings for managers. 5) Introduce incentives for meeting sales and margin objectives. 6) Include managers in annual budgeting, and require more frequent reporting on financial results. 7) Consider a larger marketing budget. The most specific recommendations are on international franchising. The VE provided drafts of some of the key documents. One specific recommendation was to include a clause in franchise agreements laying out in detail procedures for renewing the franchise. Many recommendations were patterned on the Uniform Franchise Offering Circular that is used in the U.S. to ensure transparency in dealings between corporations and potential franchisees. Client Efforts to Implement Recommendations: The suggested clause on franchise renewal was immediately adopted. A new vision statement has been developed. Other recommended management changes, some of them echoed by the TVCS consultant, are being implemented gradually. For example, new operating manuals are being prepared and distributed. As noted below, recommendations of the TVCS consultant have been implemented in many cases. Impacts on Firm Management and Financial Status: Senior managers of Firm I made an interesting point about the value of the international consultancy. The main benefit, they insist, is that they received assurance from an international expert that they are on the right track, that they had a product, the management expertise, and a proposed franchising agreement that would be able to perform adequately in international markets. The firm now has four international franchisees in the Asian region, anticipates having seven by the end of 2003, and is considering expansion to the U.S. Recommendations of the TVCS consultant are operational and nitty-gritty, and have been widely adopted. For example, menus have been revised in accordance with compiled data on customer satisfaction. And standardized operating manuals are being put in place for all branches and franchises. It is worth noting that, about the time the two consultancies were completed, the firm was able to record a profit for the first time. The firm has grown to 92 outlets in Thailand, about two-thirds of them franchises. Special Observations Drawn from this Case Study: One of the most interesting insights gained from this case study is the difference between a “true entrepreneur” and an effective corporate manager. The former may be more interested in a business breakthrough (such as being the first Thai restaurant chain to break into the U.S. market) or an aesthetic accomplishment (some rather sophisticated Thai dishes on the menu) than on controlling costs Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-19 and optimizing the bottom line. It may be worthwhile for BAC to construct a checklist or profiling device that detects when the client is a “true entrepreneur” and then design the consultancy accordingly. Another interesting point arose from the contrast between the international VE consultancy and the TVS consultancy. The former is necessarily restricted to intense interaction over a few weeks, while the latter can be carried out one day per week over a period of months. It may be worthwhile for BAC to examine more carefully the implications and best tactics for each approach. Evaluators’ Considered Judgment: Overall, these two consultancies have been effective. However, giving client firms “confidence” they are on the right track may be, in general, a questionable use of scarce U.S. government funds. Another concern is that the firm (with 92 branches and franchises in Thailand) stretches the definition of “small and medium” enterprise, and might reasonably have paid the full cost of consultancies. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-20 BAC CASE STUDY Firm Name: Firm J Line of Business: Publication of a tourist magazine and a tourism website. Volunteer(s)/Consultants: This activity was undertaken in-house by a BAC employee. Services Provided by VE/Consultant: Services focused on marketing, in particular how to increase advertising revenues both from the magazine and the website. Timing: October-November 2000. Major Elements of SOW: The magazine, established in 1986, and profitable from a few months later, had no competition until 1997. Now, there are about 10 competitors. Thus, the purpose of the consultancy was to: 1) Achieve greater name recognition for the magazine through public relations events, such as sponsorships and charitable activities; 2) Conduct research on competitor publications to better understand how to make future strategic decisions; 3) Analyze current magazine distribution and adjust to better reach target subscribers; 4) Analyze the magazine’s content to make changes that will increase readership and lower costs. Note: The consultant’s report had recommendations on management of the firm’s website that appear quite useful, although these appear to not have been included in the original scope of work. How the Consultancy Was Initiated: The magazine’s editor had attended a seminar for businesswomen, and heard a presentation about services offered by the BAC. She was quite impressed that the director knew former Prime Minister Anand. She followed up with a request to BAC for assistance. Performance of the VE/Consultant(s): The in-house consultant has written a report of variable quality. Some of the “recommendations” seem too general to be of much use: e.g., to “carry out analyses” and “examine the firm’s core competencies.” But some appear quite useful, particularly recommendations on improving the marketability of the firm’s website for scroll advertising. Working Relationship Between Firm and VE/Consultant: The client firm expressed considerable satisfaction with the fact that advisory services had become available. In a report, Kenan staff noted that positive statements were offered about the services: “it is quite rare for an organization such as Kenan to provide assistance to companies up-country, and they were very impressed with the quality of our service.” The report concludes that the firm “plans to implement many of the BAC’s recommendations shortly.” Performance of KIAsia in Supporting the Consultancy: KIAsia staff appear to have responded quickly and with enthusiasm to this request for services. Major Recommendations of VE/Consultant: The recommendations fell in two areas: 1) Undertaking efforts to ensure the magazine is widely-distributed and widely-read, and that current and potential advertisers readily recognize it is widely distributed. (Specific suggestions include reader surveys to improve content, and a survey of the competition to understand better Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-21 how to compete with them). To some extent, these recommendations “feed back” the original scope of work, rather than advancing concrete new suggestions. But, more original recommendations are offered on upgrading and improving the profitability of the web page. 2) The recommendations focus on making the website more interactive, thus increasing repeat visitors and overall usage, and in turn making the site more attractive to advertisers. Specific recommendations focused on establishing a user forum, analyzing web server statistics to determine which current content is most popular (and provide more content of that type), and surveying website visitors to determine their preferences for future content. Client Efforts to Implement Recommendations: The Evaluators note that the firm’s managers were relatively slow in implementing recommendations regarding the website. In fact, when the owners were interviewed in mid-August 2003, they noted that they found implementing these suggestions “beyond their capability” to implement, and furthermore that they didn’t want to offend their current webmaster by imposing recommendations from another source. However, the Evaluators note with surprise and pleasure, that some of the recommendations have been implemented in recent weeks. New features of the site include: opportunities for visitors to recommend new content, critique current features of the site, and request information and feedback on proposed itineraries, etc. The site has become considerably more interactive. Impacts on Firm Management and Financial Status: The firm continues to experience fierce competition from other publications, although there is no directly-competitive website. Perhaps some of the recommendations from the BAC consultancy are helping the firm to stay in business. Special Observations Drawn from this Case Study: Evaluators’ Considered Judgment: The Evaluators think of this consultancy as a “belated success,” since some recommendations appear to have been accepted almost three years after they were offered. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-22 BAC CASE STUDY Firm Name: Firm K Line of Business: This firm is an “ethical” producer of CDs. That is, they ensure intellectual property rights are protected. They receive production orders from many other firms for music or data CDs. Volunteer(s)/Consultant: International VE under a special program of IESC—the “Geek Corps” Services Provided by VE/Consultant: The VE provided IT services—a new program to integrate order taking and preparation of bills, and also offered general guidance on future software procurement. Timing: October 2002 Major Elements of SOW: The original SOW negotiated with BAC specified an expert VE to write a software program integrating the firm’s ordering, vouchering, and accounting software. When the VE arrived, it became apparent that this task was impossible. Neither BAC staff nor Firm K managers had possessed the specialized knowledge required to approach the Thai firm supporting the accounting software, and gain their agreement to release information required for the work of integration to proceed. How the Consultancy Was Initiated: Firm managers were concerned that their computer software did not effectively integrate systems for recording orders, billing, and accounting, thus requiring repeated, labor-intensive re-entry of the same information. Performance of the VE/Consultant(s): On his arrival, the VE determined that the computer firm lacked technical knowledge to adequately support its proprietary accounting software. The firm declined to provide access codes based on the argument they had proprietary rights to the software. After this setback, the VE undertook the simpler task of writing a program to integrate Firm K’s order taking with billing, enabling employees to type orders directly on the computer, and later use the same data to produce bills. Interestingly, the VE took a one week break in the middle of his consultancy (working for one week before and one after) to travel in Thailand with his wife. This was not seen as a problem. Working Relationship Between Firm and VE/Consultant: The managers of the client firm expressed great admiration for the professional skills of the VE. He, on the other hand, as noted below was disappointed that there was little interest in his proposed efforts to achieve “knowledge transfer” with the firm’s IT manager and consultant. It appears the working relationship was cordial, but not particularly close. Performance of KIAsia in Supporting the Consultancy: The MD and plant manager noted that “our relationship with Kenan was very good.” They were “very professional.” Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-23 Major Recommendations of VE/Consultant: By and large, this was not a “recommendation￾based” assessment. Instead, the VE wrote a piece of software, and imparted broad knowledge about buying software in the future. Client Efforts to Implement Recommendations: The firm is using the new software, no doubt with moderate positive results on the overall balance sheet. Impacts on Firm Management and Financial Status: The consultancy did result in introducing modest new efficiencies into the firm’s system for order taking and billing. This presumably has made some modest reductions in production costs, e.g., by reducing manpower requirements by one person or so. It has probably also introduced greater rigor into the billing process, ensuring that there are fewer overdue bills, and improving cash flow. However, as discussed below, the biggest impact in the eyes of firm management is in their improved knowledge regarding software procurement. Special Observations Drawn from this Case Study: There was a very interesting discussion with the firms MD and plant manager about a possible product that BAC could offer. The informants agreed that, in retrospect, the most useful part of the consultancy from their viewpoint was that they are now far better informed (perhaps also less naïve and more demanding) about purchase of major software systems. Not only does such a system have an initial price tag of B200,000-500,-000, it also has long range impacts on the operation of the firm, in effect “costing” or “earning” millions of baht as it performs effectively or poorly. As we talked we came up with the idea of a two- or three-pronged program for SMEs: 1) One element would be a public seminar on the general challenges and pitfalls of buying a major software system (perhaps with special emphasis on accounting software); 2) At the end of the seminar, provision of a “checklist” that could be used be attendees in dealing with software purveyors (e.g., it could be used to ask a list of specific questions); 3) Using the knowledge gained in the seminar, and information garnered through use of the checklist, SME managers could also send brief email information requests to the trainer/advisor. Evaluators’ Considered Judgment: This consultancy had mixed results. The managing director and plant manager were very satisfied with the VE’s work, and highly complimentary regarding his professional qualifications. In retrospect, they believe the main benefit was in lessons learned for the future about procuring major software systems. On the other hand the VE was relatively dissatisfied with the consultancy, expressing his dismay that Firm K’s managers discouraged him from including the IT systems supervisor and a part-time programming consultant in his work. As a result, he observed, there was no “knowledge transfer.” He noted that he had provided little more than “cheap labor” for a task that could easily have been performed by the firm’s part-time IT consultant. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-24 BAC CASE STUDY Firm Name: FIRM L Line of Business: Production of parts for assembly plants and replacement parts. Volunteer(s)/Consultants: Two BAC in-house consultants. Services Provided by VE/Consultant: General management consultancy on firm reorganization. Timing: September-November 2002 Major Elements of SOW: Firm L is a family-owned firm that manufactures parts for auto assembly, as well as replacement parts. The firm’s sales were severely affected in the late 1990s by the economic crisis, and have begun to recover significantly only in the past year. In 2002, the firm applied to the Ministry of Industry’s ITB program for assistance in developing a more appropriate marketing strategy. During assessment visits, the BAC team recommended that the consultancy not focus on marketing issues, but on the broader challenge of converting Firm L from a conservative, family-managed firm to a modern, professional corporation. This is probably an insightful judgment about the nature of the firm’s problems, and what must be done to achieve long-term success. Some family members involved in the firm agreed that the consultancy should have this focus. However, it appears to be an over-ambitious goal for a consultancy to undertaken in about 30 person-days by recent MBA graduates. How the Consultancy Was Initiated: Firm L’s ITB application was submitted in March 2002 and approved in April. After the consultancy was contracted to BAC, two in-house consultants were fielded in July. Performance of the VE/Consultant(s): Unfortunately, services provided by BAC were significantly hampered by timing problems, some the result of outside forces, some caused by slow reaction time within BAC. The pair of consultants were able to visit for only a total of 15- 20 times, visits which the informant described as half-day visits. Working Relationship Between Firm and VE/Consultant: The working relationship doesn’t appear to have been particularly close. Performance of KIAsia in Supporting the Consultancy: In this case, KIAsia provided both the support services and the consulting services. Problems of timing and an over-ambitious SOW are mentioned elsewhere. Major Recommendations of VE/Consultant: It is not clear that there were any specific recommendations beyond those encapsulated in a proposed new organization chart. Client Efforts to Implement Recommendations: Although there was general agreement among the six brothers involved in the firm on the need for reform, there was no agreement on specific steps that needed to be taken. Thus, there was no real basis for action. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-25 Impacts on Firm Management and Financial Status: Impacts on the firm were nil. The main informant referred to production on a proposed organization chart for the firm, nothing more. Special Observations Drawn from this Case Study: Any organization that desires to be successful in SME consulting in Thailand probably needs to build its professional competence in the specialized area of “converting Chinese family firms into modern corporations.” This area of practice must necessarily go well beyond normal procedures in which problems are diagnosed and recommendations are offered. The consultancy becomes a complex psycho-social process involving individual counseling; intense group processes for analysis, decision making, and commitment; and regular follow through. It also implies a substantial knowledge base—of Thai￾Chinese culture, typical family business practices, etc. It may be useful to have consultants on the roster who have been through similar processes in their own family firms. Evaluators’ Considered Judgment: This was not an effective consultancy. Soon after starting work at Firm L in July, the consultant team leader resigned from BAC. A new team was selected and began work only in September. They faced two challenges: 1) Their time was severely limited. Under Ministry of Industry regulations, ITB consultancies must be completed within the fiscal year, and the BAC consultants were required to complete their field work by November. This allowed only 15-20 half-day visits by the team. 2) Their task was over-ambitious. Firm L is operated as a family business by the founder and his six sons. While there was agreement among the younger generation on the need for reforms, each brother had a different view of what form they should take. The consultants devoted their limited time to learning the details of firm management, with little time left for the difficult task of defining workable alternatives, and the even more daunting challenge of getting six brothers and their father to agree on a single reform strategy. In the end, according to the brother most concerned with reform, the BAC produced “a new organization chart,” nothing more. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-26 BAC CASE STUDY Firm Name: Firm M Line of Business: Traditional Dance shows coupled with traditional Thai Khantoke dinners. Volunteer(s)/Consultants: International volunteer from CDC Services Provided by VE/Consultant: Marketing Timing: July 2002 Major Elements of SOW: 1) Linkage to marketing channels in the US, 2) Marketing within Thailand, 3) Marketing strategy for promotion abroad, 4) Suggest improvements in restaurant operations that will enhance marketing efforts. How the Consultancy Was Initiated: Firm M is one of a number of businesses owned by a holding company. One the holding company's directors had experienced BAC consultancy on a company the director owned. The director mentioned BAC services to the company's president who took the lead on contacting the BAC. Performance of the VE/Consultant(s): The consultant did an outstanding job in fulfilling the reasonable portions of the SOW and went above and beyond the work required. Working Relationship Between Firm and VE/Consultant: The consultant and client worked closely together to maximize the value of the consultancy Performance of KIAsia in Supporting the Consultancy: BAC provided all the support needed to make the consultancy successful. The SOW was rather fuzzy and appeared to request assistance (restaurant management) that was not necessarily consistent with the balance of the exercise. Major Recommendations of VE/Consultant: Developed the framework for a sound marketing plan with a description of the necessary elements of each part. Client Efforts to Implement Recommendations: Client appears to have leaned on consultant to do a lot of the work that should have been done by client, i.e. contacting US tour operators etc. Consultant spoon fed the client with much self help material and there is little evidence that this was followed. Impacts on Firm Management and Financial Status: The Evaluators' impressions are that the President of the holding company is the key decision maker for Firm M. Therefore it appears that there was little if any impact on firm management. While the Evaluators did not have access to financial information, their best judgment is that there was no financial impact due to non￾implementation. Expansion presently underway without market share appears to be steering this client toward financial difficulties. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-27 Special Observations Drawn from this Case Study: This business appears to be out of the purview of the definition of "small" considering that it is a part of the holding company. Considering the capital investment in the business at the time of the consultancy and the considerable subsequent investment, paying the full cost of consultancy would have been quite easy for the client. Client complained of a limited market for high end tourism dinners yet was radically expanding the capacity of the facility which is not presently fully utilized. Consultants’ Considered Judgment: The consultancy was a failure not because of the work product which was excellent but due to the lack of implementation. The funds expended on this business would have been much better directed toward more eligible clients. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-28 BSOP CASE STUDY Partnership Title: Mediation Center for Financial Disputes Thai Partner’s: Office of the Judiciary: Alternative Dispute Resolution Office Brief Overview of Partner's Role in the Thai Economy: The court system, traditionally slow and unwieldy, has become a stumbling block in clearing up NPLs and other business debt that was associated with the Asian financial crisis. Setting up mediation services to more quickly settle financial disputes is a useful way to clear up a large case backlog, and to get the disputant businesses moving again. American Partner/s: American Bankruptcy Institute Beginning Date: September 2001 Rationale for the Partnership: Two distinct circumstances that shaped Thailand in 1997—the financial crisis and approval of the new constitution—led to concerted efforts within the judiciary to undertake wide reaching reforms. In 1999, Thai court reformers in the Office of the Judiciary (OJ) and members of the American Bankruptcy Institute (ABI) with an interest in Thailand established contact. Objectives: They were seeking an appropriate starting point for reforms to speed up processing of the enormous backlog of financial cases created by the 1997 crisis. They arrived at the concept of a mediation training program, to supply non-judge mediators to work with the Alternative Dispute Resolution Office in Bangkok. Eventually, this would also lead to training a cadre of mediators to work with provincial courts across the country. Activities: While there is widespread recognition that reforms will take decades, not months or years, promising steps have been taken. A cadre of young judges have campaigned with subtle aggression, using their contacts with senior judges, carefully designed pilot studies, well-drafted professional publications, and support from U.S. partner organizations to begin shifting minds and administrative systems. Specific activities included: 1. Training of 100 “mediators and financial advisors” (with varying backgrounds in law and finance) was accomplished during two three-day workshops in September 2001. Many of those trainees have served the Mediation Center in Bangkok well. 2. A follow-on, three-day, train-the-trainers workshop was carried out in January 2002. This session was for 20 participants in the previous round who were deemed promising as trainers. Later those individuals took on the task of training hundreds of mediators who are now associated with provincial courts, in most cases as unpaid volunteers. Later rounds of training were financed under the World Bank’s ASEM project. In addition, also under ASEM auspices, a senior expert on mediation from the Federal Judicial Center in Washington took a leave of absence to join the Office of the Judiciary for a few months. He prepared a mediation handbook and organized additional training. As a result, mediation skills have been spread widely across Thailand. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-29 3. ”Mediation mentorships” provided four employees of the Mediation Center two weeks of on-the-job training in U.S. mediation centers. Performance of the Partners: While KIAsia-funded mediation training ended with these activities, as a result of excellent donor collaboration, neither the partnership nor the program has ended. Additional training sessions, using many of the same trainers, have continued under ASEM. Evidence of continuing partnership relationship, if any: There are ample indications that the Office of the Judiciary will continue their working partnership with the ABI, and other U.S. institutions such as the Federal Judicial Center and American Bar Association. Performance of KIAsia in Supporting the Partnership: KIAsia support for judicial reform in Thailand is a complex, long-term, and heartening process. KIAsia has played a crucial role by providing modest, but timely, support for several of the activities. With an OJ/ABI partnership already established, the organizations approached KIAsia for financial and administrative support. A program of three components was agreed to and implemented. Impact on the Thai Partner and Thai Economy: In 2002, settlements from 500 cases involving B505 million ($12.3 million) were concluded, In the first six months of 2003, 184 cases involving B674 million ($16.4 million) were concluded. This represents a significant reduction in case load for the formal court system, and a significant flow of funds into the economy. Special Observations Drawn from this Case Study: 1) This case reminds us there is often a degree of serendipity in the success of such partnerships. In this case, an activist American lawyer has a strong interest in Thailand because of his Thai-American wife, also a lawyer. This was an important factor in establishing the partnership in the first place, but the partnership now has a strong institutional base as well. 2) The follow-on work by the World Bank’s ASEM project is almost certainly an outgrowth of the excellent donor coordination, particularly between the Bank and KIAsia, which has led to seamless continuation of activities in this partnership. 3) Finally, the importance of the creative, energetic work carried out by a handful of youthful Thai judge-reformers in making this partnership work cannot be overlooked. However, it is important to observe, as above, that this partnership now appears to have been institutionalized. Evaluators’ Considered Judgment: This partnership has been a solid success. In addition to evidence on caseload reduction and size of judgments achieved in Bangkok that is offered above, there is evidence that there can be a strong impact on provincial courts. In a particularly significant pilot program, ASEM has provided modest (B15,000) monthly honoraria to four recently-trained mediators associated with the Chiangmai Provincial Court. Preliminary data suggest this approach can have significant impact on case loads for the courts and provide more timely justice to plaintiffs and defendants willing to try a new approach. In the brief period of four months that the mediators have taken on cases, 125 cases have been successfully mediated, nearly a quarter of the financial disputes that reached the court during the period. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-30 BSOP CASE STUDY Partnership Title: Better Process Control School Thai Partners: King Mongkut’s University of Technology, Thonburi; NECTEC; other universities Brief Overview of Partner's Role in the Thai Economy: The King Mongkut’s University of Technology is one of Thailand’s major technological universities. It is very appropriate for this institution to sponsor a program that enables Thai food processing firms to upgrade their technical capacity and enter the U.S. export market. American Partners: Purdue University; U.S. Food and Drug Administration Beginning Date: May 2002 Cost to KIAsia: $35,000 Cost to Partners: $18,220 (In fact, participants in the course are providing the bulk of funding.) Rationale for the Partnership: FDA-approved Better Process Control Schools have long been offered by land grant universities in the U.S. to train food processors on safe methods to prepare canned goods. In recent years, overseas officials have begun attending BPCSs to gain information needed by canned food exporters in their home countries to successfully enter the U.S. market. The courses are sometimes offered overseas, for example in the West Indies and Sweden. But, overseas courses are generally offered by U.S. institutions. Objectives: The idea of establishing a BPCS in Thailand, in a Thai university, grew out of the attendance of three Thai food science professors at the BPCS at Purdue University in Indiana. They discussed the idea with a Professor Emeritus at Purdue. The concept was that Purdue would offer the course the first year, and transfer needed teaching and administrative skills in the second and third year. It was decided King Mongkut’s University of Technology, Thonburi (KMUTT) would take the lead. Activities: The course was repeated three times with the involvement of Purdue University. Now, it can be delivered independently. Participants hear lectures on 16 topics, all offered by experts from Thai universities or industry, take exams, and in most cases, receive certification from the USFDA as “supervisors of thermal processing systems, acidification, and container closure evaluation programs for low-acid and canned goods.” Performance of the Partners: The partnership between Purdue University and the Thai universities has proceeded smoothly. Purdue organized and taught the first course, while the second year was transitional, and local professors and experts from the food processing industry took full responsibility for the third course. Evidence of continuing partnership relationship, if any: The emeritus professor at Purdue University and his counterpart at KMUTT continue to be in contact. In addition, the partnership with the US FDA is firmly established. E.g. The annual course at KMUTT (in May) is advertised in the FDA’s catalogue of Better Process Control Schools. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-31 Performance of KIAsia in Supporting the Partnership: The professors asked an official of the National Center for Genetic Engineering and Biotechnology, to approach KIAsia for funding of the second and third year courses, to ensure sufficient time for participating Thai professors and institutions to master their roles. BSOP staff recognized the value of this straightforward program, and agreed to modest funding to cover costs of visits by Purdue University and USFDA staff. Impact on the Thai Partner and Thai Economy: After three years, the BPCS has a firm footing in Thailand. The presence of a certified technician on the assembly line at all times is a condition for export of low acid canned goods to the U.S. In addition, knowledge gained during the course minimizes the likelihood exporters will make processing or paperwork errors that cause consignments to be rejected or delayed. Special Observations Drawn from this Case Study: In addition to provision of matching in￾kind resources by KMUTT and the other academic institutions, tuition payments by course participants --B19,500 or $460 in the first year, B15,000 later--covered a large portion of costs. However, while participants are paying fees almost sufficient to ensure full cost recovery, there are a few issues that make “closing the gap” somewhat difficult: 1)Generally, a USFDA official must attend the course to write an assessment and ensure teaching content and testing procedures are sound. Bringing an FDA official each year is a major budget item, and there are explorations of an alternative approach in which officials of the Thai FDA would play the certifying role. USFDA could continue to be involved by providing period technical/procedural updates. 2)Thai professors are somewhat resistant to full cost recovery, perhaps feeling that government￾sponsored programs (the BPCS is offered by public universities) should be subsidized to some degree. Thus the decision was made to lower tuition in the later years. 3)However, to some degree their decision may have been shaped by the presence of a similar course in the “marketplace” being offered by the Canadian government at B6000, taught by visiting Canadians. While the latter course cannot promise access to the U.S. market, it serves some of the same purposes at a lower cost. This is an instance in which one donor-supported activity is undercutting the sustainability of another. Evaluators’ Considered Judgment: This partnership has already a considerable degree of success. It will achieve full success when the course if offered on a full cost-recovery basis. The fourth annual BPCS in Thailand is scheduled for May, 2004. As appropriate, and if the market response is adequate, KMUTT can offer the course on a regional basis as well, providing training in English. The regional course could be offered every second year, alternating with the course in Thai, or offered annually, along with the Thai course. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-32 BSOP CASE STUDY Partnership Title: Capacity Building for Dispute Resolution Thai Partners: Office of the Judiciary (OJ); Alternative Dispute Resolution Office (ADRO) Brief Overview of Partner's Role in the Thai Economy: The court system, traditionally slow and unwieldy, has become a stumbling block in clearing up NPLs and other business debt that was associated with the Asian financial crisis. Efforts to speed up judicial processes in general will be a good thing for the Thai economy, and for the citizenry at large. American Partner: Federal Judicial Center (FJC) Beginning Date: October 2001 Cost to KIAsia: $121,677 Cost to Partners: $176,650 Rationale for the Partnership: Two distinct circumstances that shaped Thailand in 1997—the financial crisis and approval of the new constitution—led to concerted efforts within the judiciary to undertake wide reaching reforms. A partnership between reformers in the OJ and the FJC puts the former in touch with a large number of organizations and individuals that have been intimately involved reform of the U.S. courts in recent decades. Objectives: The concern of internal court reformers is to improve and speed up court procedures, thus ensuring more timely and transparent justice. For each innovation, this involves pilot tests, assessment of results, broad dissemination of information on results, and putting new regulations and procedures in place to spread results across the country. Some innovations that have been introduced in this way (or are currently “in process”) include: 1)introduction of a “fast track” for simpler cases; 2)introduction of pre-trial conferences for more complex cases, to agree on issues, and establish a schedule for completion of the trial; 3)introduction of “court annex” mediation services (described in case study of “mediation center for financial disputes;”4)continuous trials; and 5)use of state-of-the-art digital equipment for recording testimony. Activities: BSOP support for these activities is complex and varied, and is matched by significant inputs from the Office of the Judiciary: 1. BSOP has supported an extensive publication program that has allowed OJ reformers to publish materials in journal form that lay out the results of various pilot tests, offer comparative data from other countries, and make a case for specific reforms. 2. A visit by the head of the FJC and judges and lawyers from other courts and legal firms to inform Thai judges, administrators, and attorneys about U.S. efforts to reform and improve case management (September 2002). 3. A visit by a federal judge and federal court administrator to discuss the introduction of digital audio recording technology (DART) in the U.S. (August 2003). Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-33 Performance of the Partners: A small number of Thai judges have been empowered to undertake reform activities, and with support from the Thai government, KIAsia, and the FJC they have undertaken a series of useful steps toward reform. Officials of the FJC, and their contacts spread across the judiciary and the entire legal profession have been able to provide appropriate information as needed. Evidence of continuing partnership relationship, if any: The Thai reformers have strong English skills, a broad knowledge of reform processes from around the world, and ample confidence. It seems obvious they will continue to call on U.S. partners for advice and information well into the future. However, the working relationship will be significantly enhanced if KIAsia (or some other institution) can continue to provide modest funding for international travel, information campaigns, etc. Performance of KIAsia in Supporting the Partnership: Reformist judges cite the importance of KIAsia support on a broad range of activities, including drafting of the Secured Transaction and Class Action laws, and well as the two case studies that are included here. Impact on the Thai Partner and Thai Economy: It has already been argued in another case study that court annex mediation is beginning to have a significant impact on the justice system. It seems very likely that the other reforms discussed here — the “fast track” system; pre-trial conferences; and introduction of digital recording systems — will have further impacts, making the system more efficient, equitable, and transparent. Special Observations Drawn from this Case Study: As noted in other case studies involving U.S. government partners, the FJC lacks even modest funds to establish and maintain this type of partnership. Without very modest KIAsia support for international travel, it is unlikely this partnership could have accomplished anything significant, and indeed it is not clear it could have been established at all. Evaluators’ Considered Judgment: Much of the work of this partnership is still in process. But the evaluators are favorably impressed by what has happened so far, and optimistic that the specific activities described here will have positive impacts. The willingness of hundreds of senior judges and administrators to attend public seminars (as they did in late August to listen to a federal judge and court administrator) suggest that commitment to court reform is actually quite substantial. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-34 BSOP CASE STUDY Partnership Title: Performance Audit Workshop—Office of the Auditor General Thai Partners: Office of the Auditor General (OAG) Brief Overview of Partner's Role in the Thai Economy: The Office of the Auditor General is the supreme audit organization of the Royal Thai Government, as such responsible for ensuring that expenditures of government agencies, state-owned enterprises, and government-funded projects are consistent with regulations and effective in achieving desired results. American Partners: General Accounting Office (GAO) Beginning Date: March 2001 Cost to KIAsia: $26,410 Rationale for the Partnership: The Thai Office of the Auditor General (OAG) was given substantially expanded powers and responsibilities under the Thai Constitution of 1997 and the State Audit Act of 1999. Among the most important new responsibilities is undertaking performance audits (in addition to more traditional financial audits.) The partnership began when the OAG applied to USAID for funds to send some staffers to GAO training in the U.S. Objectives: After training, OAG auditors were expected to use the performance audit approach to improve financial governance and transparency in government agencies, state-owned enterprises, and government-funded NGOs. Activities: Utilizing materials available from the International Organization of Supreme Audit Institutions, OAG and GAO collaborated in designing a two-week performance audit workshop that was then delivered by a retired GAO professional. The course, designed as a training-of￾trainers session for 36 seasoned professionals, included exercises and casework appropriate to participants already experienced as financial auditors. A special three-day module focused on building instructional skills, so that participants could pass on their newly acquired knowledge and skills to OAG auditors across Thailand. With separate funding, the initial course was refined and offered to 40 other auditor-trainers. The training program was then delivered at OAG regional offices to 200 auditors, using the modified materials. Also with KIAsia support, the auditing standards of INTOSAI were translated into Thai for use as an OAG reference publication. One senior auditor also undertook six weeks of on the job training in the Chicago office of GAO. He now functions as a senior trainer/advisor on performance audits and related matters. Performance of the Partners: Collaboration of the partners began well in advance of the training sessions, and OAG and GAO staff jointly evaluated the INTOSAI materials and adapted them for use in Thailand. Evidence of continuing partnership relationship, if any: A study tour for OAG officials took place in Washington in July of this year. OAG officials got to know their American counterparts Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-35 well, and frequently cited their names and their views. GAO officials were active in designing the tour agenda, and hosted OAG visitors for a full day. Performance of KIAsia in Supporting the Partnership: Upon the OAG’s request, KIAsia located the U.S. General Accounting Office (GAO) as an appropriate partner to provide training on performance auditing and render other assistance. Impact on the Thai Partner and Thai Economy: OAG officials reported that since this round of training was completed in 2001, performance auditing has been well established within their organization. Performance audits are performed regularly, and their results and recommendations are frequently published on the OAG web page (). They also noted that they have expanded their overseas networks, improved public relations, and facilitated information exchange. One senior OAG official noted that as a result of performance audits “We have saved the government billions of baht.” Cases that were cited include a ring road project that was cancelled, corruption in procurement of milk for school children, an unneeded rubber curing plant, and under-utilized storage silos. Training for performance audits is ongoing. Special Observations Drawn from this Case Study: The GAO has long been involved in offering training to employees of audit agencies from around the world. This is an effective activity, both from the viewpoint of creating higher professional standards, and from the viewpoint of creating good will for the U.S. However, we noted that the GAO is constrained from supporting additional international outreach activities, by lack of even modest funding for international work (such as participating in this partnership), and prohibitions on current GAO employees being involved. It may be appropriate for the USG to review these limiting policies, with an eye to how modest funding can be made available to undertake modest international partnerships. This partnership is now effectively being extended across Southeast Asia, as OAG establishes partnerships with counterpart audit organizations in Laos, Cambodia, and China. Evaluators’ Considered Judgment: We found this activity to be an “emerging success” in Thai/U.S. partnership. The wider availability of OAG performance audits will enable a range of participants in the political and economic systems, including parliamentary committees, local governments, academic researchers, and activist NGOs to review the performance of government agencies and others who expend government funds. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-36 BSOP CASE STUDY Partnership Title: Strategic Framework and Operational Plan--OAG Thai Partner: Office of the Auditor General (OAG) Brief Overview of Partner's Role in the Thai Economy: The Office of the Auditor General is the supreme audit organization of the Royal Thai Government, as such responsible for ensuring that expenditures of government agencies, state owned enterprises, and govern-funded projects are consistent with regulations and effective in achieving desired results. American Partners: General Accounting Office (GAO); Office of the Inspector General/Department of Transportation (IG/DOT) Beginning Date: Kenan/Washington discussions with GAO began in July 2002. Rationale for the Partnership: The GAO, which was already involved in the partnership with OAG, contacted the Inspector General at DOT. The latter had formerly worked with the GAO. Since the IG/DOT had recently been through a major internal planning exercise, it was thought that their involvement would be particularly valuable. Objectives: The other OAG partnership activity with the GAO, place more emphasis on building organizational skills in a key area—performance audits. This set of activities have a broader purpose, assisting the OAG, which has a new. Broader role as prescribed in the 1997 Constitution and related legislation, in re-defining itself and its work program. When a new Auditor General assumed the office in 2001, he was very interested in undertaking strategic planning. Activities: 1) A video conference took place in January 2003, to begin planning the activity; 2) A study tour for three senior managers of OAG and a professor from Thammasat University was arranged in July 2003. Participants met with senior officials of GAO and IG/DOT, and learned about strategic planning and agenda setting in U.S. audit organizations. 3) A senior official from IG/DOT (who has been directly involved in strategic planning in her agency) visited Thailand to confer with OAG officials and make a presentation at the planning workshop. 4) The same official will also review and comment on the draft strategic framework. A possible follow-on activity would involve the GAO and OAG in broader partnership activities with ASOSAI (the Asian association of supreme audit organizations). This would be timely, since the Thais currently chair this group for a three year period. Performance of the Partners: The IG/DOT official felt her time spent in OAG offices with operational level personnel was the most useful part of her visit to Thailand in late July 2003. There was a much freer flow of information in this environment than in the lecture she delivered at the workshop. She did not have her “own” translator, as one of the OAG higher-level personnel translated. This was not appropriate, because as an advisor/instructor she had little control over content and review of translation. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-37 Another problem that arose was that, despite her personal efforts to prepare for her lecture on weekends, so she could provide the slides and materials to the OAG staff PRIOR to her arrival, timely distribution of the materials did not occur. The materials were not translated into Thai, so OAG staff who are not comfortable in English were not familiar with the materials before the session. Evidence of continuing partnership relationship, if any: It is clear that IG/DOT staff are favorably disposed toward this partnership, and anticipate that it will continue for at least another year or two. The possible involvement of the GAO in activities of the Asia regional audit organization also indicates a long term commitment to the partnership. Performance of KIAsia in Supporting the Partnership: Kenan Washington facilitated initial contacts between OAG and GAO. This was then extended to IG/DOT. A videoconference served in gathering information to prepare for the assignment and was very helpful. However, there was a problem with translation of materials for the OAG workshop in late July, and the IG/DOT counterpart wondered aloud if BSOP staff should have taken stronger steps to assure timely translation. Impact on the Thai Partner and Thai Economy: Ideally, as a U.S. partner noted, the long term impact will be “reduced corruption, and an improved economy.” For the present, though, we can only track the process of firmly establish strategic planning in OAG. Along those lines, we note that: the strategic document was scheduled to be presented to OAG staff in late September, reviewed, and then implementation of key provisions to start soon after. BSOP staff will need to monitor how carefully and effectively the plan is implemented. Special Observations Drawn from this Case Study: As has been noted in at least one other case, the U.S. partner organizations lack even modest funding to undertake international partnership activities of this type, although it is clear that they are a low cost, effective approach to providing technical inputs, and also create significant good will for the U.S. In this case, it is a good thing that KIAsia was on hand, and AERA funds were available. Evaluators’ Considered Judgment: It is too soon to make a final judgment about this partnership, or even about this stage of the partnership. The strategic framework should be under review as this summary is being written, and the process of strategy review, finalization, and implementation of new working procedures for the OAG will be unfolding over the coming months. However, based on a meeting with several OAG staff, and a phone conversation with one of the American partners, the Evaluators are fairly confident this partnership will continue, and that the OAG will become a more proficient and influential organization within the Thai governance system. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-38 BSOP CASE STUDY Partnership Title: Building Capacity and Professional Standards for Property Valuation in Thailand Thai Partner: The Securities and Exchange Commission (SEC) Brief Overview of Partner's Role in the Thai Economy: SEC is an independent government agency responsible for the supervision and development of Thailand's capital markets, both primary and secondary. It formulates policies, rules and regulations related to supervision, promotion and development of securities business and listed companies. SEC maintains an approved list of valuation firms/practitioners (currently 30) for use by any firm listed or seeking to be listed for market purposes with SEC. This is in conformance with SEC's supervisory role over publicly traded companies in Thailand (this includes banks). This list is readily used as a reference by other parties interested in valuation services such as Thai Bankers Association, Bank of Thailand, real estate companies. American Partner: The Appraisal Foundation (TAF), Beginning Date: January 2002 Cost to KIAsia: $78,000 Cost to Partners: $19,000 Rationale for the Partnership: Thailand now has two associations for valuation of property (land and buildings) which follow not only different standards, but ones which are insufficient by international standards. There is no legislation regulating the valuation business; market information is quite limited and many valuers lack proper training. The combination of these factors contributed to the economic crisis of 1997 when properties were highly overvalued. These problems were identified in a "white paper" by the Royal Institute of Chartered Surveyors" (RCIS). The paper provided impetus for the World Bank and KIAsia to approach SEC with an offer of assistance with education and standards respectively. This is an example of the close working relationship between KIAsia and World Bank to coordinate activities thereby maximizing value from their limited funds. Objectives:  Develop a single set of valuation standards, a code of ethics, and an audit checklist for supervisory bodies to control valuation standard implementation in Thailand.  Provide the users of valuation reports, valuation practitioners, and regulators with equal understanding and encourage them to use the same set of best practice guidelines.  Build up the quality and competency of valuation practitioners in Thailand.  Improve the transparency and accuracy of appraisal so that end users can make better economic decisions regarding loans, investments, tax assessment, etc. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-39 Activities: 1. Conduct a comparative study of six countries valuation standards and the International Valuation Standards. 2. Recommendation report on appropriate standards for Thailand. 3. Drafting of standards, a code of ethics and an audit checklist by Thai expert team. TAF provided consultation support and edited the draft of the English version. 4. Review and comment by Thailand's two valuation associations 5. Revisions as appropriate and then public hearings overseen by SEC. Performance of the Partners: TAF and SEC have worked well in proceeding with activities to meet the objectives of this partnership, having completed four of five activities and are on track to finish the fifth activity. This is an especially good match of partners which has helped performance. Evidence of continuing partnership relationship, if any: SEC looks to further cooperation with TAF on the areas of valuation techniques and enforcement of standards. TAF believes there is need for review appraisers at SEC and this might be an area of continuing cooperation. Performance of KIAsia in Supporting the Partnership: KIAsia has been recognized both by TAF and SEC for its support. KIAsia's coordination with the World Bank in the role of each organization's assistance (standards development and training respectively) to SEC is especially noted. Impact on the Thai Partner and Thai Economy: All parties recognize the vital importance of sound valuation standards to the financial and banking sector. The standards can contribute positively to the economy, but the real test will be for them to not only be in place, but to be utilized and enforced. This is a longer term process and there is agreement on some regulation of the valuation industry, there is disagreement whether not should entail a self-regulation organization or a government type system. Special Observations Drawn from this Case Study: There are some similarities between the savings and loan crisis in the US during the late 80's- early 90's and the Thai economic crisis of 1997; improper valuation work contributed to both. Evaluators’ Considered Judgment: This partnership is on the path to success. Implementation and effective enforcement of valuation standards will be a measure of success that will not be readily apparent for a number of years. The needs of the industry extend beyond standards only, but this is a good start. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-40 BSOP CASE STUDY Partnership Title: Upgrading the Post and Telegraph Department (DPT) Staff to Serve as National Telecommunications Department Secretariat. Thai Partner: Post and Telegraph Department. (Soon the be transformed into secretariat for the National Telecommunications Commission.) Brief Overview of Partner's Role in the Thai Economy: In its new role as secretariat to the National Telecommunications Commission, the staff of the PTD will play an important role in ensuring the telecommunications sector operates in a rational, equitable, transparent manner. American Partner: Federal Communications Commission Beginning Project Date; Late 2001 Cost to KIAsia: $106,451 Rationale for the Partnership: FCC gets a regular stream of international visitors who pass through DC. Additionally, they have a budget for international meetings (such as APEC). They meet regulators from other countries at these meetings. In 2000, the Chairman fostered a development initiative for Partner Regulators from half a dozen countries, one of which was Thailand. This limited initiative was concluded after two years, but was considered high priority because developments in other countries' communication sectors directly affect the US. Objectives: During the FCC initiative, agreement was reached on about a dozen issues for cooperation, having to do with reform of the telecommunications sector in Thailand. Once Kenan became involved, the issues were used as a roadmap for setting a fairly lengthy program. Activities: Four successive activities were undertaken: 1. Teleconference: A total of four conferences were originally planned, though it appears only one was undertaken. The session focused on getting to know each other, and deciding how FCC can help. 2. Training — Regulatory Procedures; Convergence; January 2002; 53 participants (including trainees from private telecommunications companies). 3. Training — Dispute Resolution; June 2002; 42 participants. 4. Training — Spectrum Management; August 2003; 64 participants. Performance of the Partners: There are no evaluations of activities with foreign regulators by the FCC. However, the FCC gauges prospects for working effectively with an organization by: 1)The partnership engagement itself; who proposes this relationship and the issues to be addressed. It is best if the host country proposes this. This was the case in Thailand. 2) Number of people who attend workshops etc. and their level of engagement. At the first training Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-41 sessions, there were a lot of persons attending from many different levels and there was a lot of discussion. The FCC delivered what was requested, although there were some scheduling difficulties. For the last workshop, the FCC contact person was under the impression that it was not scheduled with enough lead time. This makes preparation difficult for both sides. Materials needed to be in Thailand well before the session to enable them to be translated and reviewed. Without this, it was a challenge to discuss issues in depth, which did not occur. Evidence of continuing partnership relationship, if any: It seems this partnership may be in a period of hiatus. The third round of training has been completed, and the members of the Commission have not been selected. There are even some indications that powerful people in the government (Prime Minister’s office?) are actively influencing allocation of broadcasting licenses. If this continues, it will be bad for Thailand, and will also make the new Secretariat’s role redundant and meaningless. If these problems are ironed out, there are promising opportunities to continue and strengthen the NTC/FCC partnership. Performance of KIAsia in Supporting the Partnership: In Asia, similar work was done elsewhere within the FCC’s Development Initiative. Thailand was the only country with outside help (from Kenan), and was more successful because of it. A natural match was established before Kenan became involved, but then Kenan was very supportive. The first workshop faced scheduling difficulties. It was hard getting on the Thai government calendar. FCC counterparts weren’t sure whether this was mainly a problem of the Thai government or with Kenan. Impact on the Thai Partner and Thai Economy: Despite its misleading name, the PTD has long played a regulatory role, if only the limited one of assigning bandwidth to other government departments (for re-assignment to the private sector, etc.) Thus, PTD staff are already accustomed to playing a limited regulatory-support role, though until now they have served a Director-General, rather than a Commission. They seem mentally prepared to take on their new role. 50 persons were engaged throughout the three days of the first training session. There were expectations that the National Commission would be selected for Thailand soon. This has not occurred and apparently there are problems in the sector now regarding concessions going to licensees. The Prime Minister's Office is handling this. Therefore, unless the situation is remedied, the role of the NTC will become irrelevant. Special Observations Drawn from this Case Study: Helping a government bureaucracy to fundamentally shift its function and operating style is always a daunting task, and has no doubt failed more often than it has succeeded. However, there may be at least two favorable circumstances in this case: 1)The DPT has already played a somewhat similar role. 2)Thai bureaucrats often seem more amenable to administrative discipline, and more committed to organizational goals than is the case in most other national bureaucracies in developing countries. Evaluators’ Considered Judgment: It is too soon to predict that this effort will be a success, but also the evidence is too scanty to suggest that it will fail. In effect, the effort and the partnership hang in the balance. But, there appears to be good will on the part of professional Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-42 staff, and if the RTG makes a firm commitment to balance and transparency in managing this part of the economy, the prospects for an effective DTC are good, and the need for a continuing partnership will be readily apparent. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-43 BSOP CASE STUDY Partnership Title: The Bond Market Development Thai Partner: Thai Bond Dealing Centre (BDC) Brief Overview of Partner's Role in the Thai Economy: American Partner: The Bond Market Association (TBMA) Beginning Project Date: May 2001 Cost to KIAsia: $30,334 Cost to Partners: $31,660 Rationale for the Partnership: Both partners business activities center on bonds. TBMA is a 27 year old trade association representing the US bond market industry and has international members. It is involved in advocacy, development and promotion of standard market practices and forms, investor and industry education. BDC, (formed in 1994) unlike TBMA does not have a lobbying role. It serves as an information center on bonds (has website) works on bond market development and serves somewhat the role of a Self Regulatory Organization (SRO). TBMA faced and solved many of the issues facing BDC so the partnership was well matched. Objectives:  Develop the bond market, especially the Repurchase Agreement (Repo) market in Thailand;  Educate Thai investors to better understand fixed income investment methods and risk awareness; and  Educate bankers and corporate issuers about fundraising alternatives. Activities: 1. Visit by 4 BDC staff to the US to learn more about functioning of an advanced bond market. Organizations visited included TBMA, the Securities and Exchange Commission, the Federal Reserve, and brokerages. 2. Adaptation and translation of TBMA guides and checklists into Thai (9,000 copies in original run). 3. Securitization conference in Bangkok. 4. Assistance with developing a Repo market in Thailand. Performance of the Partners: TBMA has performed its role well on very little funding. It has leveraged some of its own travel activities to enable it to be of greater service to the partnership. BDC has discharged its role in the partnership per plan, but three of four staff members who participated in the US training visit have left the organization. This decreases the chances for maximizing benefits from the partnership and for dialogue between TBMA and BDC, especially since they were all vice presidents. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-44 Evidence of continuing partnership relationship, if any: While there is some Email contact between the organizations, the movement of trained staff out of the organization reduces the chance for continuing partnership. Additionally, activities set up for the partnership have been completed. There is clearly great potential for future partnership activities as there is intense interest in the bond market in Thailand on the part of both government and the private sector. Performance of KIAsia in Supporting the Partnership: KIAsia made the key introductions to get this partnership into place. Both TBMA and BDC praised KIAsia's support. Impact on the Thai Partner and Thai Economy: Impact on BDC has been modest as 3/4 of the key staff, those who participated in the US study visit, have left the organization. Impact on outside persons and organizations has been better as:  250 persons were educated at the Securitization Conference,  25 key stakeholders participated in a roundtable discussion with a TMBA expert, and  TMBA investors guides were customized, translated and distributed to Thai individual and institutional investors (9,000 initial run and due to demand another 2,000). This partnership was very timely as there is tremendous interest in the bond market both in Thailand and in Asia. A well-run and thriving bond market will help finance the Thai economy and can mitigate the effect of financial crises. Special Observations Drawn from this Case Study: The turnover of senior personnel may indicate some difficulties within BDC. Accordingly, a simple continuation of partnership activity without that difficulty being first addressed would likely limit the effectiveness of the activity. Evaluators’ Considered Judgment: As structured the activities were successful. There is tremendous need for further assistance with the Thai bond market. Hopefully this partnership can continue to assist the Thai economy. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-45 BTP CASE STUDY Bank Name: Siam Commercial Bank (SCB) Type of Bank, Ranking in the Banking Sector: Commercial Bank, 4th largest Thai commercial bank Consulting Company: 1) Barents Group (Barents) 2) Pricewaterhouse Coopers Risk Management Ltd. (PwC) Services Provided by Consultant: 1) Risk management training 2) Senior executive risk management training Timing: 1) June 2000 2) November 2001- December 2002 Major Elements of SOW: Risk Management Training (Barents) 1. Develop risk management course and internal training manual w/training materials including case studies, text, transparencies, and program measurements. Course topics to include: market risk management, financial risk management, and operational risk management. 2. Delivery of a training of trainers course to select bank employees who would follow through with the training. 3. Deliver a Risk Management Workshop on a pilot basis using resources developed. Areas covered included market risk, financial risk, and operational risk. Risk Management for Executives (PwC) 1. Design and deliver risk management training to senior management and board members, including Board of Directors, President and Executive Vice Presidents, Senior Vice Presidents and Vice Presidents. Topics included: Enterprise-Wide Risk Management, Corporate Governance, Credit Risk Management, Market Risk Management, and Operational Risk Management. 2. Review SCB's risk management framework and provide recommendations to improve the framework. How the Consultancy Was Initiated: KIAsia conducted a rigorous campaign to offer BTP services to all Thai banks. SCB was interested in risk management training. After the initial consultancy and the dissemination of risk management training to middle management by trained SCB staff, the need to bring the training to senior management was realized. This lead to the second consultancy, especially after KIAsia witnessed the success of providing risk management training to senior management in other banks. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-46 Performance of the Consultant(s): SCB was pleased with the performance of both consultants, indicating high satisfaction with both in the training evaluations. Working Relationship Between Bank and Consultant: All indications are that there were very good working relationships. In the case of the second consultancy, there were a number of changes occurring at the bank and this prolonged delivery of the review of the risk management framework. Performance of KIAsia in Supporting the Consultancy: KIAsia was instrumental in facilitating SCB's articulation of its training needs in a scope of work for both consultancies. The first consultancy was procured under the SEGIR IQC. Since the USAID contract officer was based in Phenom Penh, this assistance enabled USAID to expeditiously compete a task order under the SEGIR IQC. For the second consultancy, KIAsia conducted the procurement process. Although the IQC holders were invited to bid, other organizations also had the opportunity to participate. The contractor indicated that KIAsia's presence in Thailand facilitated needed modifications to the contract as work unfolded. BTP's success in initiating risk management training with senior management and concurrently developing a framework was a lesson learned from other BTP consultancies. KIAsia brought that training to SCB once the value of the approach was evident. Major Recommendations of Consultant: Barents: Establish risk management training as a year round course offering to staff. PwC: Proprietary recommendations on SCB's risk management framework. Client Efforts to Implement Recommendations: The Barents training established risk management awareness in SCB’s middle-management level through training by the bank's in￾house trainers. The risk management courses were subsequently offered year-round. PwC's risk management recommendations were to be reviewed by SCB's senior management shortly after the Evaluators visited. SCB gave every indication that they were eager to consider the recommendations and indicated that they were to be much more comprehensive than originally agreed in the SOW. Impacts on the Bank and Impact on Bank Clients: The Barents consultancy provided SCB with the internal resources to conduct training in risk management and included training 30 bank personnel to be able to conduct the training. This resulted in an awareness of risk management by the bank's middle management. The PWC consultancy increased the awareness by SCB senior management of the importance of risk management and prepared them to lead the way in implementing improvements in risk management at the bank. Both consultancies should result in the strengthening of bank procedures. While resulting in closer scrutiny of loan applications and more rigorous internal bank procedures the consultancies should result in better screening of applications (which could help sound customers) and a less risky financial condition for SCB. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-47 Special Observations Drawn from this Case Study: Training was initiated with middle management and a year and a half later brought to senior management. In retrospect, sensitizing senior management to risk management first would likely have provided greater impetus to the program within the bank. Evaluators’ Considered Judgment: Both consultancies were a success, however as indicated above, were senior management trained first, the magnitude of the success would likely have been greater. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-48 BTP CASE STUDY Bank Name: Bank of Ayudhya (BAY) Type of Bank, Ranking in the Banking Sector: Commercial Bank, 5th largest Thai commercial bank. Consulting Company: 1) Deloitte Touche Tohmatsu (DTT) 2) Pricewaterhouse Coopers Risk Management Service Ltd. (PwC) Services Provided by Consultant: 1) Risk Management Training 2) Risk Management Implementation Timing: 1) April through August 2000 2) October 2001 - December 2002 Major Elements of SOW: 1. DTT's risk-management program for BAY was divided into two parts: a) Training consisting of: 1) summary training session for top management; 2) integrated risk￾management training for middle managers; and 3) training in specific risk areas for line management and staff and b) Development of a risk management framework for BAY. 2. PwC Risk Management Implementation for BAY included the following 1) Assess and determine training needs/prepare training courses and training materials, 2) Introduce and implement credit manual through a training session for credit managers, provincial branch managers, senior credit officers and credit staff, 3) Training of Trainer course for continued implementation of the credit manual for select branch managers and senior officers in credit related functions. How the Consultancy Was Initiated: The first consultancy was initiated by BAY upon learning of KIAsia's role with AERA. BAY was the first bank to contact KIAsia on BTP. The second consultancy was requested by BAY to follow through with implementation of the risk management framework. Performance of the Consultant(s): BAY satisfaction with both consultants was very high. Some of this can be attributed to the fact that BAY knew what it wanted and articulated the requirements well, thereby receiving well-targeted training. Working Relationship Between Bank and Consultant: BAY praised the work and professionalism of both consultants. As a result of experiences with the first consultant, BAY stressed the importance of Thai language capability for the trainers. The second consultant located a skilled Thai-speaking risk management consultant from within its regional practice to conduct the training. Performance of KIAsia in Supporting the Consultancy: KIAsia was instrumental in facilitating BAY's articulation of training needs in a scope of work for both consultancies. The first consultancy was procured under the SEGIR IQC. Since the USAID contract officer was based in Phenom Penh, this assistance enabled USAID to expeditiously compete a task order Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-49 under the SEGIR IQC. For the second consultancy, KIAsia conducted the procurement process. Although the IQC holders were invited to bid, other organizations also had the opportunity to participate. Major Recommendations of Consultant: DTT: Proprietary risk management framework. PWC: Proprietary recommendations regarding credit manual and its implementation. Client Efforts to Implement Recommendations: BAY fully embraced the concepts provided by Deloitte Touche Tohmatsu. The two-day training for top management was attended by the bank’s president and its fifteen most-senior managers. They participated in an in-depth discussion of world-class approaches to credit, market, and operational risk management. The session concluded with broad agreement regarding the content and structure of the training program as well as agreement on a high-level plan for implementing a new risk management framework. After the first session was completed, training for middle management began. After the training was completed, the consultant assisted the bank to develop and implement an effective risk management framework that included an internal credit rating system. PwC: Prior to the second BTP consultancy, BAY retained PwC to assemble a credit manual from numerous directives and circulars previously in use. BAY then retained PWC through BTP to develop a series of relevant training courses and assist with initial implementation of the new credit manual. Training courses developed: a) Best Practices in Credit Risk Management for vice president and above, b) Introduction and Implementation of Credit Manual, c) Introduction to Market Risk Management for managers and staff from Treasury, Investment Banking, Risk Management, Credit, Internal Audit, d) Market Standard Techniques for managers, senior credit officers, executive vice president and above, e) Market Risk Exposure Management for managers and senior credit officers, f) Train the Trainer on the new credit manual. Impacts on the Bank and Impact on Bank Clients: Bank officials indicated that after successfully engaging with the first training consultancy, they overcame a reluctance to use consultants. Previously they had feared they would not be able to control consultants’ work. Subsequently, the Bank, using its own resources, hired a consulting firm to assemble a consolidated credit manual. Bank officials forcefully asserted to the Team that without the first BTP experience, they would not have further utilized consultants. Later they engaged in the second BTP consultancy for risk management implementation, covering all levels of the Bank. The Evaluation Team found that BAY had fully embraced risk management as part of its operations. One branch manager interviewed had participated in the TOT course and subsequently trained 40 persons. The credit manual was not found gathering dust on the manager's shelf, but with the branch credit officer, where it was being utilized daily. The branch manager asserted there had been big changes in the way loans were processed. Prior to BTP training, the prime determinants for loans were: collateral, who the prospective borrower was, and if they had a business. Little or no thought was given to avoiding non-performing loans (NPLs) by simply not making high-risk loans in the first place. Now, a careful analysis is made of the business to determine if it has the capacity to service debt. The manager asserted that the bank now helps the customer to: examine the business, decide on investment in the business, Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-50 manage funds of the business and look at sales, and where appropriate, connect to other customers in a win-win situation. Special Observations Drawn from this Case Study: BAY personnel knew of the crucial importance of risk management and were convinced that senior management should be deeply involved from the beginning. Accordingly, the training began with top management. This resulted in agreement on the balance of training and a plan for implementing a new risk management framework. This well thought out process secured the commitment of top management and paved the way for the balance of the training; it is a good lesson learned from the BTP. Another observation is the role of consultants. Prior to AERA, Thai banks avoided consultants, partially out of concern over controlling the consultancy. With this case, the bank learned through experience that it could profit from consultancy and that it could control the process to achieve the results it wanted and not be dominated by the consultant. Evaluators’ Considered Judgment: The training provided BAY was an outstanding success. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-51 BTP CASE STUDY Bank Name: Bank for Agriculture and Agricultural Cooperatives (BAAC) Type of Bank, Ranking in the Banking Sector: Specialized Financial Institution. If BAAC were a commercial bank, it would be the 8th largest Thai bank. Consulting Company: 1) Development Alternatives, Inc. (DAI) 2) National Technological University (NTU) (Thailand) and Thai Institute of Banking and Finance Association (TIBFA) 3) NTU Services Provided by Consultant: 1) Training in Responsibility Center Accounting 2) Distance learning courses for bank staff 3) Second phase of distance learning courses Timing: 1) May through September 2001 2) July through November 2002 3) May through November 2003 Major Elements of SOW: 1) Three training programs were delivered by DAI:  Executive seminar on responsibility center management: Topics covered included the theory and best practices in profit center management and skill development related to risk management. In addition, the profit center management framework, the accounting policies, and the implementation plan were included.  Managing the BAAC branch as a profit center: Provided an understanding of the profit center management framework, profit center accounting, the major components of branch financial statements, key financial indicators, key risks to be managed, and principles of asset and liability management.  Train-the-Trainer workshop on profit center management: Provided participants with training skills to carry out further branch management training on the profit center concept. The trainers were instructed how to develop a work plan for the bank-wide rollout of the profit center management framework. 2) NTU and TIBFA initiated the "New Bank Professionals Development Network” conducted via distance learning for government banking staff throughout Thailand. NTU and TIBFA developed courses with BAAC and four other participating banks and delivered them via satellite broadcast to BAAC training centers and other satellite centers across Thailand. Six courses were offered: 1) Risk Management, 2) Banking Law, 3) Internal Audit, 4) Sales and Customer Relations, 5) Asset and Liability Management and 6) Economics for Bankers. 3) NTU is implementing the "New Bank Professionals Development Network Phase II." The courses include: 1) Good Governance, 2.) Credit Analysis of SMEs and Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-52 Microfinance, 3.) Marketing and Customer Relationship Management, 4.) Knowledge of Checks and Money Laundering Protection, 5.) Banking Account Law and 6.) NPL Management and Techniques. How the Consultancy Was Initiated: KIAsia conducted a rigorous campaign to offer BTP services to all Thai banks. BAAC was interested in furthering some initial assistance GTZ had provided on Responsibility Center Accounting. Regarding the NTU/TIBFA activity, BAAC requested assistance to leverage its already existing satellite training centers. NTU also had similar training sites. The third activity was a response to the overwhelming interest in the distance learning activity. Performance of the Consultant(s): DAI successfully completed the training program for Responsibility Center Accounting, which was well received by BAAC management and staff. NTU/TIBFA performed well, training 18,000 participants, 50% above the expected 12,000. The Phase II distance learning activity is expected to at least match the phase one activity by training 18,000 participants. TIBFA was not included in the second phase perhaps due to somewhat delayed responses on its part. (Note the NTU training statistics are for five banks, not BAAC alone) Working Relationship Between Bank and Consultant: The relationship for Responsibility Center training was quite satisfactory from both parties view but there were long delays in implementation. DAI was selected for the training prior to March 2000. Due to multiple scheduling conflicts by both parties, it was not until May 2001 that BAAC and DAI were able to begin the actual training. Relationships for distance learning were excellent in both phases of the training with NTU. Performance of KIAsia in Supporting the Consultancy: KIAsia was instrumental in facilitating BAAC's articulation of its training needs (Responsibility Center Accounting) in a scope of work. Since the USAID contract officer was based in Phenom Penh, this assistance enabled USAID to expeditiously compete a task order under the SEGIR IQC. The KIAsia role with regard to distance learning was more facilitative but done quite well. Major Recommendations of Consultant: Proprietary recommendations on the implementation of a profit center management framework. Client Efforts to Implement Recommendations: BAAC has extended Responsibility Center Accounting to all of its branches. Branch results are used in evaluating performance of branches and branch results are collectively used to evaluate performance of provincial managers. After experiencing positive results with the first engagement, BAAC was eager to take advantage of further training opportunities through the distance learning program. Impacts on the Bank and Impact on Bank Clients: The training program on Responsibility Center Accounting was provided to 400 BAAC staff members, consisting of 20 top executives, 60 branch directors, 300 branch managers and 20 staff to serve as trainers. In addition, the training resulted in skill development of the BAAC staff and implementation of responsibility accounting for the entire branch network. Consequently, this helped build up the BAAC’s competitiveness for the long term. Development Associates, Inc. The AERA Program of KIAsia December 31, 2003 An Evaluation (July – September 2003) E-53 Special Observations Drawn from this Case Study: The distance learning facility has truly reached dramatic numbers of Thai bank personnel. Further study on the quality of this training is warranted. The Evaluators were not able to observe a session in progress, but suspect that large class sizes may have limited the learning experiences, especially in an environment lacking interactivity. Nonetheless, there was a dramatic market response; trainees attended the sessions on their own time in much larger than expected numbers. This is a strong indication that they valued the training. Distance training may well have value for other applications in Thailand and perhaps the region. It is especially useful in reaching audiences well outside populous capitals, saving on transportation and accommodation expense, allowing these audiences access to opportunities for upgrading skills. Evaluators’ Considered Judgment: Responsibility Center Accounting has taken root in BAAC and the training can be considered very successful. The real benefit will be realized once all personnel are familiar with the tools at their disposal and are able to use it to bring about real change in the bank. The distance training program has been a dramatic success and been proven by the market test, an overwhelming response by bank personnel seeking to increase their skills. Area Programs of KIAsia .doc/EvaIQC-36