Evaluation of the BARENTS BUMN/IBRA and CARANA JITF Technical Assistance Projects Jakarta, Indonesia October 21, 2002 USAID/Indonesia Economic Growth Team “Technical Assistance for IBRA” and “Jakarta Initiative Task Force Technical Assistance” USAID Contract No. AEP-I 00-00-00023-00 Development Associates, Inc. Management & Governmental Consultants 1730 North Lynn St. • Arlington, VA 22209-2023 IBRA / JITF Evaluation Development Associates, Inc. Page i Executive Summary We find that the USAID supported activities, at the Indonesian Bank Restructuring Agency (IBRA) and the Ministry of State Owned Enterprises (BUMN), are achieving their purpose. We realize that both IBRA and the Jakarta Initiative Task Force (JITF) were created in a response to a crisis situation. This had the consequence that the Scope of Work for each project was formulated under time pressure and in a rapidly changing environment. As a result, the team found that the activities lacked specific performance measuring criteria by which to evaluate the activities of the two consulting teams, KPMG Barents at IBRA/BUMN, and Carana/Padco (CP) at JITF. At IBRA, one of USAID’s constraints was that no taxpayer money could be used to finance technical assistance (TA) in areas where services were available from the private US commercial consulting sector. Thus, it was intended that the KPMG Barents team play an internal advisory role to IBRA management. However, with frequent changes in management, including seven different chairman in five years, (see Table 6) the team’s focus was changed frequently to accommodate the organization’s priorities. These included generating more income for the Indonesian treasury, faster disposal of assets, and the establishment of a deposit guaranty program. Despite the personnel changes and an organizational culture that appears to have hindered the advisers’ effectiveness and often limited their work to niche areas, the team has responded with flexibility and competence to the ever changing environment and priorities at IBRA. Their efforts and achievements, and the quality inputs of the short term consultants, have been noteworthy in Asset Management Investment, Bank Restructuring, deposit insurance, and finance and administration. While the Evaluation Team recognizes the inherent difficulties and challenges that each of the projects has had, we believe that the original project design could have provided clearer performance expectations, milestones for assessing progress, and performance criteria for the respective project’s key objectives. Such clearly defined objectives would have helped to provide the project with a clear focus on the expected outcomes from the TA. Due primarily to the aforementioned reasons, the Evaluation Team has concluded that, while USAID’s advisory support to BPPN/IBRA has been useful in many areas, the TA has had limited success influencing key strategic and policy issues at IBRA. JITF, after a slow start, started to take off in mid-2000. It is credited with the successful mediation of over USD 16 billion in debt, and has for each of the past two years exceeded the IMF agreed quantitative targets. The CP team has contributed to that success. Having established good relations with fellow consultants, and JITF management and staff the CP team has made several important contributions to JITF. JITF’s professional discipline has been widely acknowledged and we believe that is in no small part due to the CP team’s contributions to strengthening the alternative dispute resolution (ADR) process by applying a consistent and transparent process based upon rigorous academic standards. In addition to handling individually assigned mediation cases, team members have trained the Indonesian staff, both on-the-job and in workshops or seminars. The prevailing method of measuring JITF’s success is to compare the aggregate amount of mediated debt (original pre-mediation amount) to the targets set in the LOI agreed to between the Government of Indonesia (GoI) and the IMF. The benchmark was previously the signing of an informal debt restructuring memorandum of understanding (MOU), between the creditors and debtor. However, since the end of 2001, JITF’s formal involvement has been extended to the formal signing of the legal debt restructuring agreement between the parties. IBRA / JITF Evaluation Development Associates, Inc. Page ii As the success target for mediation is an aggregate amount, not a number of cases, JITF has focused on cases with the largest indebtedness. Recognizing that the effort and skill needed to handle a $10 million case is often no different than a $100 million case, this is clearly the most efficient approach to reaching value-based performance targets. While the CP team has contributed to JITF’s success, it is difficult to assess on the relationship of mediation skills to success rate, due to the large number of other factors that are involved in mediation. From the peer group (fellow consultants), and JITF’s management, the feed back has been positive. Taking into account that JITF was established to help overcome the stifling impact of a moribund and heavily indebted private sector, we would suggest that looking only at the value of debt restructuring agreements is an inadequate measure. The real measure of JITF’s success is whether companies -- after having been relieved of a significant part of their indebtedness (through easier terms from creditors, debt forgiveness or discounted debt buybacks)-- honor the new agreements, receive new credits, maintain or increase employment, and regain its market competitiveness. JITF has made some effort to track re-structured loans, but the response, and therefore the success of the mediation efforts, received from polled parties has so far been inconclusive. We therefore believe it is still early to declare that JITF has been an unqualified success in achieving its founding objectives. Despite the problems at IBRA, we have no doubt about the importance of the work the organization is doing, its contribution to the national treasury, and the urgency of its tasks. Due to the high degree of complexity in much of the work at IBRA, there is also a clear need for expatriate advisers. We therefore believe that continued USAID support to IBRA is warranted and can be beneficial in helping to resolve many of the remaining tasks, including selling state￾owned assets and banks back to the private sector. The continuing need for USAID support at JITF is less convincing. We have previously noted the importance of the CP team’s contributions to formalizing the ADR process. We have also noted that a locally-recruited member of the CP team is almost solely responsible for producing the IMF mandated quarterly Indonesian Corporate Debt Survey. However, with significant financial support from the World Bank, including the provision of two expatriate advisers, and a trained contingent of Indonesian mediators, the opportunity for USAID to reallocate its limited resources to projects with clearer and more immediate returns clearly exists. We therefore recommend that if continued USAID support to JITF is deemed desirable, maintaining the one expatriate adviser with strong ADR skills and the locally-recruited adviser producing the quarterly debt report would provide the greatest ongoing value to JITF. With the significant reduction in the budget level, significant cost savings could also be achieved by moving from an institutional to a personal services contractual arrangement. Note: In addition to the report that follows, Annexes 1 and 2 contain further documentation of our findings and recommendation. IBRA / JITF Evaluation Development Associates, Inc. Page iii Acknowledgements We would like to acknowledge with appreciation the time that the following people have provided to the team, while undertaking this review. 1. Chander, Suhail ABN Amro, Vice President and Head, Restructuring and Recovery 2. Ryan, Michael Asian Development Bank, Jakarta 3. Prabowo Bank Merincorp, Restructuring Officer 4. Dinning, Martin Barents, Chief of Party 5. Babcock, Lee Barents, Consultant 6. Nelson, Bernard Barents, Consultant 7. Cooke, David Barents, Project Director 8. Sutantio, Dasa BPPN/IBRA Advisor to the Chairman’s Office 9. Musa, Soebowo BPPN/IBRA, Advisor to the Chairman’s Office 10. Gozali, Hugeng BPPN/IBRA, Bank Restructuring, Vice President and Division Head 11. van Beekum, Raymond BPPN/IBRA, Communications Division Head 12. Sukadis, Harry BPPN/IBRA, Finance and Management Director 13. Slamet, Sumantri BPPN/IBRA, Vice Chairman 14. I Nyoman Sender BPPN/IBRA, Deputy Chairman, Bank Restructuring 15. Syahrial, Moh. BPPN/IBRA, Deputy Chairman, AMC 16. Salim, Felia BPPN/IBRA, ex-Deputy Chairman, BRU 17. Carmody, Bernie BUMN, ADB/PricewaterhouseCoopers Project Director 18. Thirkell, Russell Carana/ Padco, Chief of Party 19. Cavanaugh, Patrick Carana/Padco, Consultant 20. Lee, Raymond Carana/Padco, Consultant 21. Mark Edwards Commonwealth Development Corp 22. Anand, Ameesh Deloitte, Touche Tohmatsu, Manager Advisor, Restructuring 23. Chadwick, Mark Ferrier Hodgson, Associate Director 24. Sakir, Andrew Ferrier Hodgson, Director 25. Abeng, Tanri Former Minister of State Owned Enterprises 26. Edwards, Mark International Finance Corporation 27. Ruru, Bacelius JITF, Chairman, and BUMN, Secretary General 28. Tobing, Samuel JITF, Chief Operating Officer 29. Bake, Willem JP Morgan, Advisor 30. Lubis, Lana USAID, Activity Manager, Financial Institutions 31. Cornelio, Bruno USAID, Economic Growth Office, Deputy Team Leader 32. Deuster, Paul USAID, Economic Growth Office, Team Leader 33. Dinh, Quan USAID, Project Technical Officer, Economic Growth Office 34. Wight, Albert World Bank, Consultant, Institutional Development 35. Drum, Bernard World Bank, Coordinator, Private Sector Development 36. Gustely, Ed World Bank, JITF Senior Mediator 37. Watkins, Brian World Bank, JITF Senior Mediator 38. Michael Edwards World Bank, Principal Financial Advisor IBRA / JITF Evaluation Development Associates, Inc. Page iv Project Team Mr. Anton Deiters, Senior Program and Policy Analyst/Team Leader Mr. James E McCoy, Senior Program and Policy Analyst Dr. Ahmad D. Habir, Senior Program and Policy Analyst IBRA / JITF Evaluation Development Associates, Inc. Page v Table of Contents Executive Summary ....................................................................................................................................... i Acknowledgements......................................................................................................................................iii Project Team ................................................................................................................................................ iv Table of Contents.......................................................................................................................................... v Acronyms and Abbreviations ...................................................................................................................... vi I. Introduction ..................................................................................................................................... 1 II. Findings........................................................................................................................................... 1 A. General Comments.......................................................................................................................... 1 B. IBRA – KPMG/Barents ................................................................................................................... 3 C. BUMN.............................................................................................................................................. 5 D. JITF – Carana/Padco...................................................................................................................... 6 E. Role of USAID/Jakarta.................................................................................................................... 8 F. Conclusion....................................................................................................................................... 9 Table 1. KPMG/Barents Advisors at BPPN/IBRA.................................................................................... 10 Table 2. Carana / Padco Advisors at JITF ................................................................................................. 10 Table 3. Extract from USAID Barents SOW for BUMN .......................................................................... 11 Table 4. Barents Short Term Advisory Projects for BPPN/IBRA ............................................................. 12 Table 5. KPMG/Barents’ Self Reported Key Achievements at BPPN / IBRA ......................................... 13 Table 6. Extracts from USAID Policy Determination, Implementing USAID Privatization Objectives, PD-14, June 16, 1986. ............................................................................................... 14 Table 7. BPPN / IBRA Key Staff .............................................................................................................. 15 Annex 1 – Findings and Recommendations: IBRA................................................................................... 16 a. Efficiency....................................................................................................................................16 b. Effectiveness...............................................................................................................................16 c. Relevance....................................................................................................................................16 d. Practicality/Adequacy of Funding ..............................................................................................16 e. Management of Contracts...........................................................................................................17 f. Personnel ....................................................................................................................................17 g. Substantial Involvement of USAID............................................................................................17 h. Impact .........................................................................................................................................17 Annex 2 – Findings and Recommendations: JITF...................................................................................... 19 i. Efficiency....................................................................................................................................19 j. Effectiveness...............................................................................................................................19 k. Relevance....................................................................................................................................19 l. Practicality/Adequacy of Funding ..............................................................................................20 m. Management of Contracts ..........................................................................................................20 n. Personnel ....................................................................................................................................20 o. Substantial Involvement of USAID............................................................................................20 p. Impact .........................................................................................................................................21 Annex 3 – Literature Consulted.................................................................................................................. 22 Documents Related to BPPN / IBRA...................................................................................................22 Documents Related to JITF..................................................................................................................25 Annex 4 - Project Planning Calendar.......................................................................................................... 27 Annex 5 - Barents BUMN/IBRA and Carana JITF Project Evaluation Scope of Work............................. 28 Annex 6 – JITF Response to the Report ......................................................................................................35 IBRA / JITF Evaluation Development Associates, Inc. Page vi Acronyms and Abbreviations ADD Asset Disposal Division ADR Alternative Dispute Resolution ADU Asset Disposal Unit AMC Asset Management Credit AMI Asset Management Investment APD Agency Planning and Secretariat AU Administrative Unit Barents Barents Group BI Bank Indonesia BPPN/IBRA Badan Penyehatan Perbankan Nasional/Indonesian Bank Restructuring Agency BRU Bank Restructuring BUMN Ministry of State Owned Enterprises CCAS Corporate Core Asset Sales CFO Chief Financial Officer CLPS Commercial Loan Sales Program COP Chief of Party CP Carana/Padco CRI/PN Convertible Rights Issue/Promissory Note CULS Corporate Unrestructured Loan Sales EGT Economic Growth Team FAD Finance and Accounting FSA Financial Supervision Agency GOI Government of Indonesia IQC Indefinite Quantity Contract IMF International Monetary Fund IDIC Indonesian Deposit Insurance Company JITF The Jakarta Initiative Task Force LLL Legal Lending Limit LOI Letter of Intent MOF Ministry of Finance MOU Memorandum of Understanding RP Rupiah SOE State-owned Enterprises SOW Statement of Work SSD Systems and Support Division USAID United States Agency for International Development USD United States dollar WB World Bank Work Plan Strategic / Work Plan IBRA / JITF Evaluation Development Associates, Inc. Page 1 I.Introduction Development Associates, Inc. was contracted by the United States Agency for International Development (USAID) to conduct an onsite evaluation of two USAID/Indonesia Economic Growth Team (EGT) activities in Jakarta, Indonesia: “Technical Assistance for IBRA” and “Jakarta Initiative Task Force Technical Assistance.” The purpose of the evaluation is to determine if these activities are achieving their purposes, to compile success stories and lessons learned, and to make recommendations for the functioning of the technical assistance teams and potential follow-on activities. Within this context of the evaluation, the team is also expected to specifically examine questions of efficiency, effectiveness, relevance, practicality/adequacy of funding, and management of the contracts. The Indonesian Bank Restructuring Agency (IBRA) project is being implemented by Barents/KPMG while, the Jakarta Initiative Task Force Project (JITF) is being undertaken by Carana/Padco (CP). Both activities are being carried out under SEGIR Financial Services and SEGIR GBTI IQC Task Orders, with September 2002 as the completion date for both, although extensions are expected. The IBRA project has three long-term expatriate advisors, with additional support provided by fourteen short￾term advisors. The JITF project is staffed with three expatriates and one Indonesian. The review team of Anton Deiters, James McCoy and Ahmad Habir commenced the evaluations with an inception meeting and briefing at the USAID’s Jakarta office on 22nd July, 2002. This was followed by introductory meetings with the respective Chiefs of Party. As part of the data gathering process, more than fifty separate interviews were conducted with thirty-eight individuals, including project personnel, USAID staff, local agency managers and other stakeholders. The team also examined more than eighty different documents consisting of statements of work, project reports, internal reports, and both proprietary and public information . II. Findings A. General Comments IBRA and JITF This evaluation deals with two USAID commissioned and financed consultancies, one at IBRA, the other at JITF. It should be noted that the IBRA consultancy has also involved work at the Ministry of State Owned Enterprises (BUMN), the ministerial agency that oversees the work of IBRA. Both consultancies (with the exception of the special case of BUMN, dealt with below) have in common that they assist entities which were created in response to the financial crisis resulting from the Asian economic crisis in 1977 and subsequent collapse of the rupiah. The purpose of USAID’s involvement is to help the Government of Indonesia (GoI) to carry out the economic recovery plan, as set out in the IMF Letter of Intent (LOI). USAID sees this recovery being supported through the restructuring of the banking industry and sale of assets to the private sector by IBRA and the restructuring of non￾performing loans by JITF. In the case of IBRA, the sale of assets will benefit the national treasury and restructuring the banking sector is essential to the national economy. JITF’s mandate is to facilitate debt restructuring negotiations between debtors and creditors to improve the corporate sector’s viability and ability to contribute to the economic revitalization of the economy by maintaining employment, the delivery of goods and services, and by generating tax income for the national treasury. Although non-performing debt is at the heart of both IBRA and JITF, there are considerable differences between the two organizations. IBRA / JITF Evaluation Development Associates, Inc. Page 2 IBRA is a large (3,000 plus staff) government agency, which directly or indirectly owns a number of banks that became insolvent; it also owns other assets such as loans, stocks, and other forms of collateral that belonged to the banks that collapsed or to their former owners. IBRA’s’ mandate is to: 1) manage the commercial bank liabilities Government Guarantee Scheme; 2) close failed banks and to capitalize surviving banks and restructure, merge or sell banks that had been taken over; 3) transfer, restructure and liquidate the assets of failed banks, as well as the non-performing loans of recapitalized banks; and 4) sell corporate assets transferred by former bank shareholders. The assets (including restructured and un-restructured debt) are sold in international or domestic markets in order to raise cash for the Indonesian treasury. It has been reported that IBRA controlled, at one point, about 70% of the economic assets that make up the Indonesian economy. JITF, on the other hand, is not a government agency (it has even been called an NGO), but it is a special, government sponsored body created to mediate a resolution between borrowers and lenders who are in conflict over the non-payment of loans following the financial crisis. In the majority of cases at both IBRA and JITF, the debtors are Indonesian private corporations. It is on the creditor side that the difference is most manifest with IBRA, or the banks it represents, being a direct creditor, while it is mostly foreign banks who are the creditors in cases handled by JITF. The indebtedness dealt with by IBRA is denominated in both rupiah and foreign currency. As debts denominated in rupiah are essentially domestic obligations, there is little impact on the balance of payments. On the other hand, JITF is not a party to any transaction and has no assets under its control. The indebtedness dealt with by JITF is largely foreign debt, owed to non-Indonesian entities. The resolution of the indebtedness handled by JITF thus has great consequences for the Indonesian balance of payments. Moreover, IBRA is a large bureaucracy with a diverse and multi-focused agenda, and its succession of chairmen has each brought his own management style and priorities to the organization. JITF, on the other hand, is a small, single-focused entity, with a lean, hands-on management that has not changed since it was established. Such differences have major consequences for the consultancies, in terms of their SOWs, and in terms of the skills and competencies required. IBRA and BUMN The consultancy is referred to by USAID and by KB team as “IBRA/BUMN”. This may cause confusion since BUMN is a ministry with responsibility for state-owned enterprises, and IBRA is a special purpose government agency as noted above. BUMN was created in 1998 prior to the financial crisis. However, both organizations have a primary role in Indonesia’s economy, and the BUMN minister is the cabinet officer responsible for IBRA. BUMN oversees more than 150 government-owned enterprises and has sporadically attempted to privatize some of these businesses with little success. In addition, the JITF chairman concurrently holds the position of Secretary General at BUMN. It is generally believed that, after the mandate for IBRA expires late next year, the remaining assets held by IBRA will move under the direct jurisdiction of BUMN. As IBRA’s fixed life makes clear the need to quickly divest the assets that it has acquired, and with BUMN controlling a large and generally inefficient stable of companies, the volume of assets that move from IBRA to BUMN will be reflective of the former’s failure to achieve its objectives. The purpose of USAID’s involvement in IBRA is to help GOI in meeting the recovery plan, as set out in the IMF letter of intent. USAID’s involvement at BUMN appears to have evolved primarily from organizational issues that limited the involvement of the KPMG/Barents team at IBRA. However, as privatization is a key objective of USAID, and with key linkages to BUMN management, this involvement is logical. It can also have considerable future IBRA / JITF Evaluation Development Associates, Inc. Page 3 value in supporting the divestiture of BUMN￾controlled assets to the private sector in a transparent manner that will help relieve pressure on the GoI budget. The Evaluation Team is of the view that this effort should be structured as a separate project with very clear and quantifiable objectives that are consistent with USAID’s privatization policies (see Table 7). While both the IBRA and JITF consultancies appear to have complied with the requirements of their Scopes of Work (SOW) for the production of deliverables, we have noted that measurable performance objectives keyed to the desired outcomes have not been specified in terms that can be objectively measured or quantified. While the Evaluation Team recognizes the inherent difficulties and challenges that each of the projects has had, we believe that the original project design could have provided clearer performance expectations, milestones for assessing progress, and performance criteria for the respective project’s key objectives. As a general observation about both teams, the Evaluation Team has had much difficulty in identifying achievements that have had distinctly strategic importance to their respective Indonesian beneficiaries, and which can be directly attributed to the work of the TA advisers. This is not to say that the consultancies have not been important or that they have not realized important achievements at the operating level. The assistance of the CP team at JITF has clearly been valued highly by the client. In the case of IBRA, a combination of organizational culture and personnel changes at IBRA have hindered the advisers’ effectiveness and limited their work to niche areas. Unlike IBRA which has organizational objectives that are easy to quantify, JITF has essentially no clear way to measure whether it is successfully achieving the objectives, other than the quantitative targets set by the IMF.1 Consequently, in a process that records only successfully resolved cases as “completed” and does not track the reason that cases are dismissed, individual performance and anecdotal evidence becomes the primary basis for evaluating organizational performance. B. IBRA – KPMG/Barents The KPMG/Barents team (KB Team) started its work in May 2000, and was given the following tasks in the SOW: To give technical assistance to IBRA in six areas: a. Asset Management Credit; b. Asset Management Investment; c. Bank Investment and Restructuring; d. Finance and Accounting; e. Information Technology; and f. Risk Management. We find that the KB team performed overall satisfactorily under this SOW. The assistance provided to IBRA has been predominantly operational and in response to specific requests from IBRA. There appear to have been few opportunities for the consultants to become involved in strategy or policy level advisory work on a sustained basis. We have also noted that the KB team has in some areas not worked optimally as a team. Although the project does not appear to have been seriously impaired by this, we believe that a more coordinated team approach would have had a greater chance of achieving policy-level strategic impact at IBRA. All required performance, progress, and trip 1 The Jakarta Initiative Task Force (JITF) was established to serve as a mediator and facilitator of specific corporate restructuring cases, particularly those involving foreign lenders, which in turn was expected to help revive Indonesia’s business sector and jump start economic growth and employment creation. Therefore, JITF considers its work done when a loan restructuring agreement has been signed, or a case is dismissed for whatever reason. There is no subsequent tracking of restructured loans to determine whether new agreements are honored or whether the desired economic growth and employment creation is actually occurring. IBRA / JITF Evaluation Development Associates, Inc. Page 4 reports required from both long-term and short￾term consultants, have been produced according to the SOW. The Evaluation Team has noted what we believe to be serious shortcomings in the KB Team’s quarterly report however. We would therefore recommend that the report format be changed to include an executive summary and to eliminate the redundant and repetitive boiler plate text that frequently appears in succeeding reports. We would also recommend that the Work Plan agreed at the outset of the project, be maintained with all additions, deletions or changes retained in subsequent iterations. The current practice is to update the Work Plan in each quarterly report showing only the revised plan, without reference or explanation to any changes. This makes it very difficult to measure achievements to plan. We have generally found it difficult to identify where the KB Team has made significant contributions of strategic nature. For example, the KB Team has indicated that they were responsible for IBRA’s moving its valuation standard from book value to market value. However, the Evaluation Team is of the view that, while the KB Team did support this decision with technical research, the evolution of the idea was essentially obvious after failed asset sales. We have also been unable to verify that IBRA’s revised policy on selling non￾performing loans without prior restructuring resulted from the KB team’s input. Although these are examples of two important policy changes at IBRA, the KB Team’s assistance appears to have been primarily in helping to justify and implement the changes, after the need had been identified. This support was undoubtedly useful to IBRA, but it is an indication of the reactive response to IBRA’s requests that we have previously noted. In conversations with IBRA management, it was noted that the KB Team had been extensively involved in raising the awareness of the need to improve the finance and administration faction within IBRA. In addition, the Team provided significant short-term resources that were reported to have been of significant value. We believe this is one of the most prominent examples of the Team’s policy level strategic advisory work, as well as an activity that involved substantial and highly successful operational level support. The project, as designed, does not appear to have a deliberate strategic agenda. Due to the fast moving changes at IBRA, the KB Team has indicated that it had, by necessity, to be reactive. While we acknowledge the Team’s flexibility in responding to the challenges posed by the ever-changing priorities of each new new chairman, we believe a greater effort could have been made to help manage the IBRA agenda. Responding to the client’s needs is of course essential. However, we would equate the situation that has existed to responding to fires, rather than having a clearly established agenda with identified and planned support activities. The result has been a patchwork of various activities responding to the issues or management of the day, rather than a coherent program with measurable milestones leading to the ultimate objective of total asset divestiture. Our conversations with the client indicated that the long-term KB Team members have been perceived as technically competent and have produced work acceptable to IBRA. The international short-term consultants have also generally been perceived as capable, most notably in the areas of finance and administration, deposit insurance and bank merger. The Evaluation Team has found that the long￾term advisers have: • Worked under broad, general objectives, that may accommodate the organizational setting but that make assessment of the project’s success highly subjective; • Generally been focused on specific research tasks, where international contacts or technical expertise is perceived to be of value; • Had little organizational influence; • Had no frequent or substantial access or influence with the ever-changing chairmen IBRA / JITF Evaluation Development Associates, Inc. Page 5 (although this has varied over time), and have not met the responsible minister. The KB team has had to work under very difficult circumstances, and has shown resourcefulness and flexibility. It appears however, that the team members’ lack of Indonesian or Asian experience has impaired, to some degree, their effectiveness in the early stages of the project. While we agree that technical expertise should weigh more heavily than cross-cultural skills and country experience in the selection of short-term consultants, we believe that long-term consultants would have a distinct advantage if they possessed a sound understanding of Indonesian society, culture, politics, institutions and business. This is all the more important for work in a complex and important agency such as IBRA. For the past several months, one KB Team consultant has been reassigned from IBRA to BUMN. While we understand the reasons why this was done, we would recommend that project resources be refocused at IBRA. With only about one year remaining until IBRA is dissolved, there are many important remaining tasks that could be crucial to enhancing the country’s international credibility and economic well-being, if they are handled efficiently, effectively and transparently. As these are predominantly operational activities, they fall well within the consultants’ proven skills and, no less importantly, within the areas that the organization seems willing to accept. We would also caution against treating IBRA and BUMN projects as one, since we believe that this signals USAID’s implicit acceptance that IBRA will not complete its asset sales program, and that it is acceptable for assets to be transferred to BUMN. While this may in fact be the reality, any transfer of assets to BUMN should be considered a major failure of the organization that is not endorsed, implicitly or explicitly, by USAID. Similarly, it diminishes the incentive for the KB Team to focus their efforts at IBRA, since they would simply shift their attention to BUMN. By statute, IBRA is to be wound up by late next year, despite its having significant asset disposals, bank mergers and bank sales remaining to be dealt with. The organization has therefore started preparing exit scenarios that may involve the creation of new entities and vehicles that will own unsold assets. Although the Evaluation Team believes that the small team assigned to IBRA, first three and now two consultants, would have had greater impact if they could have focused on strategic level policy planning and monitoring from the beginning, the organization’s needs at this stage are primarily implementation and operations. Therefore, we are of the view that the KB Team’s efforts should focus on supporting IBRA’s management with the support necessary to ensure that all remaining assets are divested as quickly and transparently as possible. These are important and complex tasks that will undoubtedly benefit from USAID’s continuing provision of both long and short-term advisers. Please see Annex 1 for further documentation of our findings and recommendations. C. BUMN The transfer of one of the KB Team’s consultants from IBRA to BUMN was done more than halfway through the project term. It appears to have been an expeditious move. The consultant in question had successfully finished his work at Holdico Perkasa, an important holding company set up by IBRA, to hold the corporate assets of the Salim group. His move to BUMN, apparently at the request of the Secretary General of BUMN, does not appear to be the best deployment of his talents. In conversations with several BUMN staff members, we found that he is currently involved in marginal work at BUMN, with an unclear path forward until the ongoing Asian Development Bank (ADB)-sponsored consultancy work is complete. We therefore are of the view that by installing one consultant at an operational level, with no comprehensive program planned or in place, USAID would accomplish little at BUMN. IBRA / JITF Evaluation Development Associates, Inc. Page 6 This ADB consultancy is undertaking a broad￾based needs assessment and strategic plan for BUMN, which will provide a good point of reference for USAID to determine the most appropriate role at BUMN. We therefore recommend that the KB adviser currently assigned to BUMN be returned to IBRA. Future USAID involvement with BUMN can be determined early next year when the results and recommendation from the forty-person ADB￾financed consultancy are available. D. JITF – Carana/Padco The SOW for Team Carana/PADCO (“CP team”) covers the following tasks: • Provide three senior corporate debt restructuring specialists/advisors to work directly with the staff of JITF. • Advise JITF senior management and other senior government officials on all aspects of corporate debt restructuring, including specific recommendations on legal and regulatory policy, and outreach and public education. • Train local personnel to progressively take over duties held by senior advisors, with the intent of ensuring JITF sustainability. We find that the CP Team has performed these tasks in a satisfactory manner, and that along with the other consultants and staff of JITF, has helped achieve the quantitative targets set by the IMF. The CP Team has supported the application of the alternative dispute resolution (ADR) framework to JITF’s mediation efforts. This has brought about a more systematic and transparent process that appears to have enhanced the effectiveness of JITF. The CP Team was also involved in identifying the need for incentives to encourage debtors to work with JITF, and in developing the resulting “carrots and sticks” which have now been adopted. These include: tax incentives, concessions granted by the stock exchange, flexibility regarding bank’s lending ratios, and the designation of “non-cooperative” parties. We noted that the CP Team appears to work well with their Indonesian colleagues, the World Bank mediators and with each other. Feedback from our interviews was also consistently positive about the professionalism of the CP Team members. Although we were unable to interview the Indonesian member of the CP Team, due to his absence during our review, we were provided, just prior to our departure, with a recent copy of his work product, the Indonesian Corporate Debt Survey, Sixth Quarterly Report, of March 2002. We understand that production of this report is a requirement of the IMF, and that he has primary responsibility for compiling the data and publishing the report each quarter. The statistical data is comprehensive, and the report is well laid out and illustrated with informative charts and tables. The report is clearly an important document in monitoring the overall achievements and progress of JITF, as well as the broader debt situation in Indonesia. The CP Team’s involvement in producing this report and the quality of the work are commendable. Although the above noted Corporate Debt Survey was provided to us very late, our subsequent review of the document confirms our previously stated concerns that it is still too early in the post restructuring process to make any conclusive predictions about actual repayment behavior and compliance with the new restructuring agreements. The role of JITF is to facilitate a resolution of Indonesia’s non-performing private-sector corporate debt through a process of structured mediation. It is recognized that each of the CP Team’s three expatriates have their own individually assigned cases for which they are primarily responsible, and have secondary responsibilities, including supporting other mediators, acting as “second chair”, and providing training. Their efforts have undoubtedly contributed to the number of successfully concluded cases and to JITF’s IBRA / JITF Evaluation Development Associates, Inc. Page 7 achievement of the targets established by the IMF’s Letter of Intent (LOI). However, as we were not invited to participate in any actual mediation sessions, and indeed, were advised that most of the information pertaining to individual cases was confidential, we have relied on anecdotal evidence concerning the efficiency and effectiveness of the CP Team members’ mediation efforts. While we have no doubt that each member brings a wealth of experience and complementary skills to JITF, the nature of the work and the prevailing level of confidentiality precludes us from being able to directly or objectively evaluate their on-the-job performance. While the client, JITF, has expressed full satisfaction with the work of the CP Team, the direct beneficiaries of the mediation activities, namely, the respective borrowers and lenders, in so far as we have been able to interview them, have been mixed in their praise. The CP Team has conducted a number of seminars and trainings in the fulfillment of its task to perform outreach and to educate. The evaluators were present at one such event, a USAID/JITF sponsored breakfast meeting at the Shangrila Hotel on July 30, where the former Chief Justice of the New South Wales Supreme Court gave a presentation about the benefits of ADR. The meeting appeared to be well organized and attended by an audience drawn mainly from Jakarta’s legal and business community. JITF is expected to cease operations at the end of 2003. As it is anticipated that the future of a “surviving entity” of JITF may lie in making ADR available on a sustainable basis (a purely private undertaking, charging user fees), the event was particularly well timed. The participants were asked to complete a questionnaire intended to gauge their response to ADR and to the need for it in Indonesia. We have reviewed the CP Team’s report entitled, “An Exit Strategy for the Jakarta Initiative Task Force: Historical Experience and Policy Options for the Future” dated June 2001. Despite the title, we have found that the only policy option that the paper deals with substantively is the CP Team’s recommendation to transition JITF to a quasi-private sector ADR agency. While this is one option that merits consideration, we would suggest that other options such as continuing JITF as a quasi￾government sponsored agency, merging JITF with other existing arbitration/mediation organizations in Indonesia, and closing JITF, be given more thorough and objective consideration. We have also noted that in the “Exit Strategy” report, the CP Team has asserted that “JITF tends to have significantly positive effects” and that JITF has had a “high degree of … historical success,” without stating the basis for these claims. While we are aware that JITF’s claims of success usually point to the achievement of IMF targets for the amount of debt restructured, we have noted elsewhere in this report why we believe this is at best a one dimensional, interim, indicative measure. Our concern with this method of measuring success is that it is analogous to looking solely at the number of surgical operations performed by a doctor to judge his success, rather than keeping track of how many patients survive, for how long, and the quality of life they have. Other measures that would be useful indicators of performance and efficiency would include: debt restructured as a percentage of original indebtedness; average time from registration to legal restructuring agreement; amount of JITF staff time spent on each case; number of employees hired/terminated by borrower for 36 months from date of registration with JITF; amount of new credits extended to the borrower as a part of the JITF facilitated restructuring, or within 24 months of the legal restructuring documentation being signed; and the number of cases where the debtor defaults on the new restructuring agreement. Perhaps the most outstanding shortcoming in the Exit Strategy Report is the lack of any views or opinions from potential ADR users that they would support the fundamental recommendation. That recommendation -- to commit substantial time, effort and resources to transitioning JITF to an ongoing institutional agency, as opposed to the current structure as a IBRA / JITF Evaluation Development Associates, Inc. Page 8 project with a finite life -- appears to be based entirely on the CP Team’s assumption that there will be a demand for such services. We therefore believe that it is essential that an objective and professional study be undertaken to determine whether there is a real need for an institutionalized successor to JITF. Instead of using blanket assumptions, such a study would include a comprehensive survey of potential users and would include questions that would give objective data on such issues as: what would make the organization credible; would fee for service be an acceptable model and if so, how much and what type of fees; is there a need for enforceability? The CP Team has fulfilled the SOW requirements with a commendable degree of teamwork and professional competency. In the process, they have developed good relationships with the management, staff, and fellow consultants at JITF. They have helped to create a disciplined structure, provided training and guidance to improve internal capacity, and shown both enthusiasm and professionalism in their work. The CP Team appears to function well with JITF’s flat organizational structure and easy access to management. Although we believe that JITF and the CP Team have met the requirements of their SOW and have capably fulfilled their mediation role, the absence of quantitative efficiency and performance measures for both the CP Team members and the project, leaves considerable room open for judging overall performance and achievement. Aside from tracking individual case loads, we are unaware of any efforts by JITF to systematically monitor or evaluate individual contributions or job performance. And although we understand the need for confidentiality in negotiations, and that there are many factors that can influence the success or failure of mediation, we would believe that more attention should be given to establishing clear and objective efficiency and effectiveness measures for the consultants and also to systematically tracking and evaluating individual performance. We fully agree that as a process-oriented agency, JITF appears to have been successful in meeting the established IMF objectives for loan restructuring throughout. However, without knowing the rationale upon which these targets were based, and recognizing that such targets are only an interim measure of success for the process of mediation, we are reluctant to accept this as broad and conclusive evidence of JITF’s success in achieving the real goal of accelerating the pace of corporate debt restructuring and reviving the Indonesian economy. We would therefore recommend that JITF make a more diligent effort to track post-restructuring events for each case, in order to determine whether the restructuring efforts are indeed having a positive effect on the economy. Please see Annex 2 for further documentation of our findings and recommendations. E. Role of USAID/Jakarta USAID personnel have been cognizant of a number of the issues that we have described here. Agency personnel have done additional work to improve performance on several fronts: • It has developed a new SOW that provides clearer and more explicit performance expectations and indicators, focusing on a narrower and more achievable range of work (AMC, BRU), with emphasis on the transparent disposal of assets by IBRA. • USAID has taken measures since November 2001, to ensure that the Barents KB team functions as a cohesive, better coordinated unit. • USAID personnel have taken advantage of the high profile of the activity to ensure its own access to policy makers at the highest levels. • The Mission has made clear that the work at IBRA and BUMN are separate activities and has required repositioning of personnel to affect that policy. IBRA / JITF Evaluation Development Associates, Inc. Page 9 Mission officials also continue to believe in the important role of JITF. They point out that the number and size of cases has not declined. Remaining cases, totaling some US $13 billion, are among the most difficult to resolve. They contend that if USAID-funded senior advisors were removed from JITF, the program would suffer. Their role in ADR particularly was emphasized to us, as was the future role of JITF in resolving the corporate debt problem in Indonesia. F. Conclusion If USAID has adequate resources to fund both programs, given its belief in the value of the work done by JITF, and actions to strengthen efforts with IBRA, the Evaluation Team has uncovered no overriding reasons to disagree with a decision to continue the Mission's involvement with both organizations. With the reality of shrinking budgets, however, USAID-Jakarta must look carefully at all of its programs and make a determination as to where the greatest benefit can be reasonably expected. While the work of JITF has generally met with the expectations of most observers, we have questions about its true impact on the Indonesian economy. When asking these same questions about IBRA, we find that the answers are quite different: The potential impact of IBRA on the Indonesian economy is direct, measurable, and significant. Therefore, in a world of scarce resources, we would recommend giving priority to the assistance to IBRA over that to JITF. We are not proposing that USAID withdraw from JITF. But in the case that USAID would have to choose between IBRA and JITF because of scarce funding, we would recommend giving priority to continuing with IBRA. The completion of the asset disposal agenda of IBRA is of critical importance, and the need for technical support in a wide range of areas appears to us to be a much greater need. JITF could of course also continue to benefit from a USAID consultancy, but that need is, in our view, less important, since JITF already has competent foreign and local consultants, as well as Indonesian staff, who have already benefited from USAID-sponsored training, and from the daily transfer of know-how from the expatriate consultants, as had been foreseen by the SOW. Should available USAID resources be sufficient to support only one expatriate consultant at JITF, we would recommend that the person with ADR skills be retained. This would help to ensure that the remaining consultants and staff continue to develop their skills and knowledge of the ADR process. As JITF does not have internal resources to produce the Quarterly Debt Survey, it is also essential to JITF that the local adviser be maintained. IBRA / JITF Evaluation Development Associates, Inc. Page 10 Table 1. KPMG/Barents Advisors at BPPN/IBRA 1. Project Director David Cooke 2. Project Manager Jason Foley 3. Chief of Party and Bank Restructuring Advisor Long-term Martin Dinning 4. Asset Management Credit Advisor Long-term Bernard Nelson 5. Asset Management Investment Advisor Long-term Lee Babcock 6. Asset Management Credit Advisor Short-term Carl Morgan 7. Asset Management Credit Advisor Short-term James R. Hambric 8. Bank Restructuring and Merger Advisor Short-term Michael D. McNertney 9. Chief Financial Officer Advisor Short-term Gary Fechtmeyer 10. Deposit Insurance Advisor Short-term Stanley Silverberg 11. Finance and Accounting Advisor Short-term James Satterfield 12. Information Technology Advisor Short-term John Fenton 13. Information Technology Advisor Short-term Brooks Dickerson 14. Ministry of State Owned Enterprises Short-term Richard Longstaff 15. Monitoring and Supervision Advisor Short-term D. Charles Ratliff 16. Strategic Plan and Asset Management Credit Advisor Short-term Bill Dudley Table 2. Carana / Padco Advisors at JITF 1. Chief of Party and Senior Mediator Long-term Russell Thirkell 2. Senior Mediator Long-term Patrick Cavanaugh 3. Senior Mediator Long-term Raymond Lee IBRA / JITF Evaluation Development Associates, Inc. Page 11 Table 3. Extract from USAID Barents SOW for BUMN “USAID has agreed with the Minister for SOE's/IBRA that KPMGBarents will conduct an urgent review of the previous high level planning documents in relation to the GOI's asset disposition strategy and the current short-term plans. The KPMG/Advisor will produce, in agreement with the Ministry, an action plan for 2002, which will incorporate the provision of appropriate `hands-on' technical support to achieve a successful contribution to the 2002 budget. The performance of the Ministry has been constrained by the lack of specialist advice and this can now be remedied.” “Such a role would require the ability to provide simplified transaction analysis (economic and political), knowledge of privatization policy and a strong ability to form the types of working relationships necessary to working effectively in any government office in Indonesia. Deliverables would vary on day-to-day activity but, ultimately, would be measured by the ability of the Ministry to initiate and complete transactions.” IBRA / JITF Evaluation Development Associates, Inc. Page 12 Table 4. Barents Short Term Advisory Projects for BPPN/IBRA Timeframe Unit Description of Work Consultant(s) Exit Report Date Comments 1. 2000 February 26- March 31 AP Design, Implementation and Preparation of a Revised BPPN Strategic Plan for 2001 to February Dudley April 27, 2001 2. 2000 August 1- September 27 F&A Improving the Efficiency and Accuracy of Accounting Information Data Flow Scope of Work Satterfield October 2000 3. 2000 August 2-9 DC-SAA Identification of Operational Risk Associated with Outsourcing Asset Management Hambric See Note 1 4. 2000 September 7-30 AMC Policy for Holding Assets in Management Companies Until Disposition Hambric Not on file 5. 2000 September 7-30 VC/IR Communications and Investor Relations Whitney September 26, 2000 6. 2000 September 7-30 DC-SAA Follow-up on Outsourcing Risks and Review of CDO Transaction Hambric See Note 1 7. 2000 September 7- October 6 BRU Swapping Loans for Bonds and Exit Strategy for Banks Under BPPN Control Silverberg Only draft report on file 8. 2000 December 6-15 C/VC Role and Function of a Chief Financial Officer and Review of BPPN Organizational Structure Scope of Work Cooke and Hambric Not on file 9. 2000 December 10-16 C & VC AMC Review of Proposals to Setup a Supervisory Board for Asset Management and Joint Ventures Hambric See Note 1 10. 2001 January 19- February 9 C Advice on Organizational Structure, Asset Management and Disposal, and Conflict of Interest Issues Hambric See Note 1 11. 2001 March 1-May 11 F&A Improving the Efficiency and Accuracy of the Finance and Accounting Reporting Process Satterfield 11 May 2001 12. 2001 February 5-16 AP Design, Implementation and Preparation of a Revised BPPN Strategic Plan for 2001 to February Foley February 7- February 18, 2000 Report not dated but on file 13. 2001 February 26-April 27 VC Review of AMC Organizational Structure Scope of Work Hambric Not on file 14. 2001 February 26-April 27 C & AMC Strategic Plan, Assessment of Joint Venture Alternatives, Outsourcing Issues Hambric See Note 1 15. 2001 March 10-April 10 BRU Bank Monitoring and Supervision Ratliff April 5, 2001 16. 2001 March 27-April 6 C Role and Function of a Chief Financial Officer and AMC Organizational Structure Cooke April 15, 2001 17. 2001 April 1-27 C Follow on Advisory to Define the Role and Function of a Chief Financial Officer Fechtmeyer 12 April 2001 18. 2001 May 12-June 8 AP Review and Refine Draft BPPN Strategic Plan for 2001 to February 2004 Dudley June 8, 2001 19. 2001 May 30-August 4 C Follow on Advisory to Define the Role and Function of a Chief Financial Officer Fechtmeyer July 26, 2001 20. 2001 June 11-30 IT/MIS Review and Evaluation of IT Systems Supporting Financial and Accounting Applications Dickerson and Foley July 18 2002 21. 2001 June 15-July 14 /CP/IR Issues Relating to Communications and Investor Relations Olcheski July 14, 2001 22. 2001 November 26- December 14 BRU Indonesian Deposit Insurance Corporation Cooke and Silverberg December 13, 2001 23. 2002 March 15-April 14 BRU Bank Merger Satterfield and McNertney April 12, 2002 Note 1: Activities summarized in one undated memo; reference is made to other memos and documents as deliverables C – Chairman DC- Deputy Chairman AMC-Asset Management Credit AP- Agency Planning BRU-Bank Restructuring SAA-Support and Administration IBRA / JITF Evaluation Development Associates, Inc. Page 13 Table 5. KPMG/Barents’ Self Reported Key Achievements at BPPN / IBRA Deposit Insurance We have always believed that this was an important and fundamental development to the development of a healthy banking sector. Despite the `politics' between MoF, BI and BPPN we have worked for over two years with IBRA nurturing the work within the agency and now with the MoF to point where we have all three - MoF, BI and BPPN - not to mention DPR etc participating in a high level seminar on August 14, 2002. Moreover we will lead the development process through to implementation in 2004. ‘5 Bank Merger’ We have worked with the agency for over nine months on this particular issue. After suggesting that the main risks to IBRA were in the operational issues relating to the merger we reformulated the organization structure of the merger. Next we suggested a global coordinator - Arthur Anderson who was originally going to 'assist' in the merger and integration process did not have the requisite skills/experience. Finally, using our USAID contacts assisted in getting the DPR to approve the use of the '519' account for liquidity support purposes. The original request was for us to have three full time advisors working on the merger. The agency continues to want us to work full time on this issue but there is a budget constraint. Sales Strategy for Asset Sales In May 2001 we produce a sales strategy for the then Chairman and he was in agreement with the concept and that the agency should sell unrestructured loans. At the same time he also instructed the then Deputy Chairman of AMC to sell the commercial loan portfolio, value circa IDR 26 Trillion and in the main managed by four banks. Strategic Plan We started to work with Agency Planning in August 2000 on this but the then Chairman who had connections with McKinsey's brought that firm in to the Agency to prepare the plan. McKinsey's were fired in January 2001 and we were asked to assist at short notice to work with the agency staff to produce a plan. It must be borne in mind that McKinsey's had a team and our approach could only be to provide guidance and assistance. CFO USAID, the agency itself or no doubt other institutions have not appreciated the subtlety of the concept of introducing a CFO. If it had been the IMF and WB would have long ago insisted upon such an appointment as a means of breaking down the `silo mentality' that ensured that the agency operated in a dysfunctional manner. Market Value Principle This was a land mark development and in the first instance we persuaded the agency to put the idea to the MoF when they were reluctant to do so. The MoF agreed. The key point is that perceptions of what the agency have to achieve have moved downwards from the original `rule of 70%' and a realistic recovery rate of 25/30% is no longer seen negatively. CAP The USAID - KPMG/Advisors were crucial to the completion of this process in relation to the work on issues from the 1999 and 2000 y/e-audited accounts. We acted as the interface between IBRA and the WB/IMF. The WB local rep was not an accountant and although responsible for the Agency needed our technical knowledge and expertise to facilitate the work, as did the IMF person. Suspense Account We uncovered and brought to the attention of top management a massive suspense account problem. The net effect, apart from working with the agency to correct the problem was to ensure that a large team (50 people) was used immediately to lessen the potential public scandal that could have ensued. Holdiko Perkasa We had one long-term person in this part of AMI working on a hands-on basis to sell companies over many months. No Negative Press Comment about USAID Involvement in IBRA Despite all of the negative Press comment about IBRA, the number of investment bankers, financial advisors and at times anti-American sentiment our presence has attracted no attention. Navigated through the various Top Management Changes at IBRA Four Chairman plus a myriad of changes in the top management down through the ranks has made managing the project more difficult more especially given the need to accommodate the National and US Embassy political view points. IBRA / JITF Evaluation Development Associates, Inc. Page 14 Table 6. Extracts from USAID Policy Determination, Implementing USAID Privatization Objectives, PD-14, June 16, 1986. “Implementation of the privatization objective must begin with the determination of which public activities are appropriate for the private sector. The appropriateness of public versus private sector should be determined on the basis of which sector is more likely to produce a higher level of economic efficiency, innovation, and incentive, and, therefore, the greater economic benefit. Experience has demonstrated that a private enterprise (rather than a wholly or partially state-owned enterprise or parastatal), operating in a truly open and competitive environment, is usually the more likely to meet goals of economic efficiency and growth.” “Definition. For the purposes of Agency policy, privatization is defined as the transfer of a function, activity, or organization from the public to the private sector. (Related activities discussed in Section 4B of this paper, but not falling within this definition, nay be justified with reference to the revised Private Enterprise Development Policy Paper.) The major techniques for privatization, for the purpose of complying with this PD, are discussed in section 4A below. The term “privatization” is not synonymous with private enterprise. Privatization is an important and unique aspect of our private sector program in that it brings together policy reform, institutional development, and utilization of the private sector. Our private enterprise goals and program are described in the Private Enterprise Development Policy Paper.” “USAID projects designed to improve parastatal performance must have identifiable benchmarks upon which substantive progress towards divestiture can be measured.” The latter sentence is the ultimate condition upon which assistance is to be granted. In other words, the selected benchmarks must represent substantive evolutionary progress in moving the parastatal towards market-based operations and divestiture in order to qualify for USAID assistance.” “Missions have, in the past, utilized technical or capital assistance to make state-owned enterprises (SOEs) more efficient, more responsive to market forces, or more attractive for buy-outs. It should be recognized, however, that enormous amounts of donor funds committed to help SOEs meet the goal of greater efficiency have been largely unsuccessful. There is no reason to believe that new USAID resources will be better spent for that first goal unless the process is linked clearly to both making the SOE more responsive to market forces and actual divestiture. Therefore, the use of USAID funds in a manner that only improves the capability of the parastatal to respond to market forces in the absence of true policy reforms (such as improving an SOE’s accounting procedures as opposed to revising the tax code for all enterprises in a particular industry) does not comply with this policy.” IBRA / JITF Evaluation Development Associates, Inc. Page 15 Table 7. BPPN / IBRA Key Staff Period of Service Aug 2000 Nov 2000 Feb 2001 May 2001 Aug 2001 Nov 2001 Feb 2002 May 2002 Chairman Cacuk Sudarijanto Edwin Gerungan → → I Putu Gede Ary Suta Syafruddin A Temenggung Vice Chairman Arwin Rasyid Sumantri Slamet → → → → → → Asset Mgt Credit Deputy Chairman Irwan Siregar → → → Mohammad Syahrial Asset Mgt Investment Deputy Chairman Mahmuddin Yasin → → → Taufik Mappaenre Maroef Bank Restructuring Deputy Chairman Jerry Ng → Ibu Felia Salim → I Nyoman Sender Risk Management Deputy Chairman Chandra Purnama Hendy Herijanto → → System Procedure Deputy Chairman & Compliance Jusak Kazan Support and Admin Deputy Chairman Sumantri Slamet Chandra Purnama → → Junianto Tri Prijono Finance & Accounting Division Head Roni Maulana Harry Sukadis → → → → → → Treasury Division Head Edi H Sidharta Systems and Support Division Head Sri Savitri → → → Emma Sri Martini Agency Planning Vice President Noegroho Soetardjo → → → → → → → Investor Relations Division Head Eri Budi Reksoprodjo → Agency Communications Group Head Danang Kemmayan Jati → → → Raymond Van Beekum IBRA / JITF Evaluation Development Associates, Inc. Page 16 Annex 1 – Findings, Suggestions, Comments, Recommendations: IBRA a. Efficiency 1. Finding It appears that the consultancy in AMC is now working well at an operational level. After a long adjustment period, the right kind of relationship with the counterparts has been established. Recommendation Given the still heavy agenda of loan-disposal, it would be best to maintain the current consultant at AMC. b. Effectiveness 1. Finding There appear to be communication issues between the consultants and USAID. Recommendation Communications between USAID and the Barents Consultants should be improved. The Chief of Party (COP) should establish an effective dialogue with the USAID mission and where necessary, the Barents project director and USAID senior management should be involved. USAID should refrain from discussing personnel issues directly with consultants, without first consulting and involving the COP. 2. Finding There appear to be communication issues and a lack of coordination and teamwork among the consultants. Recommendation Effective measures should be taken to ensure that the Barents team functions as a cohesive, coordinated unit with a clear understanding of what is expected from the team and from each of its members regarding both long- and short term objectives and achievements. c. Relevance 1. Finding The IBRA project has been relevant in selected technical areas identified by the beneficiary, but has failed to establish itself as being a strategic partner. Consequently, the advisers have had little influence at the policy management level. Comment Where USAID advisors are used primarily as technical resources without broader access to and involvement with senior managers, there is a risk that such work could be irrelevant, if the objectives and strategies of the project deviate from acceptable norms. d. Practicality/Adequacy of Funding 1. Finding IBRA is one of the largest and most complex bank restructuring projects in the world and a critically important element in Indonesia’s economic revival. However, in the context of IBRA, the resources budgeted for this project (three long-term advisers and various short-term advisers) were relatively insignificant and contributed to, what we believe to be, the advisers’ marginal involvement and lack of strategic influence. IBRA / JITF Evaluation Development Associates, Inc. Page 17 Recommendation Unless there are overriding political reasons, we would recommend concentrating scarce resources on projects where USAID has: (a) the potential to have sustainable or important immediate impact; (b) clearly defined and measurable objectives with periodic milestones; (c) a clear understanding with the beneficiary that continued project involvement will depend on achieving periodic milestones; and (d) involvement which will have significant influence and impact from policy through implementation. e. Management of Contracts 1. Finding Barents has reported that although they had identified a need to change long-term advisers, USAID would not approve any personnel change, indicating that maintaining established relationships was a priority. Comment Where an institutional contractor recommends a personnel change, USAID should exercise caution in rejecting such recommendations, as this can diminish accountability for on-the-job performance and may result in inferior job performance. f. Personnel 1. Finding The long-term consultant’s level of experience appears to be below the level for a technically and politically complex organization such as IBRA. He had no Indonesian experience. Comments Consultants should have had some senior level managerial experience, preferably in an Asian or Indonesian context, in a large and complex organization, in order to facilitate their understanding of organizational politics and with the ability to recognize areas of opportunity and risk in their work. g. Substantial Involvement of USAID 1. Finding The contacts between USAID and IBRA appear somewhat distant. They have certainly suffered from the “embargo” ordered by the previous ambassador. With a new Chairman in command, the “one-team” or “one-front” approach of USAID and Barents toward IBRA still could be improved. Recommendation A fresh effort could be made to intensify the contacts, and to strengthen the appearance of “one-front” with the COP Barents. h. Impact 1. Finding The impact of the consultancy is very difficult to measure. Most of the impact, in day-to-day operations, appears to be felt by IBRA/AMC. There is also a positive impact felt from a number of short-term consultancies in the areas of finance and administration, bank merger, deposit insurance. IBRA / JITF Evaluation Development Associates, Inc. Page 18 Recommendation In the fast-moving IBRA environment, which is approaching its final phase, alertness on the part both USAID and Barents to IBRA’s should be intensified, in order to assist IBRA to fulfill its agenda. IBRA / JITF Evaluation Development Associates, Inc. Page 19 Annex 2 – Findings, Suggestions, Comments, Recommendations: JITF i. Efficiency 1. Finding Creditors and Debtors are not obligated to work with JITF. JITF’s involvement is voluntary. The number of cases settled outside JITF is not known; nor is it known how many cases were dropped or dismissed by JITF. JITF reports to have “completed” USD 14.2 Billion worth of debt, in 69 cases. JITF is concentrating on the largest cases of indebtedness. Comments By concentrating on the largest cases, JITF enhances its efficiency. While it is difficult to assess the real cost/benefit ratio of JITF, we can say that it is taking the most efficient approach, by giving priority to the largest amounts of indebtedness. It would be an improvement in reporting, if the docket-reports would show the number of cases dropped or dismissed over time. It should be noted that JITF has to meet aggregate dollar amount debt targets only, not a certain number of cases. It can take equally hard work to mediate and settle a $ 50 million loan as it does to mediate a $500 million loan. j. Effectiveness 1. Finding It was difficult to poll significant numbers from among the two main stakeholder groups, creditors and debtors. In a few cases, appreciation was expressed for the work of JITF. In other cases, the response was lukewarm. The effectiveness appeared to be enhanced by the “carrots and sticks”, particularly the tax credit, available to JITF assisted debtors only. Suggestion While some ten cases are reported to have taken advantage of the tax credit, it would be interesting to include in the periodic reporting how much debt has been restructured on the basis of this tax exemption, how big the tax credit in fact was. In other words, how much this cost the Indonesian treasury. k. Relevance 1. Finding JITF is concerned with reaching the quantitative goals set by the IMF. These goals have been consistently met. The work of the consultants has been crucial to reaching those goals. The relevance to the Indonesian economy and society, the “impact” of JITF, is far more difficult to assess. It has been an improvement to change the closure benchmark from MoU to Legal Closure. Comments It is true that JITF has met the goals set by IMF. In that sense, JITF’s is a success story. We recommend taking a more critical look at the meaning of the attainment of such quantitative goals. Has it created more enterprise; more employment; renewed lending; increased exports? Are there actual payments made under the new agreed upon debt? What are IBRA / JITF Evaluation Development Associates, Inc. Page 20 the cash-flows resulting from the rescheduled debt? In other words, over which period can new defaults result in more problem debts? The grace periods are such that there has been very little payment experience, if any. l. Practicality/Adequacy of Funding 1. Finding See also below under “Management of Contracts”. We find that the presence of a “contractor” has not added value to the consultancy. Recommendation As a result, the funding of this consultancy might, at least in theory, have been accomplished at a lower cost. We are, however, aware of the practical constraints. m. Management of Contracts 1. Finding Diversity of the consultants, differences in culture, skills, competences and the seemingly negligible role of any “backstopping” head office make us wonder if the contracting via Carana/Padco was indeed necessary, or preferable, and if it was not too expensive. We have found no evidence of any value-added by the head-office. One team member came from Australia; another one was already in Jakarta. There was no “corporate culture” or specific know-how of the contractor supporting them. Comment While we understand that there are historical explanations for a particular contracting situation, we wish to point out the difference between this contract, and that of IBRA, where head-office support is concerned. n. Personnel 1. Finding The team as a whole appeared energetic, enthusiastic, and competent. There was a noticeable team spirit. The team appeared to relate well and effectively with its colleagues and counterparts. The team members were varied in skills, experience and knowledge of banking and local language. This diversity appeared to be more of an asset than a liability. There seemed agreement among those immediately involved and various other observers that the member with ADR experience in particular added value to the efforts of JITF. Suggestion If USAID were to scale down its support for JITF, but wanted to keep financing just one consultant, we would recommend that the one with the ADR expertise be selected. In that case, it would be much more cost￾effective to hire the consultant directly. It would save unnecessary overhead. o. Substantial Involvement of USAID 1. Finding The link between Minister Laksamana and Bacelius Ruru and USAID appears to be an important aspect of USAID’s involvement with JITF. USAID has profiled itself prominently in its support of JITF. The recent IBRA / JITF Evaluation Development Associates, Inc. Page 21 Mediation/ADR workshop, sponsored by USAID and JITF, was a manifestation of USAID’s active role in supporting JITF and the Carana consultancy. Suggestion USAID should consider carefully whether or not to continue JITF, given its limited financing options. JITF is a success story. It may be difficult to measure its “real success”, but there is no doubt that USAID has played an important role by financing four key consultants. It may be a good moment to phase out of JITF, as that body considers its own metamorphosis to a truly private enterprise or venture. p. Impact 1. Finding There is no doubt that the consultants have had a positive impact on JITF, in its meeting the IMF quantitative targets. Whether or not JITF has had an impact on the Indonesian economy at large is another matter. It will take years of monitoring to assess that impact. Suggestion A more critical look at the “successful” results of JITF is recommended. 2. Finding The forgiveness of large portions of debt under JITF mediation has of course an effect on the Indonesian balance of payments: the debt is owed largely to foreign creditors and is denominated in foreign currency. Comment In that sense, the impact of JITF mediation on the Indonesian economy is quite serious. But it should be remembered that the debt was not being serviced to begin with -- so we are noting a theoretical impact only. 3. Finding JITF has been called a “one-stop office” for creditors. It is a convenient window. Comment This appears to be the case, although we have noted that there are many more debt case where the creditors do not avail themselves of this service . 4. Finding JITF has been trying to poll creditors who have used JITF mediation. The response has been fairly thin, as per the March 2002 Indonesian Debt Survey. Comment The matter of indebtedness is usually private and confidential. We, the Evaluating consultants, have found it similarly difficult to poll the creditors . IBRA / JITF Evaluation Development Associates, Inc. Page 22 Annex 3 – Literature Consulted Documents Related to BPPN / IBRA 1. BPPN Quarterly Project Report for Badan Penyehatan Perbankan Nasional (BPPN), August 9, 2000, KPMG Consulting/Barents Group. 2. BPPN Quarterly Project Report for Badan Penyehatan Perbankan Nasional (BPPN), November 2000, KPMG Consulting/Barents Group. 3. BPPN Quarterly Project Report No 2 for Badan Penyehatan Perbankan Nasional (BPPN), February 2001, KPMG Consulting/Barents Group. 4. BPPN Quarterly Project Report for Badan Penyehatan Perbankan Nasional (BPPN), May 2001, KPMG Consulting/Barents Group. 5. BPPN Quarterly Project Report for Badan Penyehatan Perbankan Nasional (BPPN) August 2001, KPMG Consulting/Barents Group. 6. BPPN Quarterly Project Report for Badan Penyehatan Perbankan Nasional (BPPN) November 2001, KPMG Consulting/Barents Group. 7. BPPN Quarterly Project Report for Badan Penyehatan Perbankan Nasional (BPPN) February 2002, KPMG Consulting/Barents Group. 8. BPPN Quarterly Project Report for Badan Penyehatan Perbankan Nasional (BPPN), May 2002, KPMG Consulting/Barents Group. 9. Review of BPPN & BUMN Assistance Needs a. Scope of Work, May 2002, Advisors: Mr. Jason Foley, Financial Management Specialists, Level II, Time Frame: June 10-June 19, 2002. b. Scope of Work, May 2002, Advisors: Mr. David Cooke, Financial Management Specialists, Level I, Time Frame: June 10-June 21, 2002. c. Presentation: BUMN Assistance Needs, US Agency for International Development and Barents Group of KPMG Consulting, Inc. d. Presentation: BPPN Assistance Needs, US Agency for International Development and Barents Group of KPMG Consulting, Inc. e. Presentation: Deposit Insurance Basics, KPMG Consulting. 10. BUMN Project - Privatization/Asset Disposal a. Scope of Work, November 2001, Advisors: Mr. Richard Longstaff, Financial Management Specialists, Level I, Time Frame: November 19-December 14, 2001. b. Memo, December 19, 2001, From: Richard Longstaff, Advisor-Ministry of State Owned Enterprises, To: Martin Dinning, KPMG/Barents Chief of Party-IBRA, Subject: Exit Report on Discussions with Ministry of State Owned Enterprises. c. Presentation: Tips for International Tenders for Retaining Investment Bank Advisory Services, Richard Longstaff, USAID Project Advisor, December 2001. IBRA / JITF Evaluation Development Associates, Inc. Page 23 d. Presentation: International Tender for Investment Bank Advisory Services, Richard Longstaff, USAID Project Advisor, December 2001. 11. BRU “5” Bank Merger Scope of Work, Satterfield and McNertney, Phase I Extension, March 15- April 14, 2002. 12. BRU “5” Bank Merger Exit Report on Results of Bank Restructuring Advisory Efforts, Satterfield and McNertney, Phase I Extension, April 12, 2002. 13. BRU – Indonesian Deposit Insurance Corporation Scope of Work, Cooke and Silverberg, November 26-December 14, 2001. 14. BRU – Indonesian Deposit Insurance Corporation Exit Report on Results of BRU Monitoring Project Efforts, Cooke and Silverberg, December 13, 2001. 15. BRU – Bank Monitoring and Supervision Scope of Work, Ratliff, March 10-April 10, 2001. 16. BRU – Bank Monitoring and Supervision Exit Report on Results of BRU Monitoring Project Efforts, Ratliff, April 5, 2001. 17. BPPN/IBRA Quarterly Bank Monitoring Report, Bank Restructuring Unit, February 1, 2001. 18. BPPN/IBRA Monthly Bank Monitoring Report, Bank Restructuring Unit, 31 January 2001. 19. BRU – Swappping Loans for Bonds and Exit Strategy for Banks Under BPPN Control Scope of Work, Silverberg, September 7-October 6, 2000. 20. BRU – Swappping Loans for Bonds and Exit Strategy for Banks Under BPPN Control Draft Progress Report, Silverberg. 21. AP – Design, Implementation and Preparation of a Revised BPPN Strategic Plan for 2001 to February 2004 Scope of Work, Dudley, February 26-March 31, 2000. 22. AP – Design, Implementation and Preparation of a Revised BPPN Strategic Plan for 2001 to February 2004 Exit Report, Dudley, April 27, 2001. 23. AP – Design, Implementation and Preparation of a Revised BPPN Strategic Plan for 2001 to February 2004 Scope of Work, Foley, February 5-February 16, 2001. 24. AP – Design, Implementation and Preparation of a Revised BPPN Strategic Plan for 2001 to February 2004 Report on Results of Strategic Planning In IBRA Project, Foley, February 7- February 18, 2000. 25. AP – Review and Refine Draft BPPN Strategic Plan for 2001 to February 2004 Scope of Work, Dudley, May 12-June 8, 2001. 26. AP – Review and Refine Draft BPPN Strategic Plan for 2001 to February 2004 Exit Report, Dudley, June 8, 2001. 27. AMC – Policy for Holding Assets in Management Companies Until Disposition Scope of Work, Hambric, September 7-September 30, 2000. 28. AMC – Role and Function of a Chief Financial Officer and Review of BPPN Organizational Structure Scope of Work, Cooke and Hambric, December 6-December 15, 2000. 29. AMC – Review of AMC Organizational Structure Scope of Work, Hambric, February 26-April 27, 2001. 30. AMC – Review of AMC Organizational Structure Scope of Work, Hambric, May 14-July 21, 2001. IBRA / JITF Evaluation Development Associates, Inc. Page 24 31. AMC – Review of AMC Organizational Structure Exit Report, Hambric, June 1, 2001. 32. AMC – Advantages and Disadvantages of Outsourcing Asset Management and Disposition, J. Hambric. 33. AMC – Asset Management Observations and Recommendations, J Hambric, 7 August 2000. 34. AMC – Issues Arising from the Sale of Contracted Assets (draft), J Hambric, 17 July 2002. 35. AMC – Management and Disposal of Assets, J Hambric, February 19, 2001. 36. AMC – Interim Asset Management Agreements with Contractors, J Hambric, February 21, 2001. 37. AMC – Deposit Insurance Scheme Discussion Paper (draft), J Hanbric, 17 July 2002. 38. AMC – Introduction of Joint Ventures/Equity Partnerships Discussion Paper, J Hambric. 39. AMC – Outsourcing Asset Management and Disposition Discussion Paper, J Hambric, September 25, 2000. 40. AMC – Review of Allegations of Wrongdoing Pertaining to Lehman Brothers. 41. AMC – BPPN Asset Disposition Presentation, May 31, 2001. 42. F&A – Improving the Efficiency and Accuracy of Accounting Information Data Flow Scope of Work, Satterfield, 1 August-27 September 2000. 43. F&A – Report on Information Data Flow Within the Indonesian Bank Restructuring Agency (IBRA), Satterfield, October 2000. 44. F&A – Improving the Efficiency and Accuracy of the Finance and Accounting Reporting Process Scope of Work, Satterfield, 1 March-11 May 2001. 45. F&A – Exit Report on Results of Finance and Accounting Advisory Efforts, Satterfield, 11 May 2001. 46. P/IR – Communications and Investor Relations Scope of Work, Whitney, September 7-September 30, 2000. 47. P/IR – IBRA’s Media and Public Relations Activities (memo), Whitney, September 26, 2000. 48. P/IR – Issues Relating to Communications and Investor Relations Scope of Work, Olcheski, June 15- July 14, 2001. 49. P/IR – IBRA’s Media and Public Relations Activities Final Report of Mission, Olcheski, July 14, 2001. 50. P/IR – Speech Writing Seminar Presentation, Olcheski, July 2001. 51. P/IR – Proposed Press Release: IBRA Issues Strategic Plan for 2001-2004. 52. P/IR – Notes for Website Conference, June 28, 2001. 53. P/IR – Speech to the APEC Investment Mart 2001, Investment Lecture Session, Yantai, China, 10 June 2001. 54. P/IR – Press Kit Development Samples, Olcheski. 55. IT/MIS – Review and Evaluation of IT Systems Supporting Financial and Accounting Applications Scope of Work, Dickerson and Foley, June 11, 2001- June 30, 2001. 56. IT/MIS – Report on the Evaluation of IT Systems Supporting Financial and Accounting Applications at IBRA, Dickerson and Foley, July 18 2002. IBRA / JITF Evaluation Development Associates, Inc. Page 25 57. AP – Role and Function of a Chief Financial Officer and AMC Organizational Structure Scope of Work, Cooke, March 27-April 6, 2001. 58. AP – Role and Function of a Chief Financial Officer and AMC Organizational Structure Exit Report, Cooke, April 15, 2001. 59. AP – Follow on Advisory to Define the Role and Function of a Chief Financial Officer Scope of Work, Fechtmeyer, 30 May-4 August 2001. 60. AP – Follow on Advisory Work to Establish the CFO Function in IBRA Exit Report, Fechtmeyer, July 26, 2001. 61. AP – Follow on Advisory to Define the Role and Function of a Chief Financial Officer Scope of Work, Fechtmeyer, 1 April-27 April 2001. 62. AP – Follow on Advisory to Define the Role and Function of a Chief Financial Officer Exit Report, Fechtmeyer, 12 April 2001. 63. Rising to the Challenge in Asia: A Study of Financial Markets: Volume 6 - Indonesian Banks: Survival of the Fittest, Cathrine Chou, Asian Development Bank Online Publications, 1999. 64. Corporate Governance and Finance in East Asia: A Study of Indonesia, Republic of Korea, Malaysia, Philippines, and Thailand: Volume Two (Country Studies), Asian Development Bank, Ma. Virginita Capulong, David Edwards and Juzhong Zhuang, 2001. 65. BPPN/IBRA Website. 66. USAID Policy Determination, Implementing USAID Privatization Objectives, PD-14, June 16, 1986. Documents Related to JITF 1. Quarterly Progress Report of Short-Term Technical Specialists (Senior Financial Advisors), Technical Assistance for Corporate Debt Restructuring at JITF, PADCO, 16 July 2000-15 October 2000. 2. Quarterly Progress Report of Short-Term Technical Specialists (Senior Financial Advisors), Technical Assistance for Corporate Debt Restructuring at JITF, PADCO, 16 October 2000-15 January 2001. 3. Quarterly Progress Report of Short-Term Technical Specialists (Senior Financial Advisors), Technical Assistance for Corporate Debt Restructuring at JITF, PADCO, 16 January 2001-31 March 2001. 4. Annual Work Plan, Technical Assistance for Corporate Debt Restructuring at JITF, PADCO/Carana Corporation, 8 March 2001-8 March 2002. 5. Quarterly Progress Report, Technical Assistance for Corporate Debt Restructuring at JITF, PADCO-Carana Corporation, 8 March 2001-30 June 2001. 6. An Exit Strategy for the Jakarta Initiative Task Force: Historical Experience and Policy Options for the Future, June 2001 7. The Jakarta Initiative (JITF) USAID/ECG Presentation, 15 June 2001 8. Quarterly Progress Report, Technical Assistance for Corporate Debt Restructuring at JITF, PADCO-Carana Corporation, 1 July 2001-30 September 2001. IBRA / JITF Evaluation Development Associates, Inc. Page 26 9. Training and Conference Report: Signing of Memorandum of Understanding Between Jakarta Initiative Task Force and Jakarta Stock Exchange, Technical Assistance for Corporate Debt, 29 August 2001. 10. Quarterly Progress Report, Technical Assistance for Corporate Debt Restructuring at JITF, PADCO-Carana Corporation, 1 October 2001-31 December 2001. 11. The Jakarta Initiative (JITF) USAID/ECG Briefing, 5 November 2001 12. Quarterly Progress Report, Technical Assistance for Corporate Debt Restructuring at JITF, PADCO-Carana Corporation, 1 January 2002-31 March 2002. 13. Quarterly Progress Report, Technical Assistance for Corporate Debt Restructuring at JITF, PADCO-Carana Corporation, 1 April 2002-30 June 2002. 14. Quarterly Progress Report, Technical Assistance for Corporate Debt Restructuring at JITF, PADCO-Carana Corporation, 1 January 2002-31 March 2002. 15. Guide to Restructuring in Asia, Legal Issues: Indonesia, White & Case and Ali Budiardjo, Nugroho, Reksodiputro, and Financial Issues: Indonesia, PricewaterhouseCoopers, 2001. 16. IFC Briefing Notes: Indonesia: Corporate Debt Restructuring and Indonesia: Corporate Restructuring, Jemal-ud-din Kassum, April 29, 1998 17. Indonesian Corporate Debt Survey, Sixth Quarterly Report, March 2002 IBRA / JITF Evaluation Development Associates, Inc. Page 27 Annex 4 - Project Planning Calendar Sunday Monday Tuesday Wednesday Thursday Friday Saturday 21 Jul 22 Jul 23 Jul 24 Jul 25 Jul 26 Jul 27 Jul • 1330- USAID inception meeting • 1600 - Barents inception meeting – Denning • 1400 - JITF inception meeting – Russ Thirkell • 100 - IBRA VC Slamet • 1600 - Pat Cavanaugh • 0930 - Ramond Lee - JITF • 1045 - Bernard Nelson • 1400 - Willem Bake – JP Morgan Chase • 1600 - Bernard Drum – World Bank • 1730 - Suhail Chander – ABN￾Amro • Draft report outline 28 Jul 29 Jul 30 Jul 31 Jul 1 Aug 2 Aug 3 Aug • 1400 – Mark Edwards – Commonwealth Dev Corp • 0730 - JITF breakfast • 1000 – Quan Dinh and Lanna Lubis -USAID • 1500 – Prabowo - Bank Merincorp • 0800 -Martin Dinning • 1000 - Lee Babcock • 1600 - Samuel Tobing • 1700 – Ed Gustely • 1000 – Amesh Anand - Deloitte • 1400 – B Ruru, JITF • 1600 – Dasa Sutantio – IBRA • 1700 – Hugeng Gozali, IBRA • 1400 – A Saker - Ferrier Hodgson • 1200 – Bernard Drum, WB • 1100 - Soebowo Musa • 0800 - Mike Edwards, WB 4 Aug 5 Aug 6 Aug 7 Aug 8 Aug 9 Aug 10 Aug • 0900 - USAID briefing • 10:00 - Lee Babcock • 1230 – Tanri Abeng, former BUMN minister • 2:00 - Raymond van Beekum • 3:00 – Harry Sukadis • 4:00 – Bernard Nelson • Willem Bake – JP Morgan Chase • 1500 – Brian Watkins, JITF • 1830 - Martin Dinning • 0830 - Carmodi, PWC COP at BUMN • Michael Ryan - ADB • 1030 – David Cooke, Barents 11 Aug 12 Aug 13 Aug 14 Aug 15 Aug 16 Aug 17 Aug • 0900 - USAID briefing Room 020 Draft Executive Summary • 1100 – Nalin Rathod, Bakrie • 1400 – Sumantri Slamet, I Nyoman Sender, Syahrial, IBRA • • Final USAID Meetings • Project Wrap up • 1630 – AD & JM depart Indonesia 18 Aug 19 Aug 20 Aug 21 Aug 22 Aug 23 Aug 24 Aug Finalize report in Arlington, VA 25 Aug 26 Aug 27 Aug 28 Aug 29 Aug 30 Aug 31 Aug Finalize report in Arlington, VA • Final Report Submitted to USAID IBRA / JITF Evaluation Development Associates, Inc. Page 28 Annex 5 - Barents BUMN/IBRA and Carana JITF Project Evaluation Scope of Work I. SUMMARY An evaluation is required for two USAID/Indonesia Economic Growth Team activities: “Technical Assistance for IBRA” and “Jakarta Initiative Task Force Technical Assistance.” These activities are being carried out under SEGIR Financial Services and SEGIR GBTI IQC Task Orders with BARENTS and CARANA, respectively. Both activities have completion dates in early September 2002, although USAID tentatively plans to compete follow-on activities of the same nature and would like this evaluation to inform their formulation. The existing activities, henceforth known as BARENTS BUMN/IBRA and CARANA JITF, consist of technical assistance and short-term training. Purpose: The purpose of this evaluation is to determine the extent to which the BUMN/IBRA and JITF USAID-supported activities are achieving their purposes, to compile success stories and lessons learned and to make recommendations for the functioning of the technical assistance teams and potential follow-on activities. Within this context the evaluation will specifically examine questions of efficiency, effectiveness, relevance, practicality/adequacy of funding, management of contracts, personnel, substantial involvement of USAID and impact. Contract Mechanism: The evaluation will be performed under the Development Information Evaluation Services IQC. The intent is that the chosen contractor will field a three-person team consisting of two expatriates and one local hire. They will undertake the evaluation and submit their findings within 60 days of receipt of the task order. II. BACKGROUND The “Asian Economic Crisis” and associated political events in 1997/8 highlighted and exposed serious flaws in the Indonesian banking sector and the disposal of non-performing loans (NPLs) and assets. As a result of the economic crisis, new institutions were put into place including the Indonesian Bank Restructuring Agency (IBRA) and the Jakarta Initiative Task Force (JITF). BUMN/IBRA IBRA or BPPN was created in January 1998 as a limited life agency with a sunset date of February 2004 to promote economic recovery. IBRA formerly operated as an agency of the Ministry of Finance (MOF) but has moved recently under the Ministry of State Owned Enterprises (BUMN). IBRA has four main functions: • Manage the commercial bank liabilities Government Guarantee Scheme. • Close failed banks, capitalize surviving banks and restructure, merge, or sell banks taken over. • Transfer, restructure, and liquidate assets of failed banks and non-performing loans of recapitalized banks. IBRA / JITF Evaluation Development Associates, Inc. Page 29 • Sell corporate assets transferred by former bank shareholders. To carry out its tasks, IBRA has three main divisions that cover the asset management, recovery and sale of a whole variety of assets, and two support divisions. These are Asset Management Credit (“AMC”), Asset Management Investment (“AMI”) and Bank Restructuring Unit (“BRU”). The two supporting divisions are Risk Management and Legal and Finance and Administration. The Asset Management-Credit (AMC) division is the repository of core and non-core assets of failed banks and the non-performing loans (NPLs) of state-owned and recapitalized banks. AMC’s mission is to dispose of its entire portfolio by February 2004. To date AMC is planning to offer for sale assets with face value of Rp. 126 -128 trillion within 2002. These assets will be sold through regular auctions, the formation of joint ventures, the formation of two collaterized debt obligations (asset-backed securitizations) and two holding companies, as well as through AMC’s regular, periodic loan sales via public auction, and other minor programs. Major emphasis will be placed on the joint venture, collaterized debt obligation and holding companies amounting to approximately Rp. 61.15 trillion or US$ 6.1 billion. The Asset Management-Investment (AMI) division initially had control of non-bank shares, securities, and corporate assets valued at Rp. 175 trillion. Presently, AMI has disposed of over 70% of its portfolio. At the beginning of the financial crisis, and to stop the run on the banking system, the GOI gave a blanket guarantee to all bank depositors, closed 69 banks and took over many other banks. The Bank Restructuring (BRU) division has managed the 12 banks taken over by the GOI. BRU has recapitalized and merged many banks into Bank Danamon and sold the BCA. Presently, BRU is also managing the current “blanket deposit guarantee scheme.” From March 1999 to March 2000, IBRA achieved its annual sales recovery target of Rp. 17 trillion. From March to December 2000, IBRA achieved its sales recovery target of Rp. 18.9 trillion and from Jan to Dec. 2001 IBRA achieved its goal of Rp. 27 trillion in sales recovery and Rp. 10 trillion in Bond/Asset Swaps. The BARENTS BUMN/IBRA IQC: At the broadest level, the purpose of USAID involvement in IBRA and BUMN is to help the GOI in meeting its recovery plan as set out in the IMF Letter of Intent (LOI). USAID intends to assist the GOI economic recovery through the selling of assets to the private sector in a transparent manner, as quickly as possible, to assist the reduction of the fiscal deficit. USAID involvement in IBRA and BUMN should be seen in the overall context of its involvement in finance and banking reform, other economic recovery efforts and selected institutional building. To assist Indonesia to overcome its problems, USAID has funded technical assistance to IBRA and BUMN, including advice, analyses and seminars/training. The Barents team has been in Indonesia from May 2000 to the present and has provided both short and long-term technical advisors. The SOW for IBRA covered Technical Assistance in six main areas: • Asset Management Credit • Asset Management Investment IBRA / JITF Evaluation Development Associates, Inc. Page 30 • Bank Investment and Restructuring • Finance and Accounting • Information Technology • Risk Management The Barents’ team also has offered short-term technical assistance in many other areas, such as Public relations, the design of the CFO position and TA to the Ministry of State Owned Enterprises (BUMN), which were not envisaged in the original SOW. This was particularly important in the case of BUMN where a major shift in assistance priorities occurred during the summer of 2001 in response to the shifting political situation and key changes in the leadership of BUMN and IBRA. JITF Corporate debt restructuring is one of a number of preconditions for full economic revival. As part of a multi-faceted economic recovery effort, the Government of Indonesia created the Jakarta Initiative Task Force (JITF) in November of 1998. The Jakarta Initiative was one component of the government’s strategy to deal with bad corporate debt in the financial system, the other components being: (a) enhanced bankruptcy legislation and strengthened commercial courts; (b) foreign exchange support for restructured companies through the Indonesian Debt Restructuring Agency (INDRA); and (c) strengthening Indonesia’s financial sector under reform programs supported by the World Bank and the International Monetary Fund (IMF). The government’s strategy concurrently included its creation of the Indonesian Bank Restructuring Agency (IBRA also known as BPPN) to handle bank restructuring and disposal and management of impaired assets. The significance of problems of the program during the first year of its operations were not lost on the GOI, the World Bank, IMF, or USAID. To support the JITF financially, the GOI and the World Bank signed a $31.5 million loan agreement, which became effective June 1999. It was later reduced to $10 million. USAID added the first phase of its technical assistance grant support to the program in August 1999. By early 2000, a series of revisions had been accomplished to attempt to rectify many past problems, notably (but not limited to) a clear legal basis, a meaningful incentive and sanction system, coordination with other government institutions, staffing and establishment of a Project Implementation Unit (PIU). Perhaps the most important development was the creation of the FSPC, by Presidential Decree, December 28, 1999. FSPC consists of a ministerial level committee which provides JITF its mandate for carrying out structured mediation procedures, has the final say over Indonesia’s major corporate debt restructuring schemes, and coordinates certain of IBRA activities with the JITF. Since April 2000, supported by the above revisions and with the appointment of JITF’s present chairman and its chief operating officer, JITF has consistently met its IMF numerical targets for obtaining memoranda of understanding in aggregate debt. After its initial problems, JITF appears to have been successful in developing fair, credible and transparent processes and mechanisms for assisting negotiations between debtors and creditors and obtaining memoranda of understanding in a relatively short period. JITF applies a time-bound mediation procedure. The process mandated by the GOI offers incentives for cooperative behavior, including assistance with regulatory facilitation, provision of tax incentives and protections from de-listing for companies listed on Jakarta Stock Exchange. IBRA / JITF Evaluation Development Associates, Inc. Page 31 Uncooperative behavior can result in referral to the FSPC, which may thereafter refer the matter to the Attorney General’s office for the institution of insolvency proceedings. At 30 November 2000, the sum of loans to Indonesian corporations was estimated at US$ equivalent of US$119 billion. Impaired debt at the time was estimated at US$65 billion. As of mid-December IBRA was working towards the resolution of approximately 46 percent of all distressed debt in Indonesia, while JITF was assisting to restructure an additional 17 percent. At 31 December 2000, the pace of restructuring had recently accelerated, with IBRA having reached the final stage of restructuring negotiations with 88 percent of the country’s top 21 obligors, and JITF having achieved its MOU target of US$9.4 billion. At the same time, the quality of debt restructuring had improved, with an increasing number of deals using debt-to-equity and debt-to￾convertible bond swaps at IBRA and JITF. Still, the weakening of the Rupiah threatened to bring a second round of restructuring as many deals were negotiated assuming a Rupiah-to-US$ exchange rate of 8,000. By 30 April 2001, JITF had reached commercial terms under memoranda of understanding for an aggregate total of US$11.8 billion, involving 38 cases. Of these, 19 cases (51 percent) had signed master restructuring agreements and ten cases (26 percent) had either arrived at or had made the first payment. JITF, with a professional staff of 15, presently handles 122 cases valued in excess of US$ 22 billion, of which 68 cases with debt value of $14.2 billion had signed MOUs by December 31, 2001. By April 2002 the total amount of MOUs signed had reached $15.5 billion. JITF was originally scheduled to close down by December 31, 2002. However, with the help of USAID funding, the GOI considered options for a feasible “exit strategy” for JITF that included the need to operate beyond 2002. Given the JITF late start-up, effectiveness of the program and the large amount of distressed debt still to be resolved, the GOI requested and the World Bank has agreed to extend the World Bank JITF loan through December 2003, with further extension still under discussion. The CARANA JITF IQC: The JITF mediation team comprises 8 mediators: 3 USAID, 2 World Bank, 3 nationals and 6 case managers. With a total staff of 25 persons, the overall task force collaborates together in bringing cases through the mediation process. The USAID-financed CARANA team located at JITF has provided advice and mediation services to facilitate debtor-creditor-restructuring agreements since late 1999. USAID’s team either has the lead, or substantively assists and trains Indonesians across the majority of JITF’s portfolio. USAID advisors have been essential to the JITF effort as they represent 3 of the 5 expatriate advisors located at JITF. The SOW for CARANA covers the following tasks: • Provide three senior corporate debt restructuring specialist/advisors to work directly with the staff of the JITF. • Advise JITF senior management and other senior government officials on all aspects of corporate debt restructuring including specific recommendations on legal and regulatory policy, and outreach and public education. IBRA / JITF Evaluation Development Associates, Inc. Page 32 • Train local personal to progressively take over duties held by senior advisors, with the intent to ensure JITF sustainability. III. EVALUATION REQUIREMENTS General Requirements The contractor should undertake a broad-based assessment of the extent to which the BUMN/IBRA and JITF USAID-supported activities are achieving their purposes, compile success stories and draw lessons learned and make recommendations for the improved functioning of existing technical assistance teams and potential follow-on activities. Within this context the evaluation will specifically examine questions of efficiency, effectiveness, relevance, practicality/adequacy of funding, management of contracts, personnel, substantial involvement of USAID and impact. This will include looking at the activities required in the initial SOWs and important changes made during the course of implementation. The contractor should recommend improvements for present activities and considerations for potential follow-on activities, as appropriate. Specific Task Elements The following points should be addressed, although not necessarily in the order or structure provided below. Efficiency: Are the contracts at IBRA/BUMN and JITF proving to be a cost-effective means of addressing the project objectives? In the opinion of the evaluators, could the progress in evidence to date have been achieved more efficiently with a different choice of activities within each of the two projects? Should the administration of the contracts be reconfigured in the future to accomplish project objectives more efficiently? Effectiveness: Assess the extent to which results, including the nature and quality of outputs, have been or are being produced and achieved by the two contracts. Has visible progress been made? What concrete accomplishments in IBRA/ BUMN and in JITF can be traced to their contracts? Are the IBRA/BUMN and JITF activities on target to achieve the design results? If not, what changes are required for that end? Relevance: Are the IBRA/BUMN and JITF activities relevant to the stated purpose of their contracts and the underlying projects? To what extent are the projects addressing or have addressed problems of high priority, as viewed by their stakeholders? Practicality/Adequacy of Funding: Are the IBRA/BUMN and JITF objectives reasonable in terms of their activity designs? Are the levels of resources allocated to the two contracts consistent with requirements for success as foreseen in the design process? Management of Contracts: Have the contractors effectively structured and exercised management control over the IBRA/BUMN and JITF activities and the commitment and disbursement of resources? Has required reporting been substantively of reasonable quality and provided in timely fashion? Personnel: Have the individuals funded under the contract been appropriate to the tasks? What is the result and impact from the 3 long-term advisors at IBRA and BUMN? What is the result and impact from the 3 long-term advisors at JITF? What is the result and impact of short-term advisors IBRA / JITF Evaluation Development Associates, Inc. Page 33 at IBRA and BUMN and at JITF? What are the working relationships within the two teams? What is the quality of the working relationships of the advisory teams with their counterparts? Substantial Involvement of USAID: Has USAID provided appropriate and timely input into project strategy and decisions? Are there suggested improvements? Impact: Has the project had substantial beneficial impacts on the IBRA and BUMN? Has the project had substantial beneficial impacts on the JITF? Are there any success stories to be compiled or lessons to be drawn? Recommendations: The evaluation will explicitly address the need for design and/or operational changes that could be made to strengthen the TA teams, including addressing any questions related to the quality of the advisors, management and conduct of the projects. It should also present recommendations for potential follow-on activities. IV. TEAM COMPOSITION AND METHODOLOGY The intention is to select a three–person team consisting of two highly experienced expatriates and a highly knowledgeable local consultant. The expatriate team members must have advanced degrees in Economics, Finance or Management (PhD/MBA) and substantial relevant field experience in financial system and banking sector reform. Specific experience with financial and banking reform and restructuring efforts similar to the programs described in Section II is required. Specific recent knowledge of Indonesia’s recent economic and financial reform efforts [last 3-4 years] is highly desirable. Specific relevant experience in similar overseas reform efforts is also highly desirable. The local consultant should have an advanced degree in Economics, Finance or Management (PhD/MBA/MA/MS) and be highly conversant with the history and issues involved with the institutions/bodies being assisted: IBRA, BUMN and JITF. Ideally the individual will be a local commentator on the banking and finance sectors, understand the importance of the reform and restructuring efforts, know the cast of characters involved and be able to provide the team with entrée and background not readily or quickly attainable by non-resident experts. This type of individual would probably most easily be found in university faculties, research organizations or non￾governmental organizations. Sensitivity should be paid to potential conflicts of interest in selecting the proposed team member. Ideally the skills, experience and background of the three-team members will complement one another such that the team as a whole is greater than the sum of its parts. The team will carry out the evaluation through: • Review of project documentation for both the BARENTS IBRA/BUMN and CARANA JITF teams (contract agreements, Annual Work Plans, Project Monitoring Plans, Annual Reports, Quarterly reports, etc.); • Interviews with USAID staff, BARENTS BUMN/IBRA team members, CARANA JITF team members, Indonesian counterpart personnel and appropriate IBRD/IMF staff; • Review and assessment of a sampling of studies, reports, and analyses funded under the contract. IBRA / JITF Evaluation Development Associates, Inc. Page 34 V. DELIVERABLES The evaluation team will be expected to provide the following: • Upon arrival the evaluation team will meet with the USAID Economic Growth Team Leader and/or his designees and provide its proposed evaluation plan and methodology. • One week prior to departure, the evaluation team will brief the Economic Growth Team Leader and/or his designees to present the team’s major findings, conclusions and recommendations. A draft executive summary, including key findings, conclusions and recommendations, and an outline of the report will be provided. • A draft written report, which details the team’s findings, shall be provided to USAID/Indonesia before the team leaves the country, in both electronic and written form. This draft should be in relatively final form suitable for sharing with contractors and counterparts for their comment. USAID will provide written comments, if any, to the contractor within 14 calendar days of initial submission. • A final written report, which details the team’s findings, shall be provided to USAID/Indonesia within 21 calendar days after the expatriate team leaves the country. Five hard copies should be provided to Mr. Quan Dinh, USAID/Indonesia at that time. At the same time an electronic copy in Microsoft Word format shall be emailed to Ms. llubis@usaid.gov. A copy of the final report also should be provided to PPC/CDIE/DI. The report will include all the specified requirements of the SOW, including success stories, lessons learned and recommendations for the present and future activities. VI. PERIOD OF PERFORMANCE Four weeks in Jakarta, Indonesia, to start as soon as possible. USAID/Jakarta expects that the team will begin its work by latter half of June 2002. VII. REPORTING AND SUPERVISION The evaluation team shall report to the ECG Team Leader or his designee. Further, • The team will meet with its USAID liaison officer once per week (or as otherwise agreed to by that officer) and provide a brief report of activities and progress. • The team will be responsive to USAID suggestions, and will observe any guidance given as to political sensitivities, progress reporting, and in-country travel restrictions. VIII. AUTHORIZED WORK WEEK Contractor is authorized a six-day workweek without premium pay. Clearances: ECG: QDinh (draft) date: 5/8/02 ECG: RRucker (draft) date: 5/15/02 ECG: PDeuster(draft) date: *** END *** IBRA-JITF Evaluation Report-Revised-EvalIQC-12 IBRA / JITF Evaluation Development Associates, Inc. Page 35 Annex 6 - JITF Response to the Report October 10, 2002 Via Hand Delivery Mr. Terry Myers The USAID – ECG Team Jakarta Re: JITF Response to Evaluation of Padco/Carana JITF Technical Assistance Project (the “Evaluation”) Dear Terry: The Jakarta Initiative Task Force (“JITF”) is in receipt of the revised draft Evaluation from the USAID review team which was submitted to us on October 2. The comments of the JITF are set forth below. Acknowledgment of Success. The JITF would note that the Evaluation acknowledges the JITF’s success based upon the IMF Letter of Intent performance criteria. The following passage is particularly instructive, and sets the context for the JITF’s remaining comments: [The JITF] is credited with the successful mediation of over USD 16 billion in debt, and has for each of the past two years exceeded the IMF agreed quantitative targets. The CP team has contributed to that success. Having established good relations with fellow consultants, and JITF management and staff the CP team has made several important contributions to JITF. JITF’s professional discipline has been widely acknowledged and we believe that is in no small part due to the CP team’s contributions …. Evaluation, Executive Summary, page i. Given the limited inquiry undertaken by the Evaluation team (which, to the knowledge of the JITF, only interview 1 of 120 JITF debtors, 3 of 77 JITF creditors, and did not examine JITF case files and elected not to observe any JITF mediation sessions),1 the foregoing is the only supportable conclusion which the Evaluation can reach. Indeed, the additional policy analysis 1 In their report, the reviewers say that they were told all mediations were confidential. However, the review team could have requested consent to attend from the participants. This was not done. IBRA / JITF Evaluation Development Associates, Inc. Page 36 that peppers the Evaluation is, in our opinion, both inappropriate to the JITF component and beyond the scope of work under which the Evaluation was prepared. Criticism of Performance Criteria. Following its acknowledgment of the JITF’s success under the IMF performance criteria, the Evaluation goes on to criticize the performance criteria themselves. According to the Evaluation: [W]e would suggest that looking only at the value of debt restructuring agreements is an inadequate measure. The real measure of JITF’s success is whether companies … honor the new agreements, receive new credits, maintain or increase employment, and regain its [sic] market competitiveness. Evaluation, Executive Summary, page ii. It is here that the Evaluation ignores the fundamental intent and mission of the JITF. It bears repeating that the JITF was (and is) an interim response to a crisis situation, being intended to assist with immediate issues of corporate loan default. In this regard, those who designed the JITF made the decision early on that achieving accelerated agreement between debtors and creditors was an essential element in the overall crisis-response strategy, or, in other words, that reaching such agreements constituted an end unto itself. In this light, the IMF performance criteria (MOU’s executed) make perfect sense as a tool to measure JITF effectiveness. Of course, implicit in this vision of the JITF’s mission are certain assumptions, namely, that early and quick debt restructuring improves economic output and standard of living by maintaining employment, export activity and tax base. It might be valid to question these assumptions, and the JITF is ready and willing to engage in any such inquiry. However, such inquiry must be comprehensive, and not based on a cursory examination of limited data, as was the case with the Evaluation. The JITF also questions whether (a) the cursory analysis set forth in the Evaluation was ever part of the scope of work for the Evaluation, and (b) assuming it was, whether there is any feasible way of drawing a direct correlation between any ADR program and macroeconomic fundamentals. As to the latter query, the Evaluation does correct the chief shortcoming of its predecessor draft by providing a list of variables assumed to be relevant to JITF effectiveness in place of the IMF performance criteria. Specifically, the Evaluation suggests that true JITF effectiveness should be measured by the: [N]umber of employees hired/terminated by borrower for 36 months from date of registration with JITF; amount of new credits extended to the borrower as part of the JITF facilitated restructuring or within 24 months of the legal restructuring documentation being signed; and the number of cases where the debtor defaults on the new restructuring agreement. IBRA / JITF Evaluation Development Associates, Inc. Page 37 Evaluation, Section D, page 7 (emphasis added).2 The JITF agrees that these factors are relevant. However, the JITF would note that each of these items requires monitoring far beyond the life span of the JITF. Given that the JITF only began restructuring significant amounts of debt in the second half of 2000, and given that the program is set to expire by its own terms in 2003, these criteria can therefore be used only as a post hoc method of judging JITF effectiveness, rather than as a means to determine at this point whether to continue USAID support. In other words, by arguing that future variables (i.e., employment 36 months from now) fail to support a current decision to continue USAID involvement, the Evaluation assumes its own conclusion. This argument is reminiscent of the inquisitorial practice of tossing suspected witches into a river to see if they would float, and is irrelevant to the Evaluation’s conclusion that USAID resources should be shifted away from the JITF. Other Concerns. Of course, none of the foregoing prevents an argument that either (a) the linkage between individual JITF mediators and mediation effectiveness remains to be proven, or (b) USAID should allocate its resources elsewhere as a matter of setting relative priority between programs. Indeed, the Evaluation is quick to lodge both criticisms. As to the former, the JITF wishes to observe that its doors have been, and remain, open to such inquiry. However, the fact remains that the Evaluation team never requested permission from participants to observe specific mediations, nor to examine individual case success records for the purpose of evaluating individual mediator performance. Given this lapse, it is extremely misleading for the Evaluation to point to a lack of such evidence as a reason for discontinuing USAID support. As to the later argument, resource allocation is, of course, a decision for USAID to make, and one is left wondering why the Evaluation team took this task upon itself. Nevertheless, it bears mentioning that the JITF is one of the few development programs in Indonesia that is recognized to be successful. There are, of course, higher profile programs (IBRA, for instance) that are in the position to justify many foreign consultants, and no doubt any number of new initiatives can be identified with the assistance of enterprising contractors. Nevertheless, there has been precious little demonstration that prior technical assistance has added anything to the operation of these programs, and it is questionable whether this situation will change in the future. On the other hand, as the Evaluation itself acknowledges, the CP team has made concrete contributions to the JITF, and has been partially responsible for its success. Surely, USAID must acknowledge, unlike the Evaluation team, that this factor is highly relevant in determining resource allocation. 2 As to the other variables suggested in the Evaluation, debt restructured as a percentage of total debt, average time from registration to final legal agreement and staff time allocated per case, the JITF would like to point out that information relating to such items was, and is, available to any interested parties, though the JITF doubts whether these variables impact significantly on macroeconomic fundamentals, as the Evaluation seems to suggest. IBRA / JITF Evaluation Development Associates, Inc. Page 38 As I have mentioned before, the JITF is grateful for the assistance of USAID over the past two years. This assistance has been constructive and appreciated, and we look forward to continuing the relationship over the coming year. Should you have any questions, do not hesitate to call. Very truly yours, Samuel Tobing CC. 1. Mr. Bacelius Ruru, JITF Chairman 2. Mr. Jon D. Lindborg, Deputy Director USAID Jakarta 3. Mr. Paul Deuster, USAID Jakarta 4. Mr. Bruno Cornelio, USAID Jakarta 5. Mr. Quan Dinh, USAID Jakarta 6. Mrs. Lanna Lubis, USAID Jakarta IBRA-JITF Evaluation Report-Revised – Microsoft World