FINAL REPORT CYPRUS BI-COMMUNAL DEVELOPMENT PROGRAM EVALUATION Submitted to: United States Agency for International Development Under: Evaluation IQC No. AEP-I-00-00-00023-00 In collaboration with: Nathan Associates, Inc. Evaluation team: Richard Blue, Team Leader Vivikka Molldrem Craig Webster Alejandro Gonzales Mine Yucel Yiorghos Leventis Submitted by: Development Associates, Inc. 1730 North Lynn Street Arlington, VA 22209-2023 May 25, 2004 TABLE OF CONTENTS Page No. Preface..................................................................................................................................... iv Executive Summary................................................................................................................. v I. Introduction.................................................................................................................. 1 II. Methodology................................................................................................................ 1 III. Program Purpose..........................................................................................................3 IV. Findings on BDP Management.................................................................................... 4 A. Program Structure ............................................................................................ 4 1. Stakeholder Relationships.................................................................... 4 2. Policy-Strategy Management............................................................... 6 3. The roles of the PMU........................................................................... 7 4. Characteristics of program structure and management........................ 9 B. Program Strategy and Decision-making ........................................................ 12 1. Decision making process: Operational Rules for the PMU............... 12 2. Grant Making Review and Decision Processes................................. 14 3. The Contingency Plan........................................................................ 19 C. Allocation of funds by sector......................................................................... 19 1. Changes in Spending over Time (Overview)..................................... 19 2. Governance and Civil Society Grants-Patterns and Changes............ 20 3. Special Initiatives............................................................................... 22 V. Assessment of Opinion Leader Perceptions of the Program ..................................... 23 A. Approach........................................................................................................ 23 B. Findings.......................................................................................................... 23 Cyprus Bi-communal Development i May 25, 2004 Program Evaluation Development Associates, Inc. Table of Contents (Continued) Page No. VI. Findings by Sector ..................................................................................................... 25 A. Infrastructure and Environment ..................................................................... 25 1. Background on Environment and Public Infrastructure .................... 25 2. Findings on Environment projects..................................................... 25 3. Infrastructure projects........................................................................ 27 B. Other Implementing Agencies....................................................................... 29 1. Description of the programs............................................................... 29 2. Management....................................................................................... 30 3. Strategy .............................................................................................. 30 4. Effectiveness...................................................................................... 31 C. Civil Society................................................................................................... 32 1. Description of BDP NGO Sector Support ......................................... 32 2. Use of Technology to Foster Bi-Communal Relationships............... 33 3. Management....................................................................................... 34 4. Strategy .............................................................................................. 35 5. Effectiveness...................................................................................... 36 VII. Conclusions................................................................................................................ 39 A. General Conclusions...................................................................................... 39 1. Accomplishments............................................................................... 39 2. Challenges.......................................................................................... 40 B. Performance of BDP Decision-Making and Implementation........................ 42 1. Accomplishments............................................................................... 42 2. Challenges.......................................................................................... 43 VIII. Recommendations: Making The Settlement Work..................................................... 45 Cyprus Bi-communal Development ii May 25, 2004 Program Evaluation Development Associates, Inc. Cyprus Bi-communal Development iii May 25, 2004 Program Evaluation PREFACE This evaluation began shortly before negotiations began between Greek Cypriot and Turkish Cypriot leaders on the United Nations formula for reconciliation and unification. There was great hope that a “historical moment” was at hand, by which the United Nations, using the “Annan Plan” developed under the leadership of the Secretary-General, could assist both sides to find a path to peace. The UN, supported by the international community, hoped that a settlement would be reached prior to the entry of the Republic of Cyprus into the European Union on May 1, 2004. The final corrections on this report were completed the day after the Greek Cypriot citizens decisively rejected the UN plan by their vote in the UN sponsored referendum of April 24, 2004. This rejection was in sharp contrast to the substantial majority of Turkish Cypriot citizens who voted in favor of the UN settlement. It is perhaps too easy to link the Bi-Communal Development Program’s expenditure of $60.5 million since 1998 with the outcome by concluding that the programs and projects financed by BDP with the specific purpose of promoting bi-communal collaboration and reconciliation failed to achieve their purpose. Put in the context of the approximately $450 million invested by the US Government through the United Nations since 1974, mostly for humanitarian relief and reconstruction in the Greek Cypriot sector (under the original 80-20 formula), it may be argued that a BDP type program should have begun much earlier to develop an active constituency for peace in the Greek Cypriot community, much as it was able to do among the Turkish Cypriots. Turkish Cypriot NGOs, many of them supported by BDP, were active proponents of the settlement, and helped to convince others that this was their best hope to enter into the modern world. Here too, other, more powerful motives also were in play. The US effort, along with the UN and others, has accomplished much since the inception of the program and, as we report here, since the establishment of the US-UNDP BDP. We also identify where we believe improvements could be made in the program, mainly in the context of our hope that the UN Plan would be accepted. With the Greek Cypriot rejection of the plan, it is not clear whether there will be, or should be a continuation of the BDP, or any other foreign grant program dedicated to convincing Greek Cypriots to “vote for peace”. The factors which caused these citizens to reject a settlement may well be too powerful for any such program to succeed. There was no time to revise the report to reflect this rather sad outcome. We offer our analysis, judgments and recommendations as they stood in early April when there was still hope that something might reverse what many already predicted would be a negative outcome. We have nothing but the greatest admiration for those Greek Cypriot and Turkish Cypriot citizens in government and in the civil society who believe in a different future and have worked to achieve it. Also, we applaud the unswerving commitment of the many people in the UN agencies and in the US Government who worked very hard over many years under difficult circumstances. We thank them all for sharing their knowledge, their concerns, and their hospitality. Where we have disagreed, we do so with respect and the awareness that we may be wrong. As the UN Special Envoy, Alvaro De Soto, said, we will know “in the fullness of time.” Cyprus Bi-communal Development iv May 25, 2004 Program Evaluation EXECUTIVE SUMMARY I. INTRODUCTION USAID contracted with Development Associates, Inc. to assess the strategy, attainment of objectives, and implementation of the Bi-Communal Development Program (BDP). The evaluation’s findings, conclusions and recommendations will inform the design of a follow-on “BDP-like” project, expected to begin sometime in FY 2004. The Bi-Communal Development Program (BDP) is one of three programs of the U.S. government to implement the Congressional directive (contained in a $15 million annual ESF earmark) to support bi-communal projects and measures aimed at reunification of the island and designed to reduce tensions and promote peace and cooperation between the two communities on Cyprus.1 The other two are the Cyprus America Scholarship Program implemented by the Cyprus Fulbright Commission and managed by the Department of State; and the Bi-communal Support Program, implemented by Amideast and managed by the Department of State. Prior to 1998, USAID-funded bi-communal development programs were carried out by the UNHCR, but over time the need for relief programs declined and program emphasis shifted to large infrastructure efforts. When UNHCR decided to close all but its asylum office in Nicosia, USAID signed a grant agreement for a $30.5 million program with UNDP2 . Since UNDP does hot have offices in the Republic of Cyprus (ROC), the UNDP asked the UN Office for Project Services (UNOPS) to execute the program. The BDP grant between USAID and UNDP was signed in March 1998 and UNOPS set up a Project Management Unit (PMU) in the UN Protected Area (UNPA) at Nicosia in April, 1998. The grant was later increased for a total USAID grant contribution through 2004 of $60 million. II. METHODOLOGY An evaluation team made up of 4 experienced US citizens, one Greek Cypriot citizen and one Turkish Cypriot citizen conducted the evaluation in February and March 2004. Standard USAID Rapid Appraisal methods were used, including interviews with stakeholders, extensive documentation review, assessment of policy and program management standards and procedures, small sample surveys of program grantees, and interviews with Opinion Leaders from the Greek Cypriot and Turkish Cypriot communities. Ratings by policy and program managers of the bi-communal achievements of the various projects under the BDP were provided at the request of the evaluation team. Although not rigorously scientific, these ratings do reflect the views of those who know the program best, and showed a surprising ability to discriminate between what worked and what did not. 1 Foreign Operations, Export Financing, and Related Programs Appropriation Bill, 2004 2 USAID contributed $30 million and UNDP $500,000 for the period 1998 through 2001. Cyprus Bi-communal Development v May 25, 2004 Program Evaluation Development Associates, Inc. Finally, working draft findings, conclusions and recommendations were presented to most stakeholders, resulting in a number of comments and corrections that improved substantially the team’s understanding and analysis of the program. The team made every effort to substantiate conclusions with facts and interview findings. Inevitably, our conclusions are formed in the part by judgments based on the experience and expertise of the team members. The team takes responsibility for the conclusions and recommendations made in this report. We wish to thank sincerely the many people who gave us their time, views, and expertise during the conduct of this evaluation. We are especially grateful to USAID and US Embassy staff officers, to UNOPS staff current and former, and to the UNDP leaders who took an interest in our efforts. III. MAJOR CONCLUSIONS A. GENERAL CONCLUSIONS 1. Accomplishments a. The BDP is generally well known among opinion leaders on both sides, often as UNOPS, and is perceived by most to have had an impact on promoting appropriate contacts, cooperation and the possibility of support for a peaceful settlement, especially among government officials and younger Cypriots. b. Those activities that involved substantive contacts between Turkish and Greek Cypriots about subjects important to both sides did succeed in fostering effective working relations among people from a broad range of professions and interests. c. Those activities that encouraged common infrastructure and common systems for addressing infrastructure and economic issues, even though they involved collaboration of relatively small numbers of technicians and laborers, have improved the ability of TC and GC officials to cooperate on their own should a settlement occur. This is true as well of a more limited number of NGOs sponsored by the BDP. d. BDP projects have been a material and possibly symbolic manifestation of the commitment of the United States and the international community, through the UN, to the search for a peaceful settlement. They have created venues for visibility and positive public relations for the US Embassy and for the UN. . e. BDP Civil Society programs encouraged and facilitated contacts and possibilities for interaction between Greek and Turkish Cypriot citizens that would not otherwise have been possible given the prohibitions imposed by authorities on both sides. f. Making substantial funds available to civil society organizations supported increased NGO activity in a wide variety of areas, as well as providing the material support for Cyprus Bi-communal Development vi May 25, 2004 Program Evaluation Development Associates, Inc. improved capacity by many of these organizations. The establishment of the Management Center on the Turkish Cypriot side is an outstanding example. g. The UNOPS PMU, faced with a restrictive political environment hostile to bi-communal interaction, demonstrated extraordinary creativity and energy in implementing the large public works, environment and animal health programs inherited from the UNHCR period. The compelling need to assume much of the responsibility for procurement, technical support, and financial control of these projects substantially defined the PMU’s character and operational procedures during the first two years of the program and, to some degree, through the six year life of the grant agreement. 2. Challenges a. The political purpose of the program was well understood by all stakeholders, but there were varying interpretations of how to define ‘bi-communal’ and particularly what the best means were to that end. Following several efforts to develop more directive strategies for grant making, the broad categories developed in the Flexible Framework Fostering Rapprochement were adopted. FFFR may have been useful for categorizing bi￾communal activities at a time when the political restrictions were so harsh that a narrow definition would have cut off all activities. FFFR did not serve the purpose of a strategy, however, and in today’s environment, with a settlement at hand, a more focused set of objectives and indicators is needed. b. The NGO side of the program, in part because of the restrictive political environment, never gained the level of visibility and focus achieved by the public works side. Attempts by outside consultants to develop for the BDP a more coherent civil society strategy that would support the political objective did not become fully internalized as part of the decision-making or operational criteria of the NGO program. c. The size of the grants made to many of the NGOs, especially in the first three years of the program, appears excessive by most standards used by USAID and other donors in the East European and NIS region, including in institutionally well-developed countries such as Poland. Whether driven by the high costs of the Cyprus economy, or by other factors, the dollar value of the grants (upwards of $100 000) may have strained the absorptive capacity of many smaller NGOs, as well as producing a bias towards non-profit organizations already well established. (The ceiling for maximum grant size was reduced in the latter years of the project from $100,000 to $50,000.) d. In the smaller IA projects and in NGO projects, development impact identifiable to the team has been very limited, because of the scattering of funds over many organizations and the emphasis on one-off activities rather than on strengthening the organizations that offer them. While development was not the objective of the program, seeking good development outcomes was not inconsistent with bi-communalism, and indeed may have helped to contribute to this objective, as was the case in several of the public works and animal health projects. While the NGO sector appears to be flourishing, it is not sustainable at the current cost level without significant foreign funding. Cyprus Bi-communal Development vii May 25, 2004 Program Evaluation Development Associates, Inc. e. The desire to avoid negative publicity or the appearance of political interference may also explain the predominance of grants to health (30%) and environment (19%) NGOs, particularly in the first three years of the project. As BDP became more established and the political constraints relaxed, funding shifted to peace/Mediation NGOs (11%), the latter made mostly in 2002/3. f. A political program in a highly volatile environment can be expected to experience a high failure rate in terms of activities that do not improve bi-communal tolerance and cooperation, and do not result in a significant level of enhanced capacity of value to the challenges of making a peaceful settlement work. However, absent valid indicators of program performance and even a modest effort to evaluate at the project level the extent of bi-communal results achieved, it is not possible to objectively assess whether the failure rate was either lower or higher than an acceptable level. g. The ‘strategy’ for implementation of the NGO and for much of the IA parts of the agreement was largely reactive rather than proactive. Little effort was made to direct grantees toward specific program areas or objectives through the various BDP call for proposal mechanisms. This contributed to the diverse and seemingly disconnected character of the NGO grant portfolio and to a repetitive “proposal bombardment” to the PSC by Implementing Agencies. As the possibilities of a settlement improved in 2002, the Special Initiative grant was established permitting the beginnings of a more directive program that remained within control of the Embassy-PMU decision makers. If the settlement does occur, expanded use of this facility may permit a more proactive and focused strategy of support in areas of NGO Sector development. B. PERFORMANCE OF BDP DECISION-MAKING AND IMPLEMENTATION 1. Accomplishments a. The PMU did meet the basic allocation terms of the USAID – UNDP agreement, with roughly 60% of contracts allocated to Implementing Agencies and 20% to NGOs over the life of the program. Efforts to expand the NGO program after 2001 did not significantly change the overall financial allocation program, although it did result in an increase in the number of NGOs receiving grants. . b. Interviews with IAs, NGOs and opinion leaders indicate that the PMU enjoyed a positive reputation in both communities for being evenhanded and efficient. The word “trust” surfaced in a number of interviews relating to the PMU’s role. Another term used was the “UNOPS Umbrella” suggesting the PMU role was that of a neutral but committed third party facilitator, broker, and, for larger projects, effective implementing agency. The circumstances led to the PMU becoming something akin to a “proto-governmental agency” backed by the political will and good offices of both the UN and the US Embassy. 2. Challenges a. The UNDP grant’s provision about political guidance from the Embassy provided entre’ for the USG’s representatives in the Embassy to be substantially involved in decisions Cyprus Bi-communal Development viii May 25, 2004 Program Evaluation Development Associates, Inc. throughout the project approval and implementation process. This made it difficult to establish policy, strategic objectives and implementation guidelines that could be monitored from “a distance.” The need to make quick and ad hoc decisions required the input of someone who could assess the problem from the standpoint of the Embassy on a daily basis. Policy managers became deeply involved in day-to-day implementation management. Thus there developed a potential for confusion about roles and the potential for divided loyalties. It is difficult to be a “team member” and still perform the function of policy and strategic oversight for USAID and the Embassy. Hopefully the post settlement future will permit a sharper division of labor and clarity of roles. b. The ‘strategy’ for implementation of the NGO and for much of the IA parts of the agreement was largely reactive rather than proactive. Little effort was made to direct grantees toward specific program areas or objectives through the various BDP call for proposal mechanisms. Achieving a controversial objective such as bi-communalism in a non-controversial manner is very difficult. Large infrastructure programs on common problems such as water and sewage compelled the authorities to collaborate to the degree necessary, and generally involved relatively few persons in the actual implementation process. On the civil society side, the overall record of NGO grant making suggests an understandable desire to avoid funding organizations that might arouse the opposition of one side or the other. This and the unusually large size of many of the grants may have promoted a tendency toward making grants to established NGOs that were acceptable to authorities, especially in the period before 2003. c. Frequent reorganizations and employee turnover are not unusual in international projects similar to BDP. It is of some concern that the current staff on the program side is relatively new, and there are several vacancies. On the other hand, this may be an opportunity to fashion a program that is better suited to the potential of a post settlement political environment. d. The ratio of project officers to staff and management seems disproportionate to the workload put on the line PO s, especially in 2003 – 2004. Moreover, the UNOPS policy of not training project officers on the grounds that anyone hired is already an expert presumes too much. Project Officers, especially on the NGO side could have benefited from more experienced leadership and from training for their responsibilities. e. The BDP PMU faced two very different implementation problems. The first came from the need to implement in a timely, cost effective manner infrastructure and environmental public works that produced visible and high quality solutions. The second problem was to implement an NGO grant program working with many, relatively inexperienced and fragile citizen groups on both sides. The staffing and procedural requirements for the two programs arguably were quite different. It appears that the systems put in place for managing the NGO program were largely adapted from those developed for the public works programs. This led to a documentation system focused on “contracts”, rather than a system more appropriate to a politically-oriented NGO program based on achieving bi￾communal impact. The NGO program seemed to be something of a “stepchild” to the larger “flagship” projects for much of the life of the BDP program. If civil society development in support of reconciliation is to be a theme for a successor program, it will Cyprus Bi-communal Development ix May 25, 2004 Program Evaluation Development Associates, Inc. need the flexibility to adopt decision making, management and results monitoring and reporting procedures relevant to its strategic objectives. f. Perhaps because the nature of the two programs (large IA vs. civil society) was different, there was not as much effort as there could have been to expand the scope of interaction of the large IA projects by adding a civil society component. For example, the veterinary health component could have worked with farmer organizations. This would have required a more proactive grant-making process g. The program never was able to develop consistent, well understood and documented criteria for grant making. Neither grantees nor PMU project officers can understand the reasons why projects are approved or disapproved. The perception of inconsistency undermines the credibility of the BDP, and the resulting belief that grants are given to favorites or ‘bogus’ NGOs prevents some organizations from applying. In addition, it reduces the credibility of the project officers with the grantees whose programs they monitor, and it contributes to project staff alienation from PMU management. It is possible to establish a reasonable set of approval criteria, even for a political program. This should have been done. h. Program and project monitoring and reporting focused on the “grant/contract”, rather than the recipient organization’s advancement of bi-communal goals. 77 NGOs received BDP grants (under the NGO component), 23 receiving more than one. There is no overall assessment of each of these organization’s progress toward greater interaction, collaboration or joint planning and implementation. Project files do not yield much information about bi-communal accomplishment.3 No effort was made to assess effectiveness and impact other than completion of agreed work. U.S. Embassy and PMU officers know much more about grantee effectiveness than is represented in the project reports or closeout documents. For a $6.4 million dollar investment, a better effort should have been made to evaluate and track progress on Turkish and Greek Cypriot NGOs bi-communal performance. i. The post settlement conditions should permit the introduction of strategic focus, more clearly defined grant making objectives and decision criteria, and greater transparency in announcing awards and explaining rejections. This will require a serious overhaul of grant proposal review procedures emphasizing objective rating procedures, and, possibly, participation by Cypriot experts and “wise people” in some part of the process. It is important that Cypriot government representatives in any future program steering committee understand and agree with the program’s objectives. 4 3 The US Embassy in reviewing this conclusion stated “The project files may not yield much information, but no decision was made to re-engage with an already funded NGO without an assessment of their progress to date.” The Team is pleased to learn this, but we still wonder how objective or consistent such a review process could be without some kind of evaluative documentation prepared either by the grantee or the PMU project officer as to the bi-communal achievements of the grantee. 4 We are fully aware that there are risks to greater transparency and increased Cypriot participation. On the other hand, as has been demonstrated by USAID programs in other difficult environments, US programs should “model” desirable values and behaviors in the way they are implemented. Cyprus Bi-communal Development x May 25, 2004 Program Evaluation Development Associates, Inc. IV. RECOMMENDATIONS: MAKING THE SETTLEMENT WORK Even if a solution is achieved to the Cyprus problem, experience shows that there will continue to be political bumps over the next several years. Regardless, contact between Greek and Turkish Cypriots is dramatically different than it was when the BDP began, and further freedom of movement is likely. Therefore major changes in the direction of future programs are possible that will increase their effectiveness. 1. It is no longer useful to think in terms of bi-communality. That term tends to polarize the two sides. It also fails to take into consideration the multicultural nature of Cypriot society. There is still a need to support the peace process, foster reconciliation and cooperation among the diverse Cypriot groups. This is dependent to some extent on reducing the income disparity between the Turkish Cypriot and Greek Cypriot communities. 2. The post settlement conditions should permit the introduction of strategic focus, more clearly defined grant making objectives and decision criteria, and greater transparency in announcing awards and explaining rejections. This will require a serious overhaul of grant proposal review procedures emphasizing objective rating procedures, and, possibly, participation by Cypriot experts and “wise people” in some part of the process. It is important that Cypriot government representatives in any future program steering committee understand and agree with the program’s objectives. 3. Building on work already done, several assessments should be undertaken to set the stage for the next phase of assistance. There have already been discussions about undertaking sectoral assessments and developing sectoral strategies. Any strategy work should be based on research to determine the level of need and available resources in each sector. Program baselines should be established against which to measure results. Local social science capacity exists to conduct this work. 4. Mounting a program with four major objectives, as proposed in the contingency plan, would require a much larger commitment of funding and a more diversely expert staff than the program is likely to have in the future. The analyses described above, along with further information about the EU’s plans, should factor into decision making about appropriate directions for future US assistance. 5. USAID should collaborate with local partners in performing analyses, developing strategies and implementing new initiative. The US, through the BDP and CASP, has helped to develop the capacity of many organizations and individuals. They can now participate in shaping the program of the future. 6. A future program should include a civil society component, but one more focused and strategic than the BDP, including training in effective advocacy work by NGOs. It is likely that the EU will finance many of the needed infrastructure and economic development activities. The US holds a comparative advantage in working with civil society. The BDP has fostered an active civil society with interest in maintaining their Cyprus Bi-communal Development xi May 25, 2004 Program Evaluation Development Associates, Inc. links with people on the other side. A strong civil society will be an important part of an effective, united society. In addition, there will be many issues affecting people that result from both the Annan plan and the EU accession. Civil society organizations can spur debate and increase the level of public discourse about these changes. They can also contribute greatly to the healing process of reconciliation and search for common purpose. At the same time, the large dollar size of grants made to Greek and Turkish Cypriot NGOs, even taking into account higher costs on the island, should be re-examined. The assumptions underlying the cost structure of NGO proposals needs to be carefully assessed, and efforts made to more rigorously distinguish between administrative overhead costs and cost directly related to implementing projects. 7. A second program focus should be “good governance”. Local and “state” level governments, especially on the TC side, are not sufficiently mature and developed to exercise the normal functions of government in an accountable, transparent and efficient fashion. Corruption, favoritism, cronyism is endemic in government on both sides according to most observers. Better systems for public finance management, procurement, and public participation in decision making are needed. If the Annan plan goes forward, there will also be a demand for technical knowledge about the workings of a Federal system. Although the Cypriot federal institutions will be quite limited at the beginning, substantial growth is anticipated. The United States has a comparative advantage in providing technical assistance in both of these areas. Moreover, most USAID programs throughout the Balkans and the NIS have developed strong “good governance” programs. There is now a body of knowledge, expertise and experience that can be brought to bear quickly on the problems Cyprus will encounter. Making government work has to be a critical component of the larger “making the settlement work” program. 8. Program financial and monitoring documentation should be reoriented for Results Management. From a Results Management perspective, the system for reporting on contracts and expenditures now in place is not very helpful. Expertise should be brought in to upgrade the financial and project data management system so that the implementing organizations can more effectively manage funds for results accountability, in addition to financial accountability. A Results documentation system based on systematic monitoring and evaluation will provide the necessary knowledge input for good policy as well as implementation management. Cyprus Bi-communal Development xii May 25, 2004 Program Evaluation CYPRUS BI-COMMUNAL DEVELOPMENT PROGRAM EVALUATION I. INTRODUCTION USAID contracted with Development Associates, Inc. to assess the strategy, attainment of objectives, and implementation of the Bi-Communal Development Program (BDP). The evaluation’s findings, conclusions and recommendations will inform the design of a follow-on “BDP-like” project, expected to begin sometime in FY 2004. The Bi-Communal Development Program (BDP) is one of three programs of the U.S. government to implement the Congressional directive (contained in a $15 million annual ESF earmark) to support bi-communal projects and measures aimed at reunification of the island and designed to reduce tensions and promote peace and cooperation between the two communities on Cyprus.5 The other two are the Cyprus America Scholarship Program implemented by the Cyprus Fulbright Commission and managed by the Department of State; and the Bi-communal Support Program, implemented by Amideast and managed by the Department of State. Prior to 1998, USAID-funded bi-communal development programs were carried out by the UNHCR, but over time the need for relief programs declined and program emphasis shifted to large infrastructure efforts. When UNHCR decided to close all but its asylum office in Nicosia, USAID signed a grant agreement for a $30.5 million program with UNDP6 . Since the UNDP does not have offices in the Republic of Cyprus (ROC), the UNDP asked the UN Office for Project Services (UNOPS) to execute the program. The BDP grant between USAID and UNDP was signed in March 1998 and UNOPS set up a Project Management Unit (PMU) in the UN Protected Area (UNPA) at Nicosia in April, 1998. The grant was later amended to increase the total USAID contribution through 2004 to $60 million. II. METHODOLOGY The evaluation team assembled by Development Associates for this task included two former USAID Senior Foreign Service officers, with experience in civil society evaluation and program management, a senior economist with in depth experience with Cyprus and the Annan Plan, an experienced environmental engineer, a Greek Cypriot historian and a Turkish Cypriot NGO activist and social scientist. Four days after the team’s arrival in Cyprus, the senior economist was drafted by the US Embassy and the UN for a major role in the Annan Plan negotiation process. An expatriate US citizen with suitable social science skills and in depth Cyprus experience was recruited to fill the gap. 5 Foreign Operations, Export Financing, and Related Programs Appropriation Bill, 2004 6 USAID contributed $30 million and UNDP $500,000 for the period 1998 through 2001. Cyprus Bi-communal Development 1 April 25, 2004 Program Evaluation Development Associates, Inc. The team followed data collection and analysis methods common to most USAID sponsored rapid appraisals. Program and project documentation was carefully reviewed7 , grant and contract recipients were interviewed, sites visited. The TOR (found at Annex 1) directed the team to interview all relevant stakeholders, and to conduct interviews with “opinion leaders”. In addition, the team developed several innovative approaches to organizing data and the knowledge that was in the heads of the principal players in the US Embassy and the PMU. For each of the major programs, the team addressed three questions; first, was the project sufficiently well managed to insure the desired outcome at a reasonable standard; second, did the project make sense in terms of a strategy for bi-communal objectives; third, was it effective in achieving those objectives. Effectiveness was further assessed in terms of three dimensions: material, institutional, and symbolic. Material effectiveness relates to the physical outcome, i.e., a functioning waste water treatment plant. Institutional effectiveness relates to whether those responsible for the project improved their skills, motivation, and capacity to maintain the project, or to take on new but related tasks. Symbolic effectiveness had to do with visible and emotional resonance of the project. Did people involved or affected have positive feelings about bi-communal cooperation, for example. To assess the “public works” side of the program representing 60 percent of program expenditures, 15 public works projects were selected for detailed examination based on the size, longevity, and reputed significance of the project for advancing bi-communal relations. Project files for these projects were thoroughly examined by the team’s experienced environmental engineer, in addition to conducting interviews with all relevant players in project implementation. Considered a success story, the public and animal health set of projects was selected for assessment based on advice of the PMU. On the NGO side of the program, a sample of 50 NGO grants was initially selected at random, augmented by grants suggested as “representative” by local US Embassy staff. From this list, 20 NGO interviews were conducted along with site visits. In addition, the team examined partial project files for 9 NGO grants on an intensive longitudinal basis. A list of Opinion Leaders was compiled with advice from the US Embassy staff and from the Greek Cypriot and Turkish Cypriot members of the evaluation team. 23 Opinion Leaders were interviewed. The list of all stakeholders, opinion leaders and grantees interviewed is found in Annex 2. Data from open ended questionnaires for grantees and OLs was coded and analyzed using simple descriptive statistics. Using program reports and files, an analysis of resource allocation patterns was conducted. Sample questionnaires and the coded responses to both opinion leaders and grantee questionnaires are found in Annex 5. An unusual feature of the team’s methodology was the use of two project ranking systems. After discussions with US Embassy and PMU staff, it was possible to develop a self anchoring rating system for all grants, using criteria generated from published program objectives and the Flexible 7 Based on an interpretation of the USAID agreement with UNDP, the team’s access to UNOPS files was limited to an “on request” basis. This made the documentation review somewhat difficult for two reasons; one, the team did not know what was in the files so could not make specific requests; two, the need to use broad categories for formulating requests put a heavy burden on UNOPS staff to photocopy large amounts of material, much of which was not directly relevant. Cyprus Bi-communal Development 2 April 25, 2004 Program Evaluation Development Associates, Inc. Framework Fostering Rapprochement (FFFR) ‘strategy’ developed midway through the BDP. The team asked the USAID and Embassy Representatives to classify all grants using a FFFR schema, locating grants on a continuum from “no bi-communal feature” to those that were considered Collaborative and Joint activities, the latter the highest level of bi-communal achievement. A second rating was applied by PMU staff at the request of the evaluation team. This used an A,B,C rating on two dimensions: first, level of success in achieving the bi￾communal objective of the grant, and, second, assessment of the future utility of the grantee for purposes of making the settlement work. A score of C was either not successful or of little utility. The data from these ranking systems was analyzed and used by the evaluation team to give a kind of “Delphi” assessment of the overall effectiveness of the program based on percentage of grants receiving A, or highest scores, through percentages receiving scores of B and C. Shortly after beginning the field research, the team found that BDP reporting tables were focused mainly on financial data organized by each “grant or contract”, with no corresponding system of recording progress by the grantee organization. With assistance from the PMU, the team was able to organize the grant data by name of the recipient organization, primarily on the NGO side. Grants to implementing agencies were more clearly apparent in the reporting tables. This effort led to an assessment of the degree of concentration and dispersion of NGO grant allocations, as well as a better understanding of the performance of actual organizations receiving BDP funds. The time allocated for the field work part of the evaluation was limited to the usual three weeks customary for USAID evaluations. The team used the first week for stakeholder meetings, organization of the field research and the documentation review process. During the next two weeks, 9 days were allocated to conduct interviews, site visits and documentation review, and 3 days to the preparation of a preliminary draft of main findings, conclusions and recommendations. These were presented to the US Ambassador and Embassy team, to USAID Washington CTOs via a video link, and to the PMU Management and staff in two separate meetings. Following a tight schedule on return from the field, a working draft was submitted to USAID Washington on March 19 for review by USAID staff, followed by additional briefings with UNDP, UNOPS in New York and USAID Washington. Comments were received, the final complete draft prepared and submitted to Development Associates for review and submission to USAID in April, 2004. After reviewing the final draft, UNDP/UNOPS provided the team with comments on the report and with a document updating and analyzing program indicators. The consolidated comments of UNDP and UNOPS and the evaluators' responses are found at Annex 6. The indicators paper was received too late for evaluators to review it. III. PROGRAM PURPOSE The BDP’s purpose, as stated in the grant agreement, is “to support the peace-making process in Cyprus” through “bi-communal projects and measures aimed at reunification of the island and designed to reduce tensions and promote peace and cooperation between the two communities on Cyprus.” The program “inherited” several large projects from the UNHCR period: Nicosia Master Plan (rehabilitation of the historic Venetian Walls that surround the old city and of a Cyprus Bi-communal Development 3 May 25, 2004 Program Evaluation Development Associates, Inc. community within the walls on each side of the green line8 ), rehabilitation and expansion of the Nicosia Sewerage System that serves both communities, and restoration of Pyla Village (one of the only villages on the island where Greek Cypriots and Turkish Cypriots lived together in 1998). Funds were made available for additional bi-communal projects, including projects of non-governmental and community based organizations. In 2001, the grant was amended to increase life of program funding by $30 million and extend the BDP through 2004. During most of the grant life, fostering bi-communal relations has been exceedingly tough. The Turkish Cypriot (TC) authorities banned bi-communal meetings and activities in late 1997, a ban that remained in place until April 2003. Turkish Cypriots needed special permission to cross into the South or even into the buffer zone – and that permission was frequently not forthcoming. The fact that neither the ROC, the US nor the UN recognizes the Turkish Republic of Northern Cyprus meant that activities involving TC authorities had to be implemented through other, recognized, organizations. TC and Greek Cypriot (GC) authorities could have no official relations with each other. Procurement of material for projects on the Turkish Cypriot side sometimes posed some challenges. The political situation was volatile throughout – from S-300 missile crisis to multiple failed attempts at peace talks. (Annex 8 provides a detailed timeline of the political events that influenced program management; as well as a timeline of key events in the life of the BDP.) The conditions under which the BDP operated required extreme political sensitivity; consequently the Grant Agreement specified that the UNDP would receive political guidance from the U.S. Embassy in Nicosia in selecting activities to be funded. These conditions also required innovation and dexterity from the PMU in finding ways of accomplishing the program’s objectives. Finally, it required the program to adopt a very broad definition of bi-communality. IV. FINDINGS ON BDP MANAGEMENT A. PROGRAM STRUCTURE BDP’s program structure grew out of the unique features of the Cyprus political environment. Three elements are key: the large number of stakeholders involved in the decision-making process, the inheritance of large public works projects from the UNHCR era, and the need for adaptability in an uncertain political environment where bi-communal activity is a risky business. 1. Stakeholder Relationships a. USAID USAID’s role is that of manager/monitor of the overall grant to the UNDP, rather than of decision-maker on individual projects. USAID performs its monitoring role through a PSC who reports to the US Embassy economics officer. As an Embassy employee, however, she tends to play a dual role as both project monitor – reporting on project activity to USAID headquarters - and advisor to the Embassy on project decisions. In addition, all project activities must be 8 The “green line” is the demarcation between the Greek Cypriot controlled side and the Turkish Cypriot controlled side. Cyprus Bi-communal Development 4 May 25, 2004 Program Evaluation Development Associates, Inc. reviewed by a USAID environmental officer through the Initial Environmental Review (IER) process. This process provides USAID with some ability to affect project decisions, when activities have a clear environmental consequence. b. US Embassy The task of providing political guidance to the BDP was taken on initially by the economic/commercial officer, but as the amount of project activity grew it became necessary to hire a PSC, reporting to the economic officer, to serve as the Embassy’s representative and to provide political guidance on all proposed BDP projects. Like the USAID PSC, she plays a dual role, in that she assists in monitoring certain projects, especially the large infrastructure and environmental projects, as the portfolio has become too large for the USAID PSC to monitor alone. As a result, while they play different roles, the two PSCs (Embassy and USAID representatives) work very closely together. Their different responsibilities and authorities are not easily understood to those outside the Embassy. Interviews with principal senior stakeholders, the US Ambassador and senior US Embassy officers, indicate substantial satisfaction with the achievements of the BDP program. The US Ambassador meets with grantees and, along with his predecessor, has helped to resolve implementation problems from time to time. Embassy officers reinforce the view that the BDP has been very instrumental in giving the US commitment a degree of tangible visibility, and, in their view, has contributed to the maintenance of the possibility of a peaceful settlement. UNDP senior management also values the program as an indirect expression of United Nations commitment to the cause of peace, and to a working relationship with the United States as a principal player in UN affairs. c. UNDP Since UNDP does not operate in Cyprus, it has delegated responsibility for BDP implementation to UNOPS. UNDP carries out program monitoring through visits two or three times a year to Cyprus and through electronic means. UNDP has not delegated its decision-making role on all projects, however. Although it did not do so in the early stages, UNDP now participates in the project steering committees that approve projects, and advises on other projects that do not go to the project steering committees but are valued at over $12,000. d. UNOPS UNOPS has set up a Project Management Unit (PMU) to implement the BDP. The PMU is delegated to act on UNOPS’ behalf in Cyprus, though any procurement action or grant agreement or amendment of $30,000 or more must be reviewed by UNOPS/New York before the PMU can sign. UNOPS also participates in reviews of projects at the project steering committee. UNOPS/New York carries out its program oversight and decision-making role through electronic means and occasional visits to Cyprus. e. The Greek Cypriot and Turkish Cypriot Authorities Since the Greek Cypriot (GC) and Turkish Cypriot (TC) authorities cannot meet together because of the recognition issue, the PMU arranges separate Steering Committee meetings on Cyprus Bi-communal Development 5 May 25, 2004 Program Evaluation Development Associates, Inc. each side. While authorities of both sides are aware that parallel project approval processes and funding decisions are being made on the other side, they do not know the content of those decisions, and they need take no official knowledge of them. f. The Red Cross and the Humanitarian Relief Mission Since neither the US nor the UN recognizes the Turkish Cypriot authorities, it is not possible to sign contracts with them; nor, is there legal recourse for a grant signed with any organization affiliated with or registered with the TC authorities. As the organization designated by the President of the Republic to deal with all international humanitarian assistance matters (following the events of 1974), the Cyprus Red Cross, with its Turkish Cypriot Vice Presidency as specified in the Constitution of the Republic—became the BDP’s official partner, playing an intermediary role in signing documents on behalf of the GOC. The Turkish Cypriot authorities used UNHCR’s departure from the Bi-communal Program as an opportunity to disband the Office of the Vice President of the Cyprus Red Cross through which programs had been implemented. In its place, the Humanitarian Relief Mission (HRM) was established as an intermediary between the BDP and the Turkish Cypriot authorities. HRM is fully authorized by the Turkish Cypriot authorities to sign documents on their behalf, and because of its affiliation with the BDP, it is legitimized in the South. The HRM becomes the signatory for all projects with Turkish Cypriot organizations. The HRM then passes funding on to the recipient organization, and serves as the conduit for all official communications between the TC authorities and the project. The legal issues were not a factor in the South; however, it is important for both sides to be treated in a parallel way. For this reason, the Red Cross serves as representative of the ROC and takes on a similar signatory role for activities of Greek Cypriot organizations (not NGOs) as the HRM takes for Turkish Cypriot organizations. In turn, the BDP transfers funds to the Red Cross and HRM to assist in their humanitarian programs and cover administrative and other costs. 2. Policy-Strategy Management Alice: “Would you tell me, please, which way I ought to go from here?” The Cheshire Cat: “That depends a good deal on where you want to get to.” Alice: “I don’t much care where---“ The Cat: “Then it doesn’t matter which way you go” Alice: “---as long as I get somewhere.” The Cat: “Oh, you’re sure to do that, if you only walk long enough.”9 Responsibility for policy and strategy management was divided between USAID Washington’s Europe Bureau, UNDP New York, the US Embassy in Nicosia, and the USAID local representative also in Nicosia. BDP policy was, as stated by the US Embassy officer responsible for oversight was “to take every opportunity to support and cultivate bi-communal contact…” Or, in a somewhat more 9 Paraphrased from Alice’s Adventures in Wonderland by Lewis Carroll Cyprus Bi-communal Development 6 May 25, 2004 Program Evaluation Development Associates, Inc. inclusive formulation, as put by the USAID Representative, the strategy was to “develop as many activities as a basis for bi-communal cooperation (including meetings) in as many sectors as possible”.10 This very broad policy, combined with the political constraints of the Cyprus environment, opened the door for a very wide range of grant funded activities, each of which could be justified on the general grounds of having the potential to be a “basis for bi-communal cooperation.” Apparently this very broad and flexible strategy was a cause for some concern to policy managers on the USG side. At least three major efforts were made to develop more concise and directive strategies for the environment program, for the NGO program, and finally for the entire effort, which culminated in the Flexible Framework Fostering Rapprochement (FFFR). 11 This document did help to define some of the several dimensions of the basic contact theory of bi￾communal development. Also, USG officers continued to urge UNDP and UNOPS to develop a communications strategy for the BDP, however this was delayed until early 2004, the last year of the current agreement. These efforts not withstanding, the need to be flexible, opportunistic, and responsive to a variety of political and budgetary pressures made it difficult to apply more specific strategic guidelines as a means for determining what would, and would not be funded. For example, US Embassy and USAID Nicosia officers, using the bi-communal content classification scheme they developed at the request of the evaluation team, classified 39 percent of the NGO funded activities representing $1.8 million as “Other (may be some bi-communal), Nothing bi-communal (political, NMP, island wide) or Mirror (no information sharing contact). As will be demonstrated below, some of the grants not included in this 39 percent which began as “Mirror” projects, did develop a dynamic that led to greater collaboration. The fact remains that a significant number of grants were made that had little or no bi-communal content. While many of these activities had independent value using other criterion, clearly other decision￾making factors were operating in addition to “cultivating bi-communal contact.”12 3. The roles of the PMU Over time, the PMU has assumed an increasing number of roles so the program could move faster and reach a broader range of organizations. 10 The quotations are extracted from more extensive comments made by the two US Embassy based officers responsible for policy and strategic oversight of the UNOPS implemented program. Both state that although the policies and criteria for decision making may not have been well documented, they did exist. Moreover, in the decision making process for making grants, issues of capacity building, effectiveness, and impact were considered. 11 USAID observers comment that FFFR was not so much a strategy as it was an effort to put some conceptual respectability and explanation on what was happening on the ground with the BDP program. The team agrees. 12 This classification was prepared by US staff using the UNOPS Bi-Communal Development Programme in Cyprus Report: Quarterly Report 1st October 2003 – 31st December 2003, Expenditure Report by Sector and by Project. These tables show all cumulative project Disbursements and Contracted amounts through 31 December 2003. Each funded activity was classified according to the following: S – Spirit (or In spirit) M- Mirror (no information sharing) C-Collaborative Parallel J- Joint O-Other (consultants, some bi-communal contact) N – nothing (Political)(NMP)(Island-wide) Cyprus Bi-communal Development 7 May 25, 2004 Program Evaluation Development Associates, Inc. a. Project management The PMU’s primary role is as manager of the wide range of grants funded by the BDP, and its internal organization reflects this. The PMU manages the project process from start to finish: soliciting proposals from implementing agencies (government agencies) and NGOs, making recommendations on project approvals, negotiating the final terms of the grants, monitoring them, disbursing funds and assessing performance. Once approvals have been made other stakeholders generally have no further role except during program reviews (unless an issue arises that merits interagency attention). Over the years, the program has shifted from a focus primarily on infrastructure to one of maximizing the number of bi-communal contacts throughout society. This has meant an increased emphasis on many small activities from other implementing agencies and NGOs in both Greek Cypriot and Turkish Cypriot communities. The dramatic expansion in the number of activities, from 7 in 1998 to 169 in 2003, has led the PMU to increase its staff substantially in order to manage the expanding portfolio. From an initial staff of 14 in 1998, it has grown to a current staff size of five international officers, 15 Cypriots from both sides of the island, and a handful of international experts in specialized areas. Only six of these are project officers (including vacancies). The remainders are engaged in management and administrative/financial support.13 b. Implementing agency Several factors have pushed the PMU into playing a much greater role in project implementation than one would expect from a project manager. (1) The BDP inherited large urban environment and infrastructure projects, valued at about 60 percent of total program resources, from UNHCR days. After the initial round of inherited projects was completed in 2001, BDP approved follow-on grants expanding on the initial work, so that the urban infrastructure and environment program continued to require the major share of program resources. These projects were multi-year in nature and involved large procurements and construction contracts. (2) The decision was made early on to fund implementing agency projects for one year at a time. If progress was sufficient, a second year’s funding would be added to continue the activity. Government procurement procedures, however, are cumbersome. By procuring project commodities and construction/engineering contracts directly, the PMU cut several months off the process, so that project activity could proceed on a one-year schedule. (3) Where the same types of commodities were needed by the GC and the TC sides, PMU procurement on their behalf would assure compatibility and economies of bulk purchasing. (4) PMU direct procurement made some bi-communal activity possible by bypassing the recognition issue. Additionally, the PMU stepping in to do things like pay workers directly enhanced the day-to-day bi-communality of projects in that it allowed TC and 13 Figures are based on quarterly and annual reports. Cyprus Bi-communal Development 8 May 25, 2004 Program Evaluation Development Associates, Inc. GC workers to work together on crews in Pyla and on the Venetian Walls in Nicosia. Without the PMU in between, a host of payroll and administrative issues would have made this arrangement impossible. For the same reason, the PMU has established a veterinary training and testing laboratory at its offices in the buffer zone. This has enabled TC and GC veterinary technicians to join together for training and product testing. c. Matchmaker For most of the BDP grant life, the TC authorities’ ban on bi-communal contact has made it extremely difficult for GC and TC organizations to meet, let alone collaborate. To foster bi￾communalism in this incredibly restrictive setting, the PMU had to become the intermediary that linked organizations together. As the program increased its focus towards NGOs, the PMU’s role as matchmaker has expanded. Many organizations funded by the BDP had health, youth, or environment as their primary focus. Some organizations proposed “in-spirit” or mono￾communal projects that became bi-communal as a requirement of approval. These organizations needed help from the PMU in finding partners from the other side and then in establishing a comfortable and productive working relationship. (e.g., the Girl Guides HIV/AIDS training program). d. Facilitator Promoting bi-communalism in Cyprus is a political minefield on both sides of the green line. The PMU became adept at side-stepping the mines and gaining the trust of officials by being discrete and low-profile, by not placing either side in the position of having to make decisions that would cause them political problems, and by offering both funding and quick implementation for activities that had objectives that everyone could agree with. With these tools, the PMU was able to broker agreements and overcome bottlenecks to implementation; e.g. with Embassy support convincing the TC authorities to agree to NGO projects; or assisting grantees in getting permission for members to cross the line. 4. Characteristics of program structure and management a. Quick response, quick impact Despite the large number of stakeholders involved, the BDP has maintained flexibility and the capability for quick response to changing situations. This is largely due to the good will and commitment of the main parties: USAID, UNDP, UNOPS and the US Embassy. b. “Responsive and Helpful” Among the 30 grantees interviewed by the team, one theme came through loud and clear: grantees find the PMU project officers to be helpful, responsive, and professional. Some comments: “They show flexibility, respectful relations, commitment to treat the two sides equally.” “They are efficient; they know what to do”. “They are helpful, supportive and understanding”. The only criticism was that project officers are overworked. If the activity manager is absent, no one else can fill in. Cyprus Bi-communal Development 9 May 25, 2004 Program Evaluation Development Associates, Inc. c. Who’s on first? Over the years, the BDP has experienced growing pains, as interrelationships among the US and UN agencies were not clearly spelled out or understood. Decision-making and reporting relationships have only developed over time; as the number of projects has grown it has become necessary to formalize what were once informal communications channels, such as the weekly PMU-US Embassy meeting to review progress and approve special initiatives. There are still gaps in communications that cause confusion and inconsistency in treatment of grantees. Some stakeholders interviewed by the evaluation team identified the following issues:  Absence of a formal approval process for project extensions, leading in some cases to extensions of projects that should be terminated because they no longer advance program priorities, for example.  Inadequate reporting of the full results of steering committee meetings and weekly US Embassy-PMU meetings, so that the project officer tasked with preparing the grant or contract document does not always understand the conditions agreed to.  Absence of guidance to all the project managers about items that will generally not be approved, except on an exception basis (e.g. payment of a project coordinator)  Lack of clarity as to the type of issue that requires Embassy involvement, leading PMU staff to play it safe by obtaining guidance from the Embassy on many project implementation issues. Everybody agrees that the Embassy’s political guidance was essential – on what activities to approve. Where there is disagreement is on the issue of whether their inputs on the nuts and bolts of project design and implementation, while often valuable, were worth it in terms of confusion to the grantee, over-visibility of US role, and reduction of empowerment of PMU staff. On this issue, different people had different views, but there was definitely a level of frustration with the extent of Embassy involvement on the PMU side, and an equal level of frustration on the part of responsible Embassy personnel with what they saw as persistent weaknesses in PMU management. d. “Need to know” management In the Cyprus political environment, the PMU could only operate effectively by NOT sharing full information. At times, it was better for the Greek Cypriot authorities not to know what projects the Turkish Cypriots were approving, and vice versa. Throughout, it was risky to share too much information and/or any documentation because of its potential use to either advance or discredit political positions within each community and/or between the two communities. (Examples: list of NGO projects funded on TC side provided to HRM ended up on front page of anti-settlement newspaper with names and dollar amounts; list of authorities projects on GC side led to “crack down” on GOC officials engaging in bi-communal projects.) As for the NGOs, it seemed more discrete and easier NOT to inform applicants of the list of who won awards and who did not. This “need to know” style of operation has raised suspicions and generated an element of distrust among some opinion leaders, stakeholders and grantees (both those whose proposals were approved and those whose were not). The extent to which information of various types can and should be shared needs to be revisited in light of recent and expected political openings. Cyprus Bi-communal Development 10 May 25, 2004 Program Evaluation Development Associates, Inc. e. Management decision-making and internal communications Project approvals are made by the management team and the steering committee, but the tasks of informing those whose grants were rejected, development of the grant agreement and project monitoring fall to the project officers. Absence of clear guidelines for decision-making, poor communications about project approval decisions, the “need to know” mentality and overwhelming workloads as the project portfolio has grown combine to create dissatisfaction of the primarily Cypriot project staff with the expatriate managers. f. Project management tracking systems The PMU, as its name implies, views itself as the implementer of a single project under normal circumstances 14. Hence, its reporting systems take as their management units individual contracts. The prominence of large infrastructure and environment activities in the portfolio reinforces the emphasis on managing by individual contracts. In a program containing such disparate activities, however, this does not provide managers the information they need to track progress. For example, on the quarterly and annual reports, activities are identified by their contract number and activity names, but not by grantee organizations, yet for a program focused on bi-communal contacts, understanding which organizations are furthering this objective is very important, particularly since some organizations have received multiple grants. Even for the large public works activities, the tracking system is confusing, because several contracts within a single activity have the same coding number. A review of selected project files finds that grantees submit progress reports and invoices against pre agreed dates or benchmarks for payment. If the grant includes procurement of material or services through a bidding process, the documentation is submitted to the PO for inclusion in the file. POs make every effort to visit grantees and especially to attend events organized by NGOs. On the IA side, the POs are very much involved in the implementation of the grant, including organizing technical assistance, managing procurement, and organizing meetings where necessary. POs prepare monitoring reports to the file. In its review of selected elements of files, the team noted scarcity of evaluative data on progress towards achieving bi-communal goals. This is discussed further in the following section (Program Strategy and Decision Making). Up until mid 2003, the ratio of grantees to PO was about 1 to 15 active NGO projects. With the relaxation of restraints on bi-communal contacts, the number of projects in the NGO sector has doubled, straining the capacity of the existing staff to conduct frequent site visits, while the IA side continues to account for 60% of disbursements. g. Staffing/training needs With only six project officers managing upwards of 150 projects, PMU project officers, especially for the NGO program, reported that the workload is excessive using the existing procedures. They said that it is very difficult to break away from paper work requirements of grant processing, as these requirements are the same for both large and small grants. One officer said she uses her evenings and weekends to attend events and visit grantees. Asked whether they had been given training in NGO development or other substantive areas project officers reported 14 This point was made by a Senior PMU staff person during the Team’s evaluation debriefing in Nicosia. Cyprus Bi-communal Development 11 May 25, 2004 Program Evaluation Development Associates, Inc. that UNOPS policy was not to provide staff training, as UNOPS hired only persons already qualified. This was confirmed by one of the UNOPS management staff. B. PROGRAM STRATEGY AND DECISION-MAKING 1. Decision making process: Operational Rules for the PMU The Cyprus environment’s political constraints had a profound effect on the BDP decision making process. When the project began in 1998 the separation of the island into two distinct communities had substantially hardened. At the level of the authorities, there was no political commitment to bi-communalism on either side. As much as the various stakeholders wanted to develop a focused, strategically determined development project, the political environment made the development of pre-determined criteria for grant making very difficult to formalize. Moreover, the first two years of the new project were dominated by the need to demonstrate material progress with portfolio in hand, largely concentrated on the Nicosia Master Plan, Water and Sewerage system rehabilitation and expansion, and related “big ticket” projects. Even as the NGO program got off the ground by 2000, it was at first next to impossible to find many NGOs willing to run the risk of any cooperation between the two sides, according to PMU and Embassy staff. In addition to these constraints, in 1974 the Congress of the United States had mandated through an Economic Support Fund earmark that $15 million be obligated each year for the purpose of, among other things, supporting the development of bi-communal cooperation in Cyprus that would facilitate a peaceful solution to the island’s conflict. In 1998 this $15 million did not purchase as much as it did in 1974, but for a population of some 700 to 800 thousand people, it represented a significant amount, especially as a budgetary supplement to projects that might otherwise be difficult to fund through tax revenues. The 1998 program specified that at least 20 percent of the available program funds should be spent on Non-Governmental Organizations, in an environment where the NGO community was not well developed in the Greek Cypriot side, and almost non-existent and/or highly distrusted by authorities on the Turkish Cypriot side. The humanitarian crisis of mass movements of conflict-dislocated peoples was no longer an issue by 1997, and other than infrastructure, it was becoming increasingly difficult to find appropriate projects for funding. The hardening of positions resulting from the missile crisis of the 1990s made it even more difficult to find ways to fund a program with the objective of supporting bi￾communal contacts and cooperation. Faced with the need to spend the money in an environment with multiple constraints and risks, while at the same time demonstrating progress, the program leadership had to operate in a manner that did not directly confront the political constraints, or seem to break any of the restrictions on contact and potential ‘recognition’ imposed by the TCC and GCC. After some time, some informal rules did develop that, if carefully applied, would allow work to progress. These “rules”, as developed by the PMU and US Embassy and USAID players, are as indicated in the next table. Cyprus Bi-communal Development 12 May 25, 2004 Program Evaluation Development Associates, Inc. TABLE 1 Bi-communal Program Rules under Severe Constraints Avoid the following: Political Terminology 1. Do not say bi-communal to the Turkish Cypriot 2. Do not mention TC authorities to GC authorities. Fundamental political principles/issues 1. Avoid being trapped in recognition issue. 2. Avoid voting in the Project Steering Committees. – forge a consensus. Providing too much process/grant budgetary information to PSC members and general publics on both sides. Becoming the advocate for any particular project, while doing one’s best to influence grant decisions. Do the following Be flexible and opportunistic in deciding which projects to fund as well as in implementation. Make it easy for the sides to accept the project by assuming as much responsibility for program implementation as possible; for example, do procurement, manage funds directly, and hire outside expertise to work with both sides. Act with discretion, keep a low profile. Circumvent local authorities on behalf of IA. Enhance the role of local partners without obliging them to take on responsibilities for project implementation. Encourage and facilitate contacts between Turkish and Greek Cypriots without appearing to do so. Stress economic and other benefits of the project to the TC side. In Public Works programs, use the project to fill essential gaps in an overall structure, such as Venetian Wall restoration or in the Water and Sewage program. Maintain the integrity and unity of the PMU Stakeholders, by ensuring a unified position among USAID, US Embassy, UNDP UNOPS. These were the decision rules that emerged from the experience of the PMU and the USG in trying to mount a bi-communal program in an environment where both sides lacked the political will to advance bi-communalism, and where engagement of government agencies and NGOs in anything that looked like bi-communalism carried significant risks. Cyprus Bi-communal Development 13 May 25, 2004 Program Evaluation Development Associates, Inc. 2. Grant Making Review and Decision Processes Most grant making programs have to have a process in place for advertising/informing potential grantees, reviewing and deciding on proposals, informing grant winners and losers, preparing and signing grant agreements, and monitoring and evaluating grantee performance. For each step in the process, the grant making organization is faced with a number of decisions about how to accomplish each step in the process. If the grant program objectives are broadly stated and the selection criteria poorly defined, the funding organization may have difficulty using objective processes for deciding which proposals will be approved, grantees will have trouble producing and assessing impact, and the overall program will take on a diffuse and unfocused character. The BDP is a political program with the purpose of promoting contact and cooperation between Turkish Cypriot and Greek Cypriot organizations, NGOs and citizens. Although the categories used to describe the program in Annual Reports sound like the familiar categories of a normal development program, e.g., public and animal health, environment, NGO development, all the stakeholders understand that sector objectives such as disease reduction or institutional capacity building have been secondary to the political objective of improving the possibilities of rapprochement. 15 In a political program largely financed by the United States, it is not surprising that the foreign policy interests of USG should be given primary consideration in how the program is perceived and what kinds of benefits can be expected. As we stated above, the interests of the USG and the UN in this instance are substantially convergent, the main difference being the USG was on the ground, while the UNDP was in New York. Overall, the program decision-making process has evolved into a team effort involving all the major stakeholders. It is understood by all, however, that the US Embassy is prima inter pares among the stakeholders.16 15Some stakeholders insist that capacity building as a means toward achieving bi-communal results was an important criterion in the decision making process, pointing to the substantial support given by BDP to the NGO Support Centers. The team notes that the Turkish Cypriot Support Center has provided important training to the relatively newer and arguably more robust NGO community there, but found as well that the same effort on the Greek Cypriot side was not successful. Moreover, the Greek Cypriot Center continues to remain underutilized and not very productive even after being taken over by the PMU. From the perspective of the team’s considerable experience with USAID funded Civil Society programs, it is very difficult to find much evidence in the BDP program of a serious effort to engage in NGO capacity building as an integral and consistent part of the implementation strategy. To have done so would have required the application of a strategy, much like that suggested in the Biddle Evans reports, that went well beyond the one applied by the program’s implementers and policy managers. 16 US Embassy based officers responsible for oversight of the project commented on this finding. They do not believe that their advice was listened to or taken seriously by the PMU. The evaluation team interviews with PMU staff as well as with US Embassy and USAID representatives support the finding that on matters of policy, as reflected in grant decisions, the US Embassy’s views are controlling. As noted elsewhere, US Embassy’s frustration with perceived PMU management weaknesses prompted a substantial engagement of the US staff in program implementation. This may have contributed to confusion and frustration on all sides. Cyprus Bi-communal Development 14 May 25, 2004 Program Evaluation Development Associates, Inc. a. Bi-communal explained According to interviews with policy level staff, bi-communal can be defined as broadly as any activity that has the potential for promoting tolerance, and support for a peaceful settlement of the conflict. TABLE 2 Operational Criteria for Grants Political, No Bi-communal Aspect and Other: These criteria prevailed when there was sufficient pressure or need to use BDP funds for something worth doing that did not fit well using other criteria. Island wide projects: These are mono-communal projects that have island wide implications, therefore affecting both communities. In Spirit: These activities are done by a single organization (mono-communal), but undertaken in the interest of improved tolerance and cooperation. Mirror Projects: These activities would be “In Spirit” but would be the same or similar projects in both the TC and GC communities. These projects do not involve communication between the communities. Collaborative Projects. These are activities which require some degree of communication between the two communities. The information sharing contacts are often encouraged and facilitated by the PMU. Joint activities: Projects of this kind require regular activities done together by GC and TC agencies. In an attempt to understand the distribution of the BDP funded activities in terms of the classification/taxonomy set out above, the team asked US Embassy program staff to classify the entire portfolio using the categories.. Table 3 shows the summary results. TABLE 3 A rough classification of the Portfolio* Type Number Class In Spirit (IS) 20 NGO . Mirror (M) 9 IA-NGO Collaborative C 37 IA-NGO Joint (J) 15 IA-NGO Not Bi-communal /Island Wide/Political (N) 11 IA (NMP) *The projects enumerated above do not add up to the number of grants/contracts made by the PMU. Implementing Agencies had multiple contracts, as did a number of the NGOs. Cyprus Bi-communal Development 15 May 25, 2004 Program Evaluation Development Associates, Inc. The table demonstrates that grants made for a relatively low level of bi-communal activity could develop an internal dynamic that moved the grantees into a more relevant position, and in more than half the cases, to a collaborative or joint effort. As the table below indicates, if the classification system is applied, 27 percent of grantee organizations were able to move up the scale of bi-communality from mono-communal or mirror projects to collaborative or joint projects. This is indicative of a deepening of bi-communal relations by a small but noticeable percentage of the grantees.17 TABLE 4 Project Movement toward Bi-communalism A rough classification TYPE OF MOVEMENT NUMBER N to IS 21 IS to C 3 M to C 15 C to J 7 Total Moves 46 Grantee projects showing movement towards significant bi-communality: 25 *Classifications and ratings suggested by USAID staff. The criteria for decision making about grant program selections have shifted somewhat since the program began. The agreement between USAID and UNDP called for an NGO program from the beginning, but in the conditions of 1998, it was difficult to mount. NGO grant making gradually became possible, although the formal restrictions on contacts remained in place until 2003. Various efforts were made to establish criteria to guide the decision process toward a more developmental focus during the period 2000-2003. Outside consultants developed sector strategies for a NGO development program and an effort was made to establish a more focused Environmental grant program. Finally the PMU formalized what had been evident to all. The 17 The team is fully aware of the limitations of the findings presented in Table 3 and 4 above, and appreciates the effort made by the local USAID representative to provide the reported estimates. We recognize the potential for bias of any stakeholder in making conducting such a rating. It should also be noted that our 9 days of field research covering a wide variety of Task Order issues did not permit the application of any serious social science techniques for scientifically testing hypotheses. As we became aware that very little evaluative work had been done by the PMU in assessing bi-communal results other than a one shot effort to measure “contacts”, we requested the help of those who knew the program best, the local USAID Representative and the PMU project officers (as reported in subsequent sections of this report). By so doing, our hope was to illustrate that the best information about a program is often contained within the minds of those who know the program best. When challenged, these people can and should be able to participate in qualitative evaluation processes, especially if supported by other, more systematic efforts. If program leadership takes the job of results management seriously, it will find a variety of ways to improve the relevance and validity of the information needed. Cyprus Bi-communal Development 16 May 25, 2004 Program Evaluation Development Associates, Inc. Flexible Framework Fostering Rapprochement (FFFR) was adopted which emphasized “dimensions” of bi-communality projects irrespective of the particular sector. The FFFR was UNDP/UNOPS/PMU’s attempt at an acceptable formulation for a program that was and had to be opportunistic, flexible, and politically driven. Efforts were made to develop a sector strategy for environment and for NGO development, but the team had difficulty in determining the extent to which this strategy was actually implemented, or its effects consistently monitored. The FFFR standards did help to clarify several dimensions of a strategic approach to the political goal of rapprochement.18 b. Announcing the program For IA projects, the GOC Planning Bureau and the Turkish Cypriot authorities ‘advertise’ the BDP within their organizations via circular notes. Agencies/offices then submit proposals through the Cyprus Red Cross (Planning Bureau) or the Humanitarian Relief Mission (foreign affairs) to UNOPS for processing. NGOs learn about the program through word of mouth, newspaper advertisements and more recently, from a BDP website which states purpose, broad eligibility criteria, and provides a link to an application form. Now that BDP is well known, individual NGO leaders may contact the BDP directly, and some approach US Embassy officers for consultation prior to submitting an application. c. The review process All proposals, IA and NGO, are compiled into binders by PMU staff. These are circulated to members of the Project Steering Committee(s)—UNDP; UNOPS; the US Embassy; and on the GC side, the Cyprus Red Cross/Planning Bureau and on the TC side, the HRM and the “economic director” of the “foreign affairs” office. PMU project officers conduct a preliminary evaluation of proposals and feed their recommendations into a review of proposals by the PMU senior staff and the Embassy. A recent practice is the inclusion of the project officers in this PMU-Embassy review meeting. From the Embassy’s side, proposals are reviewed by the Economic and Commercial Sections—more specifically, by the Embassy’s Advisor on Bi￾communal Programs and the USAID Program Advisor. A summary table of proposals is submitted to UNDP and UNOPS headquarters followed by a telephone conference. A consensus is reached on the BDP’s position regarding projects to be funded prior to the formal quarterly 18 In commenting on this report, US Embassy staff made the following statement: “USAID/Embassy insisted that the UNDP/UNOPS/PMU could/should develop sectoral strategies within the FFFR. There was a lengthy discussion in Nicosia and at larger BDP management meetings about how strategies in specific sectors needed to focus on activities that were the intersection of three separate and often conflicting sets of criteria—namely, (1) the BDP’s program priorities (foster bicom contacts, cooperation, etc); (2) the on-the-ground realities and constraints (political situation, institutional realities, etc); and (3) the ideal developmental goals (if Cyprus were not divided) in each sector. While this approach is outlined in detail in the “Environment Sector Strategy” documents, it was a framework that was applied across the board.” The team notes that an environmental sector strategy was developed, but if this was the framework that was applied across the board, we did not find evidence that it was applied anywhere. Nor can we discern it from the types of activities that were funded. By far, the “sector” receiving the most financial support among NGO grants was related to health issues, for which no strategy was developed so far as we can discern. The explanation given for the heavy concentration on health oriented NGOs is that these were the only fundable proposals in the early years of the project. Cyprus Bi-communal Development 17 May 25, 2004 Program Evaluation Development Associates, Inc. Project Steering Committee meetings, which include representatives of the Greek Cypriot and, separately, Turkish Cypriot communities. These meetings also include New York offices via a video link. The extent to which TCC and GCC stakeholders participate in the decision-making is limited, at least formally, to the penultimate PSC meeting. As one local participant described it, the PMU￾US Embassy internal review and consensus process assures that US and UNDP interests are generally observed. The TCC and GCC participants can say “NO” to some projects, but lack the power to approve a project without the support from the UNDP and USG decision makers. Once the grantee list has been formalized, a memorandum is sent to UNOPS Executive Director through the SPMO, who authorized the PMU to proceed with contract negotiations. In the meantime, those organizations whose proposals were not accepted receive a letter stating the same. This letter does not go into any detailed assessment of the strengths and weaknesses of the rejected proposal, but does invite applicants to call the PMU for feedback and/or to request additional information on the Program. d. The Fast Track After the 2002 Annual Review a ‘fast track’ process was created for target of opportunity projects or for ad hoc projects that might not receive the approval of the authorities. A weekly meeting was established whereby the PMU and the US Embassy approved such projects without reference to the PSC. This was further formalized as a new grant category called the Special Initiative Grant with a limit in principle of $12,000 per grant. e. Negotiating the contract After authorization is received, it is the task of the PMU Project officer to negotiate the contract with the grantee. The Project Steering Committee frequently approves proposals with conditions, which must then be negotiated and including in the actual grant documents and budgets. Once agreement is reached, the grantee may begin work according to a pre-agreed schedule of activities/deliverables which serve as a basis for payments. f. Evaluation of Program Impact Project and program files do not provide much information on the application of either rapid or more formal results monitoring procedures, whether by grantees, the PMU staff or outside experts. Although there was a mid term program evaluation, much of that effort was focused on program operations and establishment of improved management and decision making systems, rather than on program results The team found little documentation, even at the project level, to draw conclusions as to whether discrete bi-communal targets had been established or met. Absence of evaluative information in the project files, combined with the way project data is aggregated, have made it difficult for program managers to judge how they are doing and to make changes in program policy and strategy to improve program effectiveness. Cyprus Bi-communal Development 18 May 25, 2004 Program Evaluation Development Associates, Inc. 3. The Contingency Plan The contingency plan for the BDP, originally prepared in mid-2002, identifies some critical areas to be addressed in the case of a settlement. Those areas remain as valid in mid-2004 as they were two years ago. Opinion leaders as well as officials from both GCC and TCC recommended many of these same areas for future donor support. Briefly, the contingency plan recommends four strategic priorities for BDP assistance: (1) Facilitate communications through physical reconnection of the two parts of the island (roads, telecommunications, postal services, redevelopment of the Nicosia buffer zone) and through development of joint regulatory and management systems for use of common resources and facilities. (2) Support good governance through assisting in the establishment of the common state organization (the federal state), and promoting improved local government that interacts with civil society through participatory mechanisms. (3) Spur economic growth by raising competitiveness of small and medium scale industries and by developing economic linkages between the GCC and the TCC. (4) Strengthen social cohesion and mutual respect by strengthening civil society organizations, by undertaking activities that develop common values, non-discrimination, tolerance and reconciliation (e.g. through education, human rights activities, dispute resolution), and by supporting the media to promote peace and reconciliation. The problem with the contingency plan in the team’s view is that it represents a major expansion in areas of assistance, including expansion to new areas not in the BDP’s proven areas of comparative advantage such as communications infrastructure and development of economic linkages. (These two areas in particular are likely to be addressed through EU assistance.) The current structure and management of the PMU is not adequate to handle major additional interventions such as these. Subsets within some of these priority areas are appropriate, however, and these are discussed in the recommendations section. C. ALLOCATION OF FUNDS BY SECTOR 1. Changes in Spending over Time (Overview) The BDP developed out of the UNHCR program with a high level of involvement in infrastructure projects, so spending patterns illustrate a steady commitment for funding implementing agencies. The figure below illustrates that the spending over time changed somewhat due to the changing priorities of the program. As an offshoot of large infrastructure projects, the primary recipients of the funding were the implementing agencies. However, over a short period of time, the NGO program became a priority for the BDP and spending levels reflect this. Cyprus Bi-communal Development 19 May 25, 2004 Program Evaluation Development Associates, Inc. Spending Over Time - 1.000.000 2.000.000 3.000.000 4.000.000 5.000.000 6.000.000 1998 1999 2000 2001 2002 2003 Dollars IA NGO PMU Source: UNOPS After an impressive increase in spending on NGOs between 1998 and 2000, the expenditure level stabilized. The spending on NGOs fluctuated between 1.3 and 2.3 million dollars a year from the year 2000 to 2003. Implementing agencies experienced much higher levels of fluctuation in funding from the BDP over the years. Spending under the category of PMU reflects both operational costs as well as some directed expenditures such as the cost of preparing meeting rooms for Settlement negotiating teams. Although the program documentation suggests a shift in emphasis toward the NGO side of the program after 2001, the spending pattern that emerged by year 2000 remained fairly constant for IAs and NGOs for the balance of the program. The dollar value of NGO grants was reduced in 2002, while the number of NGO projects increased substantially after that. Only one year, 1999, is somewhat of an anomaly in the data. It appears in that year the PMU, itself, was the major monetary benefactor of the BDP funds in 1999. In the first year of the project, the PMU was “squatting” in UNHCR’s offices until 1999 when it refurbished the old airport premises. 2. Governance and Civil Society Grants—Patterns and Changes Although NGOs have not been the primary financial beneficiary of BDP funds, NGOs have been able to benefit a great deal from the program, especially since the year 2000. The PMU does not keep records on NGO grant spending by “sub-sector”.19 In an effort to understand better whether there were implicit sub-sector allocation preferences, the evaluation team organized all NGO grants into 8 sub-sectors, based on the titles of the grants supplemented by other descriptive data. 19 US Embassy personnel state that they had urged the PMU to keep financial and reporting records organized by NGO, but to no avail. Cyprus Bi-communal Development 20 May 25, 2004 Program Evaluation Development Associates, Inc. The chart below illustrates how spending in governance and civil society was distributed among these subject matter sub-sectors and changed over time. By May 2000, spending was concentrated in the categories of environment and public health. These two sectors remained the best funded until June 2001. By mid-2001, the data show that there was a considerable effort to fund culture, community development, peace/mediation, and education. In about one year, the allocation of funds shifted from over eighty percent allocated to the categories of environment and health to a more balanced situation in which grants were given to organizations for other activities. Spending over Time as Percent 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1998 1999 2000 2001 2002 2003 IA NGO PMU Source: UNOPS Another major shift appears in the spending patterns in 2002. Funding became available for issues in gender and the economy for the first time. In addition, 2002 saw a greater stress on spending on culture and the arts. By December 2003, spending on governance/civil society became more diversified, allowing for a richer diversity in organizations funded and projects funded. In addition, the BDP showed its interest in supporting NGO development by investing in intermediaries to assist in NGO development. Public health was the category in governance/civil society that received the most funding up until December 2003. There are many projects that received over $100,000 and dealt with some health issues of importance island-wide. Only one project received more than $150,000, a project on Dyslexia. Over the entire period, there were large sums of money invested in environmental concerns. Many projects received over $100,000 and dealt with some environmental concerns. One, Swelling Clays, cost nearly $200,000. It should be noted that while NGO funding remained fairly steady following the year 2000, the allocation by categories diversified greatly, especially since 2002. This raises administrative issues. A greater number of grants would suggest higher transaction costs and oversight costs. Cyprus Bi-communal Development 21 May 25, 2004 Program Evaluation Development Associates, Inc. unal Development 22 ation May 25, 2004 Program Evalu Percent of Disbursement of Funds for NGOs by Category 0 5 10 15 20 25 30 35 40 45 50 %May 2000 %December 2000 % June 2001 % September 2002 date of disbursement (cumulative) percent of funds disbursed Culture/Arts Community Development Environment Public Health Peace/Mediation Education Economy Gender Source: UNOPS Bi-Communal Development Programme in Cyprus CYP/98/001 Annual Reports and Qt l R t 3. Special Initiatives he special initiatives were a device that appeared as a sub-sector of the grants to allow for ibil Although the Special Initiatives appeared only in the past few months f the program, they have allowed a great number of projects that require relatively small T flex ity in the program. o amounts of capital but have a significant impact. The figure below illustrates the growth of the Special Initiatives over the past few years. Disbursement on Special Initiatives 0 20000 40000 60000 80000 100000 120000 140000 160000 as of June 2001 as of September 2002 as of September 2003 Cumulative Current Dollars Sources: UNOPS Bi-Communal Development Programme in Cyprus CYP/98/001 Quarterly Reports and Annual Reports Cyprus Bi-comm Development Associates, Inc. The projects range from the very inexpensive ($215) to the moderately expensive ($27,546). These Special Initiatives have led to bi-communal concerts, academic conferences, and other educational activities. It is hard to assess the impact of such a diverse array of “one off” initiatives. However, some seem to have potential for bringing the two communities together, even if only to discuss a common issue or problem. From the perspective of the donor, the Special Initiatives offer a great deal of flexibility and the ability to move funds quickly without the cumbersome project review and decision making process described in section 2 above. V. ASSESSMENT OF OPINION LEADER PERCEPTIONS OF THE PROGRAM A. APPROACH An open ended questionnaire was followed to structure interviews with Opinion Leaders (OL) from the Greek and Turkish Cypriot Community. The OL list was compiled from suggestions provided by the USAID Representatives and by the Greek Cypriot and Turkish Cypriot members of the team. The respondents included members of the Project Steering Committee, senior government officials, municipal leaders from both sides, union and professional association leaders, business people, and representatives of print media organizations. On the Greek Cypriot side, one Bishop of the Greek Orthodox Church was interviewed. In several cases, more than one person was in the room, but the only the answers of the principal respondent were recorded. The total number of Opinion Leaders was 23, distributed as: TC: 10; GC: 12. One informed expatriate observer was also interviewed. 6 of the TC and 3 of the GC respondents had been beneficiaries of an early BDP program, the Harvard Study Group. This may have put a positive bias on their responses…on the other hand, the relatively small size of the island’s Cypriot population produces an even smaller pool of potential Opinion Leaders, making it difficult to find someone who was not familiar with the BDP. B. FINDINGS The interview asked respondents whether they knew of the program and its objectives, their assessment of achievements, strengths and weaknesses, knowledge of other bi-communal initiatives, and their thoughts about post settlement priorities and the future role of BDP if any. 1. All respondents were knowledgeable about BDP, although 4 of the 23 had very little knowledge. These were for the most part newspaper and union leaders. 2. 7 of the 23 opinion leaders were unable to explain the purpose of the BDP, 3. Asked about the main accomplishments of the BDP, 6 cited infrastructure, 7 cited opening up contacts, and 3 cited development of civil society. 4. In reference to the main strength of the BDP, 4 respondents said making financing available; 4 GC and 1 TC respondent referred to PMU’s reputation for balance and being Cyprus Bi-communal Development 23 May 25, 2004 Program Evaluation Development Associates, Inc. trusted; while 3 TC respondents cited contributions to cultural heritage as the important strength. 5. Asked about major weaknesses, only 12 of the respondents replied, with 4 GCs and 2 TCs saying “funding of bogus NGOs” was a problem, with the same number citing “involvement of authorities” as a problem. 6. With respect to the important evaluative question of whether the BDP had made a contribution to Peace, 12 respondents said yes, while 5 said no…and the balance did not want to say. GC respondents tended to be somewhat more positive about BDP’s contribution than TC respondents. 7. The contribution of other bi-communal efforts outside BDP, primarily the EU program, was mentioned by 13 respondents, 5 persons from both sides mentioned other bi￾communal activities that had been initiated by groups without foreign funding. For example the teacher’s union leaders on the TC side described their activities with counterparts from the GC side with respect to revising textbooks. 8. Not surprisingly, most opinion leaders did not have views about the PMU as the implementing agent for the BDP, but 9 did, of which 5 were negative and 4 were positive. Several who had been involved with infrastructure and UNHCR did not see value in the NGO programs. 9. Asked about immediate future needs in a post settlement environment, 9 respondents said Reconciliation programs, 3 TCs mentioned history book revision and 2 mentioned providing legal advice and language training for the Turkish Cypriot side. The Greek Cypriot side also ranked Reconciliation, property settlement, economic and financial cost issues close behind. Respondents who participated in the Harvard Group who volunteered their assessment of that program said it had not done much to change their minds, but it did improve their understanding of the other side’s arguments. 20 Responses regarding the future role of BDP were varied and difficult to categorize. Some stated that the phrase “bi-communal”, should be dropped and a new “mantra” put in its place. Others stressed the linkage between civil society, education and reconciliation as appropriate areas for a successor program. One TC leader made the important point that after over the 40 years of separation, both sides had changed considerably, especially for the TC side. At the beginning, the TC population was largely agrarian with a tiny educated class. Today, the TC population is urban, well educated and knowledgeable about business and other professions. Also, the Turkish Cypriots are very different than the Anatolian Turks (and Kurds) who have been brought from the mainland as settlers. Now a majority in the north, the settlers are largely agriculturalists, poorly educated, traditional in belief and custom and poor, compared to Turkish Cypriots. Several of the Turkish Cypriot leaders noted that this social dualism is a source of potential serious conflict and needs to be addressed immediately if the settlement is to work. 20 The US Embassy observes that persons selected to participate in the Harvard Study Group were “hardliners” . The project was not intended to change people’s minds. The team agrees. Cyprus Bi-communal Development 24 May 25, 2004 Program Evaluation Development Associates, Inc. VI. FINDINGS BY SECTOR A. INFRASTRUCTURE AND ENVIRONMENT 1. Background on Environment and Public Infrastructure Many activities are related to the Environment and Public Infrastructure sectors that were initiated as pre-approved projects carried over to the BDP. These were rather large municipal infrastructure and environmental related projects carried out during the period February 1999 - August 2003. The two larger environmental projects developed during this period were the Nicosia Trunk E and the Mia Milia wastewater treatment facility. Two closely related environment related projects involved the study of an integrated warning system and sewage connections in Turkish Cypriot Communities (TCC). Infrastructure grants carried out as part of the pre-approved projects included several sizeable Nicosia Master Plan (NMP) related actions such as the Arab Ahmet and Chrysaliniotissa neighborhood restoration projects on both sides of the line. Pre-approved projects were discontinued in 2001 and the most recent environmental and infrastructure projects implemented under the BDP require review and approval by a Steering Committee. PMU personnel supplied information for each project as direct access by the evaluation team to project files was not allowed. The information collected in the interviews and project files review serves as the foundation for the assessment of the BDP program. In terms of dollar value, the PMU directly implemented a larger share of project resources ($6,380,834) than did the implementing agencies ($6,221,225). This enabled implementation to proceed rapidly and several grantees interviewed were impressed with the PMU’s capabilities in managing the funds and expediting procurement. Thirteen percent of the projects were contracted with NGOs (two different Greek Cypriot NGOs and two different Turkish Cypriot NGOs). Half of the projects went to the Steering Committees for approval. Thirty-two percent were either pre-approved or ad hoc agreements. Over half of the projects were implemented on both sides of the green line. 2. Findings on Environment projects a. Management Because most of the public works programs required cooperation from implementing agencies from both sides at a time when official contacts were banned, the projects could not proceed without the PMU playing a critical role as intermediary and, in many cases, as direct project implementer. To supplement its own staff skills, the PMU frequently brought in consultants to give advice in specialized areas. This worked well and was appreciated by the implementing agencies, though there were a few cases when the implementing agencies disagreed with approaches recommended by the consultants. Several technical deficiencies observed during the field inspection of the wastewater facility preclude a positive rating of the PMU management operation. Unfortunately, some of the Cyprus Bi-communal Development 25 May 25, 2004 Program Evaluation Development Associates, Inc. problems seem to have resulted from the way the PMU divided responsibilities in the interest of promoting bi-communalism. The Mia Milia wastewater treatment facility represents a significant capital investment that was not completed in agreement with standard practice as the flow meter device was not properly installed. This apparently resulted from the decision to use a Greek Cypriot supplier. Because the supplier could not work on the Turkish Cypriot side, where the flow meter was to be installed, installation fell to the Turkish Cypriot contractor in charge of the civil works. The contractor was not able to install it properly. As a result, the flow of effluent cannot be measured. The PMU is aware of this problem, but so far no action has been taken to correct it. 21 In addition, engineering oversight during the construction phase did not provided proper advice with regard to geotechnical aspects of the project and significant quantities of seepage originate from the facultative and maturation pond units. Both of these deficiencies have caused discomfort and complaints from the TC operator of the facility. Seepage from the ponds system creates conditions for environmental distress due to potential groundwater contamination. The operator of the facility also indicated that the necessary operation and maintenance training was not provided prior to the start-up of the facility operation and as a result they are unable to efficiently operate the facility. The Embassy and PMU staff state, however, that repeated training was provided for operations and maintenance as part of the project. The Embassy and the PMU emphasized to the municipality the importance of putting a decent management team in place if the program was to be continued, and that it was discontinued when the municipality would not approve it. There remains the question, however, of whether the BDP should have allowed a previous investment such as the flow-meter to fail. b. Strategy The strategy for the Nicosia sewerage program was to address a long-standing issue that required cooperation from both sides to resolve, and in doing so, foster good relations and common systems and approaches, so that the two sides would be able to work together without PMU’s presence in event of a solution. At the same time, it would produce a visible result of social and developmental consequence for residents on both sides of the city. The strategy was sound, in that it did result in common equipment, systems, and approaches. IA representatives surveyed said that the projects resulted in regular contact and good cooperation between the technical people involved in the sewerage system on both sides. 21 When we first informed the US Embassy staff of the flow meter problem, our understanding from their comments was that they were aware of the problem. However, subsequent communication informs us that they were not aware of the “details of the flow meter,” but they were aware of general management problems at the sewerage plant. Whatever the case, the BDP should have taken action to correct this serious failure in a major investment. Cyprus Bi-communal Development 26 May 25, 2004 Program Evaluation Development Associates, Inc. c. Effectiveness i. Materiality The environmental projects surveyed represent an investment of $6.6 million and have an impact on all the residents of Nicosia. They have promoted and achieved interaction between both communities and have opened the door for similar opportunities in the future. IA and NGO representatives surveyed noted that there were regular meetings, that relations had improved and they were better able to deal with their peers from the other side. In another one of the projects evaluated, the full effectiveness of the wastewater facility investment has not been fully realized because not all the necessary equipment was properly installed. It appears that no efforts have been made to complete the pending action. In general, it will be necessary to make sure that projects are not halted or abandoned before all recommended or specified actions are completed. ii. Institution-building The projects provided training in operations and maintenance for those who needed to implement the system improvements. In addition, several of the IA representatives surveyed said that their abilities to implement large projects were enhanced. The need for the PMU to take on the lion’s share of procurement and contracting itself has probably limited the capacity-building impact, and to a lesser extent has affected feelings of “ownership” of the projects and their problems by the IAs. In interviews, some IA representatives referred to the activities as being the PMU’s rather than their own. iii. Symbolic The sizes of these investments alone are an important symbol of U.S. interest in tolerance and cooperation. Unlike the Nicosia Master Plan (NMP) activities, however, water and wastewater projects are not noticed unless something goes wrong. The deficiencies noted above could in time have a negative impact if they lead to water quality problems that attract public notice. 3. Infrastructure projects a. Management Unlike the sewerage projects, the Nicosia Master Plan-related projects were implemented primarily by the IAs, except for the Buffer Zone Survey, the New Vision, and other consultancies and workshops. The major NMP initiatives reviewed, Arab Ahmet and Chrysaliniotissa neighborhood renovations, could be undertaken by one side alone, without much discussion with colleagues from the other side. The PMU played an important role in this sector by promoting a common vision between municipal authorities on both sides through the New Vision program and through partnership meetings. Information provided in the surveys of grantees contains several concerns with regard to the following aspects of the management function: Cyprus Bi-communal Development 27 May 25, 2004 Program Evaluation Development Associates, Inc.  Long delays in the approval of grants.  Need for expert advice from the PMU for the preparation of proposals.  Need for clarity of proposal submittal rules and proposal debriefings.  High turnover of personnel at the PMU.  Need for technical advisers to assist with the implementation of “sophisticated projects” such as ecological inventories. Despite the concerns expressed by the implementing agencies, there is agreement with respect to PMU strengths including its role as a facilitator for the implementation of projects. These concerns indicate that temporary technical expertise in specific fields is required to meet the intended objectives of the PMU management function. In addition, there is need for establishing a proposal review process that includes a debriefing period. b. Strategy Survey results endorse the BDP’s strategy towards the NMP as a unifying program. According to implementing agencies surveyed, the bi-communal purpose of the BDP provides the foundation of the projects being implemented in the infrastructure sector. Most projects are considered highly relevant to the physical unification of Nicosia. The general perception of the implementing agencies is that the BDP aims at closing the communication gap between the two communities. One of the interviewees indicated that the links being created as a result of the BDP are developing long lasting working relationships. c. Effectiveness i. Materiality The infrastructure projects reviewed by the team are valued at $6,018,280. They have had a substantial impact in urban revitalization. Projects such as the Arab Ahmet in the Turkish side of Nicosia and Chrysaliniotissa in the Greek side have advanced the goal of maintaining the Walled City as a residential area in preparation for a settlement and reunification of the Cyprus community. The NMP Buffer Zone Survey is an important example of creating a bi-communal group to preserve the heritage of buildings in the Buffer Zone. In the words of a Greek Cypriot interviewed for this evaluation “the purpose of the Buffer Zone Survey is to preserve the capacity of the area so it could have a role in the unification of the city.” Thus, the signal seems clear that these projects not only aid in stopping degradation of buildings in the area but also contribute to bi-communal objectives. Bi-communal working relationships fostered by this program are primarily among the architects and planners in the municipal governments. Recently, the BDP has begun fostering public￾private partnerships (e.g. the Jasmine Internet Café) to broaden the range of bi-communal contacts and to breathe life into the renovated areas. There is much scope to expand this type of activity. Cyprus Bi-communal Development 28 May 25, 2004 Program Evaluation Development Associates, Inc. ii. Institution-building All agree that their capacity to implement complex projects and to prepare proposals has improved, and some also noted that their relationships with architects and other counterparts from the other side have become close. iii. Symbolic Though they result in only a limited amount of direct bi-communal contact, historic renovations like the two neighborhoods improved under the NMP and the Venetian Walls restoration are visible symbols of US and UN commitment to improved relationships. To ensure that the symbol remains positive, it is important that the renovated communities are also revitalized, through activities that will bring people back into these areas. The public-private partnerships begun under the NMP may hold some promise to achieve this. B. OTHER IMPLEMENTING AGENCIES The evaluation team selected two of the large veterinary health projects (echinococcosus and brucellosis eradication projects) of the veterinary health departments of both sides, and the human health project (Elderly Care at Karpas) of the Ministry of Health on the Turkish Cypriot side. The two subsectors represent two very different approaches to bi-communalism. 1. Description of the programs a. Veterinary health The diseases addressed in the selected veterinary health projects, echinococcosus and brucellosis, have important impacts on Cyprus economic development. Echinococcosus had been recognized as an important problem prior to 1974 and the united veterinary service had an island-wide control program. With the separation, however, though the Republic of Cyprus maintained its echinococcosus control program in the South, the Turkish Cypriot officials in the North did not. The Greek Cypriot side had also received funds from UNHCR for echinococcosus, so by the time the BDP project was approved, the program was already past the initial “attack” phase and well into the “consolidation” phase. The infection rate was already under one percent. On the Turkish Cypriot side, however, after more than a twenty-year lapse in its echinococcosus program, it had to begin at the “attack” phase. Prevalence was high – 25 to 30 percent in sheep, 14 percent in cattle. As for brucellosis, on both sides the eradication project was new. The major effort was to prepare a common database for all the animals through ear tagging, to allow for identification of diseased animals and to pave the way for trade – both between North and South parts of the island, and to Europe. Both projects were approved by their steering committees in 1999. The BDP steering committee recognized that disease eradication is a long-term effort and was prepared to fund each activity for five years, though grants were approved annually with further assistance dependent upon sufficient progress. As of December 31, 2003, $4,411,714 has been contracted for these projects. A large percentage of expenditure has been for direct procurement by the PMU of testing equipment and supplies and pharmaceuticals, as well as an ear tagging and data base system for the North compatible with that used by the South. The grants have also financed Cyprus Bi-communal Development 29 May 25, 2004 Program Evaluation Development Associates, Inc. training and conferences for veterinary staff of both sides, an outreach program to teach farmers how to prevent disease occurrences, and establishment of a bi-communal veterinary training and testing facility in the buffer zone. b. Human health Despite efforts of the PMU to promote bi-communal health activities, early interest by implementing agencies was limited to the Turkish Cypriot side. 22 The primary activity reviewed by the team, Elderly Care, funded medical and social work services to about 200 elderly residents of six villages in Karpaz, of whom about 40 percent were Greek Cypriots. Other human health projects approved for the Turkish Cypriot health authorities were a palliative care project for cancer suffers in the Nicosia area, and a breast cancer detection project. The projects were approved in June, 2000, are now completed, and totaled $289,043. Later proposals by the GOC Ministry of Health were rejected due to high costs and low bi-communal returns expected. 2. Management a. Success depended on commitment of the Cypriot program managers For the first year after the health program was approved it faltered, primarily due to the lack of interest in bi-communal programs of the Turkish Cypriot health leadership. When new leaders more predisposed to participate in BDP activities took over, project implementation took off. As for the animal health program, the willingness of the directors of veterinary services from both sides to collaborate on data collection, training, testing and planning is due to their recognition that these economically important disease issues cannot be effectively addressed unless there is a single approach used island-wide. b. For activity that required cooperation from both sides, the PMU role as intermediary was – and is – essential The PMU provided means of communicating, sharing information, and ensuring that disease eradication programs were compatible that would not have been possible otherwise, because of the ban on direct communication between officials of the two sides. 3. Strategy Successful bi-communal strategies need to be based on important issues that require cooperation to resolve. The project strategy to promote bi-communalism among implementing agencies seemed to be: Get your foot in the door, and with luck, little by little you can wedge it open. 22 As noted earlier, public (or human) health projects organized by NGOs represent the largest single sub-sector funded by the NGO side of the BDP with more than 30 % of the total NGO allocations. Most of these projects appear to be more properly classified as public health support projects, dealing with information outreach, education, and social support to families whose relatives have a serious ailment, such as cancer or diabetes. Cyprus Bi-communal Development 30 May 25, 2004 Program Evaluation Development Associates, Inc. The elderly care and palliative care projects did help the BDP develop a relationship with the TC health authorities, but since no relationship was built on the Greek Cypriot side, the door didn’t open any further. Though there were plans to develop a strategy to promote bi-communal relations in primary health care and emergency health, it never happened. Perhaps primary health and emergency health are areas where bi-communalism is not necessary. In animal health, where there was a clearly discernible need for cooperation, the BDP strategy of wedging the door open a bit at a time worked. There was no written strategy for promoting bi￾communalism in animal health. It was a matter of building on successes, and using each successive grant amendment as an opportunity to push the door a little further. The first project consisted of mirror grants with most work done individually on each side, and a few face-to-face meetings at technical conferences. Over time, the success of the programs in reducing incidence of disease led the higher-level authorities to view them as important developmentally. Once they supported the projects, it became easier for the veterinary staff to collaborate more openly. The PMU was able to write collaborative meetings into the next round of grants. The next step was an outreach program on both sides, to train farmers throughout the island on safeguarding their animals’ health. The PMU included in farmer-level training the message that these were island-wide problems and farmers on the other side were doing their part by taking similar actions. Finally, in May 2003 the PMU opened a veterinary testing and training center at UNOPS offices in the buffer zone, to enable technicians from both sides to work together on preparation of tapeworm bait (for echinococcosus prevention) and bulk milk testing (for brucellosis identification). 4. Effectiveness In its small size and limited scope, the human health program fits better among the NGO programs than it does those of the other implementing agencies reviewed by the team. Therefore program effectiveness findings on this program will be consolidated with findings on the NGO programs. a. Materiality The veterinary health program has provided significant developmental benefits to both communities – and these benefits have enabled the program to increase its collaborative nature. Echinococcosus rates in the North have been cut in half for cattle and sheep, and have remained stable in the South. Similarly, brucellosis rates have declined greatly among cattle and goats. Aside from the messages of the outreach campaign, which have reached at least 8,000 farm families to date, substantive bi-communal exchanges have occurred among some 80 to 100 veterinary officials from both sides. The primary bi-communal benefit of this program, however, is not the exchanges that have occurred, but the fact that the common approaches, common data collection and analysis systems, and progress made in fighting these two diseases will pave the way for regular, direct cooperation between the two sides once a settlement has been achieved. b. Institution building Since the North had not had an echinococcosus control program since 1974, the institution￾building needs there were much greater than they were in the South, but TC veterinary officials Cyprus Bi-communal Development 31 May 25, 2004 Program Evaluation Development Associates, Inc. were eager to accept help. Through training, procurement of modern equipment and development of databases in both sides that can eventually be combined into one, the BDP has achieved substantial institution-building. The major factor affecting the Turkish Cypriot side’s ability to continue activities once the project has ended is financial constraints rather than institutional capacity. PMU staff recognizes, however, that handling so much of the project procurement may have prevented both sides from gaining skills in this area that they will need in the future c. Symbolic importance The veterinary program probably has very little importance as a symbol of bi-communality. The data bases and common approaches are not showy or easily described. The direct beneficiaries are technicians who happily do their thing, working with farmers on their own side of the island, except for occasional training, data analysis and discussion of recent progress. The most important benefits – reduction of disease – at the field level depends on mono-communal work between the veterinarians and the farmers. The echinococcosus and brucellosis projects could have been used to promote civil society involvement through farmer support groups that might have developed bi-communal relationships. We understand an effort to do this is being made in the ongoing dairy project. C. CIVIL SOCIETY 1. Description of BDP NGO Sector Support Based on NGO lists maintained by the BDP-funded NGO centers on both sides, there are about 141 active Turkish Cypriot NGOs and about 150 active Greek Cypriot NGOs.23 The largest categories of NGOs are professional, scientific and social. On neither side is there a legal structure that offers incentives for individuals or corporations to support non-profit and charitable organizations. Except for a few large, well-established organizations, NGOs tend to be small, fragile, and dependent on volunteers and in-kind donations from members. The initial grant agreement with UNDP allocated $2 million for NGOs. This ceiling was later removed to allow the program to expand. The philosophy behind an NGO program was twofold: that a vibrant civil society would be an important component of a successful society once a settlement was reached, and that in the meantime civil society organizations could greatly expand the number of bi-communal contacts to foster peace and cooperation. NGO projects were reviewed by the Steering Committees in the same manner as Implementing Agency projects; however, the Turkish Cypriot authorities would not initially agree to approve NGO proposals. While the steering committee on the GC side began approving NGO proposals by June 1999, the TC side did not give its approval until a year later, and then only on the stipulation that TC NGOs must have no contact with GC NGOs. Clearly, this was a major stumbling-block to bi-communal activity. As a result, the BDP funded many civil society 23 This is a somewhat lower estimate than that used in the Biddle/Evans UNDP/UNOPS NGO Strategy of July 2001. That document estimated that there are 150 NGOs on the TC side, but more like 400-500 entities on the GC side based on registration records We believe our list is more accurate, since it comes from lists compiled by the NGO centers as a result of the Biddle/Evans mission. Cyprus Bi-communal Development 32 May 25, 2004 Program Evaluation Development Associates, Inc. projects that were bi-communal “in spirit”, or concerned island-wide issues, or had mirror projects with NGOs on the other side, but where little or no actual contact between the two sides was made. The PMU encouraged GC NGOs to make contacts with individuals or groups on the TC side to increase the contacts resulting from the activities. Gradually, as the TC authorities became more comfortable with approving NGO proposals, TC NGOs also began to initiate contacts with the South, though they did so at some risk. Until April 2003, however, all on￾island contact between CG and TC NGOs was extremely difficult. Overall, the BDP has funded projects of 51 GC NGOs and 26 TC NGOs, to the tune of $6,416,800 (including assistance to NGO centers); with another $330,000 for international organizations or contracts to fund activities involving both GC and TC NGOs. On both sides, the largest number of grantees are organizations that address a medical problem such as cancer, Alzheimers, or diabetes (14 GC and 8 TC organizations); followed by professional associations and research groups. Other focus areas of NGOs were environment, advocacy, arts and culture, and youth. Only in the north were traditional community development NGOs funded. In the South, several bi-communal peace organizations received funding. Twenty-three organizations (28 percent) received multiple grants. Of these, 5 organizations, including the two NGO resource centers, received over $1 million through 2003. This concentration of grant funding indicates the difficulty BDP had in finding qualified grantees in the early years of the program. 2. Use of Technology to Foster Bi-Communal Relationships During the years of the TC bi-communal ban, the Internet was one of the few means for GC and TC organizations to communicate. Computer and Internet use is widespread throughout the South, and is common in the urban areas and some rural areas of the North. For that reason, it was potentially a very important tool to promote bi-communal relations. The BDP promoted use of communications technology by its grantees in several ways. First, the BDP created its own website (www.UNOPSpmu.org) from which potential grantees can learn about the program and other bi-communal activities, read about grant criteria and download application forms. The BDP website also contains links to other major websites funded through the BDP. Second, the BDP supported the development of several web-based resources that people interested in bi-communal peace and cooperation can access. Among these are the following key sites:  Technology for Peace (Tech4Peace.org) serves as a “portal” where organizations interested in peace and bi-communality can read or post the latest news, learn about or post upcoming events, access free e-mail and join chat-rooms on subjects of common interest, and even place their own websites, free of charge.  MediaNet (cyprusmedianet.com) offers daily translations of the key news events from major Cypriot newspapers in three languages: Greek, Turkish and English.  Cyprus Decides (cyprusdecides.org) describes the contents of the Annan Plan in plain, non-legalese language, in Greek, Turkish and English. Cyprus Bi-communal Development 33 May 25, 2004 Program Evaluation Development Associates, Inc.  The Management Center’s site (mancentre.org) announces news, events and services that it can provide to support capacity-building of TC NGOs.  The “Internet Quiz Game”, a new project, asks questions that help school-aged youth from both sides understand each other. Third, the BDP has included funds in several of its grants to enable organizations to purchase computers and software, obtain Internet access, and receive computer training. In some cases (e.g. the Neuronet project), this included specialized software to allow for professional and technical dialogue between the two sides. Many of these interventions have undoubtedly promoted the sharing of information on important issues related to bi-communality and potential solutions to the Cyprus problem. Most websites noted above are professional in appearance, user friendly, and accessible to both Greek and Turkish readers. When the Tech4Peace grant was approved, the UN and US stakeholders hoped that it could become the electronic partner to the NGO management centers, helping to encourage NGOs in a wide variety of sectors to engage with similar organizations on the other side. Tech4Peace’s director, however, has a different vision. He views the site as a resource for peace organizations rather than for the broader NGO community. Consequently, Tech4Peace has never become the tool that the BDP hoped. Except for Tech4Peace’s chat rooms, the BDP-funded web sites are informational, but do not allow for dialogue between people and groups from both sides. The latter occurs through e-mail, not websites. The PMU in its matchmaking role could have been more aggressive in initiating e￾mail dialogue among grantees and potential grantees in order to foster greater collaboration. This is reflected in the BDP’s website, which provides a good deal of general information, but few specifics about organizations whose proposals have or have not been approved or about upcoming bi-communal events and opportunities. The grantees interviewed by the team who were familiar with the BDP website had used it for one purpose – to download the application forms. A couple were confused by what they read about application deadlines. No one spoke of other parts of the site as being valuable to them. The BDP did not develop a communications strategy that could provide some direction on how the program could best use information technology until the last year of the project. 3. Management “We have no idea what’s going on.” - A grantee PMU staff is helpful and responsive and take their project monitoring responsibilities seriously. The project officers maintain excellent relations with their grantees and make frequent monitoring visits, despite the increasingly heavy workload they have taken on as the civil society program grew. The project application requirements are sufficiently comprehensive without being overly burdensome to grantees. Application instructions require grantees to identify project objectives Cyprus Bi-communal Development 34 May 25, 2004 Program Evaluation Development Associates, Inc. and beneficiaries, to provide sufficient background on the organization’s capabilities, to cover issues of sustainability, impact and bi-communality, to describe implementation arrangements, and to provide budget details. Grantees did not find the process burdensome and were comfortable requesting assistance from PMU staff when it was needed. Team review of a sample of grantee applications showed rather large differences in the amount of detail provided, particularly related to bi-communal activity. Lack of transparency and difficulties in communications are major sources of frustration, both to grantees and to applicants who do not receive grants. Those whose applications are not approved cannot understand why and are given no good explanations except for “inadequate funds”. Yet they see other NGOs with similar projects getting approvals. Some grantees became suspicious that this seemingly whimsical grant approval process meant that approvals were given to favorites, or to bogus NGOs. Some said they would not apply again for this reason. NGOs also perceive inconsistencies between the way different grantees are treated on eligible budget items (some grants have project coordinators, other grantees are told these expenses are not allowed), need for a partner organization from the other side, and project extensions. From a project management standpoint, the PMU lacks depth. The project management burden has become so great that when one project officer is away, no one else in the organization is able to handle a grantee’s issues. The institutional memory about project activities appears to reside with the project officer, not in the project files where others could access it in order to fill in for each other. 4. Strategy According to the Contact Theory lack of knowledge and information about the “other” is the source of evil. In a state of isolation, groups exaggerate their differences and fall prey to propaganda against the “other” while unfortunate events of the past can be kept alive. Creating conditions for groups to meet and get to know and see each other as human beings, and start a dialogue, helps break down negative stereotyping, prejudices and hatreds. The BDP applied the contact theory to “bi-communality”... The partial lifting of the restrictions on the freedom of travel between the two sides on April 23, 2003 was a major break through in the relations between the two communities. This naturally resulted in what the BDP was aiming to achieve: bringing the two peoples together. Even after this date it is hard to find clear criteria on which projects get approved. The PMU team has commented that the type of grants funded since April has shifted from lower levels of bi-communality to joint projects but there still are no written criteria or strategy. 24 24 The US Embassy comments on this report insist that a set of standards for promoting bi-communal activities through the BDP program did exist and were applied across the board. The team was unable to document this. Indeed, most interviews with PMU and Embassy stakeholders, including UNDP officials, USAID, US Embassy officers and Cypriot Representatives, indicated that the main decision rules were to maintain flexibility and be as opportunistic as possible. See Table 1 above for an excellent summary of these rules. Moreover, according to Embassy and PMU rankings, as high as 45% of the projects were rated as having little or no bi-communal results. A fully random process of grant making would produce similar results. The fact that some projects did succeed is not evidence for the application of criteria systematically applied. Cyprus Bi-communal Development 35 May 25, 2004 Program Evaluation Development Associates, Inc. 5. Effectiveness a. Materiality Roughly 34 percent of active GC NGOs and 15 percent of active TC NGOs were direct grantees of the BDP. Available evidence indicates that the BDP has succeeded in promoting bi￾communal contact in many cases. PMU staff rated ALL their grants on their success in achieving bi-communal goals, using a scale of A, B, and C, (high, medium and low). They gave 45 percent of the projects a “C”, suggesting little or no bi-communal achievement. 35 percent were rated “B” and 20 percent an “A”. The team also asked PMU staff to rank the organizations, as opposed to grants, on their potential to contribute to bi-communal objectives in the future. Forty-three percent received a “C” grade, 31 percent got “B”, and 26 percent got “A”. 25 The team then compared the two scores provided by the PMU staff for NGO grants with scores that the team gave to the 21 NGO projects surveyed (of which one was an international NGO and two were funded under special initiatives). In most cases, the team’s project ratings were in accord with those of PMU staff.26 Since a third of the NGO grantees received multiple grants, the team hypothesized that provision of more than one grant to an organization would be an indication that the PMU considered that organization to have high bi-communal potential. To test that hypothesis, the team prepared consolidated ratings for the 21 NGO projects that the team surveyed, consisting of the two ratings provided by the PMU (one for the project’s bi-communal impact and the other for the organization’s future potential to promote bi-communalism), the team’s rating, and the Embassy’s rating. The latter was derived by giving three points to any grant identified by the Embassy as Collaborative or Joint, two points to grants identified as mirror, and one point to grants identified as in-spirit, island-wide, or other. The four three-point rating schedules provided a maximum of 12 points for the most effective bi-communal projects and four for the lowest. The team then compared the ratings with the total funds received by that grantee organization (for all grants received). 25 This ranking includes only those grants and organizations that fall under projects listed as P07-01-XXX. Please see Annex 10 for PMU project and organizational ratings. 26 See Annex 9. Cyprus Bi-communal Development 36 May 25, 2004 Program Evaluation Development Associates, Inc. The table below shows the spread between the 21 grants in terms of bi-communal scores and value of BDP funds provided. TABLE 5 Rating $ 0-50,000 $50,001- 75,000 $75,001- 100,000 $100,001- 200,000 $200,000 and above TOTAL 11-12 2 1 3 8-10 4 2 1 2 4 13 5-7 1 1 1 3 1-4 1 1 2 TOTAL 8 4 1 3 5 21 More than half of the NGO grant recipients (13 out of 21 we looked at) were in the 8-10 rating category. But lower rated organizations have received higher amounts of money while higher rated ones received very little. All 3 of the 10-12 group received less than $ 75,000. This is because the BDP has been able to fund truly joint activity only since last April. The team then looked at those grantees (among the 21 surveyed) which had received multiple grants, to see if there was a correlation between amount received and bi-communal scores. 27 Results are shown on the following table: TABLE 6 Name of NGO Rating Amount Received % of Total Cyprus Institute of Neurology 4 $401,650 15.7 Management Center 10 389,001 15.26 Has Der 8 257,042 10 Cyprus NeuroScience and Tech 9 204,892 8.02 NGO Resource Center 8 202,618 7.9 KAYAD 8 196,989 7.7 CYMEPA 6 189,159 7.4 With two exceptions, the grantees with multiple grants did receive respectable scores from the raters. Credible informants believe that the large number of contacts fostered between organizations on both sides have reduced the level of fear and uncertainty about bi-communal relations when the restrictions on movement between the two sides were reduced. Though this opinion cannot be 27 The US Embassy comments on this table criticize the team for failing to take into account the “time dimension” of the NGO grant program, particularly with reference to the large CING grants, all of which were made at the beginning of the program when it was thought, presumably, that the institute would be able to pursue bi-communal activities. The team recognizes that grant making for bi-communal activities became easier after 2002, and is happy to learn that no more consideration was given to CING proposals. Cyprus Bi-communal Development 37 May 25, 2004 Program Evaluation Development Associates, Inc. verified empirically, it comes from people with an in-depth understanding of the Cyprus problem.28 On the other hand, development impacts from the civil society program in any sector are not apparent. Although this was a secondary objective, the documentation clearly indicates that objectives such as capacity building and material results were important in grant decisions, but the PMU had no procedures in place for assessing capacity development or for providing systematic support for this objective. This is because the program was too scattered, even within a particular sector such as health or environment, for any obvious impact. Further, neither the PMU nor NGO grantees collected consistent monitoring and evaluation data on capacity building or the development impacts of their activities. Though development impact is not the primary purpose of the BDP, it is possible for activities to have both bi-communal and developmental impacts. In fact, the presence of developmental impacts can make bi-communal activity more acceptable to authorities and skeptics. There was lack of clear agreement between stakeholders on the role of civil society capacity building in fostering bi-communal peace and understanding. The emphasis in documentation on the activity, rather than on the organization, is an indication of the secondary role attributed to capacity building. Nonetheless, some capacity building was achieved. This is described in further depth in the following section. b. Institution Building Although institution-building was not a high priority for some BDP stakeholders, some institution building did take place and dependent NGOs were formed. CS organizations in the North depend mostly on membership fees and fund raising activities for survival. Thus organizations other than unions and chambers of commerce or industry (professional associations that receive high membership fees that correspond to a percentage of their members’ incomes) were never able to establish a center or employ paid staff for their services. BDP funding provided an opportunity for this as well as creating a space for these organizations to expand their services. This did result in unintended capacity building of the grantees, but at the same time it created new expenses for the organizations while the organizations’ income levels and opportunities remained the same. Thus these young and fragile professional organizations are now dependent on BDP funding for carrying out their “new” or “expanded” services. This is more relevant for TC NGOs than GC NGOS. Only three grantees, all GC, said they had taken the one-day course offered by the NGO center on proposal writing, and one left early because the course was “too basic”. Two parallel Management/NGO Resource Centers were created to provide services to NGOs. The Management center in the North has been very successful, serving an increasing number of NGOs, while the center in the South is inactive. Some reasons were given for this including: lack of demand for the services provided, lack of interest of the managing team (board) of the Resource Center, and lack of visibility of the services of the Center. PMU took over the 28 A Post Script after April 24, 2004. In any evaluation of a controversial issue, evaluators are often told what the respondent thinks the interviewer wants to hear. In this case, the credible observers may have been right in their assessment, but the level of fear reduction on the Greek Cypriot side was apparently far too little and not sufficiently wide spread to make a difference in the outcome of the referendum. Cyprus Bi-communal Development 38 May 25, 2004 Program Evaluation Development Associates, Inc. management of the Resource Center in the South and hired a consultant to carry out the services, but this has not been enough to reverse the down cycle. There is a need for a more proactive Center in the South. During our interviews, the Center in the North raised the issue of a small market in the North and the fact that they might be interested in serving as the only center after a possible settlement. c. Symbolic Did the NGO program create the space needed for contact between the two communities? The data gathered through interviews with grant recipients and opinion leaders show that grants did indeed result in “bi-communal” activities. Thirteen out of 23 Opinion leaders questioned answered yes when asked about whether the BDP has contributed to a settlement of the political division between the TCs and GCs. Thus clearly there is a perception among most, if not all, that the program succeeded symbolically. Opinion leaders with direct experience in the public works side of the BDP expressed the view that these were by far the most important parts of the program, and did much to contribute to the establishment or renewal of working relationships among professional engineers and others. This view seems to be shared by PMU staff, who consistently gave an “A” rating to the bi￾communal impact of the major public works programs, compared to a much more mixed review of NGO projects as indicated above. During our interviews with Opinion Leaders it became apparent that many bi-communal initiatives that were already underway (especially those of the unions) were perceived as not fundable because of political affiliations that would raise objections. This issue emerged as one of the criticisms of the program. It is also evidence that groups not associated with BDP were making other efforts to advance greater contact and cooperation between the two sides.29 VII. CONCLUSIONS A. GENERAL CONCLUSIONS 1. Accomplishments a. The BDP is generally well known among opinion leaders on both sides, often as UNOPS, and is perceived by most to have had an impact on promoting appropriate contacts, cooperation and the possibility of support for a peaceful settlement, especially among government officials and younger Cypriots. 29 US Embassy commentators expressed unhappiness with these findings. Opinion Leaders interviewed gave us their perceptions of the BDP. Our reporting of those views does not necessarily indicate our concurrence. However, interviews with Embassy and PMU staff support the finding that grants could not be made to organizations that the authorities on one side or the other might object to. The more important point which does not seem to be acknowledged by the US Embassy is that OL’s on both sides indicated that there was independent support for and efforts to develop bi-communal contacts and working relationships outside the BDP. Cyprus Bi-communal Development 39 May 25, 2004 Program Evaluation Development Associates, Inc. b. Those activities that involved substantive contacts between Turkish and Greek Cypriots about subjects important to both sides did succeed in fostering effective working relations among people from a broad range of professions and interests. c. Those activities that encouraged common infrastructure and common systems for addressing infrastructure and economic issues, even though they involved collaboration of relatively small numbers of technicians and laborers, have improved the ability of TC and GC officials to cooperate on their own should a settlement occur. This is true as well of a more limited number of NGOs sponsored by the BDP. d. BDP projects have been a material and possibly symbolic manifestation of the commitment of the United States and the international community, through the UN, to the search for a peaceful settlement. They have created venues for visibility and positive public relations for the US Embassy and for the UN. . e. BDP Civil Society programs encouraged and facilitated contacts and possibilities for interaction between Greek and Turkish Cypriot citizens that would not otherwise have been possible given the prohibitions imposed by authorities on both sides. f. Making substantial funds available to civil society organizations supported increased NGO activity in a wide variety of areas, as well as providing the material support for improved capacity by many of these organizations. The establishment of the Management Center on the Turkish Cypriot side is an outstanding example. g. The UNOPS PMU, faced with a restrictive political environment hostile to bi-communal interaction, demonstrated extraordinary creativity and energy in implementing the large public works, environment and animal health programs inherited from the UNHCR period. The compelling need to assume much of the responsibility for procurement, technical support, and financial control of these projects substantially defined the PMU’s character and operational procedures during the first two years of the program and, to some degree, through the six year life of the grant agreement. 2. Challenges a. The political purpose of the program was well understood by all stakeholders, but there were varying interpretations of how to define ‘bi-communal’ and particularly what the best means were to that end. Following several efforts to develop more directive strategies for grant making, the broad categories developed in the Flexible Framework Fostering Rapprochement were adopted. FFFR may have been useful for categorizing bi￾communal activities at a time when the political restrictions were so harsh that a narrow definition would have cut off all activities. FFFR did not serve the purpose of a strategy, however, and in today’s environment, with a settlement at hand, a more focused set of objectives and indicators is needed. b. The NGO side of the program, in part because of the restrictive political environment, never gained the level of visibility and focus achieved by the public works side. Attempts by outside consultants to develop for the BDP a more coherent civil society strategy that Cyprus Bi-communal Development 40 May 25, 2004 Program Evaluation Development Associates, Inc. would support the political objective did not become fully internalized as part of the decision making or operational criteria of the NGO program. c. The size of the grants made to many of the NGOs, especially in the first three years of the program, appears excessive by most standards used by USAID and other donors in the East European and NIS region, including in institutionally well-developed countries such as Poland. Whether driven by the high costs of the Cyprus economy, or by other factors, the dollar value of the grants (upwards of $100 000) may have strained the absorptive capacity of many smaller NGOs, as well as producing a bias towards non-profit organizations already well established. (The grant ceiling was reduced from $100,000 to $50,000 in the latter years.) d. In the smaller IA projects and in NGO projects, development impact identifiable to the team has been very limited, because of the scattering of funds over many organizations and the emphasis on one-off activities rather than on strengthening the organizations that offer them. While development was not the objective of the program, seeking good development outcomes was not inconsistent with bi-communalism, and indeed may have helped to contribute to this objective, as was the case in several of the public works and animal health projects. While the NGO sector appears to be flourishing, it is not sustainable at the current cost level without significant foreign funding. e. The desire to avoid negative publicity or the appearance of political interference may also explain the predominance of grants to health (30%) and environment (19%) NGOs, particularly in the first three years of the project. As BDP became more established and the political constraints relaxed, funding shifted to peace/Mediation NGOs (11%), the latter made mostly in 2002/3. f. A political program in a highly volatile environment can be expected to experience a high failure rate in terms of activities that do not improve bi-communal tolerance and cooperation, and do not result in a significant level of enhanced capacity of value to the challenges of making a peaceful settlement work. However, absent valid indicators of program performance and even a modest effort to evaluate at the project level the extent of bi-communal results achieved, it is not possible to objectively assess whether the failure rate was either lower or higher than an acceptable level. g. The ‘strategy’ for implementation of the NGO and for much of the IA parts of the agreement was largely reactive rather than proactive. Little effort was made to direct grantees toward specific program areas or objectives through the various BDP call for proposal mechanisms. This contributed to the diverse and seemingly disconnected character of the NGO grant portfolio and to a repetitive “proposal bombardment” to the PSC by Implementing Agencies. As the possibilities of a settlement improved in 2002, the Special Initiative grant was established permitting the beginnings of a more directive program that remained within control of the Embassy-PMU decision makers. If the settlement does occur, expanded use of this facility may permit a more proactive and focused strategy of support in areas of critical importance to making the settlement work. h. Overall, the “bi-communal achievement and potential” ratings and classifications of BDP activities prepared by the PMU and by US Embassy officers directly involved with the program support the conclusion that the program was substantially successful in areas Cyprus Bi-communal Development 41 May 25, 2004 Program Evaluation Development Associates, Inc. involving common problems that the authorities could collaborate on; projects with larger budgets and fewer people organized around specific tasks, such as a common sewage project or the animal health series of projects. Projects involving Non-Governmental Organizations overall were less successful, having little or no bi-communal achievement in roughly 35 to 45 percent of the activities, especially in the period 1999 to 2001/2. Efforts to “force” a certain level of NGO bi –communal contact and cooperation during the first three to four years of the BDP, while heroic and well meaning, may have been premature and largely wasted. B. PERFORMANCE OF BDP DECISION-MAKING AND IMPLEMENTATION 1. Accomplishments a. The complex decision-making structure including USAID Washington, UNDP and UNOPS New York, US Embassy Nicosia, and UNOPS/PMU Nicosia appeared on the surface to be a recipe for stalemate. This was overcome by frequent communications, a common interest in making the program work, and, in the main, the strong leadership of the US Embassy team. This resulted in an operational mode that featured timely decision- making on grant approvals and authorizations and maximum flexibility for solving implementation problems, especially with regard to the sensitivities of the GCC and TCC authorities. b. The absence of a UNDP office in Nicosia, coupled the with more macro-political interests of UNDP leadership in promoting an overall solution to the Cyprus program meant that UNDP exercise of BDP specific program management was left substantially to the UNOPS chain of command. Only in the later years of the program did UNDP New York become more substantively involved with guiding program implementation and in developing a sound professional dialogue with USAID. c. PMU staffs are highly motivated, professional in behavior, and to the extent permitted by staffing constraints and Cyprus circumstances, accessible to their Cypriot partners. Frequent use of outside experts for implementation jobs suggests recognition of the need for additional expertise in implementing complex projects. Whether motivated by internal or external demand, the use of outside reviewers and strategic planning experts provided a steady stream of recommendations for improving performance of the program, especially on the NGO side d. The PMU did meet the basic allocation terms of the USAID – UNDP agreement, with roughly 60% of contracts allocated to Implementing Agencies and 20% to NGOs over the life of the program. Efforts to expand the NGO program after 2001 did not significantly change the overall financial allocation program, although it did result in an increase in the number of NGOs receiving grants. . e. Interviews with IAs, NGOs and opinion leaders indicate that the PMU enjoyed a positive reputation in both communities for being evenhanded and efficient. The word “trust” surfaced in a number of interviews relating to the PMU’s role. Another term used was the “UNOPS Umbrella” suggesting the PMU role was that of a neutral but committed third party facilitator, broker, and, for larger projects, effective implementing agency. The circumstances led to the PMU becoming something akin to a “proto-governmental Cyprus Bi-communal Development 42 May 25, 2004 Program Evaluation Development Associates, Inc. agency” backed by the political will and good offices of both the UN and the US Embassy. 2. Challenges a. The UNDP grant’s provision about political guidance from the Embassy provided entre’ for the USG’s representatives in the Embassy to be substantially involved in decisions throughout the project approval and implementation process. This made it difficult to establish policy, strategic objectives and implementation guidelines that could be monitored from “a distance.” The need to make quick and ad hoc decisions required the input of someone who could assess the problem from the standpoint of the Embassy on a daily basis. Policy managers became deeply involved in day to day implementation management. Thus there developed a potential for confusion about roles and the potential for divided loyalties. It is difficult to be a “team member” and still perform the function of policy and strategic oversight for USAID and the Embassy. Hopefully the post settlement future will permit a sharper division of labor and clarity of roles. b. The ‘strategy’ for implementation of the NGO and for much of the IA parts of the agreement was largely reactive rather than proactive. Little effort was made to direct grantees toward specific program areas or objectives through the various BDP call for proposal mechanisms. Achieving a controversial objective such as bi-communalism in a non-controversial manner is very difficult. Large infrastructure programs on common problems such as water and sewage compelled the authorities to collaborate to the degree necessary, and generally involved relatively few persons in the actual implementation process. On the civil society side, the overall record of NGO grant making suggests an understandable desire to avoid funding organizations that might arouse the opposition of one side or the other. This and the unusually large size of many of the grants may have promoted a tendency toward making grants to established NGOs that were acceptable to authorities, especially in the period before 2003 c. Frequent reorganizations and employee turnover are not unusual in international projects similar to BDP. It is of some concern that the current staff on the program side is relatively new, and there are several vacancies. On the other hand, this may be an opportunity to fashion a program that is better suited to the potential of a post settlement political environment. d. The ratio of project officers to staff and management seems disproportionate to the workload put on the line PO s, especially in 2003 – 2004. Moreover, the UNOPS policy of not training project officers on the grounds that anyone hired is already an expert presumes too much. Project Officers, especially on the NGO side could have benefited from more experienced leadership and from training for their responsibilities. e. The BDP PMU faced two very different implementation problems. The first came from the need to implement in a timely, cost effective manner infrastructure and environmental public works that produced visible and high quality solutions. The second problem was to implement an NGO grant program working with many, relatively inexperienced and fragile citizen groups on both sides. The staffing and procedural requirements for the two programs arguably were quite different. It appears that the systems put in place for Cyprus Bi-communal Development 43 May 25, 2004 Program Evaluation Development Associates, Inc. managing the NGO program were largely adapted from those developed for the public works programs. This led to a documentation system focused on “contracts”, rather than a system more appropriate to a politically-oriented NGO program based on achieving bi￾communal impact. The NGO program seemed to be something of a “stepchild” to the larger “flagship” projects for much of the life of the BDP program. If civil society development in support of reconciliation is to be a theme for a successor program, it will need the flexibility to adopt decision making, management and results monitoring and reporting procedures relevant to its strategic objectives. f. Perhaps because the nature of the two programs (large IA vs. civil society) was different, there was not as much effort as there could have been to expand the scope of interaction of the large IA projects by adding a civil society component. For example, the veterinary health component could have worked with farmer organizations. This would have required a more proactive grant-making process g. The program never was able to develop consistent, well understood and documented criteria for grant making. Neither grantees nor PMU project officers can understand the reasons why projects are approved or disapproved. The perception of inconsistency undermines the credibility of the BDP, and the resulting belief that grants are given to favorites or ‘bogus’ NGOs prevents some organizations from applying. . In addition, it reduces the credibility of the project officers with the grantees whose programs they monitor, and it contributes to project staff alienation from PMU management. It is possible to establish a reasonable set of approval criteria, even for a political program. This should have been done. h. Program and project monitoring and reporting focused on the “grant/contract”, rather than the recipient organization’s advancement of bi-communal goals. 77 NGOs received BDP grants (under the NGO component), 23 receiving more than one. There is no overall assessment of the organization’s progress toward greater interaction, collaboration or joint planning and implementation. Project files do not yield much information about bi-communal accomplishment.30 No effort was made to assess effectiveness and impact other than completion of agreed work. PMU officers know much more about grantee effectiveness than is represented in the project reports or closeout documents. For a $6.4 million dollar investment, a better effort should have been made to evaluate and track progress on Turkish and Greek Cypriot NGOs bi-communal performance. i. The post settlement conditions should permit the introduction of strategic focus, more clearly defined grant making objectives and decision criteria, and greater transparency in announcing awards and explaining rejections. This will require a serious overhaul of grant proposal review procedures emphasizing objective rating procedures, and, possibly, participation by Cypriot experts and “wise people” in some part of the process. It is 30 The US Embassy in reviewing this decision stated “The project files may not yield much information, but no decision was made to re-engage with an already funded NGO without an assessment of their progress to date.” The Team is pleased to learn this, but we still question how objective or consistent such a review process could be without some kind of evaluative documentation prepared either by the grantee or the PMU project officer as to the bi-communal achievements of the grantee. Cyprus Bi-communal Development 44 May 25, 2004 Program Evaluation Development Associates, Inc. important that Cypriot government representatives in any future program steering committee understand and agree with the program’s objectives. 31 VIII. RECOMMENDATIONS: MAKING THE SETTLEMENT WORK Even if a solution is achieved to the Cyprus problem, experience shows that there will continue to be political bumps over the next several years. Still, contact between Greek and Turkish Cypriots is dramatically different than it was when the BDP began, and further freedom of movement is likely. Therefore major changes in the direction of future programs are possible that will increase their effectiveness. 1. It is no longer useful to think in terms of bi-communality. That term tends to polarize the two sides. It also fails to take into consideration the multicultural nature of Cypriot society. There is still a need to support the peace process, foster reconciliation and cooperation among the diverse Cypriot groups. This is dependent to some extent on reducing the income disparity between the Turkish Cypriot and Greek Cypriot communities. 2. The post settlement conditions should permit the introduction of strategic focus, more clearly defined grant making objectives and decision criteria, and greater transparency in announcing awards and explaining rejections. This will require a serious overhaul of grant proposal review procedures emphasizing objective rating procedures, and, possibly, participation by Cypriot experts and “wise people” in some part of the process. It is important that Cypriot government representatives in any future program steering committee understand and agree with the program’s objectives. 3. Building on work already done, several assessments should be undertaken to set the stage for the next phase of assistance. There have already been discussions about undertaking sectoral assessments and developing sectoral strategies. Any strategy work should be based on research to determine the level of need and available resources in each sector. Program baselines should be established against which to measure results. 4. USAID should collaborate with local partners in performing analyses, developing strategies and implementing new initiatives. The US, through the BDP and CASP, has helped to develop capacity of many organizations and individuals. They can now participate in shaping the program of the future. 5. Mounting a program with four major objectives, as proposed in the contingency plan, would require a much larger commitment of funding and a more diversely expert staff than the program is likely to have in the future. The analyses described above, along with further information about the EU’s plans, should factor into decision making about appropriate directions for future US assistance. 31 We are fully aware that there are risks to greater transparency and increased Cypriot participation. On the other hand, as has been demonstrated by USAID programs in other difficult environments, US programs “model” desirable values and behaviors in the way they are implemented. Cyprus Bi-communal Development 45 May 25, 2004 Program Evaluation Development Associates, Inc. 6. The Scope of Works calls for up to 6 indicators that would be used to measure progress towards results. Evaluation and monitoring of program and project progress towards observable results is essential for accountability and good strategic management. The team believes that one can't establish good performance indicators until one has established clear, well-defined, operationally useful objectives. Once that has been done, performance indicators serve three purposes:  They help you to measure how successful your interventions have been.  Equally importantly, they help you to determine what activities to fund in the first place.  Finally, they help determine whether the program strategy or ‘theory’ is working, or needs adjustment. Getting ownership The team’s experience with outside consultants preparing performance indicators has not been very successful. A better approach is to engage the person in the organization who is responsible for monitoring and evaluation progress toward results in a process of “discovery”, in which what is feasible, relevant, and useful is arrived at with the consultant, if a consultant is necessary. Both USAID and UNDP have gained considerable experience with results management, and have prepared guidance documents to help implementing organizations make the right choices. Possible ways of measuring increased tolerance and cooperation between the GCC and the TCC - and other Cypriots. Priority for selection of activities should be based on criteria such as those that follow. The criteria have been introduced in the body of this report. The same criteria can be used to measure performance - at the activity-level. Materiality i - The activity is in an area where the gains from cooperation are such that there will be good reasons for close cooperation to continue even after the activity ends, and these gains can be specified at the time of start-up. ii - The activity includes substantive involvement and discussion of a large number of people, and has the potential to increase to larger numbers of people in the future. iii - The activity will achieve an economic or social objective that is important to participants of all involved communities, in which clear objectives are determined prior to activity start-up. Institution-building i - The implementing organizations have previous experience, either through BDP or through other demonstrated activity, that indicate they have has the capacity and commitment to implement the planned program. ii - The planned activities enhance the ability of the implementing organizations to expand areas of cooperation or expand the populations involved in cooperation. Cyprus Bi-communal Development 46 May 25, 2004 Program Evaluation Development Associates, Inc. iii - The planned activities involve public-private partnerships that engage civil society with government organizations in both GCC and TCC. Symbolic importance i - The planned activity and involved organizations show promise for publicity that demonstrates the value of the bi-communal or multi-cultural cooperation. This should be built into the activity plan. ii - The activity leaves a visible economic or social benefit that observers or the general public will be able to easily identify as resulting form the bi-communal or multi￾cultural cooperation. iii - The activity addresses an issue that publicly highlights cultural, historical, or political diversity and demonstrates that this issue can be worked on cooperatively through mutual understanding. The BDP attaches a weight to each of these criteria, and selects those activities that achieve a certain weight. Evaluation of individual programs should assess progress and final completion against the same criteria, using the same weights. A Delphi-type method, using four or five knowledgeable people to apply, at regular intervals, the criteria to all the individual activities and organizations, can be used to ensure consistency. Program-level achievement of tolerance and cooperation is then based on the following indicators: - Percent of activities that achieve a weighted score above ___ percent, indicating that these activities have met "tolerance and cooperation promotion" goals. (The percent selected, of course, needs to take into consideration the high-risk and consequent relatively high-failure rate expected in a political program.) - Percent of organizations that have expanded areas of cooperation and multi-cultural populations involved in cooperation. - Number of individuals from the various ethnic and cultural groups who have been substantively involved in multi-cultural cooperative activity. (Substantive involvement has to be defined, but goes well beyond attendance at a rock concert or a conference or seminar.) Ways of measuring other program objectives Assuming the team's recommendations regarding future directions are adopted, other important objectives will include some variation on the following: "Expansion of an informed, politically-active civil society" "Establishment of an effective system of governance at the Federal level" "Improved effectiveness of participatory local government." and possibly Cyprus Bi-communal Development 47 May 25, 2004 Program Evaluation Development Associates, Inc. "Establishment of an active, sustainable Institute for Governance and Civil Society in Central Nicosia." All four of the above are developmental objectives for which both USAID and the UNDP have established good performance indicators. The reader is referred to the UNDP's Handbook on Monitoring and Evaluating for Results, the SIDA – UNDP’s Measuring and Managing Results, and USAID’s Handbook of Democracy and Governance Performance Indicators (available in PDF form at USAID's website). A word on the activity-selection process The above suggestions do NOT require that the BDP continue to select all activities based on a general solicitation for proposals, as it has done in the past. In fact, the team recommends that for civil society proposals, the solicitation should be more directive in terms of the sectors or subsectors from which proposals should come, and the levels of cooperation between multicultural groups that will be expected as a minimum. For activities implemented with government agencies, these should be the product of discussion between the BDP stakeholders, rather than through a solicitation process. 7. A future program should include a civil society component, but one more focused and strategic than the BDP, including advocacy. It is likely that the EU will finance many of the needed infrastructure and economic development activities. The US holds a comparative advantage in working with civil society. The BDP has fostered an active civil society with interest in maintaining their links with people on the other side. A strong civil society will be an important part of an effective, united society. In addition, there will be many issues affecting people that result from both the Annan plan and the EU accession. Civil society organizations can spur debate and increase the level of public discourse about these changes. They can also contribute greatly to the healing process of reconciliation and search for common purpose. The large dollar size of grants made to Greek and Turkish Cypriot NGOs, even taking into account higher cost on the island, should be re-examined. The assumptions underlying the cost structure of NGO proposals needs to be carefully examined, and efforts made to more rigorously distinguish between administrative overhead costs and cost directly related to implementing projects. 8. A second program focus should be “good governance”. Local and “state” level governments, especially in the TC side, are not sufficiently mature and developed to exercise the normal functions of government in an accountable, transparent and efficient fashion. Corruption, favoritism, cronyism is endemic in government on both sides according to most observers. Better systems for public finance management, procurement, and public participation in decision making are needed. If the Annan plan goes forward, there will also be a demand for technical knowledge about the workings of a Federal system. Although the Cypriot federal institutions will be quite limited at the beginning, substantial growth is anticipated. The United States has a comparative advantage in providing technical assistance in both of these areas. Moreover, most USAID programs throughout the Balkans and the NIS have Cyprus Bi-communal Development 48 May 25, 2004 Program Evaluation Development Associates, Inc. developed strong “good governance” programs. There is now a body of knowledge, expertise and experience that can be brought to bear quickly on the problems Cyprus will encounter. Making government work has to be a critical component of the larger “making the settlement work” program. 9. Program financial and monitoring documentation should be reoriented for Results Management. Based on the information available to the team, the documentation maintained by the PMU is not only inadequate to identify bi-communal impacts, but also inadequate for effective management of funds linked to expected program results. This is not to suggest that the financial reporting is faulty from an accounting or audit perspective. Such a conclusion would be well beyond our mandate and expertise. Our point is that from a Results Management perspective, the system for reporting on contracts and expenditures now in place is not very helpful. Expertise should be brought in to upgrade the financial and project data management system so that the PMU can more effectively manage funds for results accountability, in addition to financial accountability. Cyprus Evaluation Final Report.doc/EvalQC-42 Cyprus Bi-communal Development 49 May 25, 2004 Program Evaluation