EVALUATION OF THE MACEDONIA DEMNET PROGRAM TASK ORDER NO. 805 Submitted to: United States Agency for International Development Contract No. AEP-1-00-00-00023-00 Prepared by: Thomas J. Cook Ph.D. Assisted by: Mihajlo Popovski Ph.D. Submitted by: Development Associates, Inc. 1730 North Lynn Street Arlington, VA 22209-2023 (703) 276-0677 (703) 2 76-0432 February 2002 Page No. Executive Summary ............................................................................................................... 1 I. Introdlrction ..................................................................................................................... 5 11. Purpose of the Evaluation ................................................................................................ 5 111. Methodology ................................................................................................................... 6 IV. DemNet Implementation ......................................................... ; ........................................ 8 V. DemNet Results ............................................................................................................. 14 VI. Recommendations ......................................................................................................... 30 APPENDICES Appendix A Data Collection Protocols ........................................................................... A- 1 Appendix B Contacts ..................................................................................................... B- 1 Appendix C Program Data ............................................................................................. C- 1 Appendix D Reference and Documents .......................................................................... D- 1 Appendix E Success Stories ........................................................................................... E- 1 EvaIuafion of the Macedonia Februmy 2002 DemNet Program Task Order 805 The Institute for Sustainable Communities (ISC) initiated the Democracy Network (DemNet) Program in Macedonia in April 1995, under a cooperative agreement with the United States Agency for International Development (USAID). DemNet strengthened the non-governmental organization (NGO) sector through training, technical assistance and project funding. DemNet funding has topped at $5.2 million through three phases, concluding in December 2002. Twelve months remained at the time of this evaluation, the first evaluation of the program The evaluation covers the three phases, but pays particular attention to Phases 11-111 since they reflect the main strategy of the program 11. PURPOSE OF THE EVALUATION USAID/Skopje (Mission) viewed this evaluation as the fist phase of a two-phase planning process that will guide the future direction and methodology of its NGO development program This report was prepared as an internal Mission planning document to inform the process. The evaluation sought to learn from experience in order to plan for the future. A two-person evaluation team collected information from DemNet and other NGO stakeholders during a December 2001 three-week visit to Macedonia. Data collected in Washington, D.C. and in Montpelier, Vermont, ISC's home office, augmented this information. The DemNet program in Macedonia has evolved over three phases of activities. It began Phase I as an expansive outreach to a variety of NGOs operating in the environment sector. ISC adjusted the program strategy in Phases 11-111 to intensify ISC's involvement with fewer NGOs and to add specific program components, in order to elevate the potential for project success. IV. DEMNET RESULTS A. INTRODUCTION This section presents the DemNet results in terms of both individual NGO development and the NGO sector. It relates these actual results to the DemNet expected results pledged in ISC proposals to USAID. It also identifies contextual factors that might have mediated these results, and the lessons learned from DemNet implementation that will inform the report's final recommendations. Evaluafion of the Macedonia 1 February 2002 %Ir. DemNet Program Task Order 805 Development Associales, Inc. AEP-I-00-00-00023-00 B. INDIVIDUAL NGOS DemNet NGOs credit DemNet with making their development possible. Initially, the newer NGOs generally possessed only an embryonic understanding of NGOs. DemNet taught them about NGOs and what they can do if properly developed and operated. Even with this preparation, most of the sites visited readily admitted that they needed more training. NGOs already registered reported a similar experience. Most operated as loosely organized groups of people concerned about particular issues, such as drug abuse. They acknowledged that they lacked experience in operating an NGO. DemNet took them to the next level, helping them to organize and operate more efficiently as an NGO. The report centers on DernNet "process" activities, designed and implemented to develop these new and existing NGOs to the stage where they are capable of producing significant development impacts. This is because the primary goal of the Phase 1-111 DemNet project activities (i.e., expected results) was to raise NGOs to this level. The report contends that DernNet NGOs should now move forward to actually producing and evaluating their ultimate impacts. C. NGOSECTOR The Macedonia NGO sector was described by one stakeholder as a "series of islands, disconnected from each other." NGOs do their own thing, rarely sharing with each other or learning from each other. DemNet attempted to eliminate this isolation through its training, technical assistance (TA) and outreach activities. DemNet established a network of local trainers (e.g., LEAP/CAP training; NGO development training/TA) that presumably are available to assist new NGOs. It also produced and disseminated brochures designed to provide a guidebook. Despite these efforts, many of the NGOs visited lamented the sector's enduring competitiveness. The Macedonia NGO sector suffers from the general judgment that NGOs are ineffectual in actually alleviating, or eliminating, important social problems. Persuasive evaluative documentation of NGO achievement in producing significant development results (impacts) will go a long way towards recasting the sector's image. It will demonstrate that the DemNet process results, in fact, enabled potent NGO action that indisputably produced important and measurable impacts. D. MEDIATING FACTORS Several factors influenced DemNet's implementation. The Kosovar refugee (1999) crisis fueled inter-ethnic tensions, and was followed closely by armed conflict between Albanian rebels and the Macedonian Army/Government, creating an inhospitable political environment. Evaluation of the Macedonia 2 Februmy, 2002 DemNet Program Task Order 805 The negative image of NGOs has had a detrimental impact on support for NGOs. The d proliferation of NGOs in the last decade, many of which were viewed as purely opportunistic, magnified the problem. MI' Several stakeholders also noted that the weak NGO image has inflamed the traditional NGO - government distrust, resulting in a flimsy basis for cooperation. R Most NGOs have to rely greatly on volunteers. The absence of a tradition of volunteerism in Macedonia reportedly has impaired volunteer recruitment. The frail economy also has limited private sector involvement. Small businesses are struggling just to get by; business failure is commonplace. Finally, most NGOs face a hostile fiscal and legal environment. The Law on Citizen Associations and Foundations (adopted June 25, 1998) has created a restricting fiscal atmosphere. E. LESSONS LEARNED Phases 1-111 produced helpful lessons for NGO development, growth and sustainability. Some of these lessons refer to successful practices that should be continued; others reveal shortcomings that need to be corrected. These lessons are summarized below. Lessons Learned * Sustainability training is ineffective * Application of DemNet training is weak * NGOs need to produce impacts * NGOs need to communicate impacts to improve image * NGO evaluation is inadequate * NGO Performance milestones are effective * NGO communication skills are underdeveloped * Media help amplify affirmative NGO visibility * Site visits verify Grantee qualifications J Coalitions enhance NGO success potential * Collaboration is possible and works The recommendations are presented in terms of individual NGO development and NGO sector development. They are summarized below. Evalh'on offhe Macedonia DemNef Program Task Order 805 3 February, 2002 Development Associafes, Inc. AEP-I-00-00-00023-00 Recommendations Individual NGO * Strengthen Collaboration * Focus on Impact * Require Evaluation of Ultimate Program Impacts + Improve Application Skill * Expand Communication Skill * Increase Access to Information Technology * Build Financial Management Skill NGO Sector * Build NGO Platform * Increase Sustainability Potential Synergy Potential * Integrated Intervention Pilot Concluding Comment NGOs should advance to demonstrating the ultimate impacts of their program activities on societal problems. Synergy Potential DemNet, the Local Government Reform Project (LGRP), the Community Self Help Initiative (CSHI) and the PRISMA project are all working in communities, seeking to improve conditions in their particular sectors of activity, such as NGO strengthening, small infrastructure development and quality of life improvement, local government capacity, economic development and so forth. Together, these projects address a wide range of important community development issues in Macedonia and, together, possess a very promising synergy for systematic change in these communities. The report concludes by reinforcing the point that the DemNet NGOs have been prepared to achieve significant impacts, and that the time is propitious for them to reach that goal. The LEAP and CAP projects, for example, need to demonstrate that their partnering arrangements have actually produced their intended, ultimate results. Convincing proof and then communication of these results will, in due, course establish the Macedonia NGO sector as a vital force for essential social change. Concluding Comment Some DemNet NGOs will need more assistance, as noted in this report. Most of the NGOs, however, are prepared to deal effectively with important societal problems, and to produce and document ultimate program impacts. They should begin to focus on achieving this purpose. Evaluafion ofthe Macedonia 4 February, 2002 DemNet Program Task Order 805 Non-governmental organizations (~~0s)'can play an important role in the development of civil society. When they work cooperatively with government institutions and the private sector, they provide a channel for citizen participation in the public policy process. They mobilize citizen action to promote the provision of needed services to citizens and to provide a forum for accountability. The Institute for Sustainable Communities (ISC) began implementing the Democracy Network (DemNet) Program in the Former Yugoslav Republic of Macedonia in April 1995, less than four years following Macedonia's independence, under a cooperative agreement with the United States Agency for International Development (USAID). The program was part of USAID's strategic objective to promote increased, better-informed citizen participation in political and economic decision-making. DemNet contributed to the strategic objective, by strengthening the non-governmental organization (NGO) sector, through training, technical assistance and project funding. DemNet has been funded at $5.2 million, through Phases 1-111, from April 1995 to December 2002. Phase I11 began in March 2000 and twelve months remained at the time of this evaluation, which is the first evaluation of the program The evaluation covers the three phases, but pays particular attention to Phases 11-111, since they embody the main strategy of the program today. The evaluation sought to learn from experience, in order to plan for the future. A two-person evaluation team collected information from DemNet and other NGO stakeholders, during a December 2001 three-week visit to Macedonia. Data collected in Washington, D.C. and in Montpelier, Vermont, ISC's home office, augmented this information. 11. PURPOSE OF THE EVALUATION USAIDISkopje (Mission) views this evaluation as the first phase of a two-phase planning process that will guide the future direction and methodology of its NGO development program This report was prepared as an internal Mission planning document to inform the process. At the outset of its work in Macedonia, the evaluation team ("team") met with the Mission to discuss the purpose of the evaluation. The Mission expressed satisfaction with ISC's DemNet implementation and informed the team that it was uninterested in a management review of ISC's performance. Instead, the evaluation was to focus on the DemNet process and on how it fostered 1 NGOs include a variety of community-based organizations, including professional associations and other similar groups, registered as nonprofit organizations, under the 1998 Macedonia Law on Citizen Associations and Foundations, or other applicable Macedonia laws. Evaluafion of the Macedonia 5 Februnry 2002 DemNet Program Task Order 80.5 Development Associates, Inc. AEP-I-00-00-00023-00 the development of individual NGOs and the sector as a whole, and on how this could be improved. Following its review of the draft evaluation report, the Mission further clarified the evaluation purpose. It identified two goals: 1) To assess the status of the Macedonia NGO sector and the needs of the sector, based on interviews with NGOs that participated in DemNet; and 2) to assess DemNet's contribution to the development of NGOs that participated in the program and specifically to their progress towards sustainability. The report includes identification of the lessons learned about DemNet implementation and the mediating factors that might have affected implementation. The Mission directed the team to review the three phases of DemNet, but to emphasize Phases I1 - 111, since they contained the key elements (e.g., CAP/LEAP Models) of DemNet's current NGO development strategy. The team collected information from a variety of sources. ISC's Vermont office and its MacedonidSkopje office provided extensive program documentation. The Skopje office prepared a very helpful map of DemNet Phase 111 implementation in Macedonia that identifies DemNet sites and their project activities. The team visited DemNet NGO sites to interview individually, and in focus groups, DemNet NGO staff and volunteers in order to gain an understanding of program implementation, mediating factors, lessons learned and recommendations for future NGO development. The 39 sites visited are a subset (19.7%) of the approximately 197 NGOs that have been funded by DemNet, Phases 1-111. It should be noted that there are approximately 4000 formally registered Macedonia NGOs. The main DemNet components (e.g., CAPLEAP models, NGO strengthening) were covered in the visited sites. Appendix A contains the data collection protocols used in the evaluation. The team also attended a two-day ISC-led network conference of DemNet partner grantees in Ohrid, Macedonia in order to learn about their program implementation experience and results. It conducted focus groups with NGOs that would be missed in the site visits because of logistical constraints. The NGOs and other stakeholders contacted are listed in Appendix B, which also includes the team's work schedule and the sites visited. A number of people in Skopje very familiar with the NGO development in Macedonia were also interviewed. They provided a broader perspective on the NGO sector status than the DemNet grantees, most of whom tended to answer sector-wide questions in terms of their particular projects. After consultation with USAID/Skopje, the team personally contacted people and organizations, including USAID projects such as the Community Self Help Initiative (CSHI) and Local Government Reform Project (LGRP). Evaluation of the Macedonia DemNet Program Task Order 805 DevelopmenfAssoci&s, Inc. AEP-I-00-00-00023-00 The team leader met with the Mission and with the ISC Chief of Party, midway through the field work, to discuss interim findings and review the evaluation's progress. We also discussed a report outline that presented the format and topical coverage of the evaluation. IV. DEMNET IMPLEMENTATION A. INTRODUCTION The DernNet program in Macedonia has evolved over three phases of activities. It began Phase 1 as an expansive outreach to a variety of NGOs operating in the environment sector. It learned from the Phase I experience and adjusted the program strategy for Phases 11-111 to intensify ISC's involvement with fewer NGOs and to add specific program components in order to elevate the potential for producing more significant and sustainable results. 1. Phase 1 The program began in Phase I as a broadly inclusive grant program seeking to reach as many NGOs as possible, in order to raise awareness of the NGO sector's role and responsibilities in a civil society, and in order to build the basic organizing and management skills of individual NGOs. From April 1995 to March 1998, ISC provided one-on-one technical assistance to NGOs and conducted 27 workshops that brought together approximately 750 representatives from 150 NGOs. To create an indigenous NGO training capacity, ISC established a 20-member local training team ISC also provided 70 project grants, totaling nearly $300,000, to 66 NGO grantees. Phase I resources were $1.9 million. ISC's partners in this phase were the Freedom House and the Regional Environmental Center (REC). Phase I provided several important lesson^:^ Sustained training and technical assistance for larger numbers of staff from individual NGOs is an effective way to bring about lasting change within these organizations. Intensive, one-to-one technical assistance, coupled with training, is an effective way to build organizational capacity and to bring about sustainable improvements in NGO governance and management. Small project grants to NGOs, coupled with an intensive program of monitoring and technical assistance, is an effective way to build organizational capacity and, at the same time, to make improvements in community life. For this discussion, this report drew upon the ISC DemNet Phase 11, I11 proposals to USAID. International donor suppart to NGOs is credited by some with the unprecedented growth of the sector during the 1995-2000 interval. Swnmary Report Phase 11, ISC, April 1998 - March 2000 Evalrurfr'on of the Macedonia 8 February, 2002 DemNet Program Task Order 805 Development Assochtes, Inc. AEP-I-00-00-00023-00 The more technical assistance is provided for grant management and for project implementation, the better are the results. 2. Phase II These lessons prompted a Phase Il strategy shift. ISC and USAID recognized that enhanced support, over a longer period, would be required if the sector was to become an important and sustaining contributor to Macedonia's civil society development. Macedonian NGOs were still insular, weakly organized and isolated from one another. They also lacked an awareness of the necessity of uniting with the government and with the private sector to solve community problems. In cooperation with USAID, ISC designed DemNet Phase I1 to provide more sustained training and technical assistance to fewer NGOs, and to involve more of their members, in order to affect more meaningful and sustainable change in the NGO sector. More specifically, ISC adopted a two-component strategy to: a) Assist a specific, limited number of NGOs in an NGO Strengthening Component; and b) create a Community Partnership Component. Phase I1 entailed a two-year extension (April 1998-March 2000) with modifications intended to broaden and deepen the program impact. Phase I1 was funded for $1 million. For the NGO Strengthening Component, ISC continued its basic (Phase I) approach towards NGO strengthening and concentrated on increasing the organizational capacity, project implementation effectiveness and sustainability of 23 NGOs, in order to form a networking resource for the NGO sector. By working intensively with 23 NGOs, over a two-year period, ISC hoped to strengthen this diverse group to the point where these NGOs could unite and begin providing leadership, advice and assistance to the wider NGO sector. The emphasis for the strengthening component was to build on successful Phase I experiences. Many of the supported NGOs carried out public participation projects that addressed the economic, environmental, or social needs of communities. Some of these projects encouraged various ethnic groups to work together to plan and implement project activities. ISC also encouraged projects that supported the needs of Macedonian women, in seeking to strengthen their role in public policy decision￾making. The most noteworthy aspect of the Phase I1 strategy shift was the launch of the Community Partnership Component. A more formal structure distinguishes the Community Partnership Component from the NGO Strengthening Component. Figure 1 lists the main elements of this component, hereafter referred to as the LEAP model. While ISC strongly encouraged NGO partnering with government and business interests to solve community problems, as part of the NGO Strengthening Component, the LEAP model requires a formal, municipally-adopted agreement among these parties as a precondition for future project implementation grant support. These agreements were developed through NGO, government and business collaboration, and were guided by widespread public input. Six LEAP partners were funded to develop plans; five plans were completed in Phase 11. Achievement of these partnership agreements was a first for Macedonia. Evaldon of the Macedonin 9 February, 2002 DemNel Program Task Order 805 Development Associales, inc. AEP-1-00-00-00023-00 As part of its strategy to create an NGO networking resource, ISC made the commitment to all selected NGOs to provide them with technical assistance and with training throughout the remaining 20 months of DemNet, and to provide funding for projects, based on designs that ISC would review and approve. ISC encouraged the organizations selected for the NGO Strengthening Component to design projects that would meet important needs in their communities, provide opportunities for more citizens to participate in public policy, and be completed by the end of 1999. ISC selected the organizations based, in part, on assessments of their capabilities to meet these criteria, instead of on the basis of the design of a specific project. FIGURE 1' LEAP Model 1. Proved intensive training and TA in LEAP methodology to prospective communities 2. Establish LEAP coordinating committee representing major community stakeholders and implement process with widespread stakeholder inpit at all stages 3. Develop draft commumty vision/mission statement 4. Obtain community input for preparation of final statement 5. Develop preliminary list of environmental problems 6. Conduct community survey to gain commumty input on preliminary list of environmental problems 7. Use community survey findings to set priorities for environmental problems 8. Select a number of top priority problems for action 9. Develop draft action strategy for the problems selected 10. Obtain public input on the strategy 11. Prepare final action strategy 12. Submit strategy to Municipal Council for formal approval. 13. Obtain formal Municipal approval of the strategy 14. Implement action strategy and evaluate its results. For the Community Partnership Component, ISC provided an overview of the LEAP process and what it could achieve for the lead NGOs; ISC also asked both local governments and local businesses for witten commitments to participate. ISC made it clear to alI selected NGOs that it was making a commitment to support the NGOs throughout the period of DemNet, as long as they were conducting projects that met with the general criteria of the program ISC emphasized that it was funding NGO organizations and their communities, instead of individual projects, (e.g., solid waste collection, delinquency prevention). In other words, it was interested in helping NGOs develop to the point where they could be effective agents for positive Information for Table 1 came kom the author's field interview notes and the LEAP material contained in Appendix C. Evaludon of the Macedonia DemNet Program Task Order 805 10 February, 2002 Development Associates, Inc. AEP-I-00-00-00023-00 community change. The projects were important, in that they should address important community needs, but the raison dFtre for the funding was to promote NGO development. 3. Phase 111 DemNet Phase 111 was funded for $3 million and built on the Phase I1 experience, in the pursuit of four main objectives: Expand and extend the existing NGO Strengthening and Community Partnership components; Engage new NGOs and communities in the NGO Strengthening and Community Partnership components; Assist new initiatives to support NGOs and public participation, especially those organizations that are important in USAID's private sector, and to its local government strategic objectives, as well as coalitions that arise, in response to key policy issues facing the country; and Assess the capacity of Macedonian Support Organizations (SOs) to strengthen the NGO sector and to provide seed grants to SOs showing promise of fulfilling this role. The fist two objectives aimed at building on, and expanding, Phase I1 project activities in NGO Strengthening and in the Community Partnerships. NGO Strengthening increased support for the development of the sector, as a vital player in public policy decision-making. Fifteen DemNet I1 NGOs were funded to extend their projects, expand their partnerships with other institutional actors, and provide support to other NGOs. Twenty-eight new NGOs were funded and received training and technical assistance. ISC expanded the Community Partnership Component environmental focus, (i.e., LEAP), to include a broader sustainable community action planning (CAP) process, that could also address economic and social concerns. ISC adapted the LEAP model to other types of community problems. The five CAP grantees were supported for both a planning phase, to produce an action plan, and an implementation phase to produce at least one tangible solution to a local problem. Planning and implementation grants, totaling $25,000, were awarded to each community. The CAP grantees will need the remainder of the current DemNet no-cost extension to complete project implementation, and therefore, results data on the tangible solution are unavai~able.~ The third objective, assistance for new initiatives, provided a flexible mechanism for responding to fresh opportunities that emerged during the course of the project, particularly those arising from other USAID-funded initiatives, or new policy issues. DemNet awarded 21 NGO grants, totaling $160,000, for voter education, voter mobilization and election monitoring activities, NO-Cost Extension and Program Adjustment of the Democracy Network Program in Macedonia, under Cooperative Agreement Number DHR-A-00-95-00031-00. 23 October 2001. Evaluahon of the Macedonia 11 February, 2002 DemNet Program Task Order 805 Development Associates, Inc. AEP-1-00-00-00023-00 bii through the local elections Grants Program An additional 71 NGO grants went to organizations implementing citizen participation projects. ~ri The fourth objective called for conducting an assessment of the efficacy of Macedonian SOs, as a basis for increasing their ability to help NGOs expand their capacities and to reach w sustainability. The project conducted organizational assessments of 47 of the 54 DemNet NGO partners, and produced organizational development plans in consultation with them kd This process of DemNet evolution from Phase I to Phase I11 is summarized in Figure 2. FemNet I ;rant Rounds funding many 'raining -provided project evelopment training to any xerested NGOs and then rovided organizational trengthening training to the JGOs who were given ,rants hding environmental JGOs for good project -leas . FIGURE 2 DemNet Program ~volution~ DemNet I1 .I Working - with a select -. rrouD of NGOs from rhe bemning - of the oromam . - 4 found that, in order to work on capacity building of NGOs, working through grant rounds was not the best way to do it, but that it would be preferable to focus on one group, for the length of the program Working with 2 components -LEAPS and NGO Strengthening (working with the select group of NGOs through the life of the program) 4 Added the LEAP component based on the need, (as seen through the LEAPs developed during DemNet I), to provide a more structured model for community participation and preparation of LEAPs 4 Work with NGOs of all sectors based on the growing need, (and capacity), in all sectors, not just the environmental sector Added Networking Meetings k 6 training seminars (3 days each) for all NGOs together 4 more structured approach 4 all NGOs training together 4 both project related and organizational strengthening training provided * bd ' Material for this table was provided by ISCfSkopje staff, following discussions between the staff and the Report kd author. 4 Based on the need for NGOs to have opportunities to get together to discuss issues that affect all NGOs, and to foster cooperation between NGOs Evaludon of the Macedonia 12 February, 2002 u DemNet Program Task Order 805 .CI, Development Associales, Inc. AEP-I-00-00-00023-00 2eneral training for all VGOs Selection Process - selecting VGOs for the NGO Strengthening program and hen working with them on improving their projects Lndividual technical assistance in project design improvement for NGOs Not much attention to -oalitions, except to have some of our NGO partners work together DemNet nI Continue to work with select organizations from DemNet U, based on an application process + Requiring NGOs that are continuing for a 20d phase to take the next steps in their growth.. . 4 NGO Continuation groups -requiring partnership proposals -to foster better cooperation between NGOs, local govenunent, businesses, etc 4 LEAP - to allow the plans to be implemented to show models that go through each stage, including implementation Addition of CAPS + based on the desire to broaden the focus of community action planning, not just the environment, but encompassing all aspects of the community (social, economic, etc.) - . + Some general training, but not requiring all NGOs to attend; instead, focusing on the NGOS that demotktrate a need for the general training 4 Addition of the Organizational Assessment, to allow the NGOs to participate in the determination of their training needs 4 Providing project-related training and organizational strengthening training 4 Providing more focused training, based on the organizational assessment 4 Providing funds for NGOs to obtain training or consultants, in areas related to organizational strengthening, (areas where ISC cannot provide training) Selecting groups for the program - having a probationary period during which they work on improving their projects. Final admittance to the program occurred afterthe projects were approved + Based on the lesson learned that some NGOs, that were already in the program during DernNet U, did not put much effort into improving their project proposal. .. whereas, if it were made part of the process, they would put more effon into it Provided a project design workshop, during the probationary period. and technical assistance in improving their project proposals + Provided more focused training in design, using a project idea that was already in development, (giving them areal project that they wished to implement, for them to work on) + Provided the opportunity, during the project design stage, for NGOs working on similar projects to get together and to discuss their projects -leading to a decision by some of them to work together + Focus attention on facilitating coalitions when they start to appear + Provide oppottunities for NGOs to get together, either at pre-planned meetings, or at their request, to discuss areas of mutual cooperation + Provide funds for coalitions to conduct activities together Evaluotr'on of the Macedonia 13 February, 2002 DemNet Program Task Order 805 Development Associates, Inc. AEP-I-00-00-00023-00 'uu IIemNet 11 No oppormnlties to fund good-projects if they weren't from one of our partner organizations Mentoring - reauiring - NGOs toprovide mentoring ,as a condition of the program, after they joined the program Separate training - and teihnical assistance teams working with the partners UemNet U1 + The Grants proyam was lnit~ated to provide project funding for good project ideas and to "spread the wealth" -to prohde fund& ~O~NGOS that had good ideas, but who were not ISC partners + In the selection process, (and the applications), focusing on NGOs that, along with the other criteria, were active in mentoring, or providing assistance to other NGOs + A more integrated team approach to working with the NGOs -with each NGO having a designated team, (consisting of a training person, technical assistance person and finance person from staff), working with them. + Providing a more coordinatedl comprehensiveapproach to working with NGOs The table illustrates DemNet's learning process, whereby project staff reflected on the results from certain practices, such as the mentoring activity, and made adjustments in project implementation. The team observed this on-going learning process at the network conference in Ohrid. NGO Staff discussed the lessons learned from their project experience and, in some instances, how they applied these lessons. IV. DEMNET RESULTS A. INTRODUCTION This section presents the DemNet results, in terms of individual NGO development and the NGO sector. The results discussed come from interviews with DemNet partners, during site visits and discussion, at the DemNet Network Conference in December 2001. ISC identified five "expected results" in its 1999 and 2001 proposals to USAID.' 1. Strengthened NGO Leadership - the capacity and sustainability of 15 DemNet I1 NGOs to provide leadership and support to other NGOs in Macedonia will be strengthened through continued training, technical assistance, and completion of their projects; 2. Concrete Models of Successful LEAPs - environmental improvement projects will be completed in six communities that prepared LEAPs duting the first or second phases of DemNet, consolidating models for community action planning, through the implementation phase; * DemNet 111 Proposal, July 22,1999: DemNet 111 No-Cost Extension, October, 2001 Evaldon of the Macedonia 14 February, 2002 DemNet Program Task Order 805 Development Associates, Inc. AEP-I-00-00-00023-00 Expanded Network of NGOs - 28 new NGOs will be strengthened through training, technical assistance, and project funding, and will join the current network of organizations that can provide leadership and support to other NGOs in Macedonia; Successful Models of Participatory Community Action Planning - five new communities will conclude participatory community action and implementation planning projects to address pressing economic, environmental, or social issues, and demonstrate the value of the process for community development; Cost-Qective NGO Support to Other USAID Projects - citizen groups and NGOs playing key roles in other USAID-funded projects in Macedonia, will be strengthened through training and technical assistance that ISC will provide, in coordination with other USAID project irnplementers. At appropriate points in the following discussion, the report will assess the degree to which DemNet has achieved each of these expected results. The discussion will also consider other issues, such as NGO mentoring, in the overall review of DemNet results. B. INDIVIDUAL NGO DEVELOPMENT DemNet provided funding, training and technical assistance, (TA), to relatively new, as well as to established organizations, registered as NGOs. The demand for DemNet grants exceeded the supply of project funds, and, therefore, project staff had to select the NGOs to be funded from the pool of applicants. DemNet staff acknowledged that their subjective judgment of the likelihood of NGO success played a large part in the selection process. Frankly, it is difficult to see how it could have been otherwise. These organizations were basically unknown entities, lacking any meaningful performance track record, prior to DemNet. Although they were legally registered, the registration process doesn't require objective performance data. More will be said about the selection process in the lessons-learned discussion. DemNet training is supply-driven. In order to receive a grant, NGO partners had to participate in the basic, full regimen of DemNet training modules. LEAP and CAP NGOs received additional training as well. The incentive to participate in the training is obvious. Even though it was mandatory, the NGOs visited were very grateful for the training. Many revealed in interviews that, before their DemNet experience, they were ill-prepared to begin operating as an NGO. DemNet NGOs affirm that ISC, with its resources, ideas, and training expertise, made their progress possible. The newer NGOs generally possessed promising ideas, that prompted their DemNet selection initially; however, they possessed only an inchoate understanding of the operational concept of an NGO. Some admitted candidly that the term "NGO was new to them, prior to DemNet. People interviewed credited DemNet with "getting them up and running." It taught them about NGOs and what they could accomplish if properly developed and operated. It Evaludon ofthe Macedonia 15 February, 2002 DemNet Program Task Order 805 Developmenf Assoc&s, Inc. AEP-I-00-00-00023-00 taught them how to organize an NGO, how to develop a management plan, how to develop a budget, how to promote widespread citizen participation in the NGO, and so forth. Even with this training, most of the sites visited readily admitted that they needed more training. For example, they expressed the need for more training in lobbying government agencies, in order to be more proactive in influencing legislation. Established NGOs reported a similar experience. Prior to their DemNet participation, most operated as loosely-organized groups of people concerned about particular issues, such as delinquency prevention. They acknowledged that they lacked experience in operating as an NGO, or even, for that matter, as an effective, well managed organization. They had existed as a small clique of like-minded people with little, if any, resources. Prior to DemNet, for example, most had no office space, no office equipment and no budget. They were invisible organizations. DemNet helped them to reorganize and focus. It taught them to set staffing requirements, promote citizen participation in their activities, and to adopt modern management practices that would enable them to more efficiently use their scarce resources. A good example of this is the Association of Deaf and People with Hearing Problems of Macedonia, an organization that originated in 1948, and registered as an NGO in 1998. NGO staff admitted that they misunderstood the concept of an NGO and didn't know how an NGO should be organized and operated. They were trying to promote a good cause, but felt that they were ineffective and needed help. They cited DemNet's practical training and technical assistance as very important to their chances for success. They understood that they could be effective, and gained knowledge of some of the methodologies that they could use. They also liked the continuing DemNet technical assistance, because it helped them refine their skills on the job. DemNet has also prepared NGOs to participate in the public policy decision-making process. The most successful, in this regard, are the LEAP and CAP projects, which require NGO, (six LEAP, five CAP), collaboration with local government agencies and with the business sector, in developing and implementing community improvement plans. This requirement is formalized in a municipally-approved plan that details the priority issues that will be addressed, the community improvement activities that will be implemented, and the methodology that will be used. This requirement puts the NGOs at the decision-making table, planning and deciding on the communities' future. Likewise, DemNet partners have been active in other sectors, pushing for new legislation, such as in the juvenile justice sector. The Council for Prevention of Juvenile Delinquency in Kavadarci facilitated the development of the National Juvenile Code. Several DemNet NGOs have been trained and assisted to operate as advocates for issues of concern to a variety of groups. They learned, for the first time in the DemNet training, how to campaign effectively for their specific cause. According to Evalua6on of the Macedonia 16 February, 2002 DemNet Program Task Order 805 Development Associates, Inc. AEP-I-00-00-00023-00 interviews with NGOs, this increased capacity has opened up potential channels for citizen action on behalf of the deaf and hard of hearing, schizophrenic patients, child victims of domestic violence, and so forth. An example of an NGO success story is the Organization of Consumers of Macedonia, (OCM). ISC worked with the OCM to prepare a law on consumer protection. ISC provided advocacy training. OCM organized public meetings to explain the law, and to stimulate citizen support. In July 2000, Parliament passed the Law for Consumer Protection. Other DemNet "success stories" are presented in the Report Appendix E. It should be noted, in particular, that DemNet has supported NGOs' advancement of issues important to women. DemNet reports that 27 of the 53 Phase I11 projects specifically "target women" in their activities. Review of the small grants recommendations identifies women, or young girls, as the project "beneficiaries" in numerous projects. The Small Grants component asshed a variety of NGOs In Macedonia, (not currently supported by DernNet), for projects that addressed community issues, enhanced organizational capacity, promoted widespread citizen participation on public policy dialogue and i~roblem solving. and built conflict -. resolition ca&ity within NGOs. Twenty-one NGOs received grant support and technical assistance, under the Local Elections Small Grants Program -- an initiative aimed at engaging NGOs in voter education, voter held for NGO partners on "The Role of the NGOs in the Current Crises in Macedonia" DemNet NGOs are, at this point, in 'a good position to take the next substantive, developmental step. These NGOs have been provided with a basic foundation of skills and capacity, and must now learn how to apply this with more sophistication, creativity, and innovation. NGOs have received a great deal of DernNet training and technical assistance, but many seem unable to deftly apply this training. This suggests a gap between theory and practice. NGOs were trained in the interrelated practices of lobbying and advocacy. The interviews revealed, however, that most were very uncertain as to how to actually launch a lobbying effort. Evaluolion of the Macedonia 17 Febraary, 2002 DemNet Program Task Order 805 Development Associoles, Inc. AEP-I-00-00-00023-00 w Although some have launched successful advocacy projects, as noted, many others expressed little understanding of how to operate as effective advocates. One NGO stakeholder commented bw that the NGOs can prepare a good action plan, if they are coached and guided along the way, but that, if they are left on their own, the plan produced is generally weak and lacking a sound conceptual and operational strategy. %d Another example of such a gap is in the area of monitoring and evaluation. Sound evaluation, including data on the impacts of project activities on target problems -- such as environmental ip degradation or drug abuse -- is a sine quo non of modern management practice. Sound evaluation enables accountability and informs others of your success in actually reducing or removing a pro blem. W Monitoring and evaluation is part of the DemNet training; however, the site visits exposed a dearth of objective monitoring and evaluation, (i.e. evaluation either conducted or validated by non-project evaluators), by the DemNet partners visited. They all acknowledged the importance of good evaluation, but only a few had collected anv verifiable evaluation. in the form of results - impact data. The iformation collected and reported was limited to indicators of process results; that is, information or data on the implementation of DernNet activities, such as workshops completed, coalitions formed, brochures distributed, and so forth. Absent were verifiable data on the impacts of project activities. For example, did a coalition formed to push for passage of a law to eradicate employment discrimination barriers against mentally ill people, actually get the law passed and actually "knock down" these barriers? Formation of the coalition to get the law passed represents the process; eradication of the barriers is the impact. Reportedly, each project activity is evaluated at its conclusion. NGOs collect quantitative and qualitative data, as well as video data on project implementation and results. The also collect client evaluations of services received. They are funded, in part, by the World Health Organization (WHO), which regularly evaluates the NGO's activities that it supports. Evalh'on of the Macedonia 18 February, 2002 bi DemNet Program Task Order 805 Development Associates, Inc. AEP-I-00-00-00023-00 An example of an archetypal non-evaluating NGO is the Mountaineering NGO in Skopje. This NGO's staff admitted that they neglected doing objective evaluations of the NGO's activities and that they lacked the necessary money to do an evaluation. However, the interview with NGO staff also suggested that they had only a minimal understanding of how to develop and execute a credible monitoring and evaluation plan. A critical example concerns the issue of sustainability. Sustainability is the Achilles Heel of the DemNet project and the NGO sector in general. Almost all of the staff of the visited NGOs depends on DemNet funding, and on other foreign donor support, to a lesser degree, for their employment. It is safe to conclude, as do the NGO stakeholders interviewed, that if foreign donor support ended on February 1. 2002, most Macedonia NGOs, (including the DemNet NGOs), would, shortly thereafter, go out of business. DemNet NGOs have received training on sustainability. It was also an essential agenda item at the NGO partner network conference in Ohrid, which was attended by the team Yet it was obvious from the visits, that the NGOs needed much more assistance in developing and implementing, (sustainabiity), business plans for raising operating resources from a wider network of indigenous, as well as foreign sources. They needed to think and act more creatively. They acknowledged the importance of sustainability but declared that they did not know where to begin in the Macedonia context, with its weak economy, unstable political environment, and, (according to several interviewed NGO stakeholders), ineffectual NGO public personnel. The field visits uncovered some beginning efforts. NGOs taked about raising money by various "fee for service" schemes. However, further probing revealed that even these efforts were either, "under discussion" and/or would, in any event, yield a minimal percentage of the needed funds. None of these nascent initiatives was supported by an explicit business plan or strategy. The following text boxes summarize the "process results" reported by DemNet. These results are presented in terms of the main DemNet project components9. They are presented in this section for the individual NGOs. In the next section, they are presented for the NGO sector. 9 DemNet has organized its activity in terms of five components: NGO Strengthening; Community Partnerships; Small Grants Program; Outreach: and NGO Leadership. See: DemNet 111 Proposal, July 22, 1999; DemNet 111 No￾Cost Extension, October, 2001 Evaluation of the Macedonia 19 February, 2002 DemNet Program Task Order 805 Development Associaks, Inc. UP-I-00-00-00023-00 INDIVIDUAL DEMNET PROGRAM COMPONENTS" NGO Strengthening Process Results Many NGOs, that began with only vague ideas of how to organize and operate a NGO, gained, through the DemNet experience, basic organizing skills and an understanding of the role NGOs could play in civil society development NGOs initiating the process of seeking citizen input into the community decision making process; helping citizens to see how they can participate in the process Training workshop on coalition building NGO coalition building in national campaigns, (e.g., drug abuse prevention, value added tax, juvenile delinquency prevention) NGOs using DemNet skills to leverage support from other sources NGOs using advocacy skills to raise awareness and get issues on the public agenda (e.g., mental health, diabetes), for the first time in Macedonia Training of trainers builds corps of local trainers as resource for new NGO development NGOs working to change laws affecting vulnerable populations, (e.g., deaf and hard of hearing: mentally ill; diabetics) NGOs performing public education function, (e.g., consumer rights; environmental protection) Stimulate municipal initiation of private enterprise, (e.g., solid waste disposal) NGOs seen by some, as an alternative to government, towards solving community problems Strengthening 28 new NGOs in Phase I11 through uaining, TA, and project funding Community Partnerships Process Results Completed LEAP planning process in six communities; plan implementation underway Extended and broadened the basic LEAP model to five additional CAP communities dealing with a variety of priority problems Established effective community coalitions LEAP and CAP NGO participants express greater efficacy in influencing community decision-making DemNet worked closely with group of five local conract trainers to deliver training to LEAP communities. The LEAPICAP strategy is gaining visibility and acceptance as an effective methodology for community problem solving LEAP community participants are serving as a technical resource to other NGOs interested in the LEAP methodology for community organization and problem solving. 10 The information for these Process Results text boxes came from several sources: Team notes from ISC DemNet Network Conference, Ohrid, Macedonia, December 2-3, 2001; interviews with NGO staff during site visits, interviews with ISCISkopje s*, various ISC documents: Profile: the NGO Sector in Macedonia, I March 2001; DemNet Training Modules, electronic file; DemNet I, 11, 111 and No-Cost Extension Proposals, electronic files: Funded DemNet NGO Projects Summary, electronic file; Project Descriptions for DemNet NGO Parmers, electronic files; Model of the DemNet Process in Macedonia, electronic file; Smdl Grants Program, funding rounds, electronic files; NGO Development and Continuation Grants, electronic files; Networking Conference, Working Groups, November 2001, electronic file Evaluation of the Macedonia 20 February, 2002 DemNet Program Task Order 805 Development Associates, Inc. AEP-I-00-00-00023-00 Small Grants Program Process Results 71 organizations received grant support, under the Smll Grants Program - ISC's new initiative for assisting a broader range of NGOs in Macedonia, (not currently supported by DemNet): the projects involved address community issues, enhance organizational capacity, promote widespread citizen participation on public policy dialogue and problem solving, and build conflict resolution capacity within NGOs 21 NGOs received gr~ant support and technical assistance, under the Local Elections Small Grants Program- An initiative aimed at engaging NGOs in voter education, voter motivation, and election monitoring Training workshop for NW parmers on "The Role of the NGOs in the Current Crises in Macedonia" USAID parinas meeting to consider constructive approaches to the conflict, and to examine the state of democracy in Macedonia Trainiig of trainers on conflict resolution for members of the DemNet NGO strengthening staff and for six NGO parmer organizations C. NGO SECTOR DEVELOPMENT The Macedonia NGO sector was described by one stakeholder as a "series of islands, disconnected from each other." NGOs "do their own thing," rarely sharing with each other, or learning from each other. This has led to the widespread view that Macedonia NGOs lack a platform for action, a collective mission that they could pursue, through effective common problem-solving action. DernNet has attempted to improve the image of NGOs, through its training, TA and outreach. It has sought, for example, to prepare NGOs, through the LEAP and CAP projects, to play effective roles in their communities, through partnering with government and business. Other projects have formed coalitions to reach their goals. The effectiveness of an NGO dealing with waste disposal allegedly benefited from its participation in a coalition of four environment NGOs. These nascent coalitions also serve to break down the alleged isolation of individual NGOs, reinforcing the value of collective action and contributing to sector maturity. The site visits also produced a few instances of NGOs working together within a sector; and there were examples of older NGOs mentoring newer NGOs. The HOPS NGO in Skopje is an example of one NGO mentoring other, more recent NGOs, in the drug prevention sector. The Evaluation of the Macedonia 21 February, 2002 DemNet Progmrn Task Order 805 Development Associafzs, Inc. AEP-I-00-00-00023-00 Association for Citizens with Diabetes, (Prelep), also reports mentoring three NGOs to help them prepare proposals. All of these associations suggest the potential for more extensive alliances, leading to rising sector vitality. DemNet established a network of local trainers, (e.g., LEAPICAP training; NGO development uainingTA), who presum￾ably are available to assist new NGOs. It also produced and disseminated a set of brochures designed to provide a user￾friendly guide to forming an NGO. DemNet reports a fist run of 1000 brochures; demand for the brochures led to a second run of 1000 brochures. The project also sponsored the first-ever NGO Fair in November 2001 attracting over 140 NGOs; the Fair was addressed by a representative of the President's office. Despite these efforts, many of the NGOs visited lamented the competitiveness of the NGO sector. In Macedonia's feeble economy, with national unemployment reaching 45%, and local unemployment in the communities visited reaching 85%, competition for funding to pay NGO staff salaries outweighed incentives for cooperation. ~ncentives for NGO cooperation were rhetorical; incentives for competition were monetary. The team visited a tiny fraction of the 4000 Macedonia NGOs, so any conclusions from that experience would have to be very provisional. Fortunately, information from other NGO stakeholders, and findings from a recent survey of Macedonia NGOS," was available to augment the discussion. . On the issue of NGO competition vs. cooperation, the survey reports that 85% of the respondents said that relations among NGOs were cooperative; 47% cited national level cooperation; and 60% claimed "concrete collaboration" with other NGOs on the national level. Does this mean that the sector is cooperative? - - 11 The survey was conducted in four cities in Macedonia chosen to be ethnically mized. The cities included Skopje with 60 NGOs interviewed; Gostivar with 30 NGOs interviewed, Struga and Ohrid, together, with 30 NGOs interviewed. Nationwide 120 NGOs were interviewed. Due to the more actwe NGO scene in the capital, as well as the population size of Skopje, half of the NGOs came from there. At fust, the group included the city of Kumanovo, but due to the start of the conflict at the time, this site was replaced with Gostivar. Institute for Social Research, Skopje, Macedonia Spring 2001. Evaluabbn of the Macedonia 22 February, 2002 DemNet Program Task Order 805 Development Associates, Inc. AEP-I-00-00-00023-00 The survey also reported that, even though most NGOs saw themselves as cooperating with other NGOs, only about 24% reported success in actually forming working coalitions. This finding was somewhat puzzling, because the interviewed DemNet NGOs, and the surveyed NGOs, cited numerous benefits supposedly derived from coalitions. Perhaps they were merely expressing the conventional wisdom that coalitions were good. Another possibility, probably closer to the truth, was that most NGOs were ill-equipped to actually develop and maintain a working coalition. Another interesting survey finding was that nearly 60% of NGO respondents said that they were members of an NGO "umbrella" organization, and 62% perceived their membership to be "effective." Does this suggest that there is a unified Macedonia NGO sector? Unfortunately, the survey failed to describe exactly what these umbrella organizations did. For example, did they, in any way, promote or reinforce a unifying NGO identity or mission? The stakeholders interviewed, expressed a cynical view of the Macedonia NGO sector. They argued that there were far too many NGOs and far more than Macedonia was able to support. Some characterized NGOs as "shell organizations," initially set up solely to tap into the flood of donor money for NGOs, over the last decade. They viewed NGOs as little more than employment programs for NGO staff, most of whom were unable to make money doing anything else. One stakeholder, associated with a major donor in Macedonia, went so far as to state that Macedonia lacked an NGO sector. The so-called "sector", in his view, was disorganized and unfocused, and, therefore, lacked any semblance of the cohesion generally associated with a sector perspective. This negative image may truly reflect NGO impotence in Macedonia. Maybe NGOs simply have failed to produce results commensurate with their funding. However, there are other plausible explanations as well: A few NGOs have been very effective, but many have been ineffective: NGO achievements have been communicated unconvincingly to citizens; government agencies distrust, and therefore denigrate, NGOs because they see them as competitors for political power; donors might view NGO development as an unpromising growth sector; finally the immaturity of the sector - most of the growth has occurred since 1995 - may be a limiting factor. Regardless of the reason, NGOs lack a resolute, positive image, and this should be a legitimate motivation for ameliorative action. Those familiar with DemNet acknowledge that it has done a good job in developing and strengthening NGOs, so that they are potentially capable of effective action. These observers are quick to point out, however, that the project has been less successful in raising the overall capacity of the sector. The NGO sector has yet to demonstrate that it is a source of positive Evaluafion of !heMacedonia DemNe! Program Task Order 805 23 February, 2002 Developmenf Associates, Inc. AEP-Z-00-00-0002323-00 social change in Macedonia. This judgment is understandable, since DemNet affects directly less than one percent of the registered NGOs, even if one included its outreach and leadership initiatives. The issue for this report is, practically speaking, what can DemNet do to influence the future development of the NGO sector? The following text boxes summarize the process results for the NGO sector. Outreach Process Results Produced and disseminated a set of brochures to provide step-by-step guidance for NGO development for NGO start-ups and grassroots organizations NGO network conferences document lessons learned and results, for dissemination to other NGOs Training workshop on media relations and outreach campaigns NGO Leadership Process Results Established a network of local trainers, (e.g., LEAPICAP training; NGO development trainingTA), participating NGOs and Macedonian ISC staff Series of DemNet network conferences building capacity through information sharing Organizational assessments of 47 of 54 existing NGO partners; and developed organizational development plans, in consultation with partners DemNet has identified eight SOs that it will work with and support The text boxes for the individual NGOs and the NGO sector illustrate the range of process results achieved by DemNet during Phases 1-111. They underscore the point that DemNet has established a foundation for new and existing NGOs in Macedonia, that enables these NGOs to carry out activities that should in time produce important development impacts. D. MEDIATING FACTORS Several factors should be kept in mind when assessing DemNet's implementation and results. The Kosovar refugee crisis in 1999, which fueled inter-ethnic tensions, was followed closely by armed conflict in February 2001, between a force of Albanian rebels and the Macedonian ArmyIGovernment. These events created a political atmosphere inhospitable to normal DemNet operations. Several of the NGOs visited, reported that they suspended or canceled planned Evah&on of the Macedonia 24 Februnry, 2002 DemNet Program Task Order 805 Developmenf Associales, Znc. AEP-1-00-00-00023-00 activities, in order to respond to the crisis, (e.g., problems in refugee camps), or to avoid unsafe situations. DemNet also redirected some of its small grant funds to various humanitarian projects. The negative image of NGOs has had a major damaging impact on citizen and government support for NGOs. According to several stakeholders, the sheer proliferation of NGOs in the last decade, many of which lack any discernable record of achievement, has magnified the problem Overcoming this perception, in order to forge productive partnering arrangements among NGOs, government agencies and citizens, has been a major DemNet challenge. Most stakeholders interviewed stated that this image obstacle persists. Since most NGOs have few paid staff, they have to rely greatly on volunteers. The absence of a tradition of citizen volunteerism in Macedonia, coupled with the image defect, reportedly has made volunteer recruitment much more difficult. Widespread joblessness - surpassing 80% in many communities - has also limited volunteer recruitment, since unemployed people are more worried about feeding their families than donating their time to an NGO. Also, volunteer dropout or inactivity, (e.g., people get some temporary work and drop out), runs high in most NGOs -- increasing the demands on the minimal staff and scarce resources to recruit and train volunteers. Several stakeholders noted that the NGO image has inflamed the traditional distrust between NGOs and government officials, resulting in a fragile foundation for cooperation. The LEAP and CAP projects have been able to achieve municipally-approved plans. But beyond these 11 projects, stakeholders point to the dysfunctional NGO-government relationship as a primary obstacle to NGO sector development. A related obstacle, according to one stakeholder, is the erroneous perception, (attributed by him to DernNet), that NGOs should remain independent from the government. Since the government is so omnipresent, he contends that a much more practical, and potentially effective, strategy is to work with, rather than independent from, the government. The weak economy also has limited private sector involvement. Small businesses are struggling just to get by; business failure is rife. These businesses have little, if any, "surplus capital" that they can afford to donate. Moreover, the Macedonia tax code offers zero incentives, (e.g., tax write-offs), for such businesses to support NGOs, even if they could. Finally, most NGOs face a hostile fiscal and legal environment. The Law on Citizen Associations and Foundations, (adopted June 25, 1998), regulates the registration, operation and termination of NGOs in ~acedonia." The Law has created a reskicting fiscal environment. The NGO sector, for example, generates an annual turnover of $41 million -- approximately 5.5% of the state budget. Yet, half of the $41 million goes to the state budget, in the form of customs duties, VAT and a 23% personal income tax. The personal income tax is imposed on the travel and accommodation expenses of grantees and on the salaries of full-time NGO staff. This obligation exists for most NGOs, despite the fact that certain foreign and domestic foundations and citizens' associations continue to enjoy a tax exempt status. Also, NGOs complain that they l2 Milcin, Vladiiir, "Macedonia's Fiscal Barriers: Will They Stifle the Third Sector, Open Society, Skopje, Macedonia, no date. Evoluahbn ofthe Macedonia 25 February, 2002 DemNet Program Tosk Order SO5 Development Associaies, Inc. AEP-I-00-00-00023-00 must surmount a variety of legal hurdles, equivalent to a for-profit enterprise, just to set up a money-making venture that will enable them continue operating as a non-profit NGO. E. LESSONS LEARNED DemNet Phases 1-111 produced several useful lessons for NGO development, growth and sustainability. Some of these lessons refer to successful practices (e.g., NGO selection) that should be continued. Other lessons pertain to shortcomings (e.g., sustainability training) that need to be corrected. The prescribed adjustments will be discussed in the next section of the report, under recommendations. NGO sustainability is critical. Unsustainable NGOs ultimately waste the money invested in their development. Also, they obviously contribute nothing to sector development. DemNet training recognizes the importance of sustainability, and includes it as an important part of its training regimen. As noted earlier in the report, however, few if any of the visited NGOs have prepared themselves to be sustainable. Some even voiced the naive opinion that donor funding would continue indefinitely. A couple of them said that they existed before DemNet, and that they would exist after DemNet, by just reverting to their old ways of surviving on meager resources and volunteer labor. The lesson is that the DemNet training has inadequately readied NGOs to compose and implement a practical sustainability plan. Most NGOs seem to understand that they need to build a base of post-donor support or, eventually, they will pass away. The problem, according to them is that they are unprepared to do it by themselves. Even if they are aware of some potential sources of support, such as local government, they feel unsure as to how best to approach them They are quick to express their need for DemNet assistance. The sustainability example underscores the fact that many NGOs are unskilled at applying their DemNet training. They readily admitted this during the site visits, saying that they needed more training and/or assistance in putting into action some of their DemNet training. They felt unprepared to go it alone. A prominent example of this deficiency is the absence of credible NGO monitoring and evaluation, both of which are indispensable for proper project management and accountability. Evaluation ofthe Macedonia 26 February, 2002 DemNet Program Task Order 805 Development Associates, Inc. AEP-I-00-00-00023-00 These examples suggest another lesson: perhaps DemNet trainingjassistance has been too hands￾on. NGOs might actually need more selfpractice, in which they are held accountable, in completing, totally on their own, specific assignments, such as collecting and reporting evaluation data. DemNet staff, as well as other stakeholders, contend that many NGOs still need a lot of hand￾holding. This is understandable to some degree, given the immaturity of the sector. But, those NGOs who have been exposed to the DemNet training and technical assistance should be able to strike out on their own, applying the training they have received with innovative action and adapting to new situations. It is obvious that DemNet NGOs have good intentions, proven by their sincerity and efforts to alleviate important social problems, such as drug abuse or environmental degradation. It is also evident that DernNet has done a good job in helping them to get organized and training them to run an NGO. Therefore, the absence of evaluation is puzzling. Although evaluation is a prominent element of DemNet's Organizational Development training module, the site visits revealed that the training ran short of establishing application skills, as easily evidenced in the absence of evaluations. At some point, NGOs have to demonstrate that they are capable of producing tangible and significant results. Moreover, they have to communicate these project results to key NGO stakeholders, if they are ever going to silence their critics, and prove that they can produce results that justify the investments in them The lesson indicates that NGOs are going to need more DernNet help to accomplish this. Another key lesson, reported by DemNet staff, relates to the selection of NGOs. Staff reported that, as they learned in Phase I, NGOs sometimes pumped up their experience and capacity, in their quest for funding. NGO selection therefore needed more than just a written proposal, as the basis for awarding grants. In Phase 11, the DemNet staff instituted the use of site visits to verify the NGO qualifications and capacity, alleged in written applications. These visits enabled in-person organizational assessments that focused on staff experience and knowledge, organizational development and capacity, and so forth, in order to weigh the NGO's potential for success. These site visits proved to be essential to getting behind the laudatory rhetoric found in written proposals. The staff, conducting the site visits was experienced in the operation of NGOs and, therefore, was able to focus on key indicators of organizational capacity, such as a coherent organizational plan, a reasoned statement of organizational purpose, and the NGO's ability to defend its proposal.. DemNet staff also learned that setting firm deadlines promotes timely NGO task completion. This lesson came from the LEAP experience. Some of the LEAP NGOs lagged behind in completing their action plans, putting unnecessary pressure on the project completion schedule. This delay reportedly was remedied in the CAP projects. DemNet staff held NGOs to explicit deadlines for action plan completion, using follow-on funding as the incentive, and it worked. Evaiuofion ofthe Macedonia DemNet Program Task Order 805 27 February, 2002 Development Associnfes, Inc. AEP-I-00-00-00023-00 The LEAP delays were avoided. This type of incentive, tying funding to specific performance targets, should be assessed for inclusion in all DemNet project supervision. Many NGOs are ill-equipped to communicate their activities and achievements effectively to external audiences. The scarcity of results data contributes to the problem, making it difficult to talk forcefully about achievements. Also, several NGOs said that they needed more help in just telling their story. The field interviews revealed this disadvantage to some degree; some of the NGOs interviewed were markedly unpersuasive when describing their project. This lesson suggests that DernNet training needs to fine-tune the Public and Media Relations training module, and to complete remedial training in order to improve the communication skills of NGO spokespeople. Conversely, several NGOs credited their ability to work effectively with local media as important. They utilized the media to tell their story and to boost their visibility and standing. One environment NGO recounted that positive media exposure enabled the NGO to approach the local government on firmer footing, lessening the amount of required justification for the NGO's proposal. These NGOs also utilized the media to petition mayors to contract with private companies to dispose of community solid waste. Other NGOs would benefit from technical assistance to elevate their media savvy, and, hopefully, to realize similar gains in getting a positive portrayal of their work and accomplishments before influential stakeholders. Another lesson is the value of coalition building. Coalitions helped a few NGOs leverage their meager resources, through unified efforts with other NGOs, and thereby achieved a wider impact. A good example is the LEAP project in Sveti Nikole, led by the Environmental Association, Izgrev. Several coalitions in Sveti Nikole purportedly enhanced the reputation of local NGOs. DemNet naining, however, left some NGOs unequipped to form coalitions, which meant that DemNet needed to provide remedial training. One disquieting lesson is that the DemNet training apparently was unsuccessful in allaying the chronic distrust of government held by many NGOs.. Conversations with NGOs during site visits revealed a knee-jerk negativity towards working with local governments, which were perceived as either corrupt or incompetent, or both. NGO's were markedly unenthusiastic about attempting to work beyond this mind-set. The LEAP and CAP proiects have de - - local governments in attacking community problems. The issue for the other NGOs is the degree to which they can rise above their inbuilt aversion, and build similar partnerships. They will need intensive technical assistance, and some major attitude adjustment. DemNet has stressed the need Evaluation of the Macedonia 28 February, 2002 DemNet Program Task Order 805 Development Associofes, Inc. AEP-I-00-00-00023-00 for partnering in past training. It needs to reassess its training and TA, in this regard, and apply methods that will alleviate the suspicions of the more recalcitrant NGOs. DemNet training has also emphasized the importance of creating local business support. This support has been encouraged in DemNet training, both to achieve immediate community goals, such as the LEAPfCAP methodology, and to lay the groundwork for tapping indigenous sources of support. Unfortunately, the DemNet training was unsuccessful in teaching many NGOs how to build constructive linkages to the local business community. Their detachment was strongly suggested by the absence of any local business leaders in the team's site visits. Apart from the LEAP and CAP projects, few, if any NGOs have done much to initiate alliances with local business interests. Several claimed that they were unprepared to build mutually-beneficial relationships. The detachment from business stemmed from a lack of actual experience in working with local businesses. This lesson suggests the need to bolster training in putting together coalitions with local businesses. The training should consider creating an ongoing forum for NGO-Business dialogue, to explore ways in which they can join forces in attacking community problems, such as persistent unemployment. One of the questions raised by USAID/S~O~~~,'~ under the Lessons Learned heading, was, whicht have been the most successful DemNet NGOs? The answer, of course, depends on how you define "successful." If you defme success in terms of which types of NGOs have the greatest potential as community change agents, then the answer is the LEAP and CAP projects. They are unique among the DernNet NGOs in setting up the essential partnerships with the other key players - government and local business - in the community. They have set the stage for possible and lasting social change. Are the other types of DemNet NGOs as well prepared to make a real difference? The team is unable to comment with any confidence, because of the dearth of objective indicators or evaluative data. They have been funded and have organized themselves, but the NGOs have fallen short, so far, of producing hard evidence, (evaluation or otherwise), on exactly how, or how well, they have positioned themselves, (e.g., established requisite links with government and business), to accomplish their intended goals. If you defme success as the DemNet NGOs, (LEAP and CAP included), actually achieving their expected, ultimate goals or objectives, then it is too early to tell. Moreover, the answer will remain elusive, until NGOs generate the requisite evaluation information. Testimonials from DemNet staff and NGOs are unsatisfactory substitutes for compelling impact evaluation findings. If you extend the definition of success to the NGO sector, (i.e., which type of NGOs have been most successful in affecting the overall development of the sector), then the picture becomes l3 This refers to the USAID/Skopje comments on the draft DemNet Evaluation Report, submitted in December 2001. Evalunlion of the Macedonia DemNet Program Task Order 805 29 February, 2002 Development Associates, Inc. AEP-I-00-00-00023-00 even murkier. We have noted the miniscule fraction of the total NGO sector represented by the DernNet NGOs, and, therefore, it is unrealistic to expect much of an overall impact. We have also discussed the cynical assessment of various stakeholders toward the NGO sector, in general. The lesson is that, even though DemNet has published brochures, held network conferences, held an NGO fair, and so on -- all aimed at increasing NGO visibility and acceptance -- the impact of these disparate activities on sector progress is uncertain. The field visits to DemNet NGOs and the stakeholder interviews informed these recommendations. Some of them emanate from the lessons learned, some of which signal persistent defects in NGO operations that need to be corrected. In addition, some of the recommendations draw upon the author's experience in evaluating NGOs in a variety of countries. We will discuss the recommendations, first, in terms of individual NGO development, and then discuss NGO sector development. 1. INDIVIDUAL NGO IMPROVEMENT NGO Collaboration The Community Partnership projects, which are geared to community development, require a municipally-approved agreement among the lead NGOs, the business sector and local government. This requirement paves the way for partnering to achieve community development results. The other DemNet projects lack this essential obligation. This collaboration needs to be established, if sustainable community solutions are to be achieved. The point on partnering applies also to coalition building; several NGOs credit coalitions with helping them leverage their limited resources and, thereby, improve their likelihood of being successful in reaching their goals. The report recommends that the formal agreement requirements be incorporated into all NGO grants, designed to advance community development objectives. This would serve to lessen distrust and, in so doing, facilitate productive linkages among the key stakeholders. Further, the report recommends that all NGO projects explore opportunities for productive partnerships, as a means to expand their prospects for success in those situations involving advocacy for legal change. These arrangements would also open up sources of indigenous support that promote sustainability. DemNet should assess the principal obstacles to partnering, such as mutual distrust, or lack of knowledge, and initiate curative action. Additional training and technical assistance will be necessary, to help NGOs overcome their wariness of government and their disconnection from the business sector, to negotiate workable and effective partnering agreements. Likewise, many NGOs say that they want additional help in coalition building, to enable them to unite with other NGOs in order to optimize their potential for success. Evaluation of the Macedonia 30 February, 2002 *r DemNet Program Task Order 805 Development Associates, Inc. AEP-I-00-00-00023-00 NGO Focus To-date, the focus of the project and of individual NGOs has been on process and/or internal development. The actual application of these processes and skills to achieve measurable development results, (impacts), is either lacking or unreported. Regardless, the project has given the NGOs the foundation skills necessary to act and implement activities which lead to results. It is recommended that future efforts and assistance place the highest priorities on implementing activities which produce measurable development results (impacts). This will move the NGOs to the next level, and have a wide range of positive ramifications. NGO Evaluation Proven NGO success, confirmed by reliable and compelling evaluations, would be a powerful antidote to the charge of NGO inefficacy and unaccountability. Individual NGOs would reap the rewards of greater acceptance; the sector would reach a firmer foundation for constructive growth. It is therefore distressing that NGOs have been so remiss. The report recommends that DemNet act aggressively to remedy this deficiency. All NGOs, as a condition for initial funding, should be required to submit a written evaluation plan that identifies priority evaluation auestions. The data (urocess .. and impact) that will be collected on these questions, when and how the data will be collected, how the data will be analyzed to determine results, and how, and to whom, the results will be reported. As a pre-condition for 'follow-on funding, NGOs should be required to present convincing evaluation data, (including impact data), substantiating project efficacy. This means that NGOs will have to begin focusing on the ultimate, tangible impacts that the program will produce. They will have to demonstrate that they are effective in resolving important societal problems. NGO Application Skills The report discusses the breach between theory and practice afflicting many NGOs. The report recommends that DernNet trainers work intensively with NGOs, through refresher training and technical assistance, to verify that all of them are able to apply successfully the fundamental Evaluation ofthe Macedonia 31 February, 2002 DemNet Program Task Order 805 Development Associates, Inc. AEP-I-00-00-00023-00 concepts and principles covered in the training modules. The verification process should include requiring NGOs to prepare application examples, such as a coalition building strategy, that would certify their mastery of essential skills. NGO Communication It should. be expected that the NGOs will need assistance in communicating their accomplishments clearly, convincingly and widely. The field interviews were very informative, but it took a lot of careful probing to elicit useful information. The team was left with the impression that the NGO staffs were unskillful at explaining their program to outsiders. Moreover, many of the NGOs contacted said that they needed additional advocacy and lobbying skill training. The report recommends that DemNet strengthen NGO communication skills. This can be done through additional training and technical assistance, aimed at enhancing both the speaking and writing skills of grantees. Following refresher training, they should be required to develop, on their own, a compelling project advertisement. The commercial should include a succinct and attention-grabbing written and verbal announcement that communicates the project purpose, the expected results, the achieved results, and the social significance of these results. It should be anchored, to the extent possible, by, (either actual or to be collected), persuasive evaluation information. These advertisements should be broadly disseminated within the DemNet network, and to influential NGO stakeholders nationwide to promote positive NGO sector visibility and advancement. Information Technology DemNet has sought to enable access to information technology (IT) for its NGOs. Several of the visited NGOs, however, want more assistance in this area. They see greater internet access, for example, as expanding their communication capacity. They also want to establish relationships with international NGO networks and potential foreign donors. The report recommends that DernNet devote resources to assisting, through access to computer equipment and technical assistance, those NGOs that want to upgrade their IT capacity, so that they are able to access NGO information networks and initiate contacts with a wider circle of potential grant-funding organizations, both domestic and foreign. Their need to reach these types of contacts will intensify, as current funding levels off, or declines, and subsequent pressures to identify alternative sources of funds increases. Financial management Most NGOs have to operate with very little money, barely able to cover running costs. We discussed the exacerbation of the financial pressures, resulting from the unfriendly tax code in Macedonia. These pressures will only increase, as donor funding slows down. Some of the NGOs expressed the need for technical assistance on how optimize the efficient use of their increasingly limited resources. The site visits indicated that most of them were not proficient in modern cash management methods. tr Evoluohon of the Macedonia 32 February, 2002 *r DemNet Program Task Order 80.5 Developmen! Associates, Inc. AEP-I-00-00-00023-00 The DemNet training components do not include an explicit financial management component; the discussion of resources emphasizes fund raising. The TA possibly covers the topic, to some degree. Nonetheless, the report recommends augmentation of the DemNet training by inclusion of an explicit component on financial management in the face of declining resources, to help NGOs make the best use of their scarce resources. They need training applicable to a non-profit enterprise. The component should teach how to budget scarce resources; proper cash management techniques; how to invest, (or place), non-obligated funds, to realize at least some return; how to leverage their funds though joint ventures with other NGOs; and how to account for funds expended, in terms of results achieved. 2. NGO SECTOR DEVELOPMENT NGO Platfonn Many of the 1)emNet NGOs, as well as national NGO stakeholders, lament the absence of a Macedonia NGO platform that unites and focuses them, effectively communicates their accomplishments, and lobbies for important sector issues. The Macedonia Law on Associations and Foundations, for example, offers an excellent lobbying target, to make it more NGO￾friendly. The report recommends that DemNet establish communication links with existing Macedonia NGO "umbrella" organizations, (discussed in the above-mentioned NGO Survey), to assess their potential. DemNet should also intensify its efforts to build a foundation of Macedonia NGO expertise and publicize nationwide its availability for NGO development and growth. DemNet should also develop with NGOs and influential stakeholders, a rousing NGO Mission Statement that rallies and focuses the sector. The statement should be disseminated to all NGO stakeholders, including donors, government agencies, business interests, and so forth. Finally, every actual or potential DemNet grantee should prepare a brief written statement describing exactly how their project has, or will, contribute to that mission. These statements should be disseminated widely to other NGOs, donors, government agencies and to the private sector to promote sector identity. NGO Sustainability Unsustainable NGOs ultimately waste scarce resources and are unable to enhance sector vitality and growth. That is why the author considers NGO sustainability the number one DemNet priority. Unfortunately, virtually all of the DemNet NGOs visited rely overwhelmingly on foreign donor money. Stakeholders reported this dependence for the rest of the sector, as well. Most NGOs, nevertheless, lack even the semblance of a post-donor survival strategy. The report recommends that DemNet make a much more exhaustive effort to prepare its grantees. This means identifying the most important impediments to sustainability, such as government distrust and a weak NGO identity, and then preparing, with the NGO input, an action strategy, to deal with these barriers. The strategy should rank the barriers, by importance and potential intractability, and then devise feasible and forceful actions to surmount each one. Some actions might prove to be more potent than others. That is why the strategy should Evaluation ojfhe Macedonia 33 February, 2002 DemNet Program Task Order 805 Development Associahs, Inc. AEP-I-00-00-00023-00 routinely evaluate stratagems and communicate the results to grantees and to the NGO community nationwide. Grantees should be required to prepare a written sustainability plan explaining exactly what they will do and when, to ensure their survival. DemNet should then hold them accountable by periodically verifying that they have actually taken the actions pledged. DemNet should also use the internet, and other international contacts, such as trade associations, to identify effective remedies from other countries, (e.g., other DemNet programs), suitable for import to Macedonia, and then incorporate these prescriptions into its sustainability training and technical assistance. The final recommendation speaks to the potential benefits of more fully integrating the DemNet project into the rest of USAIDISkopje's effort to advance the positive development of Macedonia's local government. Synergy Opporfunity DemNet, the Local Government Reform Project (LGRP), the Community Self Help Initiative (CSHI) and the PIUSMA project are all working in communities, seeking to improve conditions in their particular sectors of activity, such as NGO strengthening, small infrastructure development and quality of life improvement, local government capacity, economic development and so forth. Together, these projects address a wide range of important community development issues in Macedonia and, together, possess a very promising synergy for systematic change in these communities. There have been isolated instances of actual or potential collaboration. CSHI, for example, has worked with the PRISMA project to fund workshops; DemNet and LGRP are exploring possible future joint activities. The important problems communities' confront, such as local economic development, are interdependent. Success in infrastructure development promotes local economic development. But an integrated, comprehensive application of all of these community development technologies within a community, to stimulate and sustain needed changes, remains to be attempted. The report recommends that USAIDISkopje explore the potential for such an effort in a set of pilot communities. The four projects would unite to design and implement a broad intervention, such as stimulating government action to boost small business development, through infrastructure improvements, informed by widespread citizen input. The pilot sites would serve as laboratories to test the overall efficacy of these heretofore independent projects, within a coherent, integrated community improvement intervention. A CONCLUDING THOUGHT People invest their time and energy in activities that possess substantial incentives. Political theory extends this line of reasoning to citizens' political participation. Citizens will use political channels - political parties, NGOs, government agencies, etc. - if they help them solve their important social problems. Proven effectiveness is a very potent incentive. Uncertain effectiveness is unsupportive. And, obvious ineffectiveness is a defiite disincentive. Evaluntion of the Macedonia 34 February, 2002 DemNet Program Task Order 805 Development Associates, Inc. AEP-I-00-00-00023-00 Macedonia NGOs will emerge as promising channels for citizen participation, when they supply the necessary incentives through solid performance. Citizens will use and support them as they begin to realize that NGOs will reward their investment of time and energy with solutions to their problems. *i Evaluation of the Macedonia 35 Februnry, 2002 DemNet Program Task Order 805 Appendix A Data Collection Protocols Appendix B Contacts Appendix C Program Data Appendix D References and Documents Appendix E Success Stories d Evalualion of the Macedonia February 2002 DemNet Program Task Order 805 ra 3C DemNet NGO Network Meeting Interview Guide 1. Background Information: NGO name, start-up date, date received first DemNet funding, location and sector (e.g., health, environment, education) focus for each discussion group member or individual interviewee. 2. How would you describe the Macedonia NGO sector (e.g., non-existent, weak, strong) when you first became involved in the DemNet project? 3. How would you describe the NGO sector now? 4. What do you think have been the most important results/successes of the DemNet project? Of your particular NGO since you started? Datal'vidence? 5. What do see as the key factor($ of the DemNet project (as you have participated in it) that have produced these results? Of your own particular NGO? 6. What were the most important obstacles/difficulties your NGO has had to deal with in trying to start and/or continue operating? 7. Have you learned some useful lessons about what worked well and not so well'? 8. Can you think of some other things that could/should have been done to make the DemNet project (or your NGO) even more effective? 9. Looking ahead 4 to 5 years, can you recommend any things (e.g., TA, more money, equipment) that you as an NGO need to become more effective? Any recommendations for the NGO sector as a whole? 10. What has been the most important source(s) of support (e.g., $, TA) for your NGO? 11. What is your NGO doing now to prepare for the eventual end of donor support? USAID Interview Guide for 11/27/01 Meeting 1. What does USAID see as the main purpose(s) of the DernNet evaluation? 2. Who are the main stakeholders for the evaluation? What are their priority information needs'? 3. How will the results of the evaluation be used? 4. What are the priority issues/questions that the evaluation should address? 5. What does USAID see as the main type(s) of data that can be collected for the evaluation? Process data? Impact data? 6. What are the key data sources? Who should be contacted? Relevant data bases'? 7. What should be the main "unit of analysis"? The grantee? The community? Other? Sampling strategy? 8. What should be the "coverage" of the evaluation? Number of grantees, number communities? 9. Does USAID see any barriers to conducting the evaluation? 10. What does USAID prefer for the reporting of evaluation results? Mid-mission reporting? End of mission reporting? Final report? Final (completed) report? Reporting schedule? Evoluafion of the Macedonia DemNet Program Task Order 805 January, ZOO2 Development Associates, Inc. AEP-1-00-00-00023-00 Skooie Interview Guide 1. What involvement (if any) have you or your organization had with USAID's DemNet project? (Collaboration? Co-funding? Amount? What activities'?) 2. What do you think have been the major changes in the NGO sector since 1995? 3. What is your assessment of the current "visibility" of NGOs and the NGO sector in Macedonia? Are NGOs viewed as effective'? Useful? Why or why not? 4. What do you see as main strengths (if any) of the NGO sector? Main Limitations? 5. What do you thii have been the most important lessons learned (What has worked or has not worked) about the NGO sector over the last 6 - 8 years? For example, what are (have been) the most difficult challenges facing NGOs and have they figured out effective ways to deal with them? 6. What do you recommend as essential for the future development of Macedonia's NGO sector? For example, what should donors do over the next 4-6 years to significantly strengthen NGOs? 7. Do you thii any NGOs are preparing now for the eventual end of donor support? Good examples of Who? How? If no, why not? DemNet Evaluation Field Interview Guide 1. Background Data: Date of interview, NGO Name, Location of Interview 2. When and how did you (your NGO) get started? 3. What has been your main source and types of support (financial, other support)? 4. What are your main activities? How many people (e.g., citizens) do you reach with your activities? Characteristics of the people you reach? 5. What have been the most difficult obstacles (starting up, operating) you have had to deal with? How did you deal with them? 6. What have you learned about starting up and operating your NGO? What has worked? What has not worked? 7. In what ways do you think your NGO has been most effective? What do you think you were effective? 8. Are there any areas in which you think your NGO could have been more effective? 9. How do you know if your NGO is operating effectively? Has any research been done on your NGOs effectiveness'? 10. What type of future assistance would be most useful to your NGO? 11. What Have you been doing anything to prepare for the eventual end of donor support? Evaluation of the Macedonia A-2 Janmry, 2002 DemNet Program Task Order 805 U.S. Contacts Institute for Sustainable Communities, Montpelier, Vermont BriIl, Melissa, Program Advisor* Clapp, Roger, Director of Community Action Felitti, Barbara, Senior Vice-president Hamilton, George, President Kolozova, Kristina, NGO Strengthening Program Coordinator* Kusiniiova, Niica, Operations Coordinator* Mihajlova, Elisaveta, Training and Communications Coordinator* Nebiu, Besim CAP Program Coordinator* Paul Nutti, Country Director* Smith, David A., Vice-President for CEE/Eurasia Stitely, Susan, Program Officer * DemNet Staff in the ISC MacedoniaJSkopje Office International Center for Not-for-Profit Law, Washington, DC Rutzen, Douglas B., Vice President Shea, Catherine M., Program Officer Macedonia Schedule of Interviews DemNet Country Program - Evaluation team Schedule of Meetings DESIGN TEAM ARRIVAL DEPARTURE Thomas Cook 26.11.2001 16.12.2001 Mihailo Popovski local expert local expert I Tuesday, 27.11.2001 I I I Place 1 Meeting with Phone Evalunbon of the Macedonia B-1 February 2002 DemNet Program Task Order 805 9:30 USAID office Skopje Kathy Sterner - USAID & Paul Nuti - ISC office 380-446 - USAID 114-855 - ISC Wednesday, 28.11.2001 DatefTime f Place 1 Meeting with 09:OO 1 1:OO 12:30-1:30 14:OO Phone Saturday, 1.12.2001 DAILSGP UNDP Office Lunch FOSIM 070 / 583-279 Thursday, 29.11.2001 18.00 I Friday, 30.11.2001 Larry Birch Vesna Bisheva Slavica Indzevska 11.30 Frosina Sunday, 2.12.2001 Networking Meeting LEAP, S. Niole 070 1 268 946 1126 335 ext. 107 444-488 ext. 105 Faculty of Philosophy 298-220; Narcden front 19 Swiss Agency for Dev. and Cooperation LEAP Debar I"" Evaluahon of the Macedonia B-2 February 2002 DemNet Propram Ilo Trajkovski ISC organized Vladimir Gilev Ohrid Mosa Pijade, S. Niole, 0921444-600 I I I Gazmend Cami I Monday, 3.122001, Tetovo & Gostivar 8 sptemvri bb, Debar, 0961833-835 Hasan Jashari Irina Traskovska Samet Slenderi 9.00 11.30 14.30 Murat Baftjari 165, Tetovo 044138 1- 196 1200 Tetovo, 0211 20-247 Zivko Brajkovski bb, Gostivar, 0421222-271 Association for education, democracy, multi-culture Tetovo Youth Information Centre - Tetovo Roma association -El Hilal - Gostivar Development Associates, Inc. AEP-I-00-00-00023-00 Date/Time I Race Wednesday, 5.12.2001, Bitola & Prilep I meeting with Tuesday, 4.12.2001, Skop,je Phone Branko Micevski Eleonora Stojanovic Marijana Ivanova Dijana Belevska 9.00- 10.00 10.30-1 1.30 11.30-13.00 13.00-14.00 14.30-16.00 9.30-10.30 Thursday, 6.12.2001, Skopje Biology Institute, Gazi Baba, 0211 17-055 Kapaan An, lokal3, 021130-038 Ilinden bb, Sobranie na Skopje, 021121-164 Ivan Cankar, 021335- 131,0701231-900 Bird Study & Protection of Society HOPS Lunch ERINA Association support of people with Psychosis 13.30-14.30 14.30-15.30 16.00- 17.00 1 1.00-12.00 Balkan Association Zaneta Hristova Skoevska 8, (CAP Bitola) 0471252-380 12.30-13.30 Prestige,Wornen NGO Masa Diic Ivo Lola Ribar 1 1, 047122 1-370 Biosphera - Bitola Lunch (in the car) Association of Citizens with Diabetes Centre for Civic Initiative, Prilep 9.00-10.00 Nesad Azernovski 11.00-12.00 Evaluation of the Macedonin B-3 February 2002 DemNet Program Task Order 805 Partizanska 9114, 047125 1-577 Milan Baseski 070 1 259-080 Goce Todorovski I I I Union of Deaf & Mute 12.00 - 13.30 13.30 - 14.30 15.00 - 16.00 Gorce Petrov 101, Prilep 0481418-800 Borka Talevski 228, 048125-125 I I I Doverba Nikola Ognenov Lunch Consumer Protection Macedonian Inter￾ethnic Association 1 1 Oktomvri 42a, 021228-106 Slavica Knezevic Marko Oreskovic 70, 0211 14-038 Marijana Loncar-Velkova Vesna Stefanovska Vodnjanska bb pf 150, 0211 13-265 Jane Sandanski 52-2-12, 021386- 120 Dataime f Place Meeting with Phone -Friday, 7.12.2001, Ohrid, Struga I I I 10.30-1 1.30 Natura, Struga Fens Mahmudi P. Brigadi 22, Stmga, 096178 1-043 12.00 - 13.00 Toleranca, Stmga Baskim Mislimi Kuzman Sapkarev 17, 0961780-097 13.00-14.00 Lunch (in the car) 14.00-15.00 Ekolab, Labunista, Aladin Demishkoski Labunista, 096179 1-859 LEAP Labunista 15.30-16.30 Bisera - Women NGO Nade Martinoska Dimitar Vlahov 52, 0961264469 Saturday, 8.12.2001, Kumanovo 10.00-11.00 SOS Telephone Sonja Arsovska Dragojlo Dindik, 03 1122-525.75-763 13.00-14.00 Kitka, Dracevo Dom na Mtura Dracevo Pero Stojcevski, Dracevo 021594-939 Monday, 10.12.2001, Delcevo, Pehcevo I 10.00 - 11.00 PHURT, Delcevo Nejat Demirovski NJ Vapcarov bb, 0331412-337,411-137 12.00 - 13.00 Agricultural Producers Nikolco Stojkovski, Pehcevo Boris Kidric bb, 0331441-726 13.00-14.00 Lunch 14.30 - 15.30 Kladenec, LEAP Jasminka Pasaliska Boris Kidric bb, 0331441-842 Tuesday, 11.12.2001, Skopje 9.00- 10.00 Mountaineer Union Goran Titovski Oktomvri 421, Skopje 021235-540 10.30 - 11.30 SRNA-Animal Protect Dejan Krstevski Sole Stojcev 51313, I 021363-543 Evaluation of the Macedonia B-4 February 2002 DemNet Prwram Task order805 p LA. Development Assoeiofes, Ine. AEP-I-00-00-00023-00 Datflime Place 11.30-12.30 Lunch 13.00-14.00 MARS Environ. NGO Wednesday, 12.12.2001, Kocani, Stip 9.30- 10.30 I Independent Initiatives 11.30-12.30 I Javor, envir. NGO 13.00-14.00 Beekeepers, Kocani ZtIIIII Thursday, 13.12.2001, Skopje 9.00-10.30 Children's Embassy 11.30-12.30 USAID meeting 12.30-13.30 14.00-15.00 Rubion 15.30-16.30 ESMA- Roma Women kiday,14.12.2001,Kavadarci, I Strumica 9.30-10.30 Council for prevention of juvenile 13.00-14.00 Planetum- envir. NGO Meeting with Jelena Dimitrievik Daniel Segmanovic Verica Puzderlieva, Stip Blagoj Danev Zoran Atanasov Gordana Zmijanac Kathy Stermer Vladimir Karaev Kefsera Mehmedova Lazar Nanev Mitko Sopov nindenska bb, p.f. 299, 021366-648 4 vodenicki, s. Mirkovci, P.O. Box 5, Stip, 092139 1-99 1 2305, Zrnovci, Kocani, 033153-642 Osogovska 32, Kocani, 0331270-65 1 . Radika 9, Skopje, 021365-460 Partizanska B-2 153169, 021344029 Metodi Mitevskil-16- 1 I?? -no telephone Braka Hadzitefovi 28, Kavadarci, 0431412-947, 0701218-566 Maksim Gorki 3, Shunica, 0902127-783 Evaluahon of the Macedonia B -5 February 2002 DemNet Program Task Order 805 Local Environmental Action Project (LEAP) Flowchart of Project Activities PHASE IV Implementing Actions C, PHASE IV Monitoring and Evaluating Results enviromental targets and indicators process and define 2) Seek sponsors and environmental issues environmental action implementing I a) Select assessment methodology I vision 2) Establish reporting system 2) Evaluate opportunities for working with private sector [ 3) Idenafy stakeholders 1 goals andtargets. and select indicators I b) Determine scope of the assessment I baseline conditions and I 4) Initiate ~reliminaw characterize he environmental issues 3) ldentify bppormnities forwohing wilhNGO public awareness activities I gmup and work~ng cornmees 1 4) Evaluate results I management praaices IO_n I I 4) Review existing organizational stturnre I I g) Gather information I 5) Utilize evaluation 5) ! results Identify potential aaions 4 Finalize assessments S) Secure participation of implementing institutions 6) Communicate results to the cornunity * I 7) Hm projea coordinator I I 6) Prepare Project 7) Collea information Implementation Plan I and preparc "Issue Summaries" 7) Prepare implementation budget and establish accountine orocedures actions public comment 8) Secure financing institutionalize plao 9) Ensurc effective integration of EAP into statutory plnnning I)mcesser Evoluah'on of the Macedonia DemNet Program Task Order 805 NGO Partners I I I 2 NGO Association of ' Struga Eco-perspective Continuation Environmentalists 2000 (umber l Macedonia Component NGO Continuation 4 Albanian Name of NGO Association of Agricultural Producers 5 I .677.745 mkd ~acedonic Albanian NGO Continuation 32.482 I mkd Macedonian Location Pehcevo NGO Continuation Women Portion of Target Targeted population reached Sector*' fes lThe citizens of the ]Agriculture Project Title Sustainable development for Pehcevo Bird Study & Protection Society of municipality of Pehcevo; 75 agricultural producers, the local self-government, Center for Civic Initiative lthe business sector I 40 /Teachers. ouoils from lEnvironmen Evani (office Skopje) primary schools, education institutions 750 students from 15 Education towns in Macedonia Management and development of the Ezerani bid Prilep 115 schools Information/liaisa n office for NGOs and citizens within municipal govenunent Inhabitants from 8 villages Environmen in the Region of SUictly Managemen I orotected Reserve of protected Minisay of Environment and Local Government officials from Resen Prilep Between NGOs and Local Government Local officials Govenunent Evaluation ofthe Macedonia C-2 January, 2002 DemNef Program - c I Task 0 der 805 e 6 f E f C C f € E h h h F I d F Developmen(Associales, Inc. AEP-I-00-00-00023-00 Children Victims of Violence I I 15 NGO Women's Continuation Organization 16 NGO Active for Development and People with Hearing Roblem! of Macedonia Association for j~oleranca 191NGO 1 Association for I Development Improvement of /the Medical I Treaiment and Rehabilitation of I I~reast Cancer Location Project Title Grant I Ethnicity Amount :umanovo lA family for the 1402.760 mkd iMacedonian, bridge for friendship and cooperation in the region tip Prevention of 613.816 mkd Macedonian juvenile delinquency kopje Youth computer 650.000 mkd Macedonian center for deaf persons attendance at municipality (council meetings kopje /Advisory body Ifor youth and who face drugrelated problems women to prevent 55.147 mkd Albanian + 33.626 --I-- mkd Macedonian Macedonian 2.040; families from the municipalities and parents of children from the different from the region of Ohrid, Struga, Debar Population from the region of Ohrid, Struga, Debar I lo 11200 students, authorities (directly, the population of 50 youngsters who will attend the computer and language courses; representatives from the Ministry of Education and Minisny of Social Affairs; two schools in Skopje and Bitola 1.305 citizens of the municipality 250 families; 350 present at the round tables; 60 participants at the education workshops; 20 participants from the employees in the centers municipality of Stip Sector*' iccial nter-eM1c :elations 'outh ocial - lemccracy ~lture ocial; ealth ealth lucation Evaluation of the Mncedonia C-5 February 2002 DemNet Progrom Task Order 805 Development Associates, Ine. AEP-I-00-00-00023-00 Association for Children in Macedonia Matica Bankrupt Companies I I 24 NGO Association of Development Citizens with I I l~iabetes from l~unici~alit~ 25 1 NGO 1 Association of Development Citizens for I I Support to People with Psychosis - Biosfera Development Bitola Youth Location Project Title your rights? promotion and improvement of beekeeping in Kocani lmunicipality ?tabistip IEducation for members on options for self￾employment in Ismall enterprises ?rilep IEducation and monitoring resources for people with and legal protection for people with mental disabilities and problem of solid waste management in l~itola 3itola ICenter for linformation and IsnPp~rt of NGOs in Macedonia rights Grant Amount 583.521 mkd IMixed I No INatimwide IChildren's Ethnicity 518.202 mkd I I Women Tar eted Mixed, mainly Macedonian 565.147 mkd 508.566 mkd 173.864 mkd 556.167 mkd Evaluation of the Macedonia C-6 February 2002 DemNet Program E [ "wk Od-- '05 g IL f f E F $B f h F C L E E B C h Portion of Target population reached No Macedonian Macedonian Macedonian and Albanian 544.818 mkd Sector*' Roma and Macedonians Directly 150 beekeepers, 50 students, indirectly 30% of the population in the municipality of Kocani No Yes No Macedonian Economic Yes 750 laid off workers directlv and their families 4000 inhabitants of Prilep and 54 villages from the municipalities of Mariovo, Krivogastani, Dolneni i Topolcani. 50 patients, their families and 16 volunteers Yes Economic Health Health/ Education 240 households in Bitola directly involved Envirmment 100 NGOs from the region of Bitola NGO Cooperation/ Networking Development Associates, Inc. AEP-I-00-00-00023-00 Ethnieity Macedonian I threatened families through I I Grant Amount 605.533 mkd 34 35 36 lhainiig 37 INGO IHumanitarian [Debar lSocialization and 1588.744 mkd IRoma, Number 33 First Children's Embassy - Medjasi HOPS -Options for Healthy Life NGO Development NGO Development Development Organizatim of I JtheRoma- / Mesecina!Debar Name of NGO Environmental Organization Kitka Component NGO Development NGO Development integration of handicapped people in Debar Albanian Location s. Dracevo Skopje Skopje Skopje Humanitarian Organization El Hilal , /branch 381NGO I Journalists' [Skopje lDevelopment of 1598.800 mkd IMacedonian Project Title Communal waste and landfills - a challenge for the 637.005 mkd 499.122 mkd Enhancement of non￾govenunental organizations involved in the protection of children's rights in Macedonia Psycho-social club for drug￾users . - text foia law on Legal Center - public access to Mixed Macedonian Skopje linformation I Improvement of the social situation of 6000) &om the regions of Ohrid Suuga, Kumanovo and Kriva palanka (total: Women Targeted 630.715 mkd 'es 12% of population (around IEnvironment Portion of Target population reached 1294124) Albanian Iue alw their families 'cs I100 young women iu~d IWOIIICII'S Sector*' lo 'es 15 NGOs, 5-10 informal groups of citizens, 1200 250-300 drug users and users of psycho active substances; Indirect users their families inhabitants in the municipality included in I Strengthening the NGO Health; Sccial issues/econo mic 'es lthe media campaign 'es 180 participants in public lEnvironment I~epresentatives from 1 I 327 handicapped people from Debar; 65% of the Miiishies of Environment and Justice and NGOs Social Evaluation of the Macedonia C-8 Pebruary 2002 DemNet Program r E 80Sr = C C E @ P h C F C r C K e E F Development Associates, Inc. AEP-I-00-00-00023-00 Center for the Social Sector an Underdeveloped regions )evelopment '" Humanitarian from Kicevo Association of environmental is^ EkoLab Association Deshat coordination, information￾sharing, and networking raising for tolerance among families in the --P=!@= :icevo ~up~ort for handicapped children and thei families LEAP for Labunista Municipality Development of LEAP for Debar Municipality Grant I -.. . I 97.272 mkd Albanian 56.620 mkd Macedonian, i Albanian 35% Albanian 30% Turkish Macedonian Organization 62.2% Albanian 18.8% Mackdonian 11% Turkish Women I Portion of Target Targeted I population reached Vo IAround 30 NGOs participating in the NGO network 150 children (from 5 to 7 years old); 115 students from primary schools; municipalities of Kicevo, Dmgovo, Vranestica, Zajaz and Oslomej; children with handicap an, le core roup to ome e~tent*~ I lot involved irectly in the whole community *' ore group Sector*' 2ooperatiod Women's ssues iocial ocial nvironment nvironment Evaluation of the Macedonia 5k DemNel Propram C-9 February 2002 Task order 805 Developmen4 Associaies, Inc. AEP-1-00-00-00023.00 Vumbe Association LEAP for Sveti Izgrev Nikole Municipality Municipality EAP Association LEAP for Odek, Kavadarci Municipality Environmental Association Kladenec EAP Grant Women Amount I Ethnicity I Tar eted i33.450 mkd. ICommunity: 1 Women Pehcevo AP 197.62% linvolved in Development of LEAP for Pehcevo Environmental Association Zletovica I~acedonian I 78.000 mkd ICommunity: IWomen Association for Development of Underdeveloped Regions Vizija 97% involved in Macedonian the core 1.3% Roma group*' Organization Probistip Il\?acedonian I Development of 51.354 mkd ICommunity: IWomen LEAP for hobistip s. Mikovci Skopje I involved in IEZonian the core Development of Community Action Plan for Cucer Sandevo Municipality 3237% involved in group*' Albanian 33.36% Serb Organization: whole community *' whole community e2 Community whole community ** Development Evaluation of the Macedonia C-10 February 2002 DemNeI Propram F f ask 4 ' 805 g C t !fc c f tF h !E f f F L E C C- Developmertl Associales, Inc. AEP-I-00-00-00023-00 Education, Community Democracy and Action Plan for Multi-culture Tetovo I lmunicipaiity Community /Friendship and I Action Plan for Municipality 52 CAP Civic Association Krusevo Development of Citizens of the Community World Action Plan for Krusevo Municipality 53 CAP Environmental Caska Development of Association Rosa Community Action Plan for Caska Municipality 1 Albanian 600.000 mkd ICommunitv: IWomen I ivnllth Grant Amount 593.863 mkd Albanian 1.81% Turkish 1.95% Roma 1 % Vlah Organization: Mixed to some ethnic it^ Community: 80% Albanian 15% 91.74% Macedonian 2.44% Women Tar eted not involved in the core ErouD involved in the core group*3 Organization: Macedonian 557.220 mkd 177.303 mkd *' For CAP and LEAP NGOs, the sector refers to the sector focus of the NGO itself, not of the project. *2 For the CAP and LEAP NGOs, the target group of the project is the whole community and it is difficult to come up with a number of people directly involved in surveys, presentations, etc. Portion of Target population reached whole community *' whole community *2 Women involved in the core group*3 extent Community: 55% Macedonian 23% Albanian 7% Vlah *3 "Women involved inthe core group" refers to the core group of people most directly involved in coordinating the project (m 4 projects, women are in decision making positions), the surveys and public discussions include women in the community. Sector*' Youth Education . """. Educatio Macedonian Community: 64,5% Macedonian 22,595 Turkish 7.7% Albanian Evaluation of the Macedonia C-11 DemNet Proaram February 2002 whole community *2 Task order 805 Handicapped Women involved in the core group*3 whole community *' Environment mi REFERENCES AND DOCUMENTS'~ d 1. INSTITUTE FOR SUSTAINABLE COMMUNITIES DOCUMENTS: High Hopes, High Hurdles: A Needs Assessment of Macedonia's Environmental NGOs, d September 1995 d Local Environmental Action Project (LEAP) Flowchart of Project Activities; CAP Flowchart of Project Activities, electronic files J Profile: the CSO Sector in Macedonia, 1 March 2001 DemNet Training Modules, electronic fde d DemNet I, 11, I11 and No-Cost Extension Proposals, electronic files mil Funded DemNet NGO Projects Summary, electronic file Project Descriptions for DemNet NGO Partners, electronic files d Organizational Charts for ISC Vermont and ISC Skopje Offices, electronic files d Model of the DemNet Process in Macedonia, electronic file Small Grants Program, funding rounds, electronic files NGO Development and Continuation Grants, electronic files ri Networking Conference, Working Groups, November 2001, electronic file 2. INSTITUTE FOR SOCIAL, ECONOMIC AND POLITICAL RESEARCH, SKOPJE, MACEDONIA d Survey of Macedonia NGOs, Reports 1,2, electronic files l4 Note: ISC uses the term CSO as svnonmous with NGO. Except in document titles. this reoon uses the term "NGW for consistem Evaluab'on of the Macedonia D-1 February, 2002 DemNet Program Task Or&r 805 CENTER FOR CIVIC INITIATIVE - PRILEP In 1998, it occurred to the members of Center for Civic Initiative in Prilep that there was very little cooperation among the NGOs in the Prilep region. Not surprisingly, this lack of cooperation had marginalized the NGO community in the eyes of the municipal government, which simply did not see possibilities for working with NGOs. The contacts that did exist between municipal officials and NGO representatives were low-level and without a defmed protocol. In fact, the most common contacts were requests by NGOs for donated office space, a trend that appeared to frustrate both sides. Establishing a channel for information-sharing and cooperation emerged as an idea at the Center for an initial project. On the strength of this idea, CCI commenced a partnership with ISC's Democracy Network Program in 1998 aimed at bringing together key actors - 20 local NGOs, the Mayor of Prilep, the municipal council and the citizens of Prilep - to strengthen the relationships between the NGO community and local government. CCI believed that citizens in Prilep had very little access to information about the mechanics of municipal governance - how decisions were reached, what mechanisms for public participation were available, and what their rights and responsibilities were. While NGOs provided citizens with a vehicle for organizing themselves around issues, it was unclear how citizen-driven initiatives could make meaningful contributions to municipal governance on these issues. In short, citizens lacked an understanding of the "architecture" of how their government worked and how to make it work for them. In consultation with ISC, CCI developed a strategy for addressing this problem centered on the formation of a working group composed of municipal representatives (administrative employees without a political bias) and NGO members. The working group was tasked with planning the creation of an NGOfcitizen information office within the municipality. The office would be designed to provide practical information on the activities of the NGO community and guidance to citizens on public service delivery matters such as infrastructure maintenance, securing building permits, municipal council input, and many others. A CCI representative would manage the office in coordination with the municipal government, which would bear responsibility for following up on citizen inquiries and requests. Among the many challenges CCI faced in implementing this vision were: 1) building credibility with the municipal government; 2) marketing the information office with a view to building a "client base" of NGOs and citizens; and 3) empowering municipal government entities to provide efficient and appropriate follow-through on citizen requests and inquiries. In CCI's view, its success in meeting these challenges would shape a more engaged public and promote greater diversity and competition in setting priorities for Prilep. In the two years that CCI has been a DernNet partner, it has realized its vision in several ways. With ISC's package of training, technical assistance, and grant support, CCI has emerged as a credible and reputable institution in Prilep, with the capacity to assume a role as an honest broker between citizens, NGOs, and the municipal government. CCI's initiative has been formally Eval~'on of the Macedonia El February 2002 DemNel Program Task Order 805 endorsed and approved by the Mayor and the municipal council, and in October 2001, the Information Office was officially opened for business. A Citizen Council has been established and communicates ideas and concerns to the Information Office. Regular meetings are held for NGO and municipal officials to define and refine the services provided by the Information Office. Evaluation tools (surveys) have been developed to assess public opinion on the utility of the Information Office and the needs of the community. The Information Office routinely fields between 20-30 citizen inquiries and requests per week, an extremely promising beginning. Most importantly, CCI has painstakingly crafted a model of NGO-local government collaboration that other communities in Macedonia can adopt. CCI has given the citizens of Prilep a voice in defining the needs of their community. Protection of consumers has long been one of the most fundamental rights of citizens in developed societies. From its founding as an NGO, the Organization of Consumers of Macedonia has promoted the establishment of norms in this sphere of civil law in Macedonia. As the Organization commenced its work several years ago, it recognized that existing consumer protection regulations were embedded in other areas of domestic law regulating trade relations in domestic markets. As such, OCM pointed out that it was extremely difficult for the average consumer/citizen to understand consumer protection laws, to be acquainted with hislher rights in . this area, and, of course, to know how to use the system for protection. A solution was necessary. As a result of this situation, OCM advanced a proposal to create a clear, separate law for protection of consumers in the Republic of Macedonia. Momentum gathered behind this proposal when, in 1998, the European Union launched an initiative to bring Macedonian laws in line with EU common law. The regulations on consumer protection were targeted for revision. OCM then joined forces with ISC's Democracy Network Program in January 1999, and assembled a project team tasked with inventorying existing consumer protection regulations, analyzing them and developing a strategy to draft and implement new legislation. The most formidable challenge faced by OCM was that it was virtually unheard of in Macedonia at this time that an NGO could be meaningfully involved in shaping national legislation of any kind. In fact, up to this point, there had only been one previous occasion - the preparation of the Law for hotection of the Environment and Nature - when an NGO had been actively involved in doing so. OCM realized that success was possible only if the organization was viewed as an acceptable, credible partner by the Ministry of Trade, and if the organization learned how to effectively navigate and lobby the Government and the Parliament of Macedonia. A tau order indeed. Bolstered by ISC training in advocacy, strategic planning, and project management, OCM began cultivating key relationships within the Ministry of Trade, with a team of legal experts, and with media representatives who had expressed interest in consumer protection. These relationships began paying off as Ministry officials grew increasingly engaged in the efforts of the legal expert team to prepare a draft law. On top of this, OCM had successfully begun to stimulate public interest in this initiative. Upon completion of the draft law, OCM organized well-attended public Evaluation ofthe Macedonia DemNet Program Task Order 805 L.l Februaq, 2002 presentations/discussions on the draft in five towns across Macedonia and secured important citizen input. In June 1999, OCM delivered a fiial version of the working text of the consumer protection law to the Ministry for further deliberations. One year later, in July 2000, the Law for Consumers Protection was passed by Parliament. Aside from the obvious benefit realized by OCM's initiative - that protection of consumers in Macedonia is now clearly enshrined in law - the experience also generated substantial public interest in consumer protection, as follow-on educational efforts have broadened this constituency. OCM now provides training and orientation for an association of judges, and association of lawyers, and a group of government trade inspectors, all of whom bear some responsibility for successfully implementing the new law. Further, OCM's contributions reinforced the legitimacy of NGOs as partners in public policy matters, and as vehicles for safeguarding the rights of the citizens. OCM's evolution during this project stands as a textbook model for cooperation between the NGO sector and government. Quality information in a timely manner is a requirement for the success of every enterprise. For a long time, this requirement was not being met in the community of Pehcevo, a rural, undeveloped district in eastern Macedonia with a tenuous connection to municipal governmental institutions in the region. One of the many consequences of this has been that the farmers in Pehcevo, the principal source of economic activity, have not had access to resources/iiformation that could improve the quality and quantity of agricultural production. In order for a cherished way of life to remain viable, it was imperative that a reliable source of practical information be developed. The Association of Agricultural Producers - an NGO promoting the interests of farmers in the region -had an idea. The idea was relatively simple: identify the kinds of information and resources that would be valuable to the farming community in Pehcevo, and link providers of the information and resources with users in a way that was institutionally sustainable. In other words, an entity in which all sectors of the community would be invested because it provided a highly beneficial service should be formed. Although the Association was a young, inexperienced organization at the time this idea was conceived, it succeeded in securing the financial and technical support of ISC through the Democracy Network Program and prepared a project proposal for the establishment of an Agricultural Consulting Office in Pehcevo. Eight months later, in early￾2000, the Agricultural Consulting Office, managed by members of the Association, began offering education and information services to the farmers AND citizens of Pehcevo. The Association has, in retrospect, observed that the Office became a reality precisely because of the active participation and involvement of several sectors of the community - representatives of other agricultural associations, business leaders, municipal government officials, students, technical experts and successful agricultural producers - in making it happen. An important multi-stakeholder foundation was laid. Presently, the Office disseminates information, experience and knowledge to agricultural producers by offering technical support consultations, educational materials, and tailored Evald'on of the Macedonia DemNef Program Task Order 805 February, 2002 training. The Office services over 300 farmers and citizens in the region, and reaches 200 more through the bulletin it publishes. For the time being, the Office remains an entity affiliated with the ~ssociation though it is exploring ways for it to become a self-supported entity - perhaps through a fee-for-services arrangement, augmented by support from local businesses with a stake. What is clear is that the establishment of the Office has met a vital need for quality, timely information that helps farmers to improve their production and, ultimately, preserve a meaningful way of life in Pehcevo. Macedonia DemNet Program Repon-Final.doc/Evalqc-6 - Evaluntion of the Macedonia ~em~et pr&ram Task Order 805