Russian Civil Society Support Program Mid-Program Evaluation Submitted by: Chris Cavanaugh 26/11/07 Table to Contents Introduction Page 1 Section I: Methodology Page 1 Section II: Program Model and Outputs To Date Page 2 Section III: Discussion Page 4 Section IV: Conclusions and Recommendations Page 17 Appendix I – Scope of Work Appendix II – List of Interviews Appendix III – List of Source Documentation Executive Summary In its first two years, the Russian Civil Society Support Project “Dialogue” has made significant contributions to grass-roots and community-based activists’ efforts to form associations capable of lobbying local and regional governments for more responsive social policy. Over 30 local and regional NGO associations and advocacy groups, with a total of 840 constituent organizations, have been created to represent citizen interests in the areas of health care, education, women’s rights, environmental protection, housing, government transparency, and rights of people with disabilities. With CSSP support, advocacy NGOs across the country have commissioned 20 practical research studies to provide evidence for informed and professional policy making at the municipal and regional level. Exploiting the inadequacy of government structures to address urgent policy needs, CSSP advocacy funding has allowed grantees to conduct over 363 advocacy campaigns addressing 93 distinct legislative and normative acts at the local, regional, and federal levels. However, these successes in building alliances and lobbying were forced substitutions for larger ambitions of federal level advocacy posed in the original program design. First, the Russian government imposed a harsh regime of registration and reporting on the NGO community in the wake of Colored Revolutions in the Near Abroad. Second, with large infusions of government money, the Public Chamber became both the largest NGO funder in Russia as well as the sole legitimate representative of NGO interests at the federal level. Finally, a growing suspicion of USG and foreign funding capped the amount of access that US-funded NGOs across the country would have in the federal and regional advocacy process. Instead, CSSP has used its limited resources to successfully launch and spin-off several initiatives, including the Association of NGO Lawyers and the NGO Transparency Program, which have become the sustainable mainstay of USAID and other donors’ efforts to address threats to the Third Sector. Based upon review of source documentation and over 25 hours of interviews with CSSP stakeholders, this evaluation provides recommendations to maximize advocacy output over the last two years of CSSP given the current political and financial limitations. Examining successes, failures, opportunities, and challenges, this evaluation provides seven recommendations for the CSSP over the next two years to assure the legacy of strong associations that can advocate for citizen interests at the local, regional, and federal level. These seven recommendations are summarized into four key points below. To ensure maximum input over the last two years, CSSP should: • target all of its effort and funding into building the capacities of the thematic alliance grantees. Social policy research and advocacy are not ends in and of themselves, but rather resources and methods used by activists to achieve their ends. By subordinating social policy and advocacy to the goal of association building, CSSP has a greater chance of empowering sustainable structures for on-going advocacy. • be more directive in choosing alliance grantees. Independent selection committees can well vet alliance applications for need and policy relevance. CSSP should use its internal expertise to identify those alliance applicants with the greatest understanding of democratic participation and the best laid plans for alliance growth. • build the capacity of thematic alliances to search for and exploit external resources. CSSP should avoid playing intermediary between association, social policy, and working group grantees and hoping for collaboration. Rather, CSSP should use its social policy and advocacy grant funding at the discretion of the thematic alliances so that alliances may choose more relevant grantees and more directly expand their membership bases. • focus limited technical assistance funding on individual grantee mentorship programs with a specific CSSP coordinator. With the obligation for Year Three being less than was anticipated, CSSP must more precisely and economically target technical assistance. To date, investment on individualized, not blanket, activities has shown the greatest return in grantee productivity and output. Introduction In October 2007, IREX contracted former Chief of Party and IREX/Moscow Office Director Christopher Cavanaugh to conduct a mid-program assessment of the Russian Civil Society Support Program “Dialogue”. The goals and objectives of the evaluation included a general assessment of the progress of the CSSP to date with a particular emphasis on the validity of program assumptions and theory given the drastic changes in the political context. This report presents the findings of the mid-term evaluation conducted from October 21 – November 7, 2007. The first section provides a brief account of interview methodology and limitations against which the conclusions and recommendations must be weighed. The second section provides a background on the original goals, objectives, and theory of the Dialogue program to act as a baseline against which progress will be measured. To allow for a better comparison of the original program theory to implementation, the third section of the report organizes the discussion of data according to each of the three program objectives as stated in the cooperative agreement (association building, social policy research, and advocacy). The final section provides recommendations for the final two years of program implementation with additional suggestions for administration. Section I: Methodology CSSP staff provided the evaluator with over 200 pages of programmatic and grantee background reports prior to site visits in Moscow, Samara and Perm. The evaluator conducted over 25 hours of interviews with program grantees, staff, beneficiaries, and other stakeholders on-site between October 21 and November 13, 2007. CSSP staff was charged with determining the nature and scope of the interview pool based upon ease of access and diversity of experience. In total, CSSP staff was able to organize interviews with over 30 representatives from 8 grantee associations, research groups, and NGO/government working groups spread over CSSP’s first two years. Table 1 below illustrates the variety of interviewees in a cross table by grant year and grantee types. Table 1. Interview Breakdown Alliance/Association Social Policy Research Group NGO/Government Working Group Year One 0 0 3 Year Two 2 2 1 In addition to these grantees, the evaluator also met with representatives of the Higher School of Education, two members of the CSSP advisory board, and three CSSP staff members. The evaluation objectives were not to compare success or failure between grantees, but rather to explore the evolution of each grantee alliance/research/working group from inception to present, identify inputs which had a positive impact on project implementation, and assess the relationship between the intended and actual outcomes. Due to the variety of grantees and exploratory nature of the evaluation, standard protocols were not used during interviews. Rather, interviews were structured to investigate eight key factors: 1 • Project conception • Structure of participation • Project implementation • Understanding of advocacy and role of project in advocacy • Understanding of research to inform advocacy • Intended vs. unintended outcomes • CSSP inputs • Follow-on and future work The open-ended questions and discussions targeted grantees’ understanding of their own work in the advocacy process as well as how smoothly their individual projects integrated and contributed to the large CSSP. Grantee interviews averaged from forty-five minutes to one-and-a-half hours, depending upon the number of participants. One interview lasted over three hours due to the large number of participants. Where applicable, grantees were asked to invite their beneficiaries and other partners to participate in the interview process. This was applicable only in two instances. Per USAID’s request, CSSP grantees attempted to involve government officials in the interview process to obtain feedback on the advocacy process. Unfortunately, due to the upcoming election cycle and increased suspicion of USG-funded initiatives, government representatives refused to meet with the evaluator. Data analysis was conducted from November 1 – 10 with several follow-up discussions with CSSP and USAID staff. Data analysis sought to answer four evaluation questions: 1) In what context, if at all, does the CSSP theory of association building work? 2) In what context, if at all, does applied research play a successful role in advocacy? 3) What are the inputs to successful and sustainable advocacy programs? 4) What is the profile of a successful, sustainable advocacy association? The analysis presented in the report, and the consequent recommendations, are based upon the answers to the evaluation questions listed above. Section II: CSSP Model and Outputs to Date According to the program design, CSSP aimed to facilitate dialogue in the CSO community on sector-wide issues, and provide opportunities for professional networking. The Program sought to create conditions for representative third sector voices to come together to decide among themselves how to develop and advance their common agenda. Program inputs would consolidate the NGO community along thematic and professional lines, strengthen its advocacy and policy analysis skills, and promote constructive policy dialogue between NGOs and the government at the national, regional, and local levels. Finally, the program sought to support NGO-government working groups and public-private policy debate as forums for collaboration among stakeholders at national and regional levels to increase proactive and effective NGO contributions to policy formation. The program would achieve this overarching goal via three concrete objectives: Objective 1: Consolidate the NGO Community along Thematic and Professional Lines Objective 2: Improve Advocacy and Policy Analysis Skills 2 Objective 3: Promote Constructive Policy Dialogue between NGOs and the Government at the National, Regional, and Local Levels As displayed in Table 2 below, each objective was defined by a discrete battery of activities, the combined effect of which would promote an expected outcome. 3 Constructive dialogue between government and NGOs NGO Alliances improve their policy analysis capacity and learn effective international advocacy practices Objective 3: - National Parliamentary NGO Working Groups - Regional Parliamentary NGO Working Groups - Policy Development Grant Support Objective 2: - Social Policy Research Grants - Regional Social Policy Development Grants - International Advocacy Technical Assistance International N2N Partnerships Table 2. Program Objectives, Activities, and Expected Outputs Regional and national level thematic and professional associations channel citizens’ interests through broad-based membership and vertical and horizontal communication Objective 1: - NGO Alliance Grants - Professional Association Development Grants - Regional Affiliate Development Grants - Inter-Training Mentorship Program Over the course of the first two years, CSSP successfully conducted two rounds of each of the three grant competitions capturing a total of forty-five grantees working in definitive and concrete areas of association building, applied research, and advocacy. In its first two years, the Russian Civil Society Support Program “Dialogue” has made significant contributions to grass-roots and community-based activists’ efforts to form associations capable of lobbying local and regional governments for more responsive social policy. Over thirty local and regional NGO associations and advocacy groups, with a total of 840 constituent organizations, have been created to represent citizen interests in the areas of health care, education, women’s rights, environmental protection, housing, government transparency, and rights of people with disabilities. With CSSP support, advocacy NGOs across the country have commissioned twenty practical research studies to provide evidence for informed and professional policy making at the municipal and regional level. Exploiting the inadequacy of government structures to address urgent policy needs, CSSP advocacy funding has allowed grantees to conduct over 363 advocacy campaigns addressing ninety-three distinct legislative and normative acts at the local, regional, and federal levels. Despite the notable successes, CSSP has suffered its share of confusion and re-structuring due to both internal and external events. In 2005, CSSP was launched as USAID’s major initiative to help bring an infrastructure to the NGO sector and provide avenues for advocacy at the federal level. Since then, the civil society, economic, and political landscape affecting the Third Sector has changed dramatically. First, the Public Chamber, the government’s way of investing and controlling civil society, has taken the lead as they key funder for the NGO sector in Russia and therefore the most vocal driver. Several of the interviewees noted that the tone for civil society development and its direction was now firmly in the hands of the government and it was the government who would chose which umbrella groups to fund. Second, with annual economic growth at over 10%, a 30% decline in the purchasing power of the US dollar, and increased resources being funneled to the NGO community from the government and corporations, $2 million does not buy what it used to in Russia. Both of the governance board members and one of the alliance grantees noted that, in the scheme of things, the current $100,000 alliance grants are too small for the output expected in the original proposal. Finally, the new NGO regulations, suspicion of USG funding, and upcoming elections make the political environment for funding advocacy very sticky. Several grantees, especially those working at the federal level, noted that they were hesitant to apply for CSSP grants and warned that CSSP should be careful what it funds. For example, one of the social policy research grantees noted that they would not be seeking an Alliance grant this year as they are afraid of the consequences. The grantee warned that $12,500 was much less noticeable than $100,000. This does not mean CSSP has not, and will not continue, to play a vital role in the Third Sector. To date, CSSP launched the first NGO transparency and accountability program that later became the basis for USAID’s Building Trust Program. CSSP launched the Association of NGO Lawyers through which USAID and other donors can now funnel resources to help represent NGOs across the country in trying legal times. Further, the School for Housing Promotion, which arose as a direct result of CSSP programming, is fast on its way to becoming a national union of housing associations. This is not to mention the thousands of Russian citizens’ whose interests have been served by the likes of the AOMS, the Perm Anti-Narcotics Alliance, the Amur River Defense Program, to name just a few. Section III: Discussion This section describes the assumptions of each of the three components and compares these assumptions to actual practice over the first two years of the program. This comparison of intended vs. actual process and outcome allows for conclusions and recommendations to amend CSSP theory and interventions in years 3 and 4. Component I: Association and Alliance Building The key premise of the CSSP is that there is a critical mass of local grassroots NGOs across the country that are able and willing to consolidate their resources to form an association of advocates for their constituents’ needs. The assumption held that grassroots NGOs would be sufficiently self￾aware and mature that they would be able to create a participatory membership association that would be have a sophisticated tools of self-governance, standard setting, and membership outreach, similar to other umbrella organizations in the West such as InterAction and Child Welfare League of America. It was assumed that once stimulated and cultivated, alliances would be inexpensive to maintain. Targeting program theory, the assessment examined the three participating alliances across three criteria: • membership development • governance and services • sustainability 4 Below is a short profile of both the Year Two alliances examined, as well as a short resume of association linkages drawn from interviews with the other six research and working group grantees. Project for the Advancement of Inclusive Education The lead partner in this alliance is the Saratov-based Research Institute for Gender Studies (RIGS), a year-one CSSP research grantee that had long been researching inclusive education. In assembling the alliance, RIGS followed the letter of the CSSP, without quite understanding its spirit. To meet the RFA criteria, RIGS approached NGO colleagues and advocacy institutions in three regions assuring that they met the multi-regional criteria. While RIGS would be able to meet the research institution criteria, they specifically sought out two organizations in Samara (Desnitsa) and Moscow (Perspektiva) to meet the NGO criteria. To complete the picture, RIGS approached the Urban Economics Institute as a lobbying organization that would add credibility when presenting the alliance’s recommendations at the local, regional, and federal level. The alliance did not fail, as it was never a true alliance to begin with. Instead it was a large and worthwhile policy research initiative carefully painted to resemble a multi-regional alliance in a very well crafted proposal. There are three key issues that give warning to the challenges that this hodge￾podge of organizations would face in attempting to become an alliance. The first issue is that of leadership. The prime recipient RIGS is first and foremost a regional think tank. As such, its main concern is conducting and publishing research, not advocating for the educational rights of people with disabilities. Second, the four key members of the alliance (RIGS, IUE, Desnitsa, and Perspektiva) have widely differing agendas. Desnitsa and Perspektiva are pure advocacy organizations, while IUE and RIGS are research institutions. Per program theory, successful thematic alliances are made up of advocacy and service organizations supported by research institutions. Finally, RIGS as prime assumed that the advocacy organizations, Desnitsa and Perspektiva, would automatically expand the membership base of the alliance without additional resources and directives to do so. This was a false assumption as both Desnitsa and Perspektiva’s missions encompass all aspects of disability rights, not simply the advancement of inclusive education. This alliance also suffered from a lack of governance. This was difficult to foresee from the original grant application as it was extremely well written and the alliance members are all well-regarded and professional institutions. Governance problems stemmed from the various competencies of each alliance participant and their own priorities. Having established this as a research project led by a research institution, neither of the two advocacy organizations had the necessary professional competencies to participate in the governance of a research project. In terms of sustainability, this alliance grant has a very good chance of having a lasting impact with follow-on funding. Over the course of the project, Perspektiva and Desnitsa were able to make contact with colleagues in four other cities (Kazan, Izhevsk, Ulyanovsk, Ryazan) working on the same issues of inclusive education. Through regional seminars and contacts, the participating NGOs from each of these cities were able to participate in the research and look forward to using the findings in their own advocacy efforts. One of the issues that made this proposal so applicable to the Thematic Alliance competition was the sheer number of disability rights organizations that exist across all regions of Russia to take up the banner for inclusive education. The key is tapping into one of the advocacy organizations participating in this alliance and empowering them to take it to the next step in terms of membership. 5 The School for Housing Promotion Perhaps the most successful of the Thematic Alliance grantees from year two, The School for Housing Promotion (SHP) began as a consortium of six like-minded, professional NGOs that met at the CSSP-sponsored conference on housing reform in May 2006. With the help of the Eurasia Foundation, the six organizations piloted an initial housing development project funded by the Polish Ministry of Foreign Affairs in the summer of 2006. Based upon the success of this short-term project, the six key partners in the School for Housing Promotion designed a research and advocacy program that won a Fall 2006 CSSP Thematic Alliance grant. By all accounts, SHP is the flagship association for CSSP in that it fulfills almost all of the criteria laid out in the program theory for a high-impact, participatory, and sustainable coalition capable of representing citizen interests to government authority. A careful study of the capacities and structure of the association points to the reasons for its success. SHP membership is both broad and deep. While SHP is itself a working group of six to eight member organizations, together they represent a constituency of thousands of housing and community groups. SHP’s structure aims to be a middle man, supporting and providing professional development opportunities for the thousands of old and new housing associations popping up across Russia. In the words of one of SHP’s constituent housing associations in the city of Perm, “We have our own work on the ground, but we know that the School for Housing Promotion is behind us and supporting us, not just with legal information, but with moral support as well.” In that sense, it is not realistic to say that SHP has thousands of official members across the six regions. Rather, it is more accurate to say that SHP represents thousands of constituents in the six regions, providing them with resources as well as rallying them to activism when necessary. In short, each of the members of the SHP serves as a “resource center” for housing associations in their area. Importantly, SHP’s constituents are not passive about membership recruitment; indeed, their constituent housing associations are constantly seeking to expand their roles and to increase their legitimacy in advocacy. One of the more active members of the SHP in Perm stressed the importance of SHP to reach new members through outreach and activism, noting that she herself had helped to start over seventy other housing associations across the Perm region using SHP materials. SHP is currently in the process of expanding its presence through partnership organizations in three additional regions, depending upon available funding. They look forward to partnering with the RFE housing reform alliance, but are unsure about the different forms of collaboration due to drastically differing methodologies. In terms of governance, SHP presents a good model for democratic, if not openly participatory governance. The SHP was established as a true partnership between co-equals. Each organization has an equal vote on the governance board and each organization is locked into specific responsibilities and deliverables over the course of the year. Peer review and self-regulation within the association have served to assure a quality of standards during project implementation. There was excellent, daily communication between the six participants with regular face-to-face meetings. Most important, as is evident from successful governance, is that each organization, upon entering the partnership, had a clearly defined role with concrete responsibilities and tangible deliverables. Unlike the inclusive education alliance, SHP members shared a common vision and mission which was built upon a previous collaboration under the Polish-funded project. After lengthy interviews and in-depth questioning, the leader of the SHP partner in Perm, Konstantin Egorov, hit upon what he considers to 6 be the key to the association’s success. “Perhaps we work so well together because we all come from fairly democratic traditions and backgrounds,” Egorov noted. Three of the six SHP partners began under the democratically-leaning Open Russian Foundation, while a fourth organization was founded by the Eurasia Foundation. Together, these four organizations possessed a good understanding of the principles of collaboration, mutual responsibility, and participatory governance. Finally, in terms of sustainability, it appears that SHP has already laid the plans for long-term sustainability. Admittedly, they will continue to search for foreign grants, such as the recent USAID￾funded Housing Reform Project. However, they have also laid the groundwork for establishing an all-Russia Union of Housing Associations, a dues-based membership organization, the revenues from which should be able to sustain a minimal communication network. Finally, SHP has already begun to sell its wares and services to local government administrations in need of reforming their housing codes and a forum for connecting with local citizens. Given these three funding streams, SHP looks set to maintain itself to a minimal level of funding over the next few years. A key revenue source, which has yet to be tapped perhaps due to fear of “dancing with the devil,” could be the new housing management companies which will soon need new mechanisms for navigating the mine field of citizen and government interests. Commentary and Discussion In addition to data collected about organizational structures under the Social Policy Research and NGO/Government Working group grantees interviewed, the two profiles above provide three key insights into possible CSSP challenges in the coming years and strategies to deal with them. 1) Only a Small Proportion of the NGO Sector is Prepared for Democratically Governed Alliances While CSSP received over 100 applications for the Thematic Alliance grant competition in its second year, the tragic majority of these applications were rejected on two criteria. First, the proposals did not define the on-going representation of a constituency, but rather identified discrete needs of a small population that could be solved by a series of activities. Second, the applications were unable to demonstrate the ability of applicants to form a wide-reaching and sustainable voluntary association with participatory governance systems. Rather, the majority of grant proposals showed a hierarchical structure of command, control, and limitation. CSSP staff and experts worked on the RFA for Thematic Alliances for over three months and articulated the goals and objectives as clearly as CSSP thought possible. That only four proposals out of 100 merited funding raises a fundamental question about the program design, namely that either the RFA itself is poorly formulated, or that there is only a limited market of potential coalitions that fit into the above criteria. While acknowledging that the RFA did have its challenges and could have been clearer in its solicitation, the evaluator tends toward the latter explanation that the market for what CSSP seeks to foster is limited. When discussing association-building, collaboration, and umbrella advocacy with all of the interviewed grantees, the evaluator ran into the repeated challenge of grantees identifying their long-term interest in participating in an association. The overwhelming majority of the grantees placed primary focus on serving their immediate constituencies and saw little point in participating in long-term advocacy coalitions except in pursuit of a short-term goal. 7 This does not mean there is no market for NGO coalitions and advocacy groups across Russia. On the contrary, there is a very large market to bring groups of NGOs and CSOs together to challenge the status quo and improve policy affecting their constituencies. The challenge is that very few of these potential alliances will be both democratically-inclined toward an expanding, participatory membership structures and able to look beyond the discreet project-based grantee mentality that sets a discrete outcome as a measure of success rather than on-going constituent advocacy. SHP might be the exception, rather than the rule, in terms of defining the market for CSSP alliance funding. The RFE Housing Alliance lacks the democratic bent, being controlled by a single primary recipient who engages and hires grassroots NGOs across the Russian Far East to advocate citizens’ rights in housing reform. However, it has been no less successful than SHP at solving housing problems and fostering community/government dialogue in the Russian Far East. Representatives from SHP have actually praised the RFE Housing Reform project for its vision, focus, and impact. Some SHP activists even expressed envy at the simplicity of their Far-East colleagues’ governance, but recognize that the type of equal democratic governance among six equals would be difficult with so few strong partners in the RFE. 2) Alliances Should be Led by Advocates The evaluation of a wide variety of grantees allowed for the evaluator to profile a number of the grantees based upon lead organization and project outputs. A common theme that emerged was that the tone of the lead organization set the tone of the project. In other words, those projects led by activists (such as SHP, All-Russian Organization of People with Multiple Sclerosis, Nevsky Research Center, and Perm’s Center for Civic Education) had established goals of change and reform via lobbying and advocacy. By their very nature, advocates and activists understand the concept of strength in numbers and set change as their final goal. In contrast, associations (and by default social policy research and NGO/government working group) led by non-advocate institutions set their sights on different targets and do not necessarily see themselves and their alliances as change agents. Rather, they are passive researchers who assist the advocates in formulating cohesive arguments. When asked to describe the role of their projects in advocating for change, three of the grantee leaders answered to the effect that “our project was to provide decision-makers with the information they need to make good decisions.” While an admirable role, this is a passive role that is better supported under the Social Policy Research competition. That is the fate of the inclusive education association, should one of the coalition members not force the results of the research into applied policy making. RIGS has neither the capacity, nor the interest, as it remains a largely research oriented institution. Perspektiva has already started using the economic analyses in its research, with surprisingly effective results. However, as inclusive education is regional in jurisdiction, it will take a more cohesive group of disability advocacy organizations to assure the research is used to promote inclusive education. Without taking this to the next step, the valuable and high impact work might sit on the shelf as many a research has done before. CSSP might consider follow-on funding for the inclusive education alliance as the prospects for successful implementation of inclusive education across Russia are high. If so, Perspektiva and its network of twelve grassroots NGOs would give the best results. Follow-on funding could be used for advocacy campaigns and/or pilot inclusive education projects. As the research has already been conducted and the policy recommendations prepared, any follow-on funding should target the actual 8 implementation of inclusive education, either in the regions targeted in the first year or in other regions where the Perspektiva network is well connected. 3) Individualized Technical Assistance Is Necessary for Alliance Structure and Function The School for Housing Promotion was the only alliance interviewed to fully benefit from the entire battery of comprehensive, targeted assistance provided by CSSP. While SHP might be the exception rather than the rule, representatives of the SHP alliance noted that individual technical assistance plans were invaluable for opening their eyes to methods of coalition building and managing complex membership and communication structures. SHP considers CSSP’s greatest input (aside from $100,000 in grant funding) to be the individualized technical assistance plan developed after the February 2007 organizational assessment. Interviewees noted that they “did not know how much they did not know” in terms of association building and were grateful that CSSP had a higher command of the skills and capacities necessary for sustainability. They also noted that individualized trainings on PR and communications, tailored to their needs, were critically important in attracting new members and effectively engaging the press and government entities. CSSP staff also noted that the individualized technical assistance program was highly effective in helping the Russian Far East housing alliance to overcome their institutional challenges. In November 2005, CSSP staff flagged the RFE alliance structure as being highly dependent upon the prime recipient and not devolved enough to assure the creation of a true association. CSSP technical assistance plans targeted the governance and participation issues, which have apparently been solved. While the evaluator did not speak to representatives of the RFE alliance directly, the colleagues in Perm noted that there has been a tremendous amount of institutional growth in the RFE alliance that has become a very powerful network of housing advocates. CSSP credits this change to its technical assistance interventions early in the year. Grantees were less enthusiastic about large, generalized trainings conducted by CSSP over the first two years. Most grantees noted that trainings are always good, but that the diversity of topics and attendees did not lead to the alliances becoming more cohesive or better-run institutions. Other events such as the Sochi Conference in October, while enjoyable, showed little demonstrable value added. In Years Three and Four, CSSP should reduce the number of generic seminars, trainings, and meetings to a minimum and focus on individualized technical assistance plans. Component Two: Social Policy Research While Component One aimed to establish a broad based infrastructure for NGO alliances and associations, Component Two focuses on strengthening the capacity of these alliances to effectively represent the interests of their constituencies by introducing the right analytical tools to address issues and evidence-based recommendations for policy formation. To be effective advocates, alliances must come forward with data, analysis and evidence-based arguments for improving both issues critical to citizens’ lives. In its first year, CSSP placed significant attention on the development of alliances and working groups to the neglect of the social policy development component. In Year Two, CSSP came back with a vengeance to invest heavily in social policy formation with three key mechanisms: Social Policy Research Grants, Socpolitika.ru, and a series of capacity building trainings for NGO recipients. 9 The combination of these three interventions was meant to feed into the work of the alliance funded under Component One. Alliance grantees were to have a significant say in the selection of the Social Policy Research grantees to assure that research conducted under these grants would contribute to their advocacy efforts. SocPolitik.ru was envisioned as a unique clearing-house of information and research targeting grantee needs and linking researchers with NGOs. Finally, the NGO Research Capacity Building Training series was meant to increase NGO grantee appreciation and use of applied research in their advocacy efforts. This evaluation explored the role of each of these three components in supplementing and improving the work of the larger alliances and their advocacy efforts. Below is a short resume of both of the two Social Policy Research grantees who were interviewed as well as of the Higher School of Economics’ Socpolitika.ru project. These are followed by a list of conclusions and recommendations drawn both from these five interviews as well as from other grantee beneficiaries. School for Civic Education/Center for Labor Market Studies (Samara) Chosen as a research grantee under the education track in Year Two, the School for Civic Education (SCE) conducted a comprehensive local study of parental and stakeholder values on inclusive education. In partnership with the Center for Labor Market Studies, the SCE chose the research topic based upon the director’s long-term relationship to the subject as the former deputy head of the education authority for the Samara region. The research was less of an investigative effort than a summative evaluation of the Samara educational authorities’ integrated education program from 2001 – 2005. Having reviewed the research, the evaluator was impressed with both the quality of data collection and targeted nature of the research question. The data would definitely be valuable to other educational authorities or inclusive education advocates. Despite being located in the same city and targeting the same topic of inclusive education, SCE and the Inclusive Education Alliance partner in Samara, Desnitsa, collaborated very little on this project. When asked about collaboration, the SCE Director mentioned that they had attended a few conferences together, but that their topics did not overlap as Desnitsa was working primarily on administrative and economic reform in education, while SCE was working on public opinion polling. Neither Desnitsa nor SCE understood how the information and data could be used in combination to add weight to a more concerted effort. More interestingly, the lead advocacy organization for the Inclusive Education Alliance, Perspektiva, also gave little interest to the results of the SCE research. Russian Public Health Association/Institute of Sociology (Moscow) Under the second year Social Policy Research competition, the Russian Public Health Association (RPHA) and MGU Institute of Sociology banded together to conduct a two-region study of the current barriers to health care for children in cities and regional centers. This research would have fit well into the now-defunct Healthy Russia alliance’s advocacy efforts, had the alliance been able to understand and use data for advocacy. As stated by directors of both partner organizations, the research question posed is one of the critical points in the current national project for health care which runs through 2008. Both the method of research and the results could be applicable to on￾going policy formation at the local and regional levels. Most intriguingly, the lead partner at the RPHA had an uncanny understanding of the intricacies of policy advocacy and research. Having participated in a number of large-scale research projects as well as having been active as a health policy advisor in the presidential administration under Boris Yeltsin, the RPHA director was able to paint a vivid picture of the current policy development 10 process and the difference between policy formation at the regional and federal levels. He stated that evidence-based policy was not a factor for debate in federal policy making, but could still be used to inform regional implementation of federal programs. Higher School of Economics Center for Study of Social Processes (Moscow) After taking the first year reprieve from social policy, CSSP contracted with the Higher School of Economics (HSE) Center for the Study of Social Processes to act as a warehouse for research and information that would be critical to the success of CSSP alliances and other grantees. Over the first half of the second year, HSE contracted with over 20 research institutions across the country to collect, sort, and store research in CSSP’s three key areas of education, housing, and health care reform. It wasn’t until the second half of the second year that the HSE launched its website Socpolitika.ru to make the data accessible to grantees and researchers, as well as other NGO advocates throughout the country. While the number of regular users of the site climbed from 300 in February to over 2,500 in June of that year, and the overall number of hits increased from 3,000 to 11,300 in the same period, the number of regular subscribers to the Socpolitika.ru listserv has grown to 800, over 70% of which are from outside Moscow and St. Petersburg region. The website currently boasts over 700 research articles and information clippings related to the national projects and social policy research. The notorious challenge of evaluating the effectiveness of any website and/or warehouse is not the number of users accessing the information, but assuring that information is being accessed by CSSP’s targeted users, which in the case of CSSP includes advocacy NGOs and researchers working in the same area as CSSP grantees. That is not possible to do under the scope of work of this investigation. In lieu of simply tracking the number of hits to the website, the evaluator made a point of investigating CSSP grantees’ use of HSE services with overall positive results. Partners working in the area of housing reform held the highest esteem for HSE services. Members of the SHP alliance actively exploited the website and listserv, noting the success of the on-line conference that was held in May-June 2007 that attracted approximately seven new members and correspondents into the work of the alliance. Members of the Inclusive Education alliance also noted that they employed Socpolitika.ru during the beginning of their projects, but have become more regular users of the listserv and its news updates. Of the six second year grant recipients interviewed, two institutes had materials posted on the HSE website while all six had downloaded research they consider appropriate for their advocacy programs. NGO Research Capacity Building Training - Stellit Examining the results of Year One, CSSP noted that there was not only a clear lack of NGO understanding of the content of policy research, but even less of an understanding of the use of research in advocacy efforts. In September 2006, CSSP sub-contracted with Stellit research center in St. Petersburg to design a two-tiered practical training program on designing and using applied research in their advocacy efforts. These two-tiered trainings were targeted toward alliance grantees beginning in January in a cycle appropriate for each alliance grantee. Results of the trainings were mixed for two reasons. First, both the inclusive education and SHP alliances include strong research institutions within their partnerships. Not recognizing or acknowledging the Stellit seminar as a unity-building experience, these two alliances sent second tier partners to attend. The RIGS specifically opted out of the trainings as they felt that Stellit did not 11 have anything to teach them about research, applied or otherwise. Second, the Stellit trainings were a pilot program that was offered as a CSSP-mandated “extra” for alliance grantees. Grantees did not request this training, albeit necessary from the standpoint of CSSP, and therefore had little investment in its success. Several of the less research-oriented attendees (Desnitsa and Perspektiva) gave the training high ratings and increased their appreciation for applied research in advocacy. Commentary and Discussion Unlike the first year of CSSP, investment in social policy development in the second year of the program provided a great deal of information regarding both the potential for evidence-based advocacy as well how best to structure it into the larger CSSP design. Three specific recommendations follow. 1) Social Policy Research Grant Competitions Should be Designed and Selected by the Alliances Themselves. Social policy research grants were solicited for research in one of the three areas served by the four alliance grantees. While CSSP had hoped that this would allow for social policy research applicants to become functioning members or partners with the alliance grantees, this turned out not to be the case. While tangentially similar, social policy research projects and alliances passed each other in the night without the multiplier effect that was originally intended. A key exception to this was the SHP and the Ulan-Ude social policy research grantee. SHP is now looking at including Ulan-Ude as a full-time member in the SHP alliance. While seemingly aligned, alliance grantees would have a greater investment in using the research and connecting with the implanting partners if they designed the competition and chose the grantees themselves, with IREX supervision. This would allow for research to better target those questions that alliances can then use in their advocacy efforts. There would be more direct contact between the partners. However, I would recommend that CSSP monitor the situation carefully so as to prevent “advocate” leaders of the alliance from posing misleading and/or redundant questions to researchers. 2) Research Platforms Should Target the Grantees More Directly It is no doubt that HSE has been a success in year two in serving not only the program grantees, but a much wider NGO/research audience as well. After long discussion with HSE leadership, a careful examination of Socpolitika.ru and discussions with the grantees, CSSP might have purchased a larger service than it will need in the future. Now that Socpolitika.ru is up and running, the maintenance costs will be minimal. The majority of existing research is already available on the site and new publications will trickle in at a rate that does not warrant such extensive funding. IREX has rightly trimmed funding for HSE by two thirds, which will allow for HSE to maintain the Socpolitika.ru website, but also allow for HSE staff to concentrate on providing assistance to the new round of alliance grantees to be chosen in November. Instead of supporting the growth of Socpolitika.ru, funds can be more effectively used by targeting money toward partnership between new alliance grantees and HSE, so that HSE can teach them how to identify useful research, make partnerships with research institutions, and create their own web-based storehouses of research to assist their alliance members. 12 3) Tailored Trainings and/or Mentorships for Applied Research Training As noted above, the advocacy-prone alliance members that attended the Stellit training actually found them useful. However, it might not be the most efficient use of funds to continue the Stellit training seminars for such a niche market of grantees. Rather, because social policy research is such a challenging component and targets a limited number of alliance members, CSSP might be wiser to shift from trainings to mentorship as a form of technical assistance. Based upon CSSP staff opinion, SHP’s understanding of the policy process and the need for applied research increased dramatically as a result of mentorship by Professor Elena Shomina, a well-respected academic in the housing field with a keen grasp on the policy process. In Years Three and Four, CSSP should attempt to replicate this mentorship program amongst other alliance grantees. CSSP staff have excellent connections in Moscow and the regions and should be well-placed to identify and recruit high-profile specialists to assist grantees in mapping and traversing the social policy landscape. If these mentors are Moscow-based, they can provide critical insight into the federal policy-making system to which regional grantees probably have limited access. The final argument for using mentorship is that it is more economical than large trainings, which is a serious consideration, given the financial constraints of Year Three. Component 3: NGO Government Working Groups CSSP’s final component aimed to create new forums for government and NGO dialogue on pertinent social policy issues affecting citizen constituencies. The original program design hoped to fund local, regional, and national coalitions to engage legislative and executive authorities at all three levels. Due to the changing landscape and political blockades at the federal level, the second year NGO/government working group grants targeted only the local and regional levels. This component sought to support advocacy initiatives, pure and simple, in all their different forms and manifestations. The key benchmark for the success/failure of these initiatives rested upon the ability of NGOs to: a) Address an issue that was on the government agenda; b) Affect changes to specific policies; c) Monitor the implementation of these changes. Although NGO/Government working group grants only lasted seven months at most, they proved by far the most effective mechanisms for affecting immediate policy improvements. The striking variety of mechanisms used posed the most interesting of contexts. Below is a short resume of the three NGO/government working groups that were funded under Year One of the CSSP. All-Russian Organization of People with Multiple Sclerosis A first year alliance grantee, the All-Russian Organization of People with Multiple Sclerosis (further AOMS) began with the merger of seven like-minded Multiple Sclerosis (MS) affinity groups from across Russia. Although the groups had had informal ties before, the CSSP funding allowed them to increase communication and identify specific policies that needed immediate attention. Led by a charismatic doctor who had long worked with MS patients, the organization began an outreach campaign to increase its membership from seven affinity groups to over twenty in the course of just one year. 13 After careful debate, AOMS identified two specific policy issues which most drastically affected the MS population. The first issue demanded that the list of government-approved MS pharmaceuticals be updated to include the most modern medicines available in the west. The second issue demanded that more funds be allocated to MS diagnosis and treatment. With an estimated 100,000 MS sufferers in Russia, AOMS found that it had a powerful ally with deep pockets in the drug companies that could help lobby federal legislators and health authorities. Without going into details as to the specific lobbying mechanisms, in the first year AOMS was able to get eight new drugs approved by the health authority protocols for use across Russia. In Year Two, long after the CSSP grant had ended, AOMS was instrumental in lobbying for a ten-fold increase the federal budget for MS screening and treatment. With powerful allies and an established network, AOMS is now set to increase its regional network to forty-eight cities across Russia and you can rest assured that each of these affiliates will be charged with monitoring that all the money allocated for MS treatment is spent appropriately. Nevsky Research Center, St. Petersburg Unlike the AOMS, the Nevsky Research Center (NRC) targeted its efforts at solving a completely local problem, specifically the administrative reforms necessary at the municipal level for effective local self governance. The need for standard normative legislation and regulation in local self￾governance was a topic of hot debate and urgency for all 111 municipal administrations in the St. Petersburg region. The federal law on local self-governance was to take effect on January 1, 2007 and required that municipalities develop systems to cope with new responsibilities. The NRC’s leadership is comprised of former municipal deputies and policy-makers who saw the trouble coming on the horizon as all 111 municipal authorities in the region were running around like chickens with their heads cut off. Working with the executive committee of the Association of Municipalities of St. Petersburg, NRC designed a communication mechanism through which it could create a communication loop from its consortium of NGOs and citizen groups and the 111 municipalities in the region. Tackling thirteen different areas of administrative reform from urban planning to landscape architecture to competitive procurement of government services, the NRC collected a brain trust of over fourteen different organizations, each specifically targeted to one specific issue. The NRC, as the coordinating council, used its internal analytical capacity to identify reform challenges and then used its extensive network of government and NGO contacts to identify and engage local citizen groups and community-based organizations to help provide expertise and recommendations on public administration. Over the course of just six months, the NRC coalition was able to draft a comprehensive list of administrative regulations and reform necessary for public administration of government funds under the new law on self-governance. In the six months after CSSP funding ended, NRC was able to present these articles and regulations to the Association of Municipalities of the St. Petersburg Region. In the last meeting of this Association, the NRC’s recommendations were adopted by at least half of the municipalities of the region, although an exact count is currently unavailable. Over the course of its $7,000, six-month grant, the NRC was able to mobilize NGOs and citizen groups from St. Petersburg to provide input into the regulatory framework for new local self￾governance procedures. According to the NRC, the success lies in two achievements. First, the 111 municipalities in St. Petersburg have either adopted a framework for local self-governance reform or 14 have a template available to do so. Second, this is the first time that NGOs and civic groups have had such a major impact on the formulation of social policy. Unfortunately, despite continued lobbying after the grant funding wore out, NRC lacks the funds to monitor how the newly adopted regulatory framework is implemented and to recommend changes for continued improvement. Center for Civic Education and Human Right, Perm The Perm Regional Committee on Education has had a dormant, but standing Civic Education Advisory Board which has met infrequently over the last eight years. In 2006, realizing that there was a serious deficit of citizen experience in democratic electoral politics, one of the five regular members at the Civic Education Advisory Board, the Center for Civic Education and Human Rights (CCEHR) decided to search for investment to reinvigorate political investment in civic education. Applying for a CSSP NGO/Government Working Group grant, CCEHR was able to pull together a wealth of research on the benefits of civic education, expand the standing Advisory Board to include nine local education and human rights NGOs, and stimulate seven advisory board meetings over the 4.5 months of the grant. The key factor to their success, however, came when CCEHR reached access to the regional Department of Politics, which had been charged by the governor to develop a new regional program “For the Development of Political Life in Perm” to replace the program expiring in 2006. Using this access point, CCEHR and the eight other NGOs drafted a fifteen-point draft program for activities to boost civic awareness in both students and adults. Activities included teacher trainings for civics education, trainings for local election committees and monitors, and trainings on mobilization for local NGOs. Although the CSSP working group grant lasted only through July 2006, CCEHR and the rest of the Advisory Board successfully continued their lobbying through the fall legislative session and secured a five-year, thirty million ruble budget line item supporting civic education activities. Unfortunately, while the money became available on January 1, 2007, CCEHR and its colleagues have been unable to manage how this money has been spent as the local education, social welfare, political, and other departments have been pit-bulling for their own funds. Very little of the allocated funds have been given to NGOs for civic education activities. More worrying to the CCEHR is whether any of the allocated funds are going to fund party activities leading up to the December 2007 and March 2008 elections, and if so for which parties. One of the organizers noted that the success of the Civic Education program has prompted the Department of Politics to develop a new program, The Patriotic Development of the Perm Region 2007 – 2011. Commentary and Discussion Reviewing the indicators, program reports, and interviews, the NGO/Government working groups appear to be on average the most effective mechanism for affecting immediate change in local and regional politics based upon the capacity and connections of local organizations. Working groups have shown a great diversity of advocacy techniques from the use of personal relationships (Nevsky Research Center) to the leveraging of corporate sponsorship (AOMS). The $12,500 in grant funding is fairly paltry by current Russian standards, but has returned 100-fold results in terms of increased government funding for working group initiatives. The main challenge lies in the scaling up and systematizing of working group activities to harmonize them with the other two grant programs. Below are two recommendations to achieve this. 15 1) Conduct the Working Group Grants Competitions under the Umbrella of the Larger Alliance Grantees As with the Social Policy Development grants, the NGO/Government Working Group grants were originally designed to feed into larger movements. Unfortunately, due to the lack of successful larger movements, NGO/Government Working Group grants have been given out piecemeal to worthy causes that fall under the general guidelines of health, education, and housing reform. While having more immediate effect than the Social Policy Grants, working group grants have suffered from the same phenomena of “ships passing in the night” with the larger alliance grantees (again, with the exception of Ulan-Ude and SHP). To bring CSSP investment into greater harmony, the evaluation recommends that Alliance grantees be brought in on the Working Group grant design and selection. This should strengthen the link between Alliances and Working Groups and allow for Alliances to expand their advocacy efforts into new geographic regions. Making the Alliance grantees masters of the Working Group competition poses one significant risk. If anything, previous working groups have been known for their diversity of lobbying and advocacy techniques from public hearings, personal connections, media outreach, corporate pressure, and street demonstrations. The point of the Working Group competitions is not simply to fund replication of Alliance advocacy efforts, but to feed into their advocacy efforts with new methods and instruments for lobbying. CSSP should be careful during the design and selection processes not to allow the Alliances to expand their membership roles with miniature clones, but rather to engage like-minded groups with fresh ideas and enthusiasm. 2) Working Groups Need Mentorship and Technical Assistance Just as Much as Alliances As can be seen from the descriptions above, while Working Groups were effective at affecting rapid change, they were less adept at promoting well-considered change. In the example of the AOMS, only by virtue of the deep pockets of the drug companies and their financial vested interests will the MS agenda remain on the radar. Again, this extreme example of success can be treated as the exception, rather than the rule. In the other two cases, fantastic policies were put forward, championed, and adopted, but without long-term planning. Nevsky Research Center was able to put forward, and over fifty municipalities adopted, new regulations for local self-governance. Without a built-in mechanism for continued monitoring, the municipalities will be left to themselves when they face challenges in implementing the new regulations. Similarly, the Center for Civic Education and Human Rights turned a $12,500 grant into $1.2 million dollars in funding for civic and political education. This is a tremendous achievement that risks a derailment as there were no mechanisms for governance and accountability built into these funds. Because of the quick-reaction nature of the projects, NGO/government working groups should be provided with immediate attention in program planning follow-through. More often than not, a working group will hit two of the criteria for long-term success. For example, Nevsky Research Center was able to address an issue that was on the local governments’ agenda and affect immediate and tangible change. However, without built-in monitoring and vigilance, the policy changes are at risk of questionable implementation. Flexible technical assistance plans and one-on-one mentorship, either in combinations with Alliance grantees or in isolation, would help identify and overcome these issues in advance. 16 Section IV: Conclusions and Recommendations The recommendations provided in this section are not intended to evaluate the success or failure of the first two years of the CSSP. Rather, this section examines the contexts in which the CSSP, its’ partners, and its grantees have succeeded and/or failed in the goals they set for themselves. The recommendations below are broken into two different categories. The first section provides principals for selecting grantees and partners over the final two years of CSSP. The second section provides practical recommendations for grant and program management based upon grantees’ and staff feedback. Recommendations for Grantee and Partner Selection Recommendation #1: All Grant Programs Should Serve the Key Alliances In Year Two, CSSP subordinated the Social Policy Development and NGO/Government Working Group grants to inform the same general areas of the four alliance grantees: housing reform, inclusive education, and access to health care. Despite encouraging Alliance grantees to participate in the selection of grantees and attending the same trainings and seminars, this encouragement was not enough. Projects most often passed by each other without true collaboration or long-term combining of resources. Instead, in Year Three, alliance grantees should participate in both the design and selection of both groups of grantees. This active participation and responsibility should both narrow the field of research topics to be tendered, but also increase alliance ownership over the partnership and results. Further, it encourages the alliances to begin their own thought processes on research and partnership. This also encourages them to increase their ranks. Simply put, the ties between these three groups need to be much tighter and this can only be achieved by subordinating other grant programs to the needs of the alliances. Associations and Alliances Social Policy Research Grants NGO Working Groups 17 Recommendation #2: Alliances Should Be Led by Activists/Advocates, Not Think Tanks or Service NGOs The first two years of the CSSP have demonstrated that well-intentioned, energetic people can have an immediate impact on social policy at the local and regional level. And this impact has been and can be significant. CSSP has a wealth of experience under its belt and has already created one high-impact, sustainable coalition in SHP. The success of this coalition is due to two factors. First, it is led by a group of activists who have been working in the area of housing and administrative reform for the past several years. The members of the coalition have identified like-minded colleagues with whom to join forces and who share a common set of obstacles, always a good rallying point. It is these activists who will continue to drive the sustainability of this coalition in the future. Had the Moscow-based Perspektiva been the initiator, the inclusive education coalition may have turned out to be much more sustainable and effective. Second, true activists have an eye for long-term advocacy. While they understand the smaller hurdles and steps that must be taken to achieve immediate goals, they also understand that long-term strategies are required to maintain momentum and push the agenda for their constituencies in the future. The activists focus less on making recommendations, and more on the recommendations being acted upon. Recommendation #3: When Choosing Alliances, Look for Organizations with an Understanding of Democratic and Participatory Governance All of CSSP’s large successes and “spin-off” grantees to date (Association of Lawyers, School for Housing Promotion, and ASI’s Building Trust Program) have one underlying principal in common: the organizations and people involved have an understanding of participatory collaboration. For an alliance to be successful and sustainable, it must feed on new members. New members can be attracted in two ways, either by purchasing their services/allegiance or by demonstrating how their beneficiaries can benefit from participation. The inclusive education alliance demonstrated that collaboration ends when services are purchased. Open participation attracts likeminded colleagues, and tangible value added services keep them. As a disclaimer, the other successful housing alliance in the RFE began as one of the most autocratic structures supported by CSSP. However, with CSSP technical assistance and encouragement, it has become a more participatory organization that is continually attracting new members, including local governments. Again, in the larger picture of grantees, this seems to be the exception rather than the rule. Recommendation #4: Sustainability Can Only Come Through Alliances, Not Social Policy and/or Working Groups One of the key issues with any grant program is the issue of sustainability. As we have seen from the alliance, social policy, and working group grantees, CSSP needs to be most concerned with distinguishing between long-term movement building and outcome-based projects. Outcome-based projects such as the inclusive education alliance and the social policy research grants produce a tangible product with an identifiable expiration date. 18 Alliances/associations on the other hand have an on-going mission to represent the interests of citizens. The most successful alliances to date including AOMS, SHP, and the RFE Housing Alliance have proved financially robust and will continue to expand after USAID funding ends. While all three of these alliances have alternative funding sources, these sources are often for project-based work. CSSP funding might still be necessary to help mature the organizational governance and membership structures over the next year or two. Finally, one shining example of long-term sustainability is the AOMS. By tapping into the commercial sector, the association was able to bring heavy funding to bear for the benefit of their constituency. To date, neither of the two housing alliances has attempted to work with the new commercial management companies. However, given the fact that citizens and management companies have a common third party, government, it is possible that management companies may find cause for investment. Recommendation #5: Replication of Lessons Learned Comes via Expansion of the Associations and Networks Dissemination of lessons learned is actually the key to the entire CSSP, but not in the traditional sense of dissemination of best practices. CSSP’s dissemination comes in the increased membership in alliances and increased participation in alliance activities. If we look at the success of the SHP just in Perm alone we can say that the SHP model has been disseminated to over 300 housing associations across the city. One of the challenges with replication is that association structures, advocacy aims, and instruments vary widely. Given the idiosyncratic nature of association structures and advocacy environments, replication of lessons learned between associations is highly suspect. While associations may have similar standards, statutes, and rules of conduct, they are highly contextual and very idiosyncratic. Rather, CSSP might want to focus on case studies of individual advocacy efforts and instruments. Many previous grantees have lamented the lack of domestic examples of grassroots advocacy and successful lobbying. Recommendations for Program Administration Recommendation #6: Grantee Selection Should be Driven By CSSP Staff While USAID and IREX prize open competition and independent grantee selection as critical features on the competitive procurement process, these processes can derail programs when the expected output is clear. As an innovative program, CSSP is creating associations and structures that have not existed in Russia before and therefore are very challenging to outline for an RFA or selection committee in sufficient detail. Several of the criteria for selecting grantees outlined above can not be evaluated based upon an application, but have to be understood through negotiation and conversation. Having established the four criteria for successful associations (advocacy oriented, membership driven, democratically governed, sustainable) and having watched grantees rise and fall over the last two years, CSSP staff are in a much better position to diagnose potential successful grantees than any selection committee. To maintain some checks and balances against abuse of power, CSSP should employ an independent selection committee to first vet the alliance applicants for pertinence and feasibility. CSSP staff could then work with the resulting short-list of grantees to determine which alliances have the highest chance of success. 19 20 Recommendation #7: CSSP Needs to Be More Assertive With Grantees in Shaping their Alliances IREX and USAID are painfully aware of the difference between a grant and a contract. IREX’s cooperative agreement structure allows it to provide assistance to organizations wishing to contribute under the mandate of CSSP. However, under the CSSP it would be more advantageous to have a more directive style in terms of grantee management and funding. For example, the governance problems with the inclusive education alliance were foreseen, but because of CSSP’s assistance strategy very little could be done to change the nature of the alliance and grant funding to focus more on alliance structure and expansion. If IREX had had the opportunity to work in a more direct way with alliances, many of the pitfalls experienced over the last two years might have been avoided. In negotiating with alliance grantees, CSSP staff should put more emphasis on association structure, governance, and sustainability to avoid the project mentality. The more that CSSP stresses long-term sustainability and governance, the greater a chance that a genuine association will take form. Recommendation #8: Alliances Need to Be Mentored and Coached on an Individual Basis; No General Trainings For the most part, the individualized technical assistance plans designed for the alliances worked well. With the exception of the “honors student” SHP, the major challenge lies in the diverse nature of the technical assistance. CSSP offered specific seminars for particular grantees working in one field, large conferences on social policy for grantees from all fields, and Stellit and other skills trainings for grantees from all fields. This came from the need to provide technical assistance to all grantees, including social policy researchers and working groups, in some sense of equity. This blanket coverage did not prove to be the best use of resources. As an alternative, CSSP would do well to continue the individual technical assistance plans and assign each alliance grantee a staff coach and mentor that could work with the alliance grantees, and their constituent social policy research and working group sub-grantees, to assure that they receive technical assistance particular to the needs of each particular alliance. This would lessen the blur between association building, social policy development, and advocacy by incorporating each of these into one technical assistance program. CSSP should assign each coordinator a technical assistance budget that they may then use to support each of the grantees over the course of the year. All evidence points to the fact that grantee-wide seminars, trainings, and conferences are not the most effective use of funding. CSSP should avoid overlapping grantees from different alliances as much as possible. Grantees viewed the Sochi Exchange Seminar as something of a waste of time and resources with unclear output. The evaluation would also recommend reviewing the need for the Social Policy conference to be held in March. While Socpolitika.ru has been a success in helping previous grantees, it perhaps overshot its mark in terms of scope. CSSP is correct in slimming down and reorienting the HSE program, making it much more targeted toward the new grantees and fulfilling their research and intellectual needs. The benefit of the case management approach also allows for better control of information and reporting within CSSP itself as one staff member will be able to consolidate information regarding a number of grantees working toward one goal.