USAID/ECUADOR This is a redacted version of the Final Performance Evaluation of USAID/Ecuador’s Program to Support Civil Society. August 2014. Prepared independently by Mendez England & Associates. It is an approved official USAID document. Budget contained herein is for illustrative purposes. All policy and procurement sensitive information has been removed. Additional information on the evaluation can be obtained from: Gretchen Ruethling Ecuador Desk Officer LAC Bureau Email: gruethling@usaid.gov Phone: 202-712-5587 EVALUATION FINAL PERFORMANCE EVALUATION OF USAID/ECUADOR’S PROGRAM TO SUPPORT CIVIL SOCIETY [August 2014] This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Mendez England & Associates. Final Performance Evaluation of USAID/Ecuador’s Program to Support Civil Society Final Report August 2014 Prepared under RFTOP No. SOL-518-13-000009 Submitted to: USAID/Ecuador Submitted by: Melanie Reimer, Team Leader Anabella Bruch Michel Rowland Contractor: Mendez England & Associates 4300 Montgomery Avenue, Suite 103 Bethesda, MD 20814 Tel: 301- 652 -4334 www.mendezengland.com DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CONTENTS EXECUTIVE SUMMARY ........................................................................................................ 1 Evaluation Purpose, Design, and Methods ............................................................................................. 1 Findings, Conclusions and Recommendations for each Project ....................................................... 1 GF PROJECT ....................................................................................................................................... 1 PC PROJECT ....................................................................................................................................... 4 General Conclusions based on findings for both projects ................................................................. 7 General Recommendations based on conclusions for both projects ............................................. 7 1.0 EVALUATION PURPOSE & QUESTIONS .............................................................. 1 1.1 Evaluation Purpose............................................................................................................................ 1 1.2 Evaluation Questions ......................................................................................................................... 1 2.0 BACKGROUND ............................................................................................................ 2 2.1 Socio-political Context .................................................................................................................... 2 2.2 Projects’ Description ........................................................................................................................ 3 3.0 EVALUATION METHODS & LIMITATIONS ........................................................... 4 3.1 evaluation Methodology .................................................................................................................. 4 3.1.1 Qualitative Research ........................................................................................................... 5 3.1.2 Quantitative Research ........................................................................................................ 6 3.1.3 Data Analysis ......................................................................................................................... 7 3.1.4 Evaluation Limitations ......................................................................................................... 7 4.0 FINDINGS, CONCLUSIONS and Recommendations .............................................. 8 4.1 GF Project ........................................................................................................................................... 8 4.1.1 Question 1: Advocacy and Watchdog Activity ............................................................. 8 4.1.2 Question 2: Engagement ............................................................................................................... 13 4.1.3 Question 3A: Local Capacity Development ................................................................ 16 4.1.4 Question 3B: Sustainability .............................................................................................. 20 4.1.5 GF Project Recommendations ........................................................................................ 23 4.2 PARTICIPACIÓN CIUDADANA PROJECT ............................................................................ 24 4.2.1 Question 1: Advocacy and Watchdog Activity ........................................................... 24 4.2.2 Question 2: Engagement .................................................................................................. 29 4.2.3 Question 3A: Local Capacity Development ................................................................ 33 4.2.4 Question 3B: Sustainability .............................................................................................. 35 4.2.5 PC Project Recommendations ......................................................................................... 38 5.0 GENERAL CONCLUSIONS ...................................................................................... 40 5.1 Question 1: Advocacy and Watchdog Activity ......................................................... 40 5.2 Question 2: Engagement .................................................................................................. 40 5.3 Question 3A: Local Capacity Development ................................................................ 41 5.4 Question 3B: Sustainability .............................................................................................. 41 6.0 GENERAL RECOMMENDATIONS ............................................................................... 42 6.1 Question 1: Advocacy and Watchdog Activity ........................................................................ 42 6.2 Question 2: Engagement ............................................................................................................... 42 6.3 Question 3A: Local Capacity Development ............................................................................. 42 6.4 Question 3B: Sustainability ........................................................................................................... 43 FIGURES Figure 1: KII and FGD Participants by Gender Figure 2: CSO Views on Effectiveness of Monitoring and Influence Activities Figure 3: CSO Views on Quality of UCG and UTPL Training Courses Figure 4: CSO Views on Management of Grants Process Figure 5: CSO Participant Views on Quality of Coaching Activity Figure 6: CSO Views on Relative Importance of Types of Support from GF Figure 7: CSO Views on Quality of the PC Project “Marcando la Hoja de Ruta” Training Figure 8: CSO Views on Relative Importance of Types of Support from PC Figure 9: CSO Level of Confidence in Information Provided by PC Figure 10: CSO Views on Significance of Risks Facing Their Organizations ANNEXES Annex A: Scope of Work Annex B: Evaluation Matrix Annex C: Sample of Data Collection Tools Annex D: Survey Questionnaire Annex E: Description of CSO Survey Process Annex F: List of Documents Reviewed Annex G: Historical Summary of Participación Ciudadana and Its Collaboration with USAID Annex H: Certification Regarding Terrorist Financing ACRONYMS CBO Community-Based Organization CECIM Corporación Ecuatoriana de Cooperación e Inclusión de las Mujeres (Ecuadorian Corporation for Cooperation and Inclusion of Women) CI Ciudadanía Informada (Informed Citizenry) CNE Consejo Nacional Electoral (National Electoral Council) CONAIE Confederación de Nacionalidades Indígenas del Ecuador (Confederation of Indigenous Nations of Ecuador) CPCCS Consejo de Participación Ciudadana y Control Social (Council of Citizen Participation and Social Control) CSO Civil Society Organization ECOLEX Corporación de Gestión y Derecho Ambiental ECOLEX (ECOLEX Environmental Management and Law Corporation) ET Evaluation Team FEPP Fondo Ecuatoriano Populorum Progressio (Ecuadorian Fund Populorum Progressio) FGD Focus Group Discussion Fundamedios Fundación Andina para la Observación y Estudio de Medios (Andean Foundation for the Observation and Study of the Media) GF Grupo Faro GF Project Strengthening Civil Society Project implemented by Grupo Faro GoE Government of Ecuador ICNL International Center for Non-Profit Law INCLUIR Corporación Ecuatoriana por el Buen Vivir INCLUIR (Ecuadorian Corporation INCLUIR for the Good Life) KII Key Informant Interview LAPOP Latin American Public Opinion Project LEEP Legal Enabling Environment Project ME&A Mendez England & Associates NDI National Democratic Institute NGO Non-Governmental Organization OCA Organizational Capacity Assessment PC Corporación Participación Ciudadana PC Project Strengthening Democracy Project implemented by Participación Ciudadana SETECI Secretaria Técnica de Cooperación Internacional (Technical Secretariat for International Cooperation) SIEL Sistema Integrado de Educación en Línea (Integrated Online Training System) SOW Scope of Work UCG Universidad Casa Grande (Casa Grande University) UN United Nations USAID United States Agency for International Development UTPL Universidad Técnica Particular de Loja (Private Technical University of Loja) 1 EXECUTIVE SUMMARY EVALUATION PURPOSE, DESIGN, AND METHODS This final performance evaluation was designed to assess two projects supported by USAID/Ecuador: 1. Strengthening Civil Society in Ecuador (GF Project), which was implemented by Grupo Faro (GF) and three partners, all Ecuadorian civil society organizations (CSOs). The project was executed from December 2010 to September 2013, with total funding of $1,939,000. The goal was to enhance the capacity of local CSOs to advocate for democracy-related issues and support efforts to influence legislation and policies related to the CSO sector. 2. Strengthening Democracy in Ecuador (PC Project), implemented by Corporación Participación Ciudadana (PC), an Ecuadorian CSO. The period under evaluation for this project was October 2010 to September 2013; however, the activities of that period represented a continuation of long-term USAID funding of the organization that ran from April 2003 to March 2014. Support from USAID during the last three years was approximately $3,000,000. The overall goal was to promote more effective citizen participation in key democratic processes and oversight of democratic institutions. The overall purpose of the evaluation was to: 1. Determine the results of key project interventions, including successes, impediments, and lessons learned; and 2. Document and provide a record of the projects’ results as well as the history of USAID projects with the local CSOs. The evaluation was structured around four main categories of questions that apply to both projects, namely: 1) Advocacy and Oversight; 2) Engagement; 3) Local Capacity Development; and 4) Sustainability. The evaluation was conducted during the period March-May 2014, by a team assembled by Mendez England & Associates (ME&A). The team consisted of two international experts, including the Team Leader and one civil society expert, as well as one local expert and a logistics and data collection assistant. The team used a mixed-methods approach that consisted of document review and desk research, briefing meetings, key informant interviews (KIIs), focus group discussions (FGDs), a survey, and review of data previously collected by the implementing organizations. Overall, the team conducted 91 KIIs and six FGDs with 21 participants in total. Direct data collection activities covered nine municipalities in five provinces of Ecuador. In addition, the team conducted an online survey with CSO beneficiaries of the two projects, from which it received 62 responses from 16 provinces. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS FOR EACH PROJECT GF PROJECT Question 1: Advocacy and Watchdog Activity Findings  Of 24 CSO initiatives supported with small grants by the GF Project, eight included direct advocacy or watchdog activity. The duration (averaging six months) was short, which meant that some projects were not able to complete the entire advocacy process. Several grants reported results on the local level, such as drafting a new municipal ordinance with citizen input and mobilizing citizen-official committees.  CSOs report that training gave them increased confidence and knowledge of laws and skills; however, actual influence on decision makers was difficult to achieve, mainly because space for this work by CSOs was steadily diminishing at the national level and was highly variable at local government levels.  GF Project partners were unable to make much progress in their own advocacy and watchdog work, which was focused at the national level, due to political developments; however, they did mobilize CSOs to speak out on restrictive regulations. Some activities in this area were suspended in 2012 due to budget cuts and the political context. 2 Conclusions  How effective were GF Project’s interventions to strengthen advocacy and watchdog roles of other CSOs? GF Project support was largely effective in increasing confidence of targeted CSOs and their engagement in advocacy and watchdog activity, though tangible influence on government was difficult to achieve since the space for this work by CSOs was extremely limited. In spite of short time frames, some grants laid the foundations for future engagement on policy.  How effective were GF’s direct advocacy and watchdog interventions? The general political environment affecting the project made it difficult for GF and its partners to have notable influence through its advocacy and oversight work during this period, although the project laid foundations for longer term results that may eventually be achieved.  What interventions were most effective [training, work with media, work on legal environment, strategies for engagement with Government of Ecuador (GoE), etc.] and why? Training, coaching, and small grants were all effective to some extent, having had an impact on CSOs’ level of engagement in advocacy and watchdog work. Joint advocacy on the CSO legal framework has also had a modest positive effect by softening proposed regulations. Recommendations 1. Grants for advocacy and watchdog work should be long enough (at least 12 months) to enable CSOs to achieve meaningful results, and if short grants are the only option, then strategies for continuation of activities should be developed in collaboration with grantees. Question 2: Engagement Findings  One of the most significant efforts to build linkages among CSOs was El Colectivo (now the Ecuadorian Confederation of CSOs). Its support on coping with the CSO legal framework is highly valued by members. However, its sustainability is not certain, given the currently challenging environment.  The Confederation launched a collective transparency exercise known as Rendición de Cuentas, through which CSOs voluntarily share information about their activities and financing. There is growing interest among CSOs in this initiative and most informants view it positively.  The grants component strengthened connections among grantee CSOs. Sixty-nine percent of CSO survey respondents indicated that contact with other CSOs had become more frequent since 2010.  Training and grants activities also helped some targeted CSOs to engage more often and more effectively with local government officials on various issues of interest. Conclusions  How successful were GF Project’s interventions at strengthening its engagement and collaborative work with other CSOs, the media, their constituencies, and the government? The GF Project was most successful in improving its engagement with CSOs, in particular through promotion and leadership of the Confederation of CSOs.  What are the most effective and sustainable mechanisms for CSOs to constructively engage with other CSOs, their constituencies, media, and the government? The most effective mechanisms have been the Confederation and the CSO transparency exercise. The Confederation has the potential to become an important actor on the civil society stage and an interlocutor with the national government, if the government is willing to engage in dialogue with them. The CSO transparency exercise is a laudable initiative which could help boost the image of CSOs. These two mechanisms do not need significant funding to continue at current levels but financial resources would be required to expand their scope and have increased impact. Recommendations 1. The Confederation and the collective CSO transparency exercise it has spearheaded should be further supported by the GF Project implementers and donors. However, adjustments are needed to optimize processes and broaden participation and dissemination so they can achieve greater influence and impact. Question 3A: Local Capacity Development Findings  The GF Project carried out various interventions to build the capacity of Ecuadorian CSOs: 3 - University-based intensive courses reached 222 individuals with topics identified through advanced research. Training was rated highly by most informants, although the e-learning methods used in the second course proved less effective than the first course, which relied on a more traditional classroom approach. - Some grants for CSOs included funds to support networking and/or institutional strengthening, which had measurable benefits such as a new website. However, the short duration of grants forced CSOs to rush activities. - Coaching for improved sustainability was valued by targeted CSOs. - Overall, trainings organized by the GF Project were ranked as “good quality” (4 on a scale of 1 to 5) by surveyed participants. - The project developed an interactive map of over 200 CSOs and a directory of over 500 CSOs in Ecuador to help users identify relevant organizations using a variety of criteria. However, awareness of these tools was limited among CSOs. Conclusions  How effective were GF Project’s capacity development interventions with other CSOs? The work of the GF Project to build CSO capacity can be considered fairly effective, according to beneficiaries, grant reports, and post-course assessments. However, there is insufficient information available to assess actual changes in the capacity of targeted CSOs. In spite of solid support to grantees, the short duration of grants made it harder for recipients to “learn by doing”.  Specifically, what interventions in local capacity development were most effective and why? (grants, training, workshops, university training, etc.)? The coaching activity was of a high standard and responded to priority needs of CSOs; along with the first university course organized by the GF Project, it was the most appreciated activity by the participants. The training/information on the CSO legal framework and support to networking through the Confederation also had positive effects, as CSOs helped each-other cope with new regulations and expanded their circles of contacts. Some grants were also effective in building capacity, especially those that included specific resources for institutional strengthening. Recommendations 1. USAID should consider inclusion of hands-on coaching services as part of other civil society strengthening programs, especially when CSOs are facing difficult political or financial circumstances. 2. Grants for advocacy and watchdog work should be long enough (at least 12 months) to enable CSOs to achieve meaningful results, and if short grants are the only option, then strategies for continuation of activities should be developed in collaboration with grantees. Question 3B: Sustainability1 Findings  USAID has provided significant support to strengthen the financial and administrative systems of GF since 2011, contributing to more robust and practical procedures and systems.  Lack of familiarity with USAID budget cycles and processes led to misunderstandings over financial commitments and confusion within the GF consortium, especially during the first year of project implementation.  GF has a high level of technical competency and its internal systems are much improved. Financial sustainability is strong due to a diversified array of funding sources from private foundations to foreign governments, and the wide range of international linkages that GF has developed. Conclusions  Has USAID/Ecuador’s direct assistance to GF succeeded in building sustainable local capacity of the organization? USAID/Ecuador’s support has directly contributed to building sustainable capacity in a number of key areas, including financial and administration systems, and project management.  How far along the road to sustainability is GF? And in what areas do they still need support? GF, as an organization, is very sustainable by most measures. GF has an impressive array of donors and healthy annual income supporting a diverse set of activities. However, its ability to continue to support civil society strengthening is not clear given the current dearth of funding sources for that area. As a 1 The focus of the evaluation’s enquiries was on the likelihood that GF would be able to continue to function without USAID funding, with similar internal capacities and activities. 4 result, GF could benefit from support to identify income generation or funding sources to allow their continued involvement in civil society strengthening and networking. Recommendations 1. Partnership among national CSOs to execute major projects should be encouraged by USAID in countries where it works, in order to capitalize on a range of skills, experience and constituencies. Where implementers (prime and secondary) may lack strong internal systems, USAID should conduct rigorous analyses of capacities, and provide inputs to strengthen capacity of all implementers. 2. USAID should provide early and continuous training and advice on US Government (USG) budget cycles and processes to novice implementers. When reducing budgets, USAID should consider the potentially significant political, operational and financial implications for its partners. 3. The organizational policy of imposing a maximum on the percentage of overall income from a single donor (as GF does) is one that should be carefully considered by other CSOs and USAID. PC PROJECT Question 1: Advocacy and Watchdog Activity Findings  The PC Project strengthened the use at the municipal level of the Silla Vacía (Empty Chair)2 and Rendición de Cuentas (Accountability Forums) as mechanisms for citizen advocacy and watchdog work. The combined approach of providing tailored technical assistance, coaching, and quality publications/ manuals were well received by municipal leaders in targeted areas. Six municipalities adopted ordinances for implementation of Silla Vacía as a result.  A key oversight role for PC has been election monitoring for which it mobilizes thousands of volunteers. PC is the only CSO in Ecuador conducting national level observation; its quick counts over the years were considered technically impeccable and helped increase public confidence in elections.  Support by the PC Project to 30 government-funded entities to comply with new access to information laws on website transparency, including a manual and training of public officials, was highly appreciated and helped achieve high rates of compliance.  The PC Project also monitored government expenditure on mass media; their work raised awareness but also provoked tensions with the government. The method of reporting had to be changed from dollar values to air time to respond to the GoE’s concerns about accuracy. Conclusions  How effective were PC Project’s interventions to strengthen advocacy and watchdog roles of other CSOs? The project’s main focus was on direct advocacy and watchdog work as compared to supporting other CSOs. PC’s support to CSOs was primarily through the Hoja de Ruta training,3 which was effective in increasing skills and involvement in advocacy and watchdog work.  How effective were PC’s direct advocacy and watchdog interventions? Since 2011, with a changing political context, it has been difficult for PC to achieve observable effects with its advocacy—although long-term results cannot be ruled out. Election and referendum monitoring has been highly effective. Other initiatives such as monitoring compliance with the Access to Public Information Law (LOTAIP) have raised awareness and drawn government attention, though effects on policy and practice were not possible to ascertain.  What interventions by PC were most effective (training, work with media, work on legal environment, strategies for engagement with GoE, etc.) and why? PC has played a critical role in affording credibility to election processes in Ecuador, and continues to be well respected for its technical competency in this area. One of the most effective activities was monitoring of compliance with LOTAIP, which combined oversight with technical assistance to build skills and improve implementation. High rates of compliance were achieved by targeted institutions, and an innovative model for CSO-government cooperation was established. PC’s work with local governments to 2 The Silla Vacía mechanism for citizen participation was established by the Ecuadorian Constitution of 2008. It allows for an ordinary citizen to be selected to participate in debates and decision making processes of certain elected government bodies. It has since been adopted in other laws at national and local levels. 3 “Marcando la Hoja de Ruta para la Sociedad Civil” (Creating a Roadmap for Civil Society) was the main PC initiative to strengthen CSO capacity. It is discussed in more detail under Question 3A. 5 increase understanding and use of Accountability Forums has also been effective in a number of targeted municipalities.  What interventions by PC were not successful and why? Advocacy for changes to the CSO legal framework and the Citizen Participation Law, among others, did not achieve observable results. Recommendations 1. USAID should encourage civil society programs in other countries to identify non-controversial ways for CSOs to monitor and provide technical support to government (such as promoting government compliance with new laws and effective engagement with citizens). 2. International and Ecuadorian stakeholders should continue to support the role of skilled national CSOs such as PC in election monitoring, including through support of collaborative efforts by organizations in the same region. 3. PC should seek to channel more activities outside of Quito through CSOs and volunteers based in targeted areas, especially those who are already trained and have shown strong interest in promotion of democratic practices. Question 2: Engagement Findings  The PC Project has shown a strong capacity to mobilize citizen volunteers in campaigns, election monitoring and oversight activities. Participation of volunteers is still high in elections, though the level of engagement in other activities has declined since 2010.  Creation of the Red Pluralista network was initiated by CSO participants in PC training; PC was a member but opted not to take an active leadership role. At that time, the PC Project did not include funding to support this network, though in 2013 it funded small grants to two network members.  PC has effectively used civic campaigns to engage with citizens. The scale and visibility of campaigns have declined since 2012, primarily due to the political context including new legislation that restricts public demonstrations, and limited funding support for these activities.  The PC Project engaged the media through diverse channels and approaches. The digital newspaper “Ciudadanía Informada” has been a key platform with high visitor levels. Conclusions  How successful were PC’s interventions at strengthening their engagement and collaborative work with other CSOs, the media, their constituencies, and the government? The work of PC with government institutions was somewhat successful in this area, especially before 2012. The creation and maintenance of a widespread volunteer network was a significant achievement for PC. Engagement with local government and citizens in targeted areas has improved since 2012 through the project’s work on promotion of citizen participation mechanisms. Although a small CSO network (Red Pluralista) emerged spontaneously from PC’s activities, the network has been fairly dormant – as have various other Ecuadorian CSO networks in recent years.  What are the most effective and sustainable mechanisms for CSOs to constructively engage with other CSOs, their constituencies, media, and the government? The PC Project did not include a significant focus on supporting engagement by CSOs with other stakeholders. However, as a CSO itself, PC established effective mechanisms for providing technical assistance to government bodies (at national and local levels) to promote compliance with transparency laws. PC’s communications are disseminated regularly by the media and, its information – especially regarding elections – is well respected. PC’s civic campaigns and involvement of volunteers in election observation proved to be effective means of engaging citizens in public life. These mechanisms require at least modest levels of funding to be sustainable. Recommendations 1. PC should re-examine the role of volunteers in the organization and decide whether they are only to be engaged for elections, or whether a broader role is still desirable and feasible. If a wider role is envisaged, PC should make a concerted effort to reconnect with volunteers. 2. PC should continue to collaborate with the Red Pluralista’s CSO members, for example by holding joint training workshops and promoting members’ leadership in their communities. 6 Question 3A: Local Capacity Development Findings4  “Marcando la Hoja de Ruta para la Sociedad Civil” (Creating a Roadmap for Civil Society) was the main PC initiative to strengthen CSO capacity; a needs assessment in 2010 was followed by four training modules for 54 CSOs.5 Quality of the training was highly rated and CSO survey data suggests that it may have motivated CSOs to become more active in advocacy and watchdog work.  CSO survey results indicate that, on average, respondents consider that assistance from PC helped their organization improve the most in the following areas: relations with other CSOs, work with local communities, and quality of services. Conclusions  How effective were PC Projects’s capacity development interventions with other CSOs? The PC Project activities to build CSO capacity (notably, the Hoja de Ruta or Roadmap training series) can be considered fairly effective. However, the scope of PC’s work in this area was limited after 2011 due to funding constraints.  Specifically, what interventions in local capacity development were most effective and why? (grants, training, workshops, university training, etc.)? The Hoja de Ruta training series was the most effective activity due to the careful assessment of needs and the quality of the workshops. It helped organizations become more involved and effective in citizen oversight and advocacy activities. The creation of the Red Pluralista at the end of the training showed the strong motivation among some CSOs to extend the reach of the training. Afterwards, a few members replicated parts of the training in their local areas. Recommendations 1. PC should work to expand and activate their existing connections (formal and informal) with CSOs in Ecuador in order to find effective and efficient ways of extending the impact of activities. 2. PC should facilitate collaborative links between volunteers and CSOs based in the same provinces/cities, especially those who have already benefited from training in democratic practices. This strategy could help build capacity on local levels in areas that PC cannot reach directly with reduced resources. Question 3B: Sustainability6 Findings  In terms of core areas of sustainability, PC is held in high regard for its technical competency. Visibility is high for a CSO but is considered to have declined since 2010, particularly outside the major cities. Conclusions  Has USAID/Ecuador’s direct assistance to PC succeeded in building sustainable local capacity of the organization? Yes. The Mission’s support has directly contributed to building sustainable capacity in a number of key areas, including technical capacity, credibility with the public, media and civil society, visibility, and internal systems.  PC would benefit from the following support: 1) strategic planning; 2) facilitation of connections to private sector funders; 3) encouragement to develop joint projects with other organizations based in the region; and 4) knowledge management support to assist in analyzing and capturing key information, systems, methodologies, and strategies used in PC’s key initiatives, in order to mitigate the effects of reduction of staffing on technical capacity. Recommendations 1. PC should seek out suitable national and regional organizations to develop collaborative projects in order to attract funding and bolster PC’s networks of support in the region. 4 Although CSOs were not a primary target group, PC carried out several interventions to increase the capacity of CSOs, citizens, volunteer networks, and youth. 5 This initiative was co-funded by the Embassy of Canada. 6 The focus of the evaluation’s enquiries was on the likelihood that PC would be able to continue to function without USAID funding, with similar internal capacities and activities. 7 2. USAID should support regular strategic planning processes by its partners in other countries, using experienced consultants to ensure that the final product is supported by broad consensus. 3. As PC goes through its current transition, they and USAID should pay careful attention to documentation of knowledge and methods developed and refined by PC over the years, and to secure storage of valuable information. GENERAL CONCLUSIONS BASED ON FINDINGS FOR BOTH PROJECTS Question 1: Advocacy and Watchdog Activity  Space is currently very constrained for advocacy and watchdog work, especially on the national government level. The situation has changed dramatically since the projects were designed in 2009, and it has become more difficult for civil society to have a voice in or to influence public policy, with few exceptions. However, the government is still willing to work with civil society in certain sectors, notably social service delivery. Question 2: Engagement  The GF Project was able to work together with several universities in a very productive fashion although space for such collaboration with universities is also shrinking.  There has been minimal information sharing and coordination of activities between the two projects, which gave rise to reluctance to share information, especially as the funding situation became tighter. Although actual overlap was not observed, this had the effect of opportunities for synergy and shared learning being missed. Question 3A: Local Capacity Development  The major training interventions by both projects were found to have built valuable skills and knowledge in important subject areas. However, either due to funding constraints or not having included replication activities in project plans, replication and dissemination of the training modules and related publications has been minimal, so the impact to date has been limited.  Decree 16 represents a serious threat to CSOs who do not meet the new registration and reporting requirements. PC and GF Projects have both provided valuable concrete assistance to CSOs in this regard.7 Question 3B: Sustainability  Civil society in Ecuador is faced with a combination of threats: diminishing donor interest, widespread wariness about supporting civil society activity, increasingly strict regulation, and reduced opportunities for action and influence. GENERAL RECOMMENDATIONS BASED ON CONCLUSIONS FOR BOTH PROJECTS Question 1: Advocacy and Watchdog Activity 1. Donors and implementers should be realistic in their expectations of advocacy initiatives, since real impact on laws or policy can become apparent years later. 2. Civil society and donors should work to preserve and expand neutral spaces where CSOs can discuss relevant advocacy and watchdog initiatives and establish areas of collaboration. Networks and alliances of a wide variety can be effective; informal and/or temporary groupings and collaborations may be the most feasible and effective option. Question 2: Engagement 1. USAID should recognize and carefully take into account the rivalry that may exist between national organizations, especially in small countries. Although there is no surefire method of overcoming the potential resistance of implementing parties, USAID should establish detailed strategies from the outset that define how relations between projects in related areas will be managed and how information will be shared. Strategies could include regular sector-level roundtable meetings, for example, where various USAID implementers exchange information. 7 PC’s activity in this area has been funded primarily by another donor, although through this project USAID has still provided resources to cover operational expenses of PC. 8 Question 3A: Local Capacity Development 1. Since training packages developed by both projects (GF Project’s university course and PC Project’s Marcando la Hoja de Ruta) have shown positive results, these should be disseminated further by USAID and the implementers so that more CSOs can benefit, within and beyond Ecuador. 2. In future projects with an emphasis on training and/or publications for CSOs, plans and budgets should include the dissemination of learning and information beyond the primary beneficiaries, for example through replication and/or distribution of training materials in hard or soft copies. Question 3B: Sustainability 1. USAID and other donors should manage their expectations with respect to CSO sustainability. Solutions should be sought (such as development of income generation activities) to shore up the long￾term viability of organizations. CROSS-CUTTING RECOMMENDATIONS 1. When dealing with a consortium of either local or international partners, USAID normally conducts all discussions about contract and budget with the prime implementer. However, in cases of significant budget and/or activity adjustments, it is recommended that USAID staff meet directly with both prime and non-prime implementers, in order to explain the adjustments clearly to all parties. This is especially important in a situation where the prime has limited experience in managing consortia and with USAID funding, so as to minimize misunderstandings which may undermine relationships within the consortium. 1 1.0 EVALUATION PURPOSE & QUESTIONS 1.1 EVALUATION PURPOSE This is a report on the final performance evaluation of two projects funded by USAID/Ecuador: 1) Strengthening Democracy in Ecuador Project (referred to throughout the report as the PC Project), implemented by Corporación Participación Ciudadana (PC) from 2003 to 2014; and 2) Strengthening Civil Society in Ecuador Project (referred to throughout the report as the GF Project), implemented by the Grupo Faro Consortium (GF) from 2010 to 2013. The purpose of the evaluation was to: 1. Determine the results of key project interventions, including successes, impediments, and lessons learned; and 2. Document and provide a record of the projects’ results, as well as the history of the USAID projects with these local civil society organizations (CSOs).8 The findings and recommendations of the evaluation are expected to be used by: 1) USAID to inform the design of future civil society and local capacity development programs in other countries where USAID works; and 2) the two award recipient organizations for their internal learning to use to build their own capacity. The full Scope of Work (SOW) of the evaluation is available in Annex A. 1.2 EVALUATION QUESTIONS The evaluation was designed to focus on the following four categories of Evaluation Questions posed by USAID: 1. Advocacy and Oversight 1.1. How effective were PC’s and GF’s interventions to strengthen advocacy and watchdog roles of other CSOs? 1.2. How effective were PC’s and GF’s direct advocacy and watchdog interventions? 1.3. What interventions were most effective [training, work with media, work on legal environment, strategies for engagement with Government of Ecuador (GoE), etc.] and why? 1.4. What interventions were not successful and why? 2. Engagement 2.1. How successful were PC’s and GF’s interventions at strengthening their engagement and collaborative work with other CSOs, the media, their constituencies, and the government? 2.2. What are the most effective and sustainable mechanisms for CSOs to constructively engage with other CSOs, their constituencies, media, and the government? 3. Local Capacity Development and Sustainability 3.1. How effective were PC and GF’s capacity development interventions with other CSOs? 3.2. Specifically, what interventions in local capacity development were most effective and why? (grants, training, workshops, university training, etc.)? 3.3. Has USAID/Ecuador’s direct assistance to PC and GF succeeded in building sustainable local capacity of these two groups? 3.4. How far along the road to sustainability are PC and GF? And in what areas do they still need support? Each of these questions was defined in more detail by the Evaluation Team (ET), including the development of sub-questions for investigation, as shown in the Evaluation Matrix in Annex B. 8 The historical summary of PC’s work and relations with USAID is contained in Annex G. 2 2.0 BACKGROUND 2.1 SOCIO-POLITICAL CONTEXT President Rafael Correa’s rise to power in 2007 ushered in a radically different era, both politically and economically. As a result, the Ecuadorian State - historically weaker than the private sector and civil society - began to grow rapidly, driving the economy and becoming the major developer and implementer of public policies. In the same year, Correa’s party, Alianza País, convened a Constituent Assembly, which met from November 2007 to October 2008 in Montecristi. The new Constitution, drafted by the Constituent Assembly and ratified by popular referendum in September 2008, established institutions and a regulatory framework that significantly changed citizen participation and civil society’s role in Ecuador. Furthermore, it established the Transparency and Accountability Branch (also referred to as the “Fifth Power”) as a national level regulatory entity consisting of the Consejo de Participación Ciudadana y Control Social (Council of Citizen Participation and Social Control or CPCCS), the Public Defender or Ombudsman’s Office, the Comptroller General, and others. Among other functions, the Fifth Power is responsible for formulating transparent public policy, monitoring the use of public resources and government accountability, promoting citizen participation, and preventing and combating corruption.9 The new Constitution and the Law on Citizenship Participation enacted in 2010 established the CPCCS’ leading role in promoting citizen participation. The Law regulates citizen participation mechanisms previously led by civil society and local governments and includes provisions for citizen participation in budget development, formulation, and implementation of operational plans and result analysis. From the beginning of Correa’s administration, many CSO leaders have been recruited by the GoE. Many civil society groups lost strong leaders who were hard to replace, thus weakening their internal structures. The new constitution enshrined issues such as the rights of nature, collective rights, the right to wellbeing, multi-nationalism, and multiculturalism. Additionally, the GoE made innovative proposals such as keeping the Yasuní National Park oil underground and changing the traditional extractive economic model.10 The GoE has moved steadily toward increased regulation of civil society, beginning with Decree 982 issued in March 2008 to regulate CSOs. In June 2013, Correa issued Decree 16.11 The decree regulates aspects of formation, modification, and termination of CSOs, and limits the potential activity and influence of unregistered organizations. In December 2013, the GoE used its power under Decree 16 to forcibly close the Ecuadorian environmental non-governmental organization (NGO), Pachamama Foundation, on the basis that it was "interfering with public policies, undermining the State’s internal security, and affecting peace.”12 Public and private universities have seen significant reductions in their autonomy and independence as a result of the New Organic Law on Higher Education passed in October 2010, which subjected higher education centers to rigorous assessments and restructuring. Restrictive new legislation has also affected other sectors. The Communications Law, enacted in June 2013, has been applied to sanction private media outlets for publishing news and opinion pieces critical of the GoE. The Organic Penal Code, passed in January 2014, criminalized social protest and, according to several interviewees and subject specialists, instituted some penalties which are disproportionate to the acts committed.13 9 Article 206, Constitution of the Republic of Ecuador (2008) 10 http://yasuni-itt.gob.ec/quees.aspx 11http://www.icnl.org/research/library/files/Ecuador/ecuad.pdf 12 Agreement N. 125 of the Ministry of Environment, December 4, 2013. 13 “El nuevo Código Penal legitimará prácticas punitivas arbitrarias”: http://www.elcomercio.com/politica/CodigoPenal￾proyecto-discusion-debate-Correa-Ecuador-Asamblea-leyes-aborto-despenalizacion-delitos_0_1010298972.html 3 Since 2008, the GoE has increasingly sought to "ensure that programs financed by international aid agencies were aligned with the needs of the plans and policies of the government".14 This practice has intensified since 2011 under the Secretariat Técnica de Cooperación Internacional (Technical Secretariat for International Cooperation or SETECI), resulting in foreign aid agencies restricting support to non-governmental recipients of aid and discontinuing certain sensitive areas of cooperation, notably democracy and governance. Several informants also noted that since Ecuador reached a medium level of human development,15 it is no longer considered a priority country for international cooperation, resulting in several aid agencies closing or cutting back programs. This also reduced CSOs’ financial capacity to support citizen participation and continue advocacy and watchdog services. The dramatic and rapid evolution of Ecuador’s political, economic, and social context had profound effects on the two projects being evaluated as well as on the Ecuadorian organizations implementing them. The GoE’s public criticism of USAID’s support to civil society between 2012 and 2013, and the eventual non￾renewal of a bilateral agreement between USAID and the GoE, which ultimately led to the decision to close USAID/Ecuador, also contributed to the challenges faced by both projects. While some of the above changes were already underway at the time the projects were designed and began, others have been more difficult to predict. 2.2 PROJECTS’ DESCRIPTION GF Project was implemented by GF with three partners, all Ecuadorian CSOs: Fundación Esquel, Fondo Ecuatoriano Populorum Progressio (Ecuadorian Fund Populorum Progressio or FEPP), and Fundamedios. The project was executed from December 2010 to September 2013, with total funding of $1,939,000. The project’s stated goal was to enhance the capacity of local CSOs to advocate for democracy-related issues and support efforts to influence legislation and policies related to the CSO sector. GF Project Objectives 1. Strengthen the capacity of CSOs to effectively influence democratic processes, institutions, and adherence to democratic principles. 2. Strengthen the capacity of selected CSOs to develop more effective sustainability efforts and function more transparently. The project included the following key areas of activity:  Support of advocacy, oversight, and capacity building efforts of 24 CSOs through small grant funds.  Strengthen a national CSO network to build CSOs’ capacities and promote collective action.  Develop research studies and publications to enhance the work of the CSO sector.  Identify and disseminate innovative practices within the CSO sector.  Strengthen CSOs organizationally and institutionally through direct coaching and long-term formal courses.  Promote CSO sustainability through hands-on development of sustainability plans. PC Project, with the goal to promote more effective citizen participation in key democratic processes and oversight of democratic institutions, was implemented by PC, an Ecuadorian CSO. Although the evaluation covered October 2010 to September 2013, the activities of that period represented a continuation of long-term USAID funding of the organization that ran from April 2003 to March 2014. Support from USAID during the last three years was approximately $3,000,000. PC Project Objectives 1. Develop domestic observation and parallel vote tabulations of the results of national and local elections. 14 “AGECI controlará la ayuda internacional” http://www.eldiario.ec/noticias-manabi-ecuador/84756-ageci-controlara￾la-ayuda-internacional/ 15 http://www.datosmacro.com/idh/ecuador 4 2. Promote greater transparency by government institutions and elected officials at the national and local level. 3. Provide constructive input to local and national laws and regulations. 4. Promote democratic values and education through activities such as civic campaigns and democratic practices in high-schools. 5. Promote dialogue on issues of public interest. PC Project 2010-2013 specific objectives 1. Strengthen the ability of civil society to participate in and influence public policy. 2. Increase the capacity of selected CSOs in the social audit of key democratic institutions.16 3. Effectively promote democratic values, attitudes, and practices. 4. Support PC’s institutional and financial sustainability. During this period, PC focused on citizen-based oversight of democratic institutions, processes, and public spending; support to transparency and accountability by elected municipal officials; constructive input to enhance national and local legislation; consolidation of civic education programs; and promotion of dialogues between citizens and public officials or among civil society groups. In addition, the PC Project trained other CSOs in oversight and advocacy mechanisms, and promoted a CSO network to advance these issues.17 The project included the following key areas of activity:  Monitoring of and support to 30 public institutions to improve compliance with transparency law requirements.  Monitoring of use of public funds.  Promoting citizens’ understanding on key democratic issues and civic responsibility.  Advocacy for reforms to policies and legislation.  Collaboration with volunteers and other community members to promote citizen participation and oversight.  Support to a range of CSOs to promote oversight and advocacy efforts throughout the country.  Observation of elections and quick count of results. 3.0 EVALUATION METHODS & LIMITATIONS 3.1 EVALUATION METHODOLOGY The evaluation was designed to assess the effectiveness of the two projects in achieving their objectives, and to generate practical recommendations for new directions in general civil society and local capacity development programming supported by USAID. Within that broader scope, the evaluation methods and data collection tools were carefully crafted to respond as a matter of priority to the core Evaluation Questions. Fieldwork was carried out by the Team Leader, Civil Society Expert, and Local Expert in April and May of 2014. The ET used a mixed-methods approach that consisted of document review and desk research, briefing meetings, key informant interviews (KIIs), focus group discussions (FGDs), a survey, and review of data previously collected by the implementers. This approach helped to generate accurate findings about the projects context, the changes effected by the projects over time, and how different project interventions affected beneficiaries and the civil society environment more broadly, either positively or negatively. The ET gathered and analyzed the diverse 16 Note: although the specific objective reads this way, the activities proposed under this objective were primarily direct watchdog and advocacy by PC. 17 This activity was co-funded by the Embassy of Canada. 5 opinions and perceptions of government officials, USAID and project staff, and other national and international stakeholders about whether and how these projects were able to strengthen citizen participation and civil society’s role in Ecuador. The methodology was multidimensional, with the ET relying on a number of techniques that balanced each other: quantitative vs. qualitative data; individual vs. group responses; and face-to-face meetings vs. online survey. These multiple data gathering techniques also helped explore, at various levels, informants’ suggestions for priority interventions in the future. To contribute to the collection of honest viewpoints and reliable information, the ET assured informants of their anonymity. Interviewees had the option of having their names omitted from the list of informants. Because many preferred that option, the ET chose not to include that list in this report. Selection of KII and FGD participants was based on the suggested informant list from USAID, participant/beneficiary lists from various project activities, desk research, and recommendations by experts. Although implementers were asked for suggestions and contact information to facilitate the process of arranging meetings, in all cases, the ET made the final decision on informants. The selection took into account such factors as geographical location, gender, type of support received from the projects (training, funding, mentoring, network facilitation, information, etc.), type of organization (indigenous, women’s, youth, etc.), and affiliation of informant (CSO, government, media, donor, academic, etc.). The ET aimed for a reasonable balance of these factors among KII and FGD informants to ensure a wide range of voices was heard and that no relevant group was left out. When analyzing data, the ET used triangulation in order to increase the reliability and validity of its findings as well as ensure the rigor and quality of the evaluation. However, those cases where relevant information or perspectives emerged from data collection did not fit in with triangulated findings or where there was significant variation among informants on a given subject have been mentioned in the report. Although the evaluation is mainly qualitative, valuable quantitative data was sourced directly from the project implementers, and secondary quantitative data was also collected from other USAID projects and other surveys. 3.1.1 QUALITATIVE RESEARCH The collection of qualitative data was conducted primarily through:  A critical desk-top review of materials related directly to the projects, such as progress reports, annual work plans, monitoring and evaluation plans, indicator achievement tables, and project publications, as well as other materials from various sources, such as civil society studies, media articles, and independent research reports and opinion pieces. A full list of documents and websites consulted is found in Annex F.  Ninety-one KIIs with seven categories of informants including project staff, CSOs, local and national government officials, international agencies, media, beneficiaries and academics/thematic experts. The ET included gender balance and the inclusion of vulnerable groups as part of its selection criteria. KIIs were conducted through protocols and instruments with both standard and customized questions for different types of informants. Relevant direct quotations were captured in notes taken by the interviewer (or note-taker, where available).  Site visits to both PC and GF offices in Quito, nine municipalities (Alausí, Bucay, Colta, Cuenca, Guayaquil, Loja, Quito, Riobamba, and Saraguro), five provinces, and two training workshops organized by PC.18  Six FGDs to obtain additional qualitative information to strengthen analysis and understand the effects of certain activities on specific stakeholder groups. The six FGDs were composed of the following participants: 18 Since USAID funding had ceased at the time of data collection, these workshops were funded by another donor. The evaluators attended with multiple objectives, including: to gain a better understanding of the meaning of Decree 16 for CSOs, to observe the links of PC with local CSOs, to assess the reactions of participants to Decree 16, and to gauge the usefulness of the PC-prepared guide to Decree 16. 6 1. CSO members of Red Pluralista – Loja 2. CSO members of the Ecuadorian Confederation of CSOs – Quito 3. Participants in a university course implemented by GF Project – Loja 4. Community members and leaders – Bucay, Guayas province 5. PC volunteers – Guayas province 6. Parish council members – Colta Figure 1: KII and FGD Participants by Gender Type of Informant Male Female Total Key Informant 54 37 91 Focus Group Participant 15 6 21 Total 69 43 112 Detailed methodology for FGDs  Each FGD was designed for up to eight participants who engaged in an open discussion structured around predetermined questions (discussion guide) led by a moderator. In all cases, the moderator was assisted by a note taker.  Discussion guides included 9-12 questions designed for a 90 minute discussion, starting with broad questions and moving into narrower or key questions. The types of questions used were: opening, introductory, transition, key, and ending. Questions were neutrally worded and neutral probes were used (an example of the guides can be found in Annex C).  The note-taker recorded the key words, expressions, and non-verbal communications of participants. FGD reports included a general summary of participants’ responses to each question and highlighted the range of responses, experiences, etc. 3.1.2 QUANTITATIVE RESEARCH Quantitative research for this evaluation primarily consisted of an online survey of CSOs that benefited from the projects. After analysis of project documents, the ET selected CSOs that participated in the following core project activities: GF Project-organized university courses, GF-funded coaching of CSOs, PC-organized training series “Marcando la Hoja de Ruta” (Creating a Roadmap for Civil Society), sub￾grantees of both projects, and members of two networks supported by the projects Red Pluralista and El Colectivo. The survey targeted 221 CSO representatives and received 62 responses (34 women, 28 men) from 16 provinces. The survey questionnaire is found in Annex D, and a detailed description of the survey process is found in Annex E. Based on the understanding that internet access was high among CSOs that participated actively in the projects, the survey was conducted using the Survey Monkey platform19 to ensure confidentiality, ease of access, and speed in collecting and analyzing data. The link to the survey, accompanied by a letter of introduction, was sent via email - which assured confidentiality of responses and did not require respondents to identify themselves personally - to the targeted CSOs. The email survey was heavily supplemented with follow-up telephone calls to boost response rates, which were found to be necessary due to the poor quality of email contact information and the tendency of recipients to pass over the email message as spam, possibly containing a virus. While many questions were the same for all respondents, the survey was designed with specific questions for the three sub-sets of CSOs – sub-grantees, training participants, and CSO network members – in order to receive more in-depth feedback on those areas of intervention and their effectiveness. In this manner, the ET avoided burdening CSO respondents with repeated requests to complete separate surveys and at the same time facilitated and enhanced the data analysis process since all responses automatically became part of the same data set and could easily be compared. 19 https://www.surveymonkey.net/ 7 The survey instrument was initially developed in English and, after being approved by USAID, was translated, pretested, revised, and implemented. Survey questions were closed-end responses, thus removing the need for translating responses back into English. The questions took into consideration cultural sensitivities and were carefully worded to ensure clarity while avoiding potentially controversial or sensitive terminology. The survey design took into account gender issues and aimed for optimal inclusion of women and women-focused CSOs. 3.1.3 DATA ANALYSIS To conduct the data analysis, the ET used a mix of quantitative and qualitative methods. To increase the rigor and quality of the evaluation, the ET triangulated its findings. Triangulation was made possible by the various methods of data collection used for each major evaluation question (see Evaluation Matrix in Annex B). In addition, the ET asked similar questions to different stakeholders involved in the same issue. The different data collected was analyzed, including through the use of a matrix where each KII response on six core issues was summarized, and the results have been related to each other as appropriate. The ET worked closely with the Mendez England & Associates (ME&A) Data Analyst to prepare, conduct, and analyze the survey results. Data from survey responses was used to confirm and substantiate the findings from document review, KIIs, and FGDs. In addition to examining the descriptive information and scale of CSO responses to different questions, the ET cross-tabulated various responses based on the type of involvement with the projects (grantees vs. non-grantees, for example).20 The results were used to enrich data analysis and allow comparisons between survey responses and data obtained using other methods. The ET also looked for any significant differences in responses based on respondent gender. Once data collection was completed, the ET held three separate briefing meetings with the implementers of each project and with USAID to discuss preliminary findings, conclusions, and recommendations. That process was a valuable opportunity to receive additional information and clarifications from the key project stakeholders and to adjust the analysis as required. 3.1.4 EVALUATION LIMITATIONS Several limitations presented challenges to this evaluation, the most serious of which included: 1. Most people asked to be informants were willing to participate, although many requested anonymity. Taking into account informant’s concerns, a list of informants is not included in this report. Some individuals or institutions declined to participate (or simply did not respond to invitations), therefore, there is a possibility of selection bias, i.e. those who chose to participate might differ from those who did not in terms of their attitudes and perceptions, affiliation with government/non-government structures, socio-demographic characteristics, etc. 2. It proved difficult to secure meetings with national government. This constrained the ET’s ability to fully understand and take into account the viewpoints held within national government circles. 3. Reaching the targeted numbers for FGDs was challenging for various reasons. In most FGDs, the number of participants was either three or four, as compared to the planned six to eight. The ET sought particular individuals to participate in a variety of FGDs in order to investigate some specific aspects of the two projects. In some cases, the number of potential participants with the desired characteristics in a given location was only six or seven, not all of whom were available.21 This factor limited the scope of the exchange of experiences and viewpoints in FGDs, although the discussions were still fruitful. 4. Since a number of questions dealt with issues that took place in the past or changes that took place since the projects began, recall bias may have occurred. On occasion, informants found it difficult to offer opinions on activities or the implementing organizations when the relevant interaction dated back 20 It should be noted that due to the relatively small total sample of 62 responses, and the smaller numbers of respondents that belong to particular sub-sets such as grantees, in some cases the data generated by cross-tabulations may not be statistically significant. 21 In one case, the team worked with the PC volunteer coordinator to invite participants, but later learned that he had not maintained contact with the volunteers in that city, and in fact was no longer the coordinator. 8 more than a year. There was also some confusion of the two projects and their different phases among informants. 5. There is a known tendency among informants to adapt responses to approximate what they perceive as the social norm (halo bias). Informants may not have revealed their sincere opinions in all cases, especially when questions required them to assess the work of colleagues, friends, or allies, or comment on GoE actions. To mitigate this limitation, the ET consistently guaranteed informant confidentiality and anonymity and did not insist on names being included in a list of informants nor in the completed surveys. As well, the ET made every effort to arrange interviews and FGDs in comfortable and private settings, and worked to establish rapport between the interviewer and respondent. 6. Researching the details of two projects carried out over more than three years (as well as the complex context) involved the collection and analysis of a large amount of data from a wide variety of informants and documents. The projects’ various modifications also required intensive investigation. The task was further complicated by the fact that PC has not yet had time to prepare its project’s final report, and the GF’s final report focused primarily on its project’s last year. These factors meant that extra time was required to comb through quarterly reports, workplans, and other documents to extract the necessary information. 7. Given the lack of experience in Ecuador with online surveys, it was anticipated that the response rate for the CSO survey would be relatively low. The ET assured survey recipients of confidentiality and worked to obtain responses using email and telephone follow-ups in order to address this factor. The follow-ups were hampered by the fact that many contacts in the databases provided by implementers were no longer correct. Ultimately, 62 surveys were completed, representing a response rate of 28 percent. While there are many variables that affect survey response rates, this can be considered a good response rate according to expert views sought by the ET. However, it should be noted that the data on some sub-sets of respondents is not statistically significant. 4.0 FINDINGS, CONCLUSIONS AND RECOMMENDATIONS 4.1 GF PROJECT 4.1.1 QUESTION 1: ADVOCACY AND WATCHDOG ACTIVITY 1.1 How effective were GF’s interventions to strengthen advocacy and watchdog roles of other CSOs? 1.2. How effective were GF’s direct advocacy and watchdog interventions? 1.3. What interventions were most effective (training, work with media, work on legal environment, strategies for engagement with GoE, etc.) and why? 1.4. What interventions were not successful and why? 4.1.1.1 Findings Strengthening of CSO advocacy and watchdog roles The GF consortium dedicated considerable efforts to strengthening CSO’s capacities, including in the areas of advocacy and watchdog activities, in accordance with the overall project objective. The project supported a wide range of CSO initiatives through two rounds of small grants of up to $10,000 each. According to project documents, the first round in 2012 was aimed almost entirely at promoting advocacy and citizen participation, including through building the knowledge and skills of CSOs. The second round in 2013 included those themes but broadened the scope of eligible projects to include organizational development of CSOs (including grantees), and encouraged a focus on women and children/youth. Thirteen projects were funded in the first round and 11 in the second (8 projects in all included advocacy or watchdog activities), all lasting an average of six months. Originally intended to last up to 18 months (according to the consortium’s proposal), the grants were cut short by a combination of factors: the slow first year of implementation, uncertainty among partners about roles, and the breakup of the consortium. 9 As a result, the project had two 6-month rounds of small grants. Grant documents and interviews with grantees indicated that a variety of activities related to advocacy and government oversight were implemented, ranging from a multi-stakeholder observation of human trafficking problems on a national level to the development of proposals to local authorities in the Andean canton of Sigchos to combat violence against women and ensure equal access by women’s groups to community facilities. CSOs that benefited from GF Project small grants reported some interesting results from their initiatives. In some cases, the grant funds contributed to an ongoing or wider process so not all results can be attributed to GF Project’s support. A number of grantees mentioned that the implementation period was short, which meant that some activities could not be followed through effectively to ensure that the expected results were achieved. It was found that some projects were able to lay foundations for advocacy but not necessarily carry the process to its logical conclusion within the grant period. The ET received firsthand information from the following grantees regarding their achievements in relation to advocacy and oversight:  Jatun Kawsay Foundation22 in Saraguro used its first grant to build the capacity of various indigenous community groups to promote citizens’ rights, and the second grant to develop concrete proposals for community-based management of natural resources. A new ordinance on natural resource protection in Saraguro communities was drafted in a consultative manner and delivered to the municipal government. Although the mayor at the time did not adopt the ordinance, Jatun Kawsay hopes that the incoming mayor will support their initiative.  Corporación Ecuatoriana por el Buen Vivir (INCLUIR)23 (Ecuadorian Corporation for the Good of Life) worked closely with La Merced Parish in Quito, Pichincha to support the expansion and consolidation of citizen participation mechanisms. The parish council has established six citizen committees to develop solutions to the community’s priority problems. There is a high likelihood, according to INCLUIR, that the example of La Merced will be replicated by two neighboring parish councils that have just joined with La Merced to form a mancomunidad (formal grouping of parishes). According to the CSO survey data, whether or not a CSO had received or applied for a grant from one of the projects appeared to have little or no effect on their frequency of contact with local and national government officials. However, grantees were much more likely to engage in activities that aimed to influence or monitor the actions of the national government compared to those who did not apply for or receive a grant. A full 75 percent of CSO grant recipients said that their activities aimed at influencing and monitoring government were either more effective or much more effective now than in 2010 – a surprising result given the obstacles that such activities face in the current context.24 Figure 2, next page, illustrates this finding for both projects (although only two grants were made by PC). In addition to the grants, support to citizen participation in (and oversight of) government decision-making processes was highlighted in various GF-organized training workshops, according to project documents and interviews with CSOs. Additionally, advocacy was the focus of one entire module of the training courses for CSO leaders organized by the project in collaboration with Universidad Casa Grande (Casa Grande University or UCG) and Universidad Técnica Particular de Loja (Private Technical University of Loja or UTPL) (described in the section on Local Capacity Development). CSO representatives interviewed by the ET repeatedly affirmed that the training improved both their confidence and skills for planning and carrying out advocacy and watchdog work.25 22 http://www.kawsay.org/home.aspx 23 http://www.incluirecuador.org/ 24 Since this data only became available at the end of data collection, it was not possible to explore this response pattern further with CSOs. 25 This data is supported by the course assessment completed by consultants in 2013 which found that 92% of surveyed participants rated their knowledge of advocacy and participation as sufficient or moderately sufficient – this topic received the highest knowledge rating among those surveyed. 10 Two informants mentioned that they used the training materials in their community work to promote citizen watchdog activities, enabling them to more easily pass on what they learned. The results of the CSO survey indicated that participants in the UTPL course had significantly lower levels of engagement in advocacy and monitoring activities (in terms of both frequency and effectiveness) than respondents from the UCG course (and PC’s Marcando la Hoja de Ruta training). In addition, project reports indicate that the GF Project provided funding and advisory support to CSOs to implement at least three mechanisms to monitor local government bodies (veedurías) in different provinces. CSOs that participated in GF-supported activities reported that they had improved knowledge of the legal frameworks for civil society and citizen participation, which helped them comply with legal norms and inform citizens and government officials about their rights and obligations. Some interviewed CSO representatives indicated they had increased their engagement with the government after receiving support from the GF Project. The CSO survey shows that 56 percent of respondents supported by the GF Project have increased the frequency of their advocacy and monitoring activities since 2010, while 45 percent reported being more effective or much more effective.26 Others described difficulties or concerns with pursuing such engagement due to political factors, especially after Decree 16 was promulgated. With respect to influence on government policies and practices, CSOs’ experience was also mixed. A few CSOs stated they found receptivity at local government levels, and thus had some input in decision-making through mechanisms ranging from the “Silla Vacía” to regular roundtables that included government officials and civil society bodies. Others reported that local government leaders, even those (such as accountability forums) that are now obligatory according to the Law on Citizen Participation, were resistant to their attempts for dialogue and to promote the use of watchdog and accountability mechanisms. Those CSOs who tried to engage the national government reported very low levels of receptivity. Generally, CSO respondents supported primarily by the GF Project indicated that due to the project, their relations with local governments had changed by a factor of 2.28 [slightly better on scale of 1 (no improvement) to 5 (great improvement)] and relations with the national government were also slightly better (2.03). However, nearly half of these respondents reported no improvement at all in government relations. 26 Advocacy activity is one of the few areas of questioning where notable differences emerged between male and female survey respondents. Female respondents were much more likely to say their organization conducted no activities aimed at influencing or monitoring government (36% chose this response), while only 18% of males indicated no activities of this nature. Although the sample group for the following question on advocacy effectiveness became much smaller because of the response, 36% of females indicated that their activities had become more effective or much more effective since 2010, while 57% of males selected these two responses. Note: This cross-tabulation combined responses of CSOs working with both projects. 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% Yes No 8.33% 14.29% 16.67% 14.29% 66.67% 57.14% 8.33% 14.29% How has the effectiveness of such activities changed since 2010? Did your CSO receive a grant from either PC or GF since 2010? Figure 2: CSO Views on Effectiveness of Monitoring and Influence Activities Much Less Effective Less Effective No Change More Effective 11 Advocacy by GF Project partners The GF Project selected the following public policy fields in early 2011 as their focal areas for advocacy attention: the Communications Law, regulations under the Access to Public Information Law, and CSO legislation. With respect to the Communications Law, the GF Project carried out an in-depth analysis and submitted proposals to the GoE, but ultimately they were not adopted. The GF Project was ultimately unable to achieve measurable results related to access to information due to political developments that limited their involvement (see the section on Watchdog activity.) The GF Project’s most significant advocacy work focused on the legal framework for CSOs. Due to budget constraints and potential political sensitivities around the involvement of a foreign organization, the GF consortium ultimately decided against having the International Center for Non-Profit Law (ICNL) participate in the project. As a result, GF itself assumed responsibility for activities aimed at improving the legal environment. In 2009, GF was instrumental in rallying CSOs to respond to Decree 982 (with ICNL’s technical assistance), with the formation of El Colectivo a direct result of that process. In early 2011, GF played a leading role in drafting a manifesto by El Colectivo in response to a GoE-drafted proposed law on civil society. (Although not a GF Project activity, the manifesto and other statements laid the foundation for later activities supported by GF). The GF Project then supported El Colectivo’s preparation of a draft law on CSOs in 2011-12 through lengthy consultative processes with other CSOs and analysis of the experiences of other countries. GF and some CSOs believed this was the best way to address the fragile situation of civil society in view of increasing tendencies toward tight government control. However, the draft law was not presented to the government nor publicly circulated at that time; GF staff indicated this was due to divided opinion within civil society and financial constraints. In June 2013, when Decree 16 was promulgated, GF assumed a key role in the consultative process to draft a pronouncement on behalf of Confederation members which proposed the development of CSO legislation based on dialogue between the GoE and CSOs. To date, the GoE has not given any indication of its intention to enter into such dialogue. For the time being, the draft law prepared by the Confederation has been shelved. Although Decree 16 contains many elements that GF (and most other informants) considers highly prejudicial to civil society interests, the decree in fact incorporated some recommendations made by El Colectivo prior to the GF Project. For example, it refers to the duty of the government to promote the development of civil society and its entities, establish grant funds for joint government-CSO projects, and promote transparency among CSOs. Watchdog activities of GF Project partners The GF Project’s watchdog or government oversight activities were focused on establishing two observatories: the Observatory of Public Policy, and the Observatory of Rights and Liberties. The Observatory of Public Policy, led by GF, began in May 2012. First, the documentation and analysis of GF on transparency and access to information was collected. This resulted in the development of a methodology for monitoring the provision of public information by government institutions. Subsequently, the Ombudsman invited GF join a small working group to collaborate in two areas: 1) a consistent methodology for monitoring public body websites for compliance with Article 7 of the Access to Public Information Law (LOTAIP); and 2) recommendations for reforming the legal framework on transparency and access to information. Shortly after the working group was formed, the Ombudsman suspended the activity for political reasons related to changes in key government officials, according to GF staff informants. The Observatory of Rights and Liberties, led by Fundamedios, focused on analyzing events as well as laws and policies related to freedom of expression and communication. It worked in coordination with Fundamedios’ Media Observatory (a separate entity). In April 2012, the Observatory of Rights and Liberties published a detailed analysis of the proposed Law on Communications with the aim of protecting freedom of expression in Ecuador. Due to the increasing political difficulties facing the project implementers in mid-2012, the GF Project reports and personnel indicated that both observatories had to adjust their activities and operations. In 12 the case of the Observatory of Public Policy, activities ceased completely because there was a high risk of political misinterpretation. The Observatory of Rights and Liberties first reduced its public profile, and later suspended operations. 4.1.1.2 Conclusions 1. In spite of significant contextual constraints on advocacy and watchdog work, the GF Project had some positive results, notably achieved by CSOs working at the local government level. Although it was not feasible for most CSO grantees to launch and achieve concrete results with oversight or advocacy activities within short timeframes, especially given the constrained space for interaction with government, a number of grants helped lay foundations for future CSO and citizen engagement on policy issues by strengthening networks and building awareness. While the eventual outcomes of that preparatory work cannot be predicted with any accuracy, the combination of grants and training encouraged targeted CSOs to either get involved or stay involved in advocacy and watchdog work. 2. Efforts spearheaded by GF Project achieved some modest influence on the CSO legal framework, though the current situation is far from ideal and the Confederation has not persuaded the national government to discuss their concerns. The difficulties of the project in making progress on policy influence at national level cannot be attributed to any weakness in project design, methods or strategies. 3. It is not possible to determine whether ICNL’s continued technical assistance to the GF Project would have made a difference in the endeavors to enhance the CSO legal framework in this context, though an international perspective and expert view on best practices can often be helpful in crafting possible solutions and building greater consensus among civil society actors.27 Although local understanding and expertise are essential, based on the ET’s experience, an objective external voice can add significant value in such delicate situations. On the other hand, the overt participation of ICNL could have had negative repercussions given the highly sensitive political context in 2012. In sum: How effective were GF’s interventions to strengthen advocacy and watchdog roles of other CSOs? GF Project support was largely effective in increasing the confidence of targeted CSOs and their engagement in advocacy and watchdog activities, though tangible influence on government was difficult to achieve since the space for this work by CSOs was steadily diminishing at the national level and highly variable (but generally limited) at local government levels. How effective were GF’s direct advocacy and watchdog interventions? The general political environment affecting the project made it difficult for GF and its partners to have notable influence through advocacy and oversight work, which was focused at the national level, although longer term results may eventually be achieved. What interventions were most effective (training, work with media, work on legal environment, strategies for engagement with GoE, etc.) and why? Training, coaching, and small grants were all effective to some extent, having had an impact on CSOs’ level of engagement in advocacy and watchdog work, in spite of the obstacles inherent in the environment. Advocacy on the CSO legal framework, based on development of common positions with other organizations, has also had some positive effect. What interventions were not successful and why? As the project’s planned work on freedom of expression was suspended in 2012, due to the political environment, it was unable to achieve any real impact. Attempts through the Confederation of CSOs to engage with the GoE to discuss the effect of Decree 16 on civil society have not, to date, met with receptiveness. 27 Note: ICNL collaborated closely in 2009 with GF and other CSOs in Ecuador through the USAID-funded Legal Enabling Environment Project (LEEP) providing support to civil society’s response to government initiatives to introduce tight sector regulation. ICNL is currently working with PC to further train Ecuadorian civil society on the national legal framework. 13 4.1.2 Question 2: Engagement 2.1. How successful were GF’s interventions at strengthening their engagement and collaborative work with other CSOs, the media, their constituencies, and the government? 2.2. What are the most effective and sustainable mechanisms for CSOs to constructively engage with other CSOs, their constituencies, media, and the government? 4.1.2.1 Findings Role of GF GF is perceived by most informants as a hybrid entity that combines elements of a think tank (research and analysis functions) with characteristics of an advocacy/capacity building organization. This combination of functions is illustrated by its slogan: “ideas and collective action.” Generally, the ET found that GF’s public profile is low, especially outside of Quito. Among those who know the organization, GF and its leaders are seen as diplomatic and skilled at maintaining civil relations with government officials and international stakeholders. Success of GF Project interventions One of the most significant GF Project-supported efforts to build linkages and strengthen engagement among CSOs was the Ecuadorian Confederation of CSOs. The origins of this initiative (as El Colectivo) are described in detail in the Section 4.1.1 above. (See below under “Mechanisms for CSO engagement” section for additional findings on the Confederation). The GF Project served to expand and strengthen GF’s network of contacts among Ecuadorian CSOs to some extent, especially in provinces outside of Pichincha and Guayas. This was achieved as the Consortium members implemented activities targeting a broad range of CSOs across the country, especially through training and grant initiatives. However, it should be noted that the Ciudadanía Activa brand (the original name of the project) and partners Esquel Foundation and FEPP continue to be more familiar to CSOs and other informants outside of Quito since those two organizations have long track records of working in various parts of the country, and were responsible for executing the CSO training and grants activities in the first two years of the project. The grants component strengthened connections among grantee organizations, whose representatives were brought together for training on grant management, as well as to share experiences at the end of each of the two grant phases. Grantees reported that these events helped them become aware of other CSOs doing interesting work in their own provinces and beyond, and learn from their experiences. Some grantees were motivated to join the Confederation, while others have maintained links with GF without joining the Confederation. When asked, “How has the frequency of your organization’s direct contact with other CSOs changed since 2010?”, 41 out of 59 CSO survey respondents (69 percent) indicated that contact had become more frequent or much more frequent. Among respondents who stated that they had received more support from the GF Project, 72 percent indicated increased frequency of contact while only 6 percent said it had become less frequent. Respondents that received support from both GF and PC were even higher on the scale, with 80 percent mentioning more frequent contact. Respondents said that the networks to which they belong were most useful for obtaining information, receiving training, and connecting with other CSOs, but least useful for gaining access to funding. Most CSO grants also contributed to some extent to establishing new links and strengthening existing links between grantee CSOs and their constituencies or target populations. A number of CSOs supported with grants (direct recipients as well as their beneficiaries) mentioned that the grant project strengthened their interaction with the government, especially at local levels. For example, Corporación Ecuatoriana de Cooperación e Inclusión de las Mujeres (Corporation for Cooperation and Inclusion of Women or CECIM) utilized grant funding to forge strong new bonds with community members in La Merced Parish which continue to this day. At the same time, they significantly boosted their engagement with the parish council, becoming a source of training and information for members. The ET found that engagement, through information sharing and coordination, between implementing CSOs and the GF and PC Projects was minimal notwithstanding similarities between their respective objectives and activity areas. According to informants and ET observations, the willingness of these 14 prominent CSOs to coordinate was limited, partly due to competition (both were involved in bids for the GF Project) and partly owing to later disagreements in strategy over the legal framework for CSOs, which led PC to leave El Colectivo. While USAID did encourage the implementers to meet periodically, there was no system in place to facilitate coordination or direct information exchange, and roundtable meetings of USAID Democracy and Governance Program implementers were infrequent. The ET found little duplication of effort, although both projects prepared and published similar CSO manuals related to Decree 1628 (reflecting each organization’s particular approach) and participation overlap was minimal. The development and delivery of two courses for CSO leaders, in collaboration with UTPL and UCG universities, contributed to stronger relations between Esquel Foundation and the GF consortium on the one hand, and universities on the other hand. While GF itself played a secondary role to Esquel in design and delivery of the training, it actively engaged with at least five universities in Ecuador in the closing months of the project to share the final curriculum and explore future collaboration on similar trainings. Project engagement with the media was minimal. Engagement by the GF Project with regional and international CSOs was limited; the international advisory group that was planned to serve as a sounding board for consortium members as they implemented the project never became active as a group. GF staff indicated that convening the members proved difficult due to their busy schedules, though they did consult with individual members of the group from time to time. Mechanisms for CSO engagement with other actors The Confederation is one of the most significant mechanisms for engagement supported by the GF Project, although its membership of 63 CSOs, some of which are networks whose members are also considered Confederation members, is relatively small considering that there are reportedly more than 45,000 CSOs in Ecuador. Further, membership is heavily concentrated in Quito and Guayaquil. The Confederation’s activities have so far focused on exchange of information and viewpoints on the evolving legal framework (notably Decree 16), for which members have expressed unanimous appreciation in interviews and FGDs. Members especially value the Confederation’s role in connecting a variety of CSOs working in different sectors whose leaders are now realizing their commonalities and how they can help each other cope with difficulties. Some informants criticized the Confederation for trying to speak for all CSOs and for being slow to react to issues such as the closure of Pachamama, an environmental and indigenous CSO. However, Confederation members generally commented that decision-making processes on such issues were consultative and inclusive, and that it was not easy to reach consensus on sensitive topics – which understandably affected the speed with which the body was able to speak out publicly. Members interviewed by the ET believe that the Confederation is playing an important role as a rallying point for CSOs with shared concerns (especially smaller ones who otherwise would be voiceless). Members who participated in FGDs highlighted the additional confidence that belonging to the Confederation gave them as a key benefit, as well as the importance of having a credible national platform for CSOs that crosscuts thematic areas and thus plays an advocacy role with the government on issues affecting the entire civil society sector. The CSO survey data indicates that 36 percent of the respondents who chose the Confederation as the network in which they were most active rated it as very useful or extremely useful. On the other hand, 24 percent considered it not useful. The Confederation has no direct funding and no Secretariat; although some activities were funded by the GF Project, it now survives on volunteer labor and in-kind support from its members. Its sustainability is far from assured due to current challenges in funding civil society initiatives in Ecuador, although members have shown a firm commitment to its existence. Another notable new mechanism for CSO engagement with other sectors of society is the collective transparency exercise, known as Rendición de Cuentas, where CSOs voluntarily share information about activities, beneficiaries, and financing which is then amalgamated in a detailed report and presented publicly. The first exercise was carried out in 2011 by El Colectivo and included 37 CSOs; the second was organized 28 The two publications were: Guía para la Aplicación de Decreto 16 by the PC Project, which was co-funded by ICNL, and Manual Legal para Organizaciones de la Sociedad Civil by the GF Project. 15 by the Confederation in 2012 and involved 102 CSOs.29 In both cases, GF guided the process and channeled funds from the GF Project. Informants reported that there is growing interest among CSOs in this transparency initiative, especially since there has been no negative reaction by the Government to the first two reports. However, several interviewed CSOs did not consider the process to be beneficial for them or for civil society generally. The Government’s reaction has not been easy to measure, but some national level officials attended the second event and a few CSOs reported that their government counterparts seemed to appreciate the effort to improve transparency. Most informants view the process in a positive light and believe that additional transparency by CSOs is a step towards enhanced credibility and social responsibility by the sector. For the third round of transparency reporting in 2014, participating CSOs were asked to pay a fee to cover the costs of the process. Half the required amount was committed at the time of data collection, which suggests that there is enough interest in the activity to sustain it without donor funding (which is also being sought). A number of obstacles have been reported for both of these mechanisms, including: lack of resources to fund Confederation services, hire a coordinator, and recruit new members; and difficulty in drawing in CSOs from outside Quito and Guayaquil. Additionally, some CSOs have simply opted for other strategies and platforms to advance the interests of civil society and their own organizations, rather than joining these mechanisms. 4.1.2.2 Conclusions 1. The Confederation is a promising initiative that has attracted a small but committed core group of members and has the potential to become an important actor on the civil society stage in Ecuador. It clearly has the potential to act as an interlocutor with the national government but success in that area will depend almost entirely on the GoE’s willingness to engage in dialogue with civil society through it – which to date has not been demonstrated. For now, the Confederation has provided an important forum for a diverse set of civil society actors to come together, exchange views, share problems and develop joint strategies on legal frameworks. 2. The CSO transparency exercise is a laudable initiative in response to government criticism of the sector. While still limited in scope, this kind of proactive effort is a useful way for CSOs to boost their collective and individual images with the public and government and demonstrate their value and contribution to society when this has been repeatedly questioned by the government. The methods used for the exercise could, however, be refined further; the current method combines administration and operating costs and reports that they account for more than 40 percent of CSO expenditures. This figure that could readily be misunderstood to mean that this amount goes to “overhead” and not activities, which could have negative repercussions for civil society. 3. The GF Project’s engagement with regional and international CSOs was limited, due to the difficulties with activating the international advisory group and the decision by the consortium to proceed without ICNL’s participation in the project. In sum: How successful were GF’s interventions at strengthening its engagement and collaborative work with other CSOs, the media, their constituencies, and the government? GF was most successful in improving its engagement with CSOs, in particular through promotion and leadership of the Confederation. Interaction with the Government was primarily at the national level; however, relations deteriorated from 2012 onwards due to the political environment. GF had limited engagement with media and the public, in part due to the same political situation. What are the most effective and sustainable mechanisms for CSOs to constructively engage with other CSOs, their constituencies, media, and the Government? The most effective mechanisms used by the GF Project have been the Confederation and the CSO transparency exercise. These mechanisms 29 Participating CSOs were mostly members of the Confederation, but other CSOs also showed interest and were included in the process. 16 do not need significant funding to continue at current levels but financial resources would be required for them to expand in scope and have increased impact. Sustainability may also be jeopardized if strong leadership by GF were not able to continue. 4.1.3 QUESTION 3A: LOCAL CAPACITY DEVELOPMENT 3.1. How effective were GF’s capacity development interventions with other CSOs? 3.2. Specifically, what interventions in local capacity development were most effective and why? (Grants, training, workshops, university training, etc.)? 4.1.3.1 Findings As part of their core mandate under the project, the GF consortium designed and carried out a series of interventions to build the capacity of Ecuadorian CSOs. The most important of those activities are outlined below, together with an analysis of their effectiveness. University-based courses In partnership with UCG in Guayaquil, Esquel Foundation led the design of an innovative intensive course for CSO leaders, the pilot of which was delivered in 2012 to 30 participants using a classroom learning approach. The course was then analyzed, modified, and scaled up in late 2012 for eight modules to be delivered by UTPL using a methodology heavily based on e-learning and other distance learning methods in order to reach a larger number of participants across the country. A total of 192 people participated in the course, although the 2013 post-course assessment found that there was a high level of participant dropout.30 The ET conducted interviews and FGDs with participants in the UTPL course, who were satisfied with the quality and relevance of the course. Some of those interviewed had used the course’s information and materials in their organization’s work. However, informants noted some difficulties with online participation including limited interaction with professors and the strict schedule for participation. GF indicated that the e-learning technology had indeed posed problems for many participants, even those who were already accustomed to computer use. Although the student evaluation process used in the courses was found to be quite rigorous, there was no system for monitoring the use of learning after the course or the effect of the course on the operation of the CSOs in which participants were active. The survey of CSOs indicated that the respondents who received any kind of training from the GF Project rated its importance to their organization at 3.6 on a scale of 1 to 5 (with 3 being moderately important, 4 very important). Training in general was ranked highly by survey respondents and as one of the most important types of support provided to civil society by the GF Project. The survey asked respondents who worked closely with GF to choose what they believed were the project’s two most important services from a list; 53 percent (18 of 34) selected training as one of their two choices. More specifically, among the 34 survey respondents who participated in one of the university courses, 83 percent rated the quality of the UCG training as either good or very good (4 or 5 on a scale of 1 to 5), while 58 percent rated the UTPL course at those levels (see Figure 3, next page). These ratings correspond with the findings of GF post-course assessments, indicating that UCG’s classroom approach was more effective. The analysis of survey responses suggests that course participants’ organizations have engaged more frequently in activities aimed at influencing or monitoring government since 2010 and that they see those activities as more effective. However, the sample size is too small to draw any solid conclusions, since 14 of the 34 respondents who took these courses said their organizations were not involved in this kind of activity at any level of government. GF refined its curriculum and methods again after this second experience and tried to interest several Ecuadorian universities in offering all or part of the course. Those efforts were constrained by end of project funding shortly after the new curriculum was finalized and by the challenges that universities are currently facing under new education sector 30 The assessment report contracted by GF in 2013 indicates that the assessors were unable to reach many participants with the contact information in project files for a rigorous followup regarding the reasons for dropout or effects of the course. This evaluation attempted to include all those who completed the courses in the CSO survey but had similar difficulties in contacting them; only 23 responded to the survey. 17 regulations, as mentioned in the context section. Grants As mentioned under Question 1 on Advocacy, the GF Project supported various CSO initiatives with small grants, several of which focused specifically on networking and institutional strengthening, while the remainder emphasized activities on awareness raising, civic education, participation, and advocacy. All 10 GF Project grantees that responded to the survey rated the grant they received as either very useful or extremely useful to their organization (4 or 5 on scale of 1 to 5), and rated their own grant activities as very or extremely useful. Perhaps owing to the small size and short duration of the grants, only two of 34 respondents that worked with the GF Project (10 of which had received GF grants) selected grants as one of the most important types of support from the GF Project. On the other hand, interviewed grantees unanimously pointed to the grants’ short duration as an obstacle to achieving meaningful results. Nevertheless, the evaluation found evidence that some grants had a modest direct positive effect on the organizational capacity of recipients; according to grantees and GF Project implementers, change was especially marked when specific activities, such as website development, were included in the funding. CSO beneficiaries of some grant activities also saw indirect capacity benefits, for example, eight grassroots organizations were assisted by CECIM to improve their structures and systems in compliance with the law in preparation for seeking official Confederation registration and membership. However, these processes have been slowed and, in some cases, suspended as groups grapple with the implications and costs of registration, and worry about being associated with the Confederation.31 Informants reported that some CSO grantees were adversely affected by political fallout from the “crisis” that affected the consortium and USAID/Ecuador. Two grantees described how certain activities had to be abandoned once the relevant government entities suddenly refused to cooperate further. It is worth noting that USAID and GF were very flexible with grantees in terms of branding activities and did not require the use of USAID branding once the government began to criticize the project. According to document review and grantees, the grants were well managed and closely monitored by the GF consortium during both phases, and especially well supported with mentoring from GF during the second round. Those grantees rated the management of the grants by GF Project as 4.42 (on average) on a scale of 1 to 5. Figure 4, below, shows the breakdown of ratings by percentage of respondents that chose each response. 31 The ET attempted to organize a FGD with representatives of these community organizations to better understand the positive aspects and challenges of the grant-supported project but was advised that it would be difficult to arrange, partly due to the anxiety of those organizations about being associated with USAID and the GF Project implementers. 0% 10% 20% 30% 40% 50% 60% 70% Cursos de la Universidad Casa Grande (Guayaquil) Cursos de la Universidad Técnica Particular de Loja (UTPL) 0% 0% 0% 10.71% 16.67% 32.14% 66.67% 39.29% 16.67% 17.86% How would you rate the quality of the training? What type of training did you receive? Figure 3: CSO Views on Quality of UCG and UTPL Training Courses Very Poor Quality Poor Quality Satisfactory Quality Good Quality Very Good Quality 18 Coaching The GF Project, in recognition of the difficult situation facing many CSOs in securing their futures, contracted experienced consultants to deliver a combination of training and one-on-one coaching to 22 selected organizations with a focus on planning for sustainability and social responsibility. The first round of support in 2012 was focused on analysis and planning and the second round was based on lessons learned from the first phase and included significant support for implementation of plans. Reactions by the nine interviewed CSO beneficiaries was very positive, although the 12 survey respondents that received coaching gave it a moderate endorsement at an average rating of 3.7 (on scale of 1 to 5, with 4 being good quality and 3 being satisfactory) and its relevance a 3.58 (5 being extremely relevant), and its impact as 3.42 (5 being extremely important). Figure 5 shows the breakdown of quality ratings by percentage of respondents that chose each response. The fact that the interviewed CSOs and project staff were unable to point to tangible effects on the financial sustainability of beneficiary organizations needs to be understood in the current Ecuadorian context, which is characterized by severe reductions of civil society funding from international donors and the private sector. More time will be needed to properly assess the effects of the coaching activity, which aimed to build sustainability on more than just financial levels. There was no apparent effect of coaching on the risks that CSO survey respondents perceived for their organizations; of the 12 surveyed CSOs that received coaching, nine (75 percent) identified lack of funding as the most significant risk. 46 percent of other CSO respondents identified lack of funding as their most significant risk. However, it is worth noting that none of the 12 that received coaching pointed to internal problems as their most significant risk, compared to 10 percent of CSOs that had not received coaching. Legal framework The GF Project provided assistance to CSOs to understand and comply with the Ecuadorian legal framework, with particular emphasis by GF in late 2013 after Decree 16 was proclaimed. A legal manual, based on consultations with various CSOs, was already underway at that time so its final form included guidance on Decree 16 and the effect on the regulation of CSO affairs. Activities also included meetings with Confederation members to brief them on the new requirements, discuss their impact, and agree on steps to be taken in response. Confederation members and other CSOs commented that meetings facilitated by GF (both during the final months of the GF Project and subsequently) were very useful in helping them comprehend the changed legal requirements. Although some informants also appreciated the legal manual, most CSO representatives interviewed, including some active members of the Confederation, had not seen the document (in either physical or electronic format). 16.67% 58.33% 25.00% Figure 4: CSO Views on Management of Grants Process How would you rate the grant-giving organization’s management of the grants process, based on your experience of applying for, receiving and reporting on the grant? very bad bad satisfactory good excellent 16.67% 16.67% 50% 16.67% Figure 5: CSO Participant Views on Quality of Coaching Activity How would you rate the coaching activities’ quality? very poor quality poor quality satisfactory quality good quality very good quality 19 Networks As described in the section on Engagement, the GF Project and GF in particular were instrumental in creating the Confederation and promoted the involvement of a cross-section of Ecuadorian CSOs in the network and in the Rendición de Cuentas collective transparency process. The overall effects of these processes on involved CSOs (and on broader civil society) are discussed in that section. Although the project plans included GF establishing an “Information and Communications Center for CSOs” (in collaboration with the Confederation), and some project funding has gone towards equipment and furniture for the Center, to date it has not advanced far beyond preparatory stages.32 The project developed an interactive map33 of over 200 CSOs and a directory34 of over 500 CSOs in Ecuador to help users identify relevant organizations using a variety of criteria. Both are hosted on the Confederation website. At least 10 CSO informants were asked about these resources but none were aware of them, even though some were members of the Confederation, so it was unclear how useful they had been. In general, the GF Project’s capacity building interventions were well researched and based on careful assessment of needs, and activities were adapted to reflect lessons learned through earlier activities. Overall, trainings organized by the GF consortium were ranked as good quality (4 on a scale of 1 to 5) by surveyed participants. CSO survey respondents that were primary beneficiaries of the GF Project chose “advocacy to promote civil society participation in matters of public interest” as an important type of support provided by the GF Project, second only to training. Following closely behind was “sharing of information.” Figure 6, below, shows the breakdown of CSO responses about the most important types of support from GF Project. 4.1.3.2 Conclusions 1. University courses: Participants were generally satisfied with the courses, though the e-learning experience posed difficulties for many learners. It is not possible to judge the overall effect the training may have had on the capacity or operations of the CSOs in which participants were active without extensive additional research. 2. Grants: In spite of the GF consortium’s solid support to grantees, the short duration of grants was a significant constraint on their ability to design and implement projects that could achieve meaningful results. The rushed process also made it harder for recipients to “learn by doing” through reflecting on 32 GF is currently fundraising to establish permanent premises for the Center. 33 http://www.confederacionecuatorianaosc.org/confederacion/mapa-interactivo-osc 34 http://confederacionecuatorianaosc.org/confederacion/content/directorio-georeferenciado-de-organizaciones-de-la￾sociedad-civil 32.4% 52.9% 5.9% 38.2% 20.6% 26.5% 5.9% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% Figure 6: CSO Views on Relative Importance of Types of Support from GF What are the two most important services or types of support provided to civil society by Grupo Faro? 20 and adjusting activities, which would have added value to the grants component. However, the grants did enable some recipient CSOs to broaden their networks of contacts and gain experience in new areas of activity. The few GF grants that included modest resources for institutional strengthening led to specific benefits for some recipient CSOs, such as a new website or a new consensual action plan for a network. These were valuable benefits of a kind that are not often possible to achieve with the project-based funding approach preferred by many donors. 3. Coaching: The GF Project’s coaching methodology was of a high standard and responded to priority needs of targeted CSOs, particularly in the second phase when the activity included hands-on support for implementing plans. This was especially useful to CSOs in an environment with reduced donor funding and confined space for action. The coaching helped CSOs envision a future where they would not depend completely on international donors and identify creative options for internal restructuring and income generation. Although only a limited number of CSOs benefited, the coaching was a positive innovation of the GF Project although in fact it was not part of the original project plans. 4. General: The survey results indicate that, on average, respondents consider that GF and GF Project support helped their organizations improve the most in the following areas: relations with other CSOs, work with local communities, and quality of services. Least improvement was noted in CSOs’ relations with the national government and access to funding. Although the project has developed some potentially useful resources, such as the map/directory of CSOs and the legal manual, their effect appears to be limited, at least in part due to inadequate systems for dissemination of information (but also probably affected by the end-of-project funding). In sum: How effective were GF’s capacity development interventions with other CSOs? The GF Project’s work to build CSO capacity (including university courses, coaching, training on the CSO legal framework, and small grants) can be considered fairly effective based on the perceptions of CSO beneficiaries, grant reports, and post-course assessments. However, there is insufficient information available to assess actual changes in the capacity of targeted CSOs. Specifically, what interventions in local capacity development were most effective and why? (grants, training, workshops, university training, etc.)? The coaching activity and first university course at UCG were the types of activities most appreciated by participants, while the training/information on the legal framework for civil society and support to networking through the Confederation also had positive effects on capacity. Some grants were effective in building capacity. 4.1.4 QUESTION 3B: SUSTAINABILITY 3.3. Has USAID/Ecuador’s direct assistance to GF succeeded in building sustainable local capacity of this group? 3.4. How far along the road to sustainability is GF? And in what areas does it still need support? To assess organizational sustainability, the ET focused on the likelihood that GF could continue to function with internal capacities and principal activities roughly similar to the situation that existed prior to the termination of USAID funding. The ET took into account the following aspects of sustainability: technical competency, credibility (especially with the government and other CSOs), public visibility, current funding and future funding prospects, internal governance, and external linkages. In responding to this question, the ET considered only the situation of GF as a direct recipient of USAID funding and did not analyze the sustainability of GF consortium members. 4.1.4.1 Findings Impact of USAID assistance In terms of the support provided by USAID, several aspects of the GF Project were directly aimed at promoting the sustainability of GF. First and foremost, USAID provided significant support since 2011 to strengthen GF’s financial management and administrative systems and procedures. The assistance took several forms but of primary importance were: 1. Consultant facilitation of two Organizational Capacity Assessment (OCA) processes in September 2012 and August 2013. GF staff participated actively and reported that the assessments were very 21 helpful in identifying priority areas and strategies for improvement. 2. Consultant support to the GF finance team to prepare for annual audits of both the project and the organization. GF and USAID recognized this assistance as having made a significant contribution to obtaining clean audit reports each year since 2011, despite what informants described as a highly disorganized financial and administrative records within the organization. 3. Regular financial reviews and ongoing mentoring by USAID staff. In addition to consultants paid with project funds, members of the Mission’s local capacity development team (especially the finance personnel) provided considerable advice and mentoring support to GF throughout the Project to ensure that procedures and reports were in line not only with USAID rules but also with international accounting standards. GF and USAID staff agreed that although the process was difficult at times, the end result was a much stronger set of internal procedures and systems that have made the organization more able to handle funding conditions from other donors. In addition to the foregoing types of support from USAID, taking a leadership role in the GF Project gave GF a platform from which to establish and strengthen links with CSOs across the country, bolstered GF’s role as a key player in El Colectivo and later the Confederation, and generally raised the organization’s profile among civil society actors – although that effect was limited during the first two years by the consortium approach and use of the brand Ciudadanía Activa. As a first time USAID prime implementer, GF struggled to understand USAID’s budgeting cycles and budget modification processes. USAID and GF staff reported misunderstandings about the possibility of budget reductions during the first year of project implementation. GF and consortium members described a series of discussions with USAID staff over pending budget cuts that led to complex budget and activity revision exercises. When USAID obligated $1.4 million in September 2011 as per the Modification #2 to the GF Project, GF understood that the project budget had been cut and that no further funds would be available for the project. This amount represented only 58 percent of the original budget. These changes coincided with increased political sensitivities and were exacerbated by weaknesses in GF’s financial management staffing and systems. The uncertainty that accompanied these budget and activity revisions contributed to tensions and distrust among consortium members. Some consortium members viewed the budget reductions and requests by USAID to modify activities as a sign of wavering USAID commitment to the project and to Ecuadorian civil society at large. They felt that support was being reduced just at the moment when they (and broader civil society) were at greater risk, although USAID explained to GF that this was part of global cost-cutting and not driven by local considerations. Level of sustainability According to the second OCA in August 2013, GF’s strongest areas of institutional capacity were human resources and financial management, both of which improved significantly since the 2012 assessment. The areas that were still in need of the most attention were administration (especially procurement and logistics) and program/project management. In terms of the core areas of sustainability examined by the evaluation, the following are the key findings: 1. GF’s technical competency appears to be high, based on observations by various informants. Although it should be kept in mind that consultants performed a significant part of the work implemented by GF under the Project, the ET found the documents and analyses produced to be of a high standard. Beneficiaries, such as grantees and training participants, were generally of the view that GF and its consultants were competent and professional. The organization has a strong capacity for innovation, which is demonstrated by the diversity of their current project portfolio, as well as the fact that the Executive Director is part of the Ashoka Fellows35 network and benefits from the innovative discussions on leadership and social entrepreneurship it nurtures. 2. GF’s credibility with the national government was initially high (as demonstrated by the Ombudsman’s invitation to join the working group on the LOTAIP) but has suffered a decline due to the political circumstances that have affected the entire consortium and USAID since 2011-2012. 35 More information available at https://www.ashoka.org/fellows 22 3. With respect to CSOs and the public at large, GF has a low profile and is not well known beyond Quito, although among most Confederation members that the ET consulted, its credibility is high. Most of the 34 CSO survey respondents who received support primarily from the GF Project indicated a high level of confidence in the information it provided, with an average rating of 3.74 on a scale of 1 to 5. With respect to the importance to civil society in Ecuador of GF’s work, the same respondents gave an average rating of 3.76, or very important. Its primary vehicle for financial transparency and accountability to the public is its website, where information on annual income, budget, and donors is posted. Although this data was more complete than the information shared by many Ecuadorian CSOs, it was at least a year out of date in some key areas and donor information was not clearly stated. The misperception that GF was receiving substantial funding from the GoE (discussed in a previous section) underlines the importance of ongoing transparency by CSOs in relation to their financial backers, especially in this type of tense environment.36 4. GF’s visibility with the public, as noted above, is low. Since the organization works in a wide variety of subject areas that do not necessarily fit with the typical role of a think tank, it does not have a clear identity among external stakeholders. 5. With respect to internal systems and governance, GF has made significant progress in recent years. Initially identified as weak areas, the Board of Directors’ involvement and internal decision-making processes have been strengthened since 2012 according to OCA records and informants. Several sources commented on the unusually high rates of staff turnover, some linked to the political difficulties that affected GF and its partners but also to evidence that unclear roles and procedures were contributing factors. Regular performance evaluations are now being implemented and job descriptions are kept updated, which should contribute to increased stability. Internal financial and administrative systems and procedures have also improved greatly according to staff and advisors and should no longer pose difficulties for the organization’s future. The lack of a comprehensive strategic plan was identified as an area for action by the 2012 and 2013 OCAs but it has yet to be developed. At present, the organizational direction is guided by annual operating plans and a research agenda. The OCA process identified the challenge that may be posed by an eventual change of leadership of GF, a concern that was echoed by various interviewees. 6. Insofar as financial sustainability is concerned, GF has an extremely diversified funding base of about 30 donors, primarily consisting of international bodies that range from bilateral and multilateral government agencies to a variety of foundations. GF has a policy that specifies that any single donor cannot fund more than 30 percent of their entire budget. This policy has been respected during the years for which financial information was obtained (2010-2014). GF conducts consulting and research work for various parties such as private foundations and other think tanks which somewhat reduces their dependency on traditional donors and project grants. 7. In addition to the linkages within Ecuador that have been described in the Engagement Section 4.1.2 of this report, GF has extensive international contacts, on the regional level and beyond, which have been cultivated through frequent travel and other contacts by the Executive Director in particular. Those formal and informal networks include a significant number of past and current donors and a wide range of other allies that work in complementary areas. Those contacts have proven valuable in helping GF deal with the challenges of the last few years, for example, by identifying a highly skilled regional consultant to assist with their communications strategy during the crisis period in 2011-12. 4.1.4.2 Conclusions 1. Steady USAID finance staff support, periodic consultant expert assistance funded by the project, and two OCAs processes were instrumental in GF being able to put its finance and administration systems in good order, and have thus made a significant contribution to the organization’s sustainability. 36 This misperception may have been based in part on some confusing information on GF’s website which listed several major projects that appeared to focus on work with the government. In reality, the funds for those activities do not come from the GoE, and the ET has notified GF of this possible source of confusion. 23 2. GF’s determined efforts since it was established to have a diversified funding base have paid off with an impressive array of donors and healthy annual income. Thus, the end of the GF project and closure of USAID/Ecuador will have minimal effect on their overall financial health. However, it is far from certain that they will be able to continue playing a leadership role in strengthening Ecuador’s civil society, given the current dearth of funding sources for that area of activity. To date, GF has not been able to obtain new donor resources to support the Confederation nor the CSO accountability exercise. 3. Misunderstandings based on lack of experience and possible miscommunications about budgets and funding obligations made it difficult for GF to cope well with potential budget reduction scenarios. The uncertainty that accompanied these budget discussions and cuts were key factors in the weakening and eventual dissolution of the consortium. 4. Human resources is a persistent challenge for GF, which struggled to maintain a consistent corps of staff for a variety of reasons, including weak internal management systems, a well-educated and very mobile staffing pattern, and the high-level political criticism of the organization and its partners. This is their primary sustainability challenge, along with ensuring continued strong leadership. 5. Taking into account all of the above findings, the evaluation considers that GF is a more sustainable organization than when the GF Project began, in part owing to USAID’s support. In sum: Has USAID/Ecuador’s direct assistance to GF succeeded in building sustainable local capacity of the organization? Yes, USAID’s support has directly contributed to building sustainable capacity in a number of key areas, including financial and administration systems, and project management. How far along the road to sustainability is GF? And in what areas do they still need support? GF, as an organization, is very sustainable by most measures. They could benefit from support to: 1) improve retention of personnel and mitigate the impact of potential transitions in key leadership positions; and 2) identify income generation or funding sources to allow their continued involvement in civil society strengthening and networking. 4.1.5 GF PROJECT RECOMMENDATIONS The following recommendations are intended to inform future GF programming in Ecuador, as well as future interventions by USAID in other countries. 4.1.5.1 Question 1: Advocacy and Watchdog Activity 1. In the future, grants aimed at supporting advocacy and watchdog work should be of long enough duration to enable CSOs to plan and carry out activities that have time to culminate in meaningful results. While the exact duration will depend on the capacity of grantees and nature of proposed activities (among other factors), at least 12 months is recommended for this type of grant. Where short-term grants are unavoidable, donors, implementers, and grantees should collaboratively seek strategies (and other sources of funding as needed) to support complementary activities that build on the foundation of earlier activities. 2. The GF Project implementers and others (including CSOs, donors, think tanks, academics and other experts) should continue careful monitoring of the CSO legal framework in Ecuador, and seek to develop creative strategies for advocacy to improve the current law and/or alleviate its impact on civil society. Where serious differences about strategy emerge among CSOs, such as the advisability of presenting a draft CSO law, it is important that spaces for open discussion and exchange on those key issues be maintained, in order to minimize the risk that disagreements will weaken the sector.37 4.1.5.2 Question 2: Engagement 1. The Confederation and the collective transparency exercise it has spearheaded should be encouraged 37 In this respect, Grupo Faro and other CSO leaders could consider use of the deliberation methodology developed by Kettering Foundation, among other tools. For more information, see http://kettering.org/periodicals/deliberative￾pedagogy-an-education-that-matters/ 24 with ongoing support by donors and leaders in the CSO community. Adjustments are needed to optimize processes and include more CSOs in order for both efforts to be consolidated and have increased impact. For example, the manner in which CSO expenses are presented in the transparency report should be changed in order to clarify the information that suggests that CSOs are spending unduly large amounts on administration. 4.1.5.3 Question 3A: Local Capacity Development 1. USAID should consider inclusion of hands-on coaching for individual organizations as part of civil society strengthening programs in other countries, especially when CSOs are facing difficult political or financial circumstances. Such coaching could include financial planning and legal advice, depending on the context and local priorities. 4.1.5.4 Question 3B: Sustainability 1. In cases where implementing organizations may not yet have strong internal systems and procedures, USAID should conduct a rigorous pre-award analysis of all project implementers (prime and otherwise) and provide resources for periodic analysis of capacities and for inputs to strengthen institutional capacity in crucial areas. This is especially important in situations where one organization is expected to manage a consortium or partnership. In this case, only the prime implementer was assessed, and then heavily supported by USAID’s local capacity development team and consultants once internal weaknesses came to light during the first year. 2. Partnerships or consortia among capable national CSOs to execute major projects should be encouraged by USAID when appropriate, in spite of the grave difficulties that arose in this situation. Well-structured alliances can capitalize on the skills, experience and constituencies of several organizations to achieve results that would never be possible with a single implementer or with a foreign consortium. However, the allocation of project roles needs to be clear and correspond to the individual capacities of each organization. Having a very capable consortium management staff it is also very important.. 3. USAID should provide early and continuous training and advice in US Government budget cycles and processes to novice prime implementers (and other consortium members, if any) so they can better understand the implications of budget reduction discussions and plan accordingly. 4. The organizational policy of imposing a maximum on the percentage of overall income that can be sourced from a single donor should be carefully considered by other CSOs and USAID. While this may not prove realistic in all cases, such a practice can serve as a constant reminder of the importance of diversified funding sources for CSOs. 5. GF should continue to strengthen its human resources management systems, including consideration of incentives and other mechanisms to ensure that personnel remain with the organization for longer periods of time. In case of internal conflicts, it may be advisable to seek professional mediation or counseling services. In order to enhance management of the organization and all of its internal and external relations, consideration should be given to creating a deputy director position or similar role. The division of leadership responsibilities would promote internal stability and make the organization better able to cope with future leadership changes. 4.2 PARTICIPACIÓN CIUDADANA PROJECT 4.2.1 QUESTION 1: ADVOCACY AND WATCHDOG ACTIVITY 1.1 How effective were PC’s interventions to strengthen advocacy and watchdog roles of other CSOs? 1.2. How effective was PC’s direct advocacy and watchdog interventions? 1.3. What interventions were most effective (training, work with media, work on legal environment, strategies for engagement with Government of Ecuador, etc.) and why? 1.4. What interventions were not successful and why? 4.2.1.1 Findings Strengthening of CSO advocacy and watchdog roles The Silla Vacía and Accountability Forums are mechanisms for citizen and civil society participation in government decision-making and, as such, are ways in which citizens can perform advocacy and watchdog 25 roles. PC has sought with this project to strengthen their use at the municipal level throughout the period under evaluation. Although CSOs do not have specifically allocated roles in these participation mechanisms under the Law on Citizen Participation and were not the primary targets of the PC Project’s work, community-based organizations (CBOs) were involved in some of the Project’s activities. Key PC Project activities included training for municipal officials, citizens and community organizations on how to use the Silla Vacía mechanism, and technical assistance in preparing regulations at the local level.38 As a result of these activities, six municipalities adopted ordinances for the use of Silla Vacía. The PC Project also assisted three local authorities at the provincial level, 18 at the canton level, and five at the parish level, to hold Accountability Forums, after training public officials on their duties to hold forums and coaching them to prepare information. In parallel, training was provided to citizens and community organizations on their rights to information.39 FGD participants valued PC Project’s support and recognized their role in starting these forums. However, the forums’ effectiveness was limited, as informants noted that local government authorities delivered activity reports with limited opportunity for dialogue. Many civil society and other informants commented that the Silla Vacía mechanism had become more bureaucratic since being entrenched in legislation, which has to some extent discouraged its utilization and citizen participation. Although a good number of CBOs and CSOs participated in activities supported by the PC Project, the ET was not able to assess the impact of these activities on their capacity and roles in advocacy and oversight.40 The combined approach of providing tailored technical assistance, coaching, and quality publications/manuals were well received by municipal leaders in targeted areas. Respondents commented on the personalized support provided by the PC Project, which they found more useful than the assistance received from other entities such as the Municipal Association and the CPCCS. It is important to note that several municipal and national government informants with responsibilities for citizen participation mechanisms expressed openness to collaboration with CSOs like PC, and recognized the need among local authorities for more support than the CPCCS is currently able to provide. CSOs’ role in advocacy and watchdog activities was also supported by the PC Project through the design and delivery of the Marcando la Hoja de Ruta training in 2011-12 (co-funded by the Embassy of Canada and described further in the section on Local Capacity Development). The training focused heavily on themes related to advocacy and oversight, which were included in every module. Interviewees highlighted the relevance and importance of these topics as well as the training quality. Of the survey respondents supported primarily by the PC Project, 55 percent have increased the frequency of advocacy and monitoring activity since 2010, and 55 percent reported being more effective or much more effective in that work. Generally, CSO respondents supported primarily by the PC Project, indicated that the project’s support had helped them change their relations with the local government by a factor of 3.14 [moderately better on a scale of 1 (no improvement) to 5 (great improvement)] thanks to support from PC. As for relations with national government, they felt that relations were only slightly better – 25 percent reported no improvement at all, and the average rating was 2.42 (with 2 being slightly better, 3 being moderately better). In the final stage of the PC Project, two CSOs (both members of the Red Pluralista) were given short-term grants of $5,000 each to carry out activities related to citizen oversight and participation. The Federación 38 Targeted municipalities were Latacunga and Cuenca in 2011, Machala, Tulcan, Riobamba, Morona, Cuenca (reformed) in 2012, Pallatanga, Cumandá, Bucay and Santa Lucía in 2013 39 In 2010, this process started in Machala (El Oro province) and continued in Tulcán, Riobamba, Morona and Latacunga. In 2012, target municipalities were Pimampiro, Mira, Pastaza, Santa Lucía, Daule, and Playas. In 2013 Cumanda, Bucay, Santa Lucía and Pallatanga were targeted. Three provincial authorities: El Oro, Carchi and Chimborazo; 18 cantonal authorities: Milagro, Ibarra, Lago Agrio, and Tena, Pimampiro, Mira, Espejo, Tulcán, Latacunga, Chambo, Colta, Pallatanga, Gonzalo Pizarro, Cuyabeno, Loja, Playas, Santa Lucía and Daule; and five parish level authorities: Alluriquin, Puerto Limón, San Isidro, La Libertad, and El Goaltal. 40 Since CSOs were not a specific target group for PC in these activities, no records were available to verify the level of CSO participation, and securing contact details in order to seek more information directly from participating CSOs proved difficult. 26 Clasista de Trabajadores de El Oro implemented the “Strengthening Civil Society Organizations for the Implementation of Mechanisms of Citizen Participation” project, and the Fundación de Ayuda por Internet (FUNDAPI) carried out the “Territorial Networks and Citizenship: the Cantonal Oversight Systems Citizen Participation of the Province Guayas” project.41 Advocacy and watchdog activities by the PC Project The PC Project played an important role in expanding and enhancing the role of citizens as watchdogs in several ways. According to informants, its work in election monitoring is especially well respected and highly visible. In addition, PC is the only CSO in Ecuador that performed national level election observation and quick counts. Interviewees highlighted the importance of these activities to the country’s democracy at a time when the government, in general, and the Consejo Nacional Electoral (National Electoral Council or CNE) in particular, had little credibility. Results reported by PC’s quick counts (in at least six elections, before and during this Project) were considered reliable, technically impeccable, and helped increase confidence in the election processes. These activities served to mobilize thousands of volunteers, mostly youth, some of whom reported that they still value this experience as a first step to involvement in democratic practices. Notwithstanding its long track record of monitoring, the PC Project’s role in the February 2014 local elections was minimized by budget constraints as its final USAID funding was drawing to a close42. Informants described how PC volunteers and other citizens waited for PC’s reports on those polls and wondered why they did not come. The CNE itself has now begun to perform quick counts, adopting some of PC’s processes, and evaluation sources indicated that the CNE no longer sees the need for PC or other CSOs to monitor elections. Another important activity of the PC Project in the area of advocacy and watchdog work was centered on efforts to promote compliance by government institutions with Article 7 of the LOTAIP,43 which specifies a long list of information that government-funded bodies are required to keep updated on their websites. Prior to 2010, 18 public and private institutions were monitored and a manual for monitoring indicators was developed by PC at the request of the Ombudsman’s Office. The manual was validated by the targeted public institutions (and by the Ombudsman) in 2011 for obligatory compliance of all public institutions. In 2010, 12 new national public institutions were added to the combined monitoring and mentoring process led by PC. In terms of quantitative impact, project reports indicate that in January 2011, the average compliance of the 30 institutions was 87.92 percent, but by 2013 most had reached 100 percent compliance.44 On a qualitative level, interviews with relevant staff of these entities highlighted PC’s technical expertise, the importance of knowing their work was being monitored, and the usefulness of the technical manual, all of which helped them improve their website’s compliance. Some expressed a sense of abandonment when the PC Project’s support ended, but stated their commitment to continue the work. Senior government officials at times publicly boasted of the high levels of compliance by these institutions. Much of this initiative was led from PC’s Guayaquil office, where many young volunteers had the opportunity to become involved and learn about the LOTAIP in the process. On the other hand, some informants raised questions regarding the quality of the information on the websites and the commitment to real transparency by government-funded institutions. In another major watchdog initiative, PC has monitored nine TV channels with national coverage in order 41 As of the time of report writing, the ET has insufficient information to assess these activities. 42 USAID support to domestic observation in Ecuador had been mainly focused at national elections. 43 LOTAIP available for download at http://www.ecuadorestrategicoep.gob.ec/images/leytransparencia/LOTAIP.pdf 44 Based on the high rates of compliance achieved by 2012, PC agreed with USAID to shift the focus of LOTAIP support to local government authorities in 2013. At that time, the political situation had changed for USAID and PC, a different Ombudsman had taken office, and there seemed little chance of PC having further impact on national level institutions, although it was hoped that the Ombudsman’s Office would replicate the work of PC based on methods and materials already developed. It is unclear to what extent that occurred, since the Ombudsman has not shown willingness to meet with either PC or the evaluation team. 27 to promote the appropriate use of government funds on mass media.45 A team of three monitors produced monthly reports and PC sent press releases to a wide range of media outlets. A highlight of this work was the process in which the PC Project tracked and publicized detailed information on the official publicity used in the 2011 Constitutional Referendum. According to KIIs conducted by the ET, that process helped build public awareness of the government’s use of public funds during the campaigns. The PC Project’s regular press releases were disseminated widely by the media, which created discomfort in government circles and provoked tensions between them and PC. The organization has defended its legal right to monitor this issue, but had to change its methods from reporting expenditures to reporting minutes of air time in order to ensure accuracy, since it cannot obtain complete data on the (discounted) cost of air time purchased by the government. In terms of direct advocacy, PC had some notable successes in earlier years,46 but during the period evaluated it faced more challenges in its efforts to promote public policies, laws, and regulations or procedures at the national and local levels. For example, the PC Project’s recommendations on the Organic Law on Citizen Participation, on the reform of rules for election for members of the CNE, and on questions to be posed by the Referendum were not included in the final documents approved by the government.47 These difficulties were attributed by informants to the reluctance of the government to recognize the voice of civil society in public policy matters. On the positive side, the PC Project has successfully advocated some aspects of the local regulations on the use of the Silla Vacía and for the adoption of several ordinances at local levels related to accountability forums. The PC Project also tried to carry out oversight of other key entities at the national level but those efforts had little impact. Monitoring the National Assembly was not considered effective by informants since the legislative body is now weak relative to the executive branch. The PC Project sent a permanent oversight delegate to the CPCCS as well, and monitored the process of appointments to the Council. However, the Project’s recommendations were not ultimately adopted by government. 4.2.1.2 Conclusions 1. Over the years of USAID support, PC built an engaged and knowledgeable citizen-base that participated in effective advocacy and watchdog campaigns and initiatives, strengthening democratic values and processes in Ecuador. PC succeeded in placing oversight of and respect for the use of public funds on the radar of the general public, government, donors, and media through its work on monitoring spending on mass media. However, the impact of the activity in terms of public interest has diminished now that reporting has to be done by minutes of air time rather than expenditure. The awareness generated by this initiative also generated negative attention from the government and thus may have impeded the PC Project’s other efforts to advocate for change. 2. The monitoring of compliance with the LOTAIP served as a mechanism for civil society (specifically PC) to engage meaningfully with public sector institutions and to help them achieve high levels of compliance with these new regulations. This initiative was successful in encouraging and assisting these organizations in complying with new transparency regulations, and although longer term impact is hard to predict, motivation to maintain high compliance seems high in at least some of those entities. This activity was especially valuable in a political context where direct interaction by CSOs with the public sector has been significantly reduced; this type of cooperation kept the doors open for PC to work with some national government bodies in a productive fashion. 3. PC has played a critical role in affording credibility to election and referendum processes since its creation in 2002, a time of democratic instability in Ecuador. It continues to be well respected for the technical competency of its work in this area. This involvement has strengthened PC, civil society and the 45 Since 2012, most funds for implementation of this activity were provided by NED. TV stations monitored included: ECUAVISA, Teleamazonas, Gama TV, TC Televisión, RTS, Canal del Estado, RTU, Canal UNO, and Telerama. 46 For details of influence on earlier legislation, refer to Annex G, point 1.5. 47 PC also presented suggestions for the adoption of other laws, such as the Communications Law, the presidential veto on the Law of Citizen Participation and Social Control, the referendum in relation to the Water Act, and the regulations for oversight committees by the Council on Citizen Participation and Social Control (CPCCS) – which took into account nine out of 11 recommendations by PC. 28 government, and ultimately democratic processes and values. PC’s quick counts produced results that were very close to the final official results, and therefore did not give rise to controversy. Some might therefore question the utility of the exercise, especially given that recent elections in Ecuador have been declared largely free and fair. 48 However, having prompt results from a reliable source served to bolster public confidence in elections, since election authorities have generally been slow to announce results. 4. Although the PC Project’s support to the implementation of Silla Vacía and Accountability Forums by targeted local authorities has been welcomed by citizens and officials in most cases, these mechanisms have had mixed results. Some of PC’s strategies for mobilizing participation have been adopted by government bodies; for example, the “Clean Walls, Honest Candidates” campaign49 was replicated in 2013 and 2014 by the CPCCS and CNE. In a few cases, the methods taught by the PC Project have been adopted for ongoing implementation. In others, the authorities have chosen less interactive ways of reporting on their work to the public. The effectiveness of these mechanisms and others - and thus the effectiveness of the project’s support - has depended on the political will of elected officials, understanding of the mechanisms by the government and/or civil society. Although change has been slow and there have been obstacles, there is evidence that the work of the PC Project has made a difference and that there is demand for more activities of this nature. There is a definite role for PC and other CSOs to play in promoting broader and more effective use of these mechanisms across the country, especially in conjunction with local governments. 5. Regardless of the limited results of the PC Project’s advocacy efforts, it is important for civil society to maintain a dialogue with relevant government institutions and remain involved and proactive in the debate and formulation of relevant laws and policies. The ET found no evidence of weakness in the PC Project’s design, technical analysis or strategies for advocacy, but there was no willingness on the other side of the table to give credence to their inputs. On the positive side, PC gained valuable experience in advocating and monitoring institutions and processes created under the new Constitution. In sum: How effective were PC’s interventions to strengthen advocacy and watchdog roles of other CSOs? The project’s main focus was on direct advocacy and watchdog work, as compared to supporting other CSOs. PC’s support to CSOs was primarily through the Hoja de Ruta training, which was effective in increasing skills and involvement in advocacy and watchdog work. How effective were PC’s direct advocacy and watchdog interventions? Since 2011, with a changing political context, it has been challenging for the PC Project to achieve observable effects with its interventions, especially in advocacy - although long-term results cannot be ruled out. Election and referendum monitoring has been highly effective. Other oversight initiatives have raised awareness of important issues and drawn government attention, though effects on policy and practice were not possible to ascertain. What interventions by PC were most effective (training, work with media, work on legal environment, strategies for engagement with GoE, etc.) and why? One of the most effective project activities was monitoring of compliance with the LOTAIP, which was particularly useful due to the combination of oversight with technical assistance to build skills and boost compliance. High rates of compliance were achieved, but perhaps more importantly, an innovative model for CSO-government cooperation was established. At the local level, the PC Project’s work with municipal governments to increase the understanding and practice of Accountability Forums has also been effective in a number of targeted municipalities. What interventions by PC were not successful and why? Advocacy for changes to the CSO legal framework and the Citizen Participation Law, among others, did not achieve observable results. The 48 See for example http://www.state.gov/documents/organization/186722.pdf and http://www.freedomhouse.org/report/freedom-world/2013/ecuador#.U4IpRYVMyOI 49 This civic campaign was organized by PC beginning in 2008 to promote greater transparency and respect for public and private spaces by candidates for public office, and to press for candidates to remove their publicity materials after elections. 29 difficulty did not lie with PC Project’s design, positions or strategies, but rather with the resistance by government to recognize the role of civil society in public policy. Government criticism made it necessary to change the methods for monitoring public spending on media, which has diluted its impact. 4.2.2 QUESTION 2: ENGAGEMENT 2.1. How successful were PC’s interventions at strengthening their engagement and collaborative work with other CSOs, the media, their constituencies, and the government? 2.2. What are the most effective and sustainable mechanisms for CSOs to constructively engage with other CSOs, their constituencies, media, and the government? 4.2.2.1 Findings Role of PC PC has played an important role within Ecuador in building citizen involvement. Several informants noted that PC is a political organization but not a political party and, as such, it contributes to developing citizen skills in democratic processes and educating them on their rights and responsibilities. Its presence and multiple activities have provided thousands of citizens with the opportunity to debate and learn about issues of national and local importance. Informants noted that PC had an enormous convening capacity in its earlier years, and that their annual conventions were massive; this was an uncommon achievement for NGOs in Ecuador. Some informants, especially outside of Quito, noted that with a decrease in civic campaigns and volunteer activity in the last three to four years, PC’s visibility and, to a lesser degree, its credibility in their communities, had decreased.50 Some lamented PC’s transformation from a “citizen movement to becoming just another NGO.” Key informants commented on this shift as a response to the political context that made citizen campaigns a higher political risk for volunteers and the organization. In addition, the transformation reflects the different styles and priorities of PC’s leaders over time, given that the original director was very much a citizen activist and mobilizer, while the current director can be considered a more technical and professional manager. Success of PC Project interventions As described in the Advocacy Section, PC was successful in engaging citizens in its initiatives through an effective grassroots citizen base. PC was able to mobilize citizen volunteers in campaigns, election monitoring, and oversight activities such as compliance with the LOTAIP, as noted previously. Volunteers form an integral part of PC’s organizational structure and have representation on its Board of Directors, according to the organization’s statutes. Volunteers were organized by provincial coordinators (in all 24 provinces), and in the case of Guayas, this structure also included three additional canton-level coordinators.51 Volunteers were offered periodic training (including through the PC Project’s online training platform) and until 2010, each provincial coordinator received modest funding for operating expenses. More recently, PC’s engagement with volunteers has been focused on election monitoring, though some long-time volunteers said they were not informed about PC’s plans regarding the 2014 local elections, which were limited to Azuay province due to funding constraints.52 Although regular involvement in PC activities has declined, at least some volunteers are continuing to promote citizen participation in their communities. A number of former PC volunteers have become civic and political leaders, and several currently hold public office. When interviewed, they credited their commitment and skills in citizen participation to what they learned as PC volunteers. (However, some of these leaders consider that the government has responded to civil society demands, and see a diminished role for CSOs in the country.) 50 As described below under Sustainability, the LAPOP Americas Barometer surveys have shown a slightly increasing level of public confidence in PC, among those who have heard of the organization. 51 Coordinators were provided funds for internet, office space and computer equipment 52 PC did mobilize approximately 10,000 volunteers for the 2013 election, which is a testament to its well developed election monitoring systems and methodologies. Several informants noted that there is a notable difference between long-term engaged volunteers and those that participate in one-time activities. The on-going volunteer network is what is discussed in this section. 30 The Guayaquil PC office, in particular, involved and trained youth in political leadership and democratic values through youth clubs and the “Vive la Democracia” (Live Democracy) initiative.53 PC coordinated with public and private schools to recognize the students’ activities with the PC Project as part of their community service requirements, and coordinated with local universities to recruit students for election monitoring. In examining PC’s current linkages to other sectors and CSOs, the ET found that engagement by PC with other CSOs was most prominent through the “Marcando la Hoja de Ruta” initiative, which included a civil society needs assessment and a training program that led to the creation of the Red Pluralista by CSO participants. This network started with enthusiasm but has not been very active. It recently elected new leadership and, according to network members who participated in the evaluation, there is new hope for increased levels of interaction and activity. From the outset, PC decided that the network should have full autonomy and promoted a horizontal style of organization, especially since PC’s financial resources were diminishing. As a result, the PC Project provided minimal support to the network. The CSO survey data indicates that 58 percent of the 12 respondents that chose the Red Pluralista as the network in which they were most active, rated it as very useful or extremely useful. Only eight percent considered it not useful. As stated earlier, 69 percent of CSO survey respondents said that their contact with other CSOs had become more frequent or much more frequent since 2010. Among respondents who indicated having received more support from the PC Project, 64 percent indicated increased frequency of contact, while 27 percent said it had become less frequent. The significant variation in responses may be related to the fact that the PC Project did not emphasize support to CSOs or networking among them since 2010, although there are other factors that could be at play such as PC’s lower profile in recent years. An examination of PC’s current linkages to networks in Ecuador found them to be relatively weak at present. PC has participated in a number of national level networks, such as the Justice Network, which is now barely active, and the Access Coalition, which is no longer operational. PC was initially part of El Colectivo but left due to differences of opinion over the advisability of proposing an NGO law and the best way to respond to Decree 16. Informants identified some currently active thematic networks, in the environmental sector for example, but few remain active in areas of relevance to PC’s priorities, such as justice and governance. The PC Project engaged the media through diverse channels and approaches, and through the media, engaged a broad cross-section of the population. The main vehicle for media-related activities has been its digital newspaper, Ciudadanía Informada (Informed Citizenry or CI), online since 2004, which has the objective of providing citizens with quality information on topics of public interest and it is one of only five Ecuador-based digital newspapers. PC hired part-time correspondents in five provinces to generate articles for the newspaper that would capture news from various parts of the country and presented the stories from a citizen perspective. PC Project records, based on Google Analytics, show that the site generated a progressive increase of visitors during the 2006 to 2012 period. In 2006, the site received 493,263 page hits and reached 1,360,461, its highest number, in 2012. The site had 14 percent fewer hits in 2013, and data as of March 2014 showed a similar number of hits for this year. Other communication products included frequent press releases and bulletins, which many informants described as well received by the media and widely disseminated. PC is frequently cited by the mass media54 and its information is generally considered credible and relevant by the media and the public, according to KIIs. However, informants outside of Quito noted that media coverage had diminished in the last couple of years. Among CSO respondents to the survey, awareness of information products was low; 65 percent were unaware of the Pulso Ciudadano (Citizen’s Pulse) bulletin55 produced by PC, and 54 53 In 2010, PC worked with 13 schools in Guayas; in 2011, 672 students trained in democratic values, supported 13 student government elections, held 45 workshops and completed 13 citizenship campaigns. In 2012, PC trained 1,212 youth in political leadership, held workshops for 43 youth facilitators and completed a publication on the results. 54 In 2011, PC and PC events were mentioned or PC staff interviewed by media 515 times; in 2012, 453 times and in 2013, on average twice a week (according to data provided by PC staff). Figures include TV, radio and print media. 55 Pulso Ciudadano is a bulletin or newsletter produced by PC to update citizens on developments in the political life of the country, including new or amended legislation, and to share PC’s analysis of developments related to democratic 31 percent were unaware of PC’s website. The most likely reasons are that the PC Project has not focused in recent years on working with CSOs, and that PC has not prioritized high visibility in the current political context. Through the CI branch of its operations, PC also engaged with the media through provision of training on topics such as social networking, new communication laws, and ethics in journalism.56 Initially focused on the PC correspondents, training was expanded to include other journalists and journalism students, which increased the reach of the project as well as provided correspondents and CI with valuable linkages to the professionals and media outlets that disseminated their reports and articles. Informants praised the training received from PC for its quality and the topics covered, as well as for the opportunity to network with other journalists. PC correspondents highlighted the importance of the training offered by PC in their personal and professional development, and the value of training to other journalists in order to interest them in civil society issues. As discussed in the Advocacy Section, PC has maintained its pre-2010 ties and developed new links with certain government institutions at the national level, and in 2013 forged new links with selected local level governments through its watchdog and related activities. Activities by Silla Vacía and Accountability Forums by government bodies to support implementation of access to information obligations have been particularly important in this regard. A PC mechanism that cut across several areas of engagement was Sistema Integrado de Educación en Línea (Online Integrated Training System or SIEL), which was established as a training platform in 2010. The PC Project used this online system to offer training to volunteers in areas such as public access to information, alternative dispute resolution, and citizen participation mechanisms. SIEL was also used to train 70 representatives of organizations being monitored by the PC Project. Over the time in which platform was used for training purposes (2010-2012), a diverse set of subjects were covered in its courses.57 Interviews found a variable level of satisfaction with SIEL; some informants found the topics interesting and the platform user-friendly, while others did not. The ET did not observe any process for assessing the effectiveness of this training mechanism or satisfaction of users. The SIEL website was no longer active at the time of the evaluation due to the pending termination of USAID funding. Mechanisms for CSO engagement with other actors Over the years PC has led various national civic campaigns with high levels of citizen interest and participation. The “Being on Time”58 and “Clean Walls” campaigns were mentioned repeatedly by informants as examples of successful PC activities, highlighting the effectiveness of these mechanisms in constructively engaging citizens, especially youth, in issues of public concern. Most recently (in 2010, 2011 and 2012), the Ojo Ciudadano (Citizen Eye) campaign of the PC Project provided training to citizens in several topics, including tolerance, transparency, freedom, justice, and security. Some respondents and the PC Project technical staff noted that the current political context and new legislation that restricts public demonstrations (and in fact, any activity that may “disrupt public order”59) make this type of activity more difficult for CSOs to undertake effectively. As a result, PC decided in 2012 to reduce the scale and visibility of its civic campaigns. In 2013, the PC Project supported small-scale local civic awareness activities as part governance. Examples can be seen on the website at http://www.participacionciudadana.org/pc10/index.php?option=com_virtuemart&page=shop.browse&category_id=10& Itemid=75 56 PC offered five workshops for 183 journalists in 2011; five in 2012 with 200 participants; and several workshops on journalism and elections in 2013. The total number trained between 2010 and 2013 was 758, in Azuay, Imbabura, Guayas, Manabí y Pichincha provinces. 57 (2010) Political Advocacy, Accountability, Political Communication and Leadership, Democracy and Communication. (2011) On-line Studies and use of the platform, Conflict Resolution, Research. (2012) Project Development (Basic and Intermediate). 111 participants. Political and Social Use of New Communication Technologies (Basic and Intermediate). 12 participants. 58 This popular campaign led by PC in 2003 raised awareness among citizens of the need to respect the time of others and of the importance of meeting their commitments on time. 59 For example, the Penal Code of Ecuador (Art. 559) now makes it a crime to disseminate messages (through any means) that incite the commission of offences of disturbance of public order. 32 of two grants to CSOs, rather than conducting any campaigns directly. A role that PC has continued to play under this Project is that of preserving spaces for civil society to participate as an advocate and watchdog in light of an ever-stronger government presence. The use of monitoring of media as a mechanism for informing the public on important issues (ranging from the legal use of public funds in publicity to violence against women), has been made effective through collaboration with the mass media, as many outlets have disseminated PC’s press releases and reports. Various informants commented that the public and government officials have greater awareness of the use of public funds on publicity due to PC’s work in this area. Informants both within and outside of PC noted that the PC Project’s strategy, which is careful to focus on compliance with laws and regulations in its reports and publications, has allowed PC to continue its work in a difficult political context where criticism of the government may bring harsh consequences. The PC Project’s support of and engagement with municipal governments in the use of Silla Vacía and Accountability Forums has been well received in localities where there has been political openness, but results have been modest, as discussed in the Advocacy Section. 4.2.2.2 Conclusions 1. PC was professional and technically competent in the civic campaigns, watchdog work and advocacy initiatives it promoted. Its combination of regular citizen engagement across the country and strong technical capacity afforded the organization a high level of legitimacy and visibility, which few other Ecuadorian CSOs could claim. Lately, its level of engagement with citizens has declined, primarily due to the political context and budgetary constraints. 2. The creation and maintenance of a widespread volunteer network was a significant achievement for PC. However, the network is not as strong now as in PC’s early years and has lost momentum since this Project began in 2010. Since that time, and given funding constraints, the organization’s management decided to allocate less human and financial resources to these areas of activity and focus volunteer engagement primarily in election monitoring related activities. 3. The spontaneous emergence of the Red Pluralista network was an indication of a strong motivation among a core group of CSOs to strengthen civil society in their areas and work with the PC Project to extend the reach of its training. For CSOs, the network also represented a shared space to discuss common needs and challenges. This represented an opportunity for PC and the Project to take a supporting role in a small network of interested organizations, which could potentially have collaborated with PC volunteers or otherwise contributed towards achievement of project objectives. However, PC’s strategy was to allow the network to grow “under its own steam” without a more active participation and leadership from PC, in part due to funding constraints; therefore, it did not capitalize on (or perhaps recognize) that opportunity. 4. PC has been effective in maintaining and establishing links with some government institutions at national and local levels through its access to information support activities, both preceding and during this Project. Such activities have helped establish positive and collaborative relations between public sector institutions and PC. 5. The PC Project was effective in attracting citizen and media interest through its different media outreach initiatives. It generated high levels of visitors to its website and regular media presence. This media presence was stronger in Quito and less evident in other parts of the country in the last year. 6. The sustainability of many of the PC Project mechanisms for engagement is largely dependent on the context; the scarcity of resources to replicate training and support networking, the fear of engaging in advocacy activities (or even linking to others involved in advocacy), and regulatory challenges of conducting civic campaigns, all pose constraints to mechanisms used by PC in the past. In sum: How successful were PC’s interventions at strengthening their engagement and collaborative work with other CSOs, the media, their constituencies, and the government? The work of PC with government institutions was somewhat successful in this area, especially before 2012. Outreach to grassroots constituencies, volunteers, CSOs and the media were also limited during this period by 33 declining financial resources. However, engagement with local government and citizens in targeted areas has improved since 2012 through the project’s work on promotion of citizen participation mechanisms. What are the most effective and sustainable mechanisms for CSOs to constructively engage with other CSOs, their constituencies, media, and the government? The PC Project did not include a significant focus on supporting engagement by CSOs with other stakeholders. However, as a CSO itself, PC established effective mechanisms for providing technical assistance to government entities (at national and local levels) to promote compliance with LOTAIP. The PC Project’s communications are disseminated regularly by the media, and its information – especially regarding elections - is well respected. PC’s civic campaigns proved to be effective means of engaging citizens in public life, especially when based on the mobilization of young volunteers. These mechanisms require at least modest levels of funding to be sustainable. 4.2.3 QUESTION 3A: LOCAL CAPACITY DEVELOPMENT 3.1. How effective were PC’s capacity development interventions with other CSOs? 3.2. Specifically, what interventions in local capacity development were most effective and why? (Grants, training, workshops, university training, etc.)? 4.2.3.1 Findings Although CSOs were not a primary target group of the PC Project, PC carried out several capacity development interventions to further the project’s objectives and increase the capacity of CSOs, citizens, volunteer networks, and youth. Key initiatives are described below. Marcando la Hoja de Ruta para la Sociedad Civil was a PC initiative (co-funded by the Embassy of Canada and USAID) that aimed to strengthen civil society in Ecuador through several activities. PC carried out an assessment in 2010 to identify CSOs’ key needs60 and then designed a series of four training modules to respond to those priorities.61 Fifty-four CSOs (as well as some universities and government institutions) received training in diverse subjects that included citizen participation and the new constitutional mechanisms for participation, new information technologies, legal framework, and advocacy, among others. Training participation was variable, averaging 25 CSOs per module; 30 organizations completed at least two modules. The ET received positive feedback on the training from participants who considered it to be highly relevant. The survey of CSOs indicates that respondents who received any kind of training from PC on average rated its importance to their organization at 4, on a scale of 1 to 5. More specifically, among the 10 survey 60 Marcando la Hoja de Ruta para la Sociedad Civil: Una investigación cualitativa para determinar el estado actual de las OSC y contribuir a su fortalecimiento 61 The four modules were entitled 1) Constitutional Principles and Citizen Participation, 2) Tools for Citizen Oversight and Monitoring of Government, 3) Leadership and Policy Advocacy, and 4) Strategies for Communication and Policy Dialogue. 0% 20% 40% 60% Very Poor Quality Poor Quality Satisfactory Quality Good Quality Very Good Quality 0% 0% 30.00% 20.00% 50.00% Figure 7: CSO Views on Quality of the PC Project “Marcando la Hoja de la Ruta” Training How would you rate the quality of the training? 34 respondents who participated in this particular series of workshops, 50 percent rated the training quality as “very good,” the maximum on a scale of 1 to 5, while the rest considered it to be either satisfactory (30 percent) or good (20 percent). Eighty-two percent of those respondents said they engage in activities aiming to influence or monitor the actions of government. Of those, 64 percent said they had increased the frequency of these activities since 2010, and 55 percent said the impact of these activities was either more effective or much more effective since 2010. Although this sub-set of survey respondents was only 10 individuals, and many other factors could have affected responses, this data suggests that this training series may have motivated CSOs to increase their focus on this type of engagement. More surprising is the fact that respondents perceived their advocacy and watchdog work as more effective - especially given the complex environment for such activities since 2010. At least two organizations replicated the training for other organizations in Cuenca and Loja provinces, and had the opportunity to present their experience in a workshop organized by the PC Project with 50 CSOs. The formation of the Red Pluralista is attributed to the above-described training, as a group of 12 participating CSOs wanted to continue the process of addressing and discussing shared areas of interest with other CSOs. Members have received assistance from the PC Project in the form of materials for replicating trainings, and some have collaborated in training events offered by PC. Afro-Ecuadorian women training initiative Consistent with the project’s objective to increase the knowledge of afro-descendent groups regarding citizen rights and obligations, the new Constitution, and international legal instruments to strengthen democracy, PC contracted with the Ecuadorian NGO Corporación de Gestión y Derecho Ambiental ECOLEX (ECOLEX Environmental Management and Law Corporation or ECOLEX) for afro-descendent women.62 In 2010-11, the project trained 103 female leaders in Guayaquil, Ibarra and other locations with significant afro-descendent populations, and some trainees went on to provide legal aid in their communities as part of a wider ECOLEX program. Key informants highlighted the importance of this activity as a way to strengthen relationships among these women (most of whom were CSO leaders) and develop increased confidence levels in their abilities to represent their constituents. One participant gave credit to this training for her organization having an international representative and a broader understanding of where they could go for support and funding. This initiative represented the only project activity focused specifically on this minority group. Training on Decree 16 Training on Decree 16 was another notable PC activity that has aimed to directly contribute to CSO capacity.63 Activities have included a series of one-day workshops in various cities to brief CSOs on the contents and implications of Decree 16, accompanied by a guide or manual that presents the most pertinent aspects of the decree and a new registration system in a relatively simple format and language. Feedback from participating CSOs indicates that these events have been helpful, but the ET observed that considerable confusion was still evident among participants – apparently because many aspects of how the decree will actually work are still unclear, so PC facilitators were hard pressed to explain the inexplicable. With respect to the general effect of the PC Project’s support to CSOs, the survey results indicate that, on average, respondents believe that assistance by PC helped their organization improve the most in the following areas: relations with other CSOs, work with local communities, and quality of services. Least improvement was noted in the organizations’ relations with the national government and access to funding. CSO survey respondents that benefited primarily from the PC Project chose training as an important type of support provided by the organization, second only to “advocacy to promote civil society participation in 62 Note: This activity was essentially a pass-through of funds by PC to ECOLEX, undertaken at the request of USAID/Ecuador. 63 This work is primarily funded by Konrad Adenauer Foundation with additional support from ICNL under the global LEEPII project funded by USAID. However, until March 2014 USAID funding of the PC project covered most indirect costs. 35 matters of public interest.” The third highest ranking was for “sharing of information.” Figure 8, below, shows the breakdown of CSO responses about the most important types of support from the PC Project. 4.2.3.2 Conclusions 1. The PC Project’s key CSO capacity building intervention, “Marcando la Hoja de Ruta,” was well researched and responded to CSOs’ needs at that time. Although there was variability in attendance and few organizations completed all modules, the strong positive reaction from interviewed and surveyed CSOs indicates that the training was relevant and useful. There is convincing evidence that the additional knowledge and confidence helped participating organizations become more involved and more effective in citizen oversight and advocacy activities. 2. The spontaneous emergence of the Red Pluralista at the end of the training series was an indication of the training’s relevance and a strong motivation among a core group of CSOs to strengthen other CSOs and work with the PC Project in this area. However, the initiative – which did not receive funding support from the PC Project, except $10,000 for grants to two network members in 2013 – did not get off the ground in any meaningful way. It is worth nothing that two additional network members mobilized resources to implement some training replications. In sum: How effective were PC’s capacity development interventions with other CSOs? The PC Project activities to build CSO capacity (notably, the Hoja de Ruta training series) can be considered fairly effective, overall, although the scope of their work in this area was limited after 2011 due to funding constraints. Specifically, what interventions in local capacity development were most effective and why? (grants, training, workshops, university training, etc.)? The Hoja de Ruta training series was the most effective activity, due to the careful assessment of needs that preceded them and the quality of the workshops. It helped organizations become more involved and effective in citizen oversight and advocacy activities, and afterwards a few members replicated parts of the training in their local areas. 4.2.4 QUESTION 3B: SUSTAINABILITY 3.3. Has USAID/Ecuador’s direct assistance to PC succeeded in building sustainable local capacity of these two groups? 3.4. How far along the road to sustainability is PC? And in what areas does it still need support? With respect to organizational sustainability, the evaluation focused on analyzing the likelihood that PC could continue to function with internal capacities and principal activities roughly similar to the situation that existed prior to the termination of USAID funding. The ET has taken into account the following aspects of sustainability: technical competency, credibility (especially with government and other CSOs), public visibility, current funding and future funding prospects, internal governance, and external linkages. 31.6% 42.1% 10.5% 63.2% 21.1% 15.8% 0.0% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% Figure 8: CSO Views on Relative Importance of Types of Support from PC What are the two most important services or types of support provided to civil society by Participacion Ciudadana? 36 4.2.4.1 Findings Impact of USAID assistance PC recently celebrated its 10th anniversary. Heavily supported by USAID since the moment it was created [originally for election monitoring in collaboration with the National Democratic Institute (NDI)], PC has developed unique technical skills in various areas and is held in high regard for its technical competency. Over the years, this organization has developed and grown into an important, respected, and competent Ecuadorian CSO. Direct assistance to the institutional strengthening of PC has included fundraising and strategic planning consultants, OCA exercises in which some USAID staff participated, and travel to the USA for promoting alliances. Discussions with various informants indicated that some donors may have been disinclined to fund PC, precisely because it was so steadily supported by USAID, and others simply did not consider PC to be a priority, especially in times of shrinking resources for Ecuador cooperation projects. Level of sustainability The following are the key findings regarding PC’s financial sustainability and its ability to continue playing the same role for civil society: 1. PC’s credibility with the national government was high in the past, as demonstrated by its close relationship with the Ombudsman’s Office in the LOTAIP monitoring initiative, its ability to provide inputs to various pieces of legislation, and close coordination with the CNE in elections. This credibility has suffered to some extent as the President has criticized PC for certain activities (notably, monitoring of public spending on publicity). These developments in the political context have limited PC’s ability to influence national-level institutions. With respect to CSOs, the media and the public at large, PC’s reports, press releases, and bulletins are considered reliable and credible. Its election monitoring information is highly regarded for its reliability and informants consider its methodology impeccable.64 In addition, Americas Barometer surveys conducted in Ecuador by the Latin American Public Opinion Project (LAPOP) indicate that public confidence in PC is relatively high; in 2014, those respondents who knew of PC gave it a rating of 58.8 points out of 100, on a scale ranging from no confidence to great confidence. That proportion has been increasing modestly but steadily since 2006, when the confidence rating was 48.1 out of 100.65 The 19 CSO survey respondents that indicated receiving support primarily from PC indicated a relatively high level of confidence in information provided by the organization, with an average rating of 3.53 on a scale of 1 to 5. With respect to the importance of PC’s work to civil society in Ecuador, the same respondents gave an average rating of 3.89, or very important. Figure 9, above, shows the breakdown of CSO responses by percentage of those who selected each possible response. In terms of transparency practices, which are important for an organization that promotes these values, since 2008 PC has produced an annual report on its activities that is available on its web site 64 The high quality of the methods being used by PC for Parallel Vote Tabulation (Quick Count) were verified by an independent expert in election monitoring contracted by USAID in 2009. 65 Data sourced from LAPOP Americas Barometer 2004-2014, draft report shared with USAID/Ecuador in June 2014. 15.79% 26.32% 36.84% 21.05% Figure 9: CSO Level of Confidence in Information Provided by PC What level of confidence do you have in the information provided by the implementing organization? no confidence little confidence moderate confidence much confidence absolute trust 37 and convenes an annual meeting where results are presented. Beyond these, the organization does not currently provide information to the public regarding its donors or financial status. 2. PC’s technical competency is highly regarded by informants. PC has acquired unique technical skills in a broad set of areas that range from election observation and quick counts to understanding the citizen participation mechanisms created under the new Constitution. It has proven capacity in leading advocacy and watchdog initiatives, mobilizing civil society campaigns, and engaging citizens in democratic processes. 3. PC documentation and informants indicated that although its public profile has diminished over the last few years, the organization still enjoys significant levels of visibility. That is especially true in Quito and Guayaquil where the mass media regularly covers the organization’s activities (most recently the use of government funds for publicity in media), and in areas where election monitoring activities and civic campaigns have been strongest. The LAPOP Americas Barometer study mentioned above found that 39 percent of Ecuadorians surveyed in February 2014 had heard of PC (595 out of 1512 interviewees).66 4. With respect to internal systems and governance and management structures, PC’s internal systems are relatively strong according to the OCA self-assessment performed in 2013. The organization rated its overall capacity at 3.6 out of 4, with 4 demonstrating high capacity. The assessment process also identified weaknesses in key areas. The organization performs very well in the financial management and project management areas. Areas of concern include organizational structure and strategic planning. The assessment noted “there is no strategic institutional plan that is not linked to a specific donor. Decisions for new projects are based on USAID’s plans...and there is no sustainable financial planning.” The ET also observed some weakness in the preservation of important documents related to PC’s past work and methods, which were not readily accessible.67 Although not investigated in depth by the evaluators, this suggests that as PC downsizes its activities and staff, and possibly relocates its offices, valuable institutional memory may be at risk. PC’s volunteer network, which was a unique and important part of its organizational structure, has been somewhat weakened since 2010 by a combination of factors, including USAID budget cuts and changes in PC’s priorities. The level of regular engagement by volunteers has declined in recent years, communications are weak between volunteer coordinators and PC’s headquarters, and information sharing has diminished according to interviewed volunteers. 5. Interviews with PC leaders and staff revealed considerable efforts towards resource development, with 20 potential donors identified, contacted, and numerous proposals submitted in the past year. Prospects for grants are uncertain, as the current funding environment in Ecuador is very difficult for bilateral/multilateral donors as well as the private sector, especially for activities related to promotion of enhanced democracy and governance. Two consultants were contracted to support PC with developing a financial sustainability plan, the first in 2010 and the second in 2013. Some PC staff noted that the first produced a good diagnostic of the situation but did not offer useful ideas regarding funding, whereas the second one generated some innovative ideas that may be pursued. PC has also considered fee for service projects, such as the sale of media monitoring services to the private sector, the marketing of training seminars with university backing, training local governments in the new citizen participation mechanisms, and conducting research, among others. To date, efforts to diversify funding have not been successful and, currently, staffing is significantly reduced with some key personnel working on a pro bono basis. 66 Informants at LAPOP provided the following figures for those survey respondents who responded “yes” to the question “Have you heard of the NGO Participación Ciudadana?” 35% in 2004; 39% in 2006; 40% in 2008; 32% in 2010; 48% in 2012; and 39% in 2014. 67 Many reports and other documents were only listed on the PC website, but not downloadable—only a few publications are available for viewing on the website. 38 6. An organization’s national, regional and international support networks can increase their resiliency and sustainability. PC belongs to several international networks, including the Latin American Network for Democracy and the Acuerdo de Lima. Membership in the latter has provided PC the opportunity to participate in election monitoring in other countries and to gain recognition for its expertise in that subject. PC’s competency in election monitoring has been recognized by the Organization of American States, which has consulted with it on several occasions. In addition, PC has been part of several thematic networks in Ecuador, although those are minimally active at present. 4.2.4.2 Conclusions 1. PC has a limited window of opportunity to identify funding to carry on its work of promoting citizen participation, accountability, and democratic processes in Ecuador. It cannot currently be described as sustainable in terms of the role it has played in Ecuador to date. 2. Limited efforts to collaborate on funding proposals with other organizations (nationally or regionally based) may have contributed to ongoing dependency on USAID funding, although efforts to secure funding in the last two years have been laudable. Donor reluctance to fund PC has also been a significant factor in inability to secure major funding. 3. PC’s robust governance structure, with well respected members on their Board of Directors, strong credibility, and committed leadership are all important assets for its future. 4. A strategic plan with organizational priorities based on a consultative process with staff and volunteers would have been an asset for PC as it faced reduced levels of funding since 2010. The lack of clearly defined priorities for development of the organization and its programming has impacted its effectiveness and ability to cope with shrinking resources. In sum: Has USAID/Ecuador’s direct assistance to PC succeeded in building sustainable local capacity of the organization? Yes. The Mission’s support (under this Project and previous ones) has directly contributed to building capacity in a number of key areas, including technical capacity, credibility with the public, media and civil society, visibility, and internal systems. How far along the road to sustainability is PC? And in what areas do they still need support? PC is still far from being financially sustainable, partially due to external circumstances. It would benefit from the following support: 1) strategic planning to clearly identify its priorities and the way forward; 2) facilitation of connections to private sector funders who prioritize the type of work done by PC; 3) encouragement to develop joint projects with other organizations based in the region; and 4) knowledge management support to assist in analyzing and capturing the information, systems, methodologies, and strategies used in PC’s key initiatives, in order to mitigate the effects of reduction of staffing on technical capacity. 4.2.5 PC PROJECT RECOMMENDATIONS The following recommendations are intended to inform future PC programming in Ecuador, as well as future interventions by USAID in other countries. 4.2.5.1 Question 1: Advocacy and Watchdog Activity 1. PC should assess the costs and benefits of the monitoring of media spending activity in its current format, and consider whether the method or focus needs to be adjusted to generate more public interest and/or discussion about change in government practices. The impact of this activity outside of electoral periods needs to be weighed against the political costs, as the balance between them is not evident. 2. In countries with a similar political environment, USAID should encourage civil society programs to identify non-controversial ways in which CSOs can monitor and provide technical support to government entities. An initiative such as monitoring fulfillment of transparency obligations by state￾funded institutions or helping government officials interact effectively with citizens can be an important avenue to keep doors open between the sectors, and can help develop mutually beneficial institutional and personal relations between actors on both sides. 39 3. International and Ecuadorian stakeholders should continue to support the role of skilled CSOs such as PC in election monitoring, including pre-election processes. Such organizations provide a means of mobilizing ordinary citizens to engage in locally-owned oversight of some of the most significant democratic processes in their country. As well, the combination of national and international actors tends to lend more credibility to observation findings and has the potential to have more influence on government and election authorities. 4. PC should capitalize on existing contacts with and capacity of CSOs and volunteers based in other provinces to extend the reach and impact of their citizen participation work so they can be translated into real participation at the grassroots level. Ways of further collaborating with locally based CSOs and volunteers (both existing and new ones) in this work, rather than direct implementation by PC in communities, should be actively explored. While PC has taken some steps in this direction (including through two small sub-grants in 2013), CSOs and volunteers already trained by PC and/or GF in citizen oversight and advocacy could play a more regular and active role in carrying out work at community level to support use of participation mechanisms by local government bodies. These kinds of strategies can be both effective and economical, while also providing a boost to legitimacy and strengthening PC’s base of citizen support. 4.2.5.2 Question 2: Engagement 1. Volunteers have been an integral part of the organization, while their role has changed over time. This role should be re-examined to determine if the sole purpose of volunteers is to support election monitoring initiatives, or if they should have a broader role within PC. If a larger role is desired, efforts should be made to reconnect with volunteers to inform them of current priorities and plans of the organization, as this was last done in 2010, and to re-energize them. 2. PC should continue to collaborate with the Red Pluralista by holding joint training workshops and promoting members’ leadership in their communities, to the extent possible with available funding. Periodic virtual (via Internet) gatherings of the members of the network should be promoted (but not necessarily convened) by PC to keep lines of communication alive among the members. 4.2.5.3 Question 3A: Local Capacity Development 1. PC should build on local capacity development initiatives, such as the Marcando la Hoja de Ruta training series, in order to expand and activate their existing linkages with CSOs in Ecuador and broaden the impact and dissemination of their activities. One commendable strategy that PC has recently begun to use is the co-hosting of training events with locally-based CSOs of the Red Pluralista. 2. PC has built capacity among a broad network of former and current volunteers, especially on topics related to election monitoring, and successfully trained a core group of CSOs in certain subjects, notably related to citizen participation and advocacy. PC should consider ways that the volunteers and CSOs could complement each other’s skills, resources and networks, for example by collaborating on election monitoring or civic awareness campaigns. Some CSOs might wish to take on the role of “hosting” PC-mobilized volunteers in their area, for example by allowing them to use some CSO facilities, and in exchange may gain access to a valuable source of human capital for supporting their own activities. In this way, PC could deepen and broaden the reach of its initiatives and facilitate cross￾linkages in various provinces. 4.2.5.4 Question 3B: Sustainability 1. PC should actively seek out national and regional organizations with complementary skills and experience, in order to identify and develop collaborative projects that could form the basis for attractive funding proposals. Those partnerships based on co-execution of projects would not only open up new financing opportunities (as some donors prefer to support such collaborative efforts), but would also bolster PC’s networks of support. 2. USAID should strongly encourage and support regular strategic planning processes (and reviews) by its implementing partners, using experienced consultants to facilitate the processes and ensure that the final product is supported by broad consensus. Those exercises should go beyond financial planning, to examine closely the current priorities, risks and opportunities of the organization, and to provide a solid foundation for programming and fundraising decisions. 40 3. As PC goes through its current re-structuring and adjustment of office arrangements, which may include the departure of key staff members, they should pay careful attention to the documentation of knowledge and methods that may not yet have been captured, in order to ensure valuable knowledge capital is not lost. 5.0 GENERAL CONCLUSIONS 5.1 QUESTION 1: ADVOCACY AND WATCHDOG ACTIVITY 1. Space is currently very constrained for advocacy and watchdog work, especially on the national government level. Monitoring government bodies such as the National Assembly and advocating for changes in national policies such as the Communications Law or Decree 16 has had minimal impact, due to the government’s increasing reluctance to dialogue with CSOs or take into account civil society inputs - or, in some cases, to even acknowledge its role in contributing to legislation or public policy. It has become more difficult for civil society to have a voice in or to influence public policy, with few exceptions. This situation has deteriorated over the life of the projects. 2. In many instances, the ability of CSOs to influence on or even dialogue with local government depends heavily on personal relationships and the inclinations of individual leaders, as well as their political affiliations. However, there is also evidence that the government, on various levels, is willing to engage with and listen to civil society in certain sectors - notably those related to social service delivery, where in some cases the state is contracting with CSOs to tap into their expertise in order to build the government’s own human resource capacity. While that type of engagement has definite advantages for organizations struggling to make ends meet, there is also a risk that CSO advocacy will be undermined by financial links to the government, and that the relevance or “niche” of some CSOs will diminish as they share their expertise with civil servants. 5.2 QUESTION 2: ENGAGEMENT 1. The evaluation found substantial evidence that CSO networks, as mechanisms to support engagement among organizations with shared interests, are facing challenges in the current political context. Others are struggling to survive and simply do not have the capacity or resources to undertake outside activities. All of these factors present obstacles to the active and open engagement of CSOs, including PC and GF project implementers, with other actors. 2. The collaboration of the GF consortium with two universities took advantage of existing resources and had positive outcomes for all parties, at a relatively low cost. This was a good example of the synergies that can be developed between CSOs and institutions of higher learning such as universities in Ecuador, many of which are highly respected, have good relations with government, and maintain a neutral image. 3. Rivalry between PC and the GF consortium organizations, and disagreements on strategy for improving the legal framework for CSOs, were key obstacles to open information sharing between the projects. USAID’s low-key efforts to encourage interaction had little effect, especially in the tense political context and tighter funding situation that prevailed from 2012 onwards. According to informants, due to the difficult nature of this situation, it was more feasible for USAID to focus on avoiding overlaps between the projects (in which they succeeded), as compared to insisting on coordination or sharing of information among the reluctant parties. This had the effect of opportunities for synergy being missed – for example, in the research and design of CSO training modules, and in the preparation of manuals to guide CSOs on legal issues.68 In many respects, the isolation of the two projects divided the range of relevant actors between those who worked with the PC Project and those who worked with the GF Project. Although overlap was largely avoided by this division of “territory,” the projects and 68 Although the approaches of each project to their respective legal manuals/guides were slightly different and their funding sources were different for manuals related to the application of Decree 16, the evaluation found that much of the content was similar, thus there was some duplication of effort. 41 implementers would have benefited from having more awareness of what their counterparts were doing. 5.3 QUESTION 3A: LOCAL CAPACITY DEVELOPMENT 1. The major training interventions by the GF Project (university courses) and the PC Project (Marcando la Hoja de Ruta) were highly valued by CSO participants and were found to have built valuable skills and knowledge in subject areas that were important to civil society at the time. Unfortunately, the universities have not adopted the university training curriculum at this point and further replication of that course is uncertain. Likewise, there has been little replication of the PC Project training by participating CSOs nor project financial resources to support these activities. In both cases, the training topics were relevant to CSOs, but the impact of these capacity building initiatives has been confined to a narrow group of targeted organizations. As well, publications produced by the projects have not been disseminated to CSOs (either in hard or soft copy) on a systematic basis. 2. Decree 16 represents a potentially serious threat to CSOs who do not meet the new registration and reporting requirements. PC and GF have both provided valuable concrete assistance and helped CSOs cope with the new environment by providing manuals, training, networks, and forums for discussion. 5.4 QUESTION 3B: SUSTAINABILITY 1. The current environment in Ecuador for CSO sustainability can only be described as extremely difficult. Civil society is faced with a combination of threats, including most notably: diminishing level of interest among international donors generally, widespread wariness among donors and the private sector about supporting any civil society activity that may not be valued by government, increasingly strict regulation of CSO activity and international cooperation channels, and diminishing space for citizen and civil society action (in many fields) and influence. Figure 10, below, indicates the breakdown of survey responses among CSOs with respect to the significance of risks facing their organizations. It shows that more than half of respondents consider lack of funding to be their greatest risk at present. 2. It must be recognized that some CSOs that are now aligned with government actors and priorities, especially those working in areas such as service provision to vulnerable populations, have found new opportunities for government funding in the form of grants and/or service contracts. Therefore, some CSOs are benefiting in certain ways from the changes that the government has instituted in recent years. 52.8% 15.1% 22.6% 7.5% 1.9% 0.0% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% Figure 10: CSO Views on Significance of Risks Facing Their Organizations What is the most significant risk faced by your organization at present? Lack of funding Political developments The legal framework for CSOs Internal conflict Lack of skills Other 42 6.0 GENERAL RECOMMENDATIONS The following recommendations have been derived from the preceding general conclusions as well as the conclusions related to each project. They are intended to inform future programming of the implementing partners in Ecuador and of USAID in other countries. 6.1 QUESTION 1: ADVOCACY AND WATCHDOG ACTIVITY 1. Donors and implementers should be realistic in their expectations of advocacy initiatives since real impact on laws or policy can become apparent years later. The mere act of putting alternative viewpoints into the public sphere is important, whether or not the state ultimately accepts them (which will often be unrealistic). Likewise, the lessons learned from a failed initiative can serve to inform a subsequent successful advocacy strategy. 2. Civil society and donors should work to preserve and expand neutral spaces where CSOs can discuss relevant advocacy and watchdog initiatives, share what works and doesn’t work, and establish alliances and areas of collaboration. Informal gatherings as well as more formal networks are useful in this regard, whether at the national or local level. It would be useful, for example, for forums to be convened where CSOs with different viewpoints on Decree 16 could openly debate their ideas, with the support of moderators. Networks and alliances of a wide variety can be effective - they should not be considered weak or failures just because they are not formalized, or because they do not last for a long time, since some collaborations have a specific purpose that can lose relevance after a certain period. 6.2 QUESTION 2: ENGAGEMENT 1. USAID should recognize and carefully take into account the rivalry that may exist between national CSOs such as GF and PC in this kind of context, especially in small countries with relatively few major NGOs in the democracy and governance sector. In the process of making award decisions, the potential impact on such organizations should be evaluated carefully, especially in cases of similarity of objectives and potential overlap of activities. USAID should establish detailed strategies from the outset that define how relations between projects in related areas will be managed, and that make clear to what extent implementers will be required to share information and coordinate with other projects. 2. There is no surefire method of overcoming the potential resistance of implementing parties, and actual collaboration may not be achievable. The use of indicators or conditions to insist upon mechanisms for information sharing and coordination should be considered and, if needed, USAID staff should act as a go-between to ensure that relevant information (including planned activities and published materials) is shared among various projects in order to reduce possibilities for overlap and confusion. The regular convening of roundtable meetings on a sectoral level for Democracy and Governance implementers is another strategy that can be very effective, if meetings are well organized and facilitated. In extreme cases, professional mediation or counselling services could be considered to facilitate the process of opening channels of communication. 3. PC, GF and other CSOs should continue to seek collaborative relationships with reputable and capable universities and other institutions of higher learning. Such strategic alliances can improve outreach capacity and reduce political vulnerability of CSOs, although in the current environment, educational institutions are also subject to increasing government regulation. 6.3 QUESTION 3A: LOCAL CAPACITY DEVELOPMENT 1. Since both the GF-supported university course and the PC-supported Marcando la Hoja de Ruta training packages are the fruit of intensive research of needs and have shown positive results, these should be disseminated through USAID networks and platforms so that CSOs in other countries can benefit. Implementing organizations should also disseminate these training packages throughout their own networks so that more CSOs in Ecuador can be supported with this kind of training. Those who 43 offer the courses on an e-learning basis should consider including introductory classes on how to take such courses, in order to minimize dropout rates and optimize the learning experience for students. 2. In projects that place significant emphasis on training and/or publications for CSOs, project plans and budgets should take into account (as much as possible) the need for strategies to share the information/learning tools beyond the primary beneficiaries, whose number is usually fairly limited due to budgetary constraints. The strategies for extension of training may include a certain amount of replication by targeted CSOs, or they may focus on the distribution of training materials, in either electronic or hard copy, which ideally would be adapted for the use of secondary beneficiaries. With respect to publications, there should be plans to ensure that project-developed resources are systematically made available on relevant websites, and that interested audiences are made aware of those resources. 6.4 QUESTION 3B: SUSTAINABILITY USAID and other donors should manage their expectations with respect to CSOs. Creative solutions should be sought to shore up the long-term viability of organizations that are seen as important parts of the counter-balancing role played by civil society in democratic societies. For example, donors could support the establishment of affiliated income generation “branches,” and the creation of endowment funds or the investment of funds that would normally be spent on office rental in the purchase of permanent premises. 6.5 CROSS-CUTTING RECOMMENDATIONS 1. When dealing with a consortium of either local or international partners, USAID normally conducts all discussions about contract and budget with the prime implementer. However, in cases of significant budget and/or activity adjustments, it is recommended that USAID staff meet directly with both prime and non-prime implementers, in order to explain the adjustments clearly to all parties. This is especially important in a situation where the prime has limited experience in managing consortia and with USAID funding, so as to minimize misunderstandings which may undermine relationships within the consortium. 1 ANNEXES 2 ANNEX A: SCOPE OF WORK 3 SECTION C – STATEMENT OF WORK FINAL PERFORMANCE EVALUATION OF USAID/ECUADOR’S CIVIL SOCIETY PROGRAM: 2010-2013 I. BACKGROUND Civil society in Ecuador has been significantly weakened from its much more advanced position in past decades, when Civil Society Organizations (CSOs) made significant improvements to the legal framework in many areas, including human rights and the environment. The sector has faced a series of challenges over the last few years. Namely, (i) a leadership crisis, following an exodus of many CSO leaders to take on government positions; (ii) a financial crisis driven by a reduction of donor funds; and (iii) a political crisis as the operating environment for CSOs has become constrained by challenging regulations and increased controls. USAID/Ecuador has a long history of support to local Ecuadorian CSOs who seek to promote citizen participation in advocacy and oversight of democratic institutions and processes (including elections). This final performance evaluation will assess the results of the Mission’s two main civil society programs: 1. Strengthening Democracy in Ecuador Project Dates: April 4, 2003 – March 31, 2014 Project Funding: $ 10,387,400 Period to be evaluated: October, 2010 – September, 2013 Estimated Project funding during last 3 years $3,000,000 Implementing organization: Participacion Ciudadana (PC) 2. Strengthening Civil Society in Ecuador Project Dates: December 29, 2010 - September 30, 2013 Project Funding: $ 1,939,000 Implementing organization: Grupo Faro (GF) USAID’s Strengthening Democracy in Ecuador project has been active for over ten years. However, only the work of the last three years is to be evaluated recognizing the difficulty of conducting a retrospective evaluation over a decade of work and given the more recent political changes in Ecuador. USAID’s Strengthening Civil Society in Ecuador project overlapped with the Strengthening Democracy in Ecuador project. Strengthening Democracy in Ecuador (SD), Participacion Cuidadana (PC), 2003-2014 USAID/Ecuador began its support to PC in 2003. It is an early example of mission support for Local Capacity Development (LCD.) The Strengthening Democracy in Ecuador project sought to promote more effective citizen participation in key democratic processes and oversight of democratic institutions. The project supported PC’s efforts to (a) develop domestic observation and parallel vote tabulations of the results of national and local elections, (b) promote greater transparency by government institutions and elected officials at the national and local level, (c) provide constructive input to local and national laws and regulations, (d) promote democratic 4 values and education through activities such as civic campaigns and democratic practices in high￾schools, and (d) promote dialogue on issues of public interest; among others. From 2010-2013, PC’s project has mainly focused in citizen-based oversight of democratic institutions, processes, and public spending; transparency and accountability by elected municipal officials; constructive input to enhance national and local legislation; consolidation of civic education programs, and promotion of dialogues between citizens and public officials or among civil society groups, and events on issues of public interest. Also, the project supported PC’s institutional strengthening. In addition, PC trained other CSOs in oversight and advocacy mechanisms and promoted the establishment of a CSO network to advance these issues. This last activity was co-funded by the Embassy of Canada. During the period 2010-2013 the award included the following specific objectives: ● To strengthen the ability of civil society to participate in and influence public policy; ● To increase the capacity of selected civil society organizations in the social audit of key democratic institutions; and ● To effectively promote democratic values, attitudes and practices. In addition, the award included activities to support PC’s institutional and financial sustainability. The project had the following key achievements: ● Increased the compliance rate of 30 public institutions according to transparency law requirements reaching 98% compliance (from a baseline of 36%); ● Improved the advocacy and oversight capacity of 36 local civil society organizations; ● Provided effective monitoring of use of public funds, producing useful information for public control institutions to trigger some investigations; ● Increased citizens understanding on key democratic issues; ● Advocated and introduced reforms to policies and legislation; ● Built a cadre of provincial volunteers, promoting citizen participation and oversight; ● Helped established a network of 12 organizations throughout the country to promote oversight and advocacy efforts. ● Pioneered Parallel Vote Tabulation (PVT) in Ecuador. ● Mobilized volunteer election observers, deploying roughly 10,000 volunteers in 2013; Please refer to Annex A for project’s Performance Monitoring Plan (PMP) and/or project indicators/data. Strengthening Civil Society in Ecuador (SCS), Grupo Faro (GF), 2010- 2013 In 2010, USAID/Ecuador partnered with implementer GF on this activity. The Strengthening Civil Society in Ecuador project aimed to enhance the capacity of local CSOs to advocate for democracy￾related issues and support efforts to influence legislation and policies related to the CSO sector. The project also used formal courses and coaching sessions to enhance targeted CSOs' organizational capacities. The project had two specific objectives: ● Strengthen the capacity of CSOs to effectively influence democratic processes, institutions, and adherence to democratic principles; and ● Strengthen the capacity of selected CSOs to develop more effective sustainability efforts and function more transparently. 5 The project had the following key achievements: ● Supported advocacy, oversight and capacity building efforts of 24 small CSOs through small grant funds (13 during Phase 1 and 11 during Phase 2); ● Strengthened a CSO network (El Colectivo) to build CSOs’ capacities and promote collective action on behalf of the CSO sector. ● Developed research studies/publications to enhance the work of the CSO sector ● Disseminated innovative practices within the CSO sector through a call for CSO best practices; ● Strengthened CSOs organizationally and institutionally through direct coaching and long- term formal courses; and ● Promoted CSO sustainability through hands-on development of sustainability plans II. PURPOSE OF THE EVALUATION Conduct a final performance evaluation1 for two civil society projects: (1) Strengthening Democracy in Ecuador, Participacion Cuidadana (PC), 2003-2014; and (2) Strengthening Civil Society in Ecuador, Grupo Faro, 2010- 2013. The purposes of this final performance evaluation are to: • Determine the results of key project interventions, including successes, impediments, and lessons learned; • Document and provide a record of the projects’ results, as well as the history of the USAID projects with these local civil society organizations (CSOs); In turn, the final performance evaluation is expected to be used by: - USAID to enhance the design of future civil society and local capacity development programs in the region or in other countries; - The two award recipient organizations for their internal learning, to use the evaluation findings to build their own capacity. The results will also be useful for USAID/Washington’s Center of Excellence on Democracy, Human Rights, and Governance and the Bureau for Latin America and the Caribbean, which collects findings on best practices and lessons learned globally, and regionally, on both civil society strengthening and LCD; and USAID/Washington’s Bureau for Policy, Planning, and Learning, which encourages USAID project evaluation globally. The contractor must comply with the USAID Evaluation Policy (www.usaid.gov/evaluation), also codified in USAID’s Automated Directives System Chapter 203 (www.usaid.gov/ads/policy/200/203). 1 Per USAID Evaluation Policy (www.usaid.gov/evaluation), and USAID’s Automated Directive Systems 203.1.1 (http://www.usaid.gov/ads/policy/200/203): “performance evaluations often incorporate before-after comparisons, but generally lack a rigorously defined counterfactual. Performance evaluations focus on descriptive and normative questions: what a particular project or program has achieved; how it is being implemented; how it is perceived an valued; whether expected results are occurring; and other questions pertinent to program design, management, and operational decision-making.” III. EVALUATION QUESTIONS The evaluation will answer the following questions: 1. Advocacy and Oversight 1.1. How effective were PC’s and GF’s interventions to strengthen advocacy and watchdog roles of other CSOs? 6 1.2. How effective was GF’s and PC’s direct advocacy and watchdog interventions? 1.3. What interventions were most effective (training, work with media, work on legal environment, strategies for engagement with Government of Ecuador, etc.) and why? 1.4. What interventions were not successful and why? 2. Engagement 2.1. How successful were PC and GF’s interventions at strengthening their engagement and collaborative work with other CSOs, the media, their constituencies, and the government? 2.2. What are the most effective and sustainable mechanisms for civil society organizations to constructively engage with other CSOs, their constituencies, media, and the government? 3. Local Capacity Development and Sustainability 3.1. How effective were PC and GF’s capacity development interventions with other CSOs? 3.2. Specifically, what interventions in local capacity development were most effective and why? (grants, training, workshops, university training, etc.)? 3.3. Has USAID/Ecuador’s direct assistance to PC and GF succeeded in building sustainable local capacity of these two groups? 3.4. How far along the road to sustainability are PC and GF? And in what areas do they still need support? IV. EVALUATION METHODOLOGY, DESIGN, AND LIMITATIONS This performance evaluation will be conducted in accordance with USAID’s Evaluation Policy, to produce reliable data to support evidence based findings, and provide a sound basis for analysis and recommendations for future programming. The evaluation will be participatory to include stakeholder values but at the same time unbiased and transparent. The contractor will actively engage USAID/Ecuador and the implementing partners, as well as other stakeholders as needed, through all phases of the evaluation, including: identification of questions and informants; selection of the appropriate data collection methods; identification of relevant documentary sources; finalization of the timeline; and reaching consensus about findings, conclusions, lessons learned and recommendations. This participatory approach will allow beneficiaries’ and stakeholders’ views to be incorporated in the design, which can result in improved acceptance and buy-in of the final conclusions and recommendations. In addition, this approach will allow the contractor to determine and refine the most important evaluation questions and come up with valuable recommendations for civil society and local capacity development programs. The evaluation will be multi-dimensional. The contractor will collect quantitative and qualitative data from a broad range of stakeholders, beneficiaries and non-beneficiaries to ensure independence of the process, as well as accuracy and completeness of the findings, conclusions, recommendations and lessons learned. To gather data, the contractor will use a number of techniques. Moreover, it will combine techniques that balance each other: quantitative vs. qualitative data, individual vs. group responses, in-depth interviews vs. direct observation, mini-surveys, etc. Such techniques will help the contractor capture the diversity of opinions and perceptions of beneficiaries and stakeholders about SD-SCS’s strengths and weaknesses, as well as the degree of their satisfaction from participation in these two projects. They will also help uncover unexpected positive or negative impacts of the SD and SCS projects, what has occurred because of their implementation, and what would have occurred in their absence. Finally, they will assist in generating sound and evidence￾based conclusions and recommendations on important issues to the projects and USAID. Given the length and scope of the projects, ME&A will proceed systematically when conducting the evaluation. First, it will map all major activities under each project. Second, it will prioritize the key intervention areas. Third, within each of these areas the contractor will identify the stakeholders and beneficiaries that need to be interviewed. Because both projects have worked with a large number of 7 public institutions, CSOs, media outlets, NGOs, journalists, etc., the team will conduct interviews with random samples of assisted beneficiaries. The contractor will communicate with USAID on the selection of samples to make sure that they are reasonably representative. Throughout the evaluation, the contractor will collect - whenever possible - sex- disaggregated data and observe gender-sensitive indicators in order to measure SD-SCS’s impact on both genders. If a gap or difference is noted, the contractor will examine what has caused it and seek to identify the barriers that have prevented men or women to realize equal outcomes from the projects. In addition, ME&A will highlight best practices and models as lessons learned and potential for replication. When determining the projects’ impact, the contractor will keep in mind that the SCS project was designed to have long-term benefits, some of which may have not been fully realized yet given the short time of implementation and the political environment in Ecuador. In addition, the contractor will consider the challenges faced by the implementers and their impact (or not) on the implementers’ ability to achieve project results. Throughout the evaluation, the contractor will adjust the data collection methodologies in order to respond to limitations or opportunities presented during fieldwork, while also taking care to maintain methodological rigor in the evaluation. To increase the rigor and quality of the evaluation, the contractor will triangulate its findings. Triangulation will be made possible by the various methods of data collection that will be used for each major evaluation question. In addition, the contractor will ask similar questions to different stakeholders that are involved in the same issue. Whenever possible, it will link qualitative with quantitative techniques. The different data collected will be analyzed, and the findings will be related to each other. However, even though the triangulation method may yield convergent findings, this does not mean that these findings are unquestionable. For this reason, the Evaluation Team will present the preliminary findings - as a basis for further consultation and gaining further information and evidence from the Mission - before drafting the Final Report. EVALUATION DESIGN This evaluation will be completed through two sub-activities to be carried out simultaneously: qualitative and quantitative data collection. The work for the qualitative data collection will be primarily conducted through interviews and other interactions with those organizations and individuals, as well as other stakeholders and partners that were involved in the activities of the two projects. In addition, the contractor will conduct a comprehensive review of historical information and reports pertaining to both projects since 2010, and earlier as necessary, for the PC project. This information will be analyzed and the results will be tailored to answer the main evaluation questions outlined above under Section III Evaluation Questions. Qualitative data will be collected by using a number of methods including: • A critical desk review of materials related to SD, SCS and other relevant projects, as well as any material that will be provided by USAID such as project reports and annual work plans, project performance management plan, data on achievement of performance indicators, etc. This review will also extend to documents external to the projects that are identified by the team through its own research or through informants which have a bearing on the evaluation questions. • Interviews with USAID/Ecuador staff and staff of project implementers (PC and GF). • In-depth, semi-structured interviews with representatives from civil society and public sector institutions knowledgeable of the civil society, citizen participation, and other DG areas in which these projects have been active. The final list will be determined after the careful desk review of materials and after the discussion with USAID and the implementers. 8 • In-depth interviews with a wide variety of representatives of the CSOs, CSO coalitions and networks, think tanks, NGOs, volunteer groups, former CSO leaders, universities, and public sector institutions who participated in each project. • Field visits and meetings with project partners in Quito, Guayaquil, and at least 2 additional cities. These visits will be an opportunity to meet some of the beneficiaries, conduct field observations, and gather best practices and lessons learned from project implementation. Particular attention will be paid to interviews with women who participated in the projects in order to gauge the impact of SD-SCS activities on them. • Review of project outputs against objectives and performance indicators. • Interviews with other international donors or implementing agencies, especially those involved in supporting civil society development in Ecuador, as well as representatives of other USAID projects. • Focus Group Discussions (FGDs) to obtain qualitative information to strengthen the contractor’s analysis and understand the relationship between the projects and the results they have achieved, within the given context in the country. The evaluation team will invite up to 12 participants for each FGD, keeping in mind refusals and no-shows. ME&A will aim at having a minimum of 4 participants to conduct the FGD; if fewer than 4 participants show up, it will change to having a key informant interview (KII) instead. The contractor will focus on the following categories of people for FGDs. This list may change after the desk review and discussions with USAID. • Representatives of CSOs • Representatives of women’s or women-focused organizations • Citizens belonging to youth, indigenous and afro-descendant population groups • Media journalists • NGOs Quantitative data collection methods will consist of: • Review of data sourced from the projects on indicators such as number of reports and articles with clear information on democracy strengthening issues. • Collection and review of secondary data from the analysis of the legal framework for citizen participation and access to information. • Review of data from available surveys and other studies. • Mini-surveys (See additional details below). Mini-Surveys Information from small grant recipients, training participants, members of CSO networks and coalitions, and other beneficiaries will be key to understanding the projects’ successes and challenges in strengthening democratic institutions and processes. To solicit this information from the broadest possible range of informants in the limited time available, the evaluation team will develop three different mini-surveys. The first mini-survey will target the recipients of small grants primarily to explore in what ways they have benefited from project support and measure the reach of their small grant activities. The second mini-survey will be focused on gathering perceptions and feedback regarding capacity building activities implemented within projects. The focus will be on any changes in structures and operations of organizations that have happened as a result of their participation in the project(s). The third mini-survey will reach out to members of the CSO networks and coalitions that PC and GF have promoted and supported, with a view to understanding the range of perspectives that members may have on the utility of the networks and coalitions, and the results and challenges related to support offered by the project(s). 9 The survey instruments will first be developed in English for approval by USAID, before being translated into Spanish. If the respondents fail to send the surveys back within the given time, the evaluation team will attempt to reach them via phone and conduct phone interviews. Survey questions will have closed responses, thus removing the need for translating responses back into English. The questions will take into consideration the country’s political context and clearly assure respondents that their responses will be anonymous to encourage frank and open answers. Mini-surveys will be conducted within a limited timeframe and will correspond with the level of technical resources of CSOs and other respondents. As internet access is high among medium-sized CSOs that participated in the project, at least one mini￾survey will be conducted by using an internet survey tool, SurveyMonkey, to ensure confidential, easily accessible, and fast methodology of collecting and analyzing data. Another method of data gathering that will be used will be phone interviews for those respondents that do not have access to internet and are in remote areas. Questions Questions for the interviews, FGDs and mini-surveys will be formulated after conducting a thorough review of materials that will be provided by USAID. While the main thrust of the questions can be identified before the start of the mission based on evaluation questions outlined above (Section III), the refinement of questions will happen based on desk review of relevant documents and after the team members have had a chance to identify the exact nature of the interventions, by activity or component. Sampling Methodology Given the large number of project beneficiaries, ME&A will apply different selection mechanisms for different mini-surveys. The two projects worked on building advocacy and oversight capacity for a total of 60 CSOs (36 worked with SD and 24 with SCS), making up the universe of the sample for the first mini-survey. The contractor will include all targeted CSOs through an online mini-survey using the Survey Monkey tool. The mini-survey will consist of different questions for the SD and SCS projects but, at the same time, will also have general questions applicable to both projects. The second mini￾survey will reach out to members of the CSO networks and coalitions that SD and SCS have promoted and supported, with a view to understanding the range of perspectives that members may have on the utility of the networks and coalitions, and the results and challenges related to support offered by the project(s). The mini-survey will reach out to 12 organizations that are members of the SD-initiated network working on promotion of oversight and advocacy efforts, while the exact number of CSOs that are members of the SCS-supported CSO network (El Colectivo) will be determined during the inception phase. Gender When designing the evaluation, the team will pay special attention to methodologies that will allow it to collect information on the extent to which the projects have integrated gender considerations throughout the activities. Some of the methods and applications that will be used are described below: Methods Application Data Collection Quantitative Methods Number of news stories and programming on gender-based themes Review of Data Number of training sessions received on addressing gender issues Review of Data, PMP plan Has the coverage of gender issues increased in the new content? FGDs combined with mini- survey Disaggregation of received data by gender, particularly through mini surveys Mini surveys 10 Qualitative Methods How did your project contribute to increase coverage of gender issues? Semi-structured interviews, Round Table Discussions Did the training contribute to your increased attention to covering gender issues in your work? Semi-structured interviews, Focus Groups Comparison and contrast between beliefs and practices between genders Group interviews Gender differences in accessing information Semi-structured interviews Differential access by gender to markets and services Semi-structured interviews Opportunities and barriers for women in the civil society sector FGDs Men’s and women’s role in civil society sector Semi-structured interviews/FGDs Data Analysis After finishing data collection, the evaluation team will process and consolidate the raw quantitative and qualitative data collected. The analysis will be based on the Evaluation Matrix developed during the first phase of the evaluation process. More specifically, the standard evaluation criteria adopted by the Development Assistance Committee of the Organisation for Economic Co-operation and Development (OECD-DAC) will be approached as follows: a) Effectiveness: Using the project’s Logframe and Performance Management Plan, the evaluation team will analyse to what extent the results obtained following the implementation of activities have contributed to the attainment of the planned objectives. Using evidence collected during the first two phases of the evaluation, the report will explain the factors that contributed to or hampered the achievement of results in terms of mobilization and capacity building of selected CSOs, increase of access to information and the development of new services or further expansion of existing ones. Analysis of coordination and synergy between activities on national and local levels will be also done. The report will discuss the additional/indirect effects (positive or negative) of the project. b) Sustainability: The evaluation report will review sustainability factors in terms of project design, process, implementation and national context. Sustainability will be analyzed from various perspectives: legal, institutional, capacity building, and financial. The report will highlight the factors that facilitated or decreased the sustainability prospects of the results of the project (legislation, synergy with similar initiatives, engagement and ownership by local stakeholders, etc.). Quantitative data collected will be analyzed using established evaluation techniques and industry standard data analysis tools. These tools will enable evaluators to evaluate not only descriptive statistics (such as the number of CSOs assisted by USG programs, number of CSO advocacy campaigns supported by the USG, percentage of citizens that participate in public opinion debates regarding democracy issues, etc.) but also more advanced analytical exercises such as measures of correlation (say, between geographic region and success rate). For qualitative data resulting from stakeholder interviews, where much of the evidence may be anecdotal or inferred, the team will use triangulation to identify any inconsistencies and ensure reliability. Triangulation will assist the team to reduce the “response bias” in which respondents tend to tell the evaluators what they want to hear. Throughout the analysis process, the team members will share and compare notes taken during the interviews and FGDs, identify any variations in the information provided to them by different 11 stakeholders, and reveal their different expectations and opinions about the projects. Evaluation findings and recommendations for civil society interventions will be presented to the Mission for comment before the team’s departure. The final report will be submitted after comments of the Mission are fully integrated. EVALUATION LIMITATIONS There are several limitations inherent to the design of this evaluation. 1. As some informants may decline to participate, there is a possibility of selection bias, i.e. those respondents who choose to participate might differ from those who do not in terms of their attitudes and perceptions, affiliation with government/non-government structures, and socio￾demographic characteristics and experience. This may apply to in-person interviews, FGDs and mini￾survey. 2. Since a number of questions will deal with issues that took place in the past or changes that have taken place since the projects began, recall bias cannot be excluded. As SD was launched in 2003, and SCS activities began in 2010, some respondents may find it difficult to accurately compare organizational arrangements/capacity three or more years ago to the current situation. 3. There is a known tendency among respondents to under-report socially undesirable answers and alter their responses to approximate what they perceive as the social norm (halo bias). The extent to which respondents will be prepared to reveal their true opinions may also vary for some questions that call upon the respondents to assess the performance of their colleagues or people on whom they depend for the provision of services. To mitigate this limitation, ME&A will: provide the respondents with confidentiality and anonymity guarantees, where possible; conduct the interviews in settings where respondents feel comfortable; and establish rapport between the interviewer and the respondent. 4. The large number of modifications for the projects will require an intensive review of project materials. 5. Differentiation/separation of results of the two projects. Informants may confuse them, or both projects may have targeted the same beneficiaries. 6. Results of the SD project in the first seven years and in the last three years may not be very easily separated. 7. The large amount of data that is going to be collected and analyzed. The team is expected to collect and analyze program information from before 2010 on PC activities. V. ESTIMATED LEVEL OF EFFORT LOE by Task / Deliverable Level of Effort (Days) Tasks/Deliverables Team Leader 2 Technical Specialists Planning and Document Review Phase Desk research and familiarization with projects Orientation/planning meeting with USAID Evaluation Design (Work plan/Timeline/Other data) Document Review 3 1 3 2 6 2 6 8 Data Collection Phase Data collection including meetings, interviews, surveys, and focus groups. 20 48 12 Analysis and Reporting Phase Data analysis Mid-term briefing with USAID Drafting of evaluation narrative (core report without annexes) Final Briefing with USAID and presentations Submission of draft evaluation report, revisions, submission of final evaluation report 10 1 5 2 10 10 2 5 2 10 Total 57 99 13 ANNEX B: EVALUATION MATRIX 14 Research Questions & Sub-Questions Key Areas of Enquiry Data Sources Data Collection Methods Data Analysis 1. Advocacy and Oversight 1.1. How effective were PC’s and GF’s interventions to strengthen advocacy and watchdog roles of other CSOs? 1.2. How effective was GF’s and PC’s direct advocacy and watchdog interventions? 1.3. What interventions were most effective (training, work with media, work on legal environment, strategies for engagement with Government of Ecuador, etc.) and why? 1.4. What interventions were not successful and why? 1.1. To what extent did targeted CSOs (from local to national level) improve their capacity in these areas? Did they improve their knowledge of the relevant laws and policies? Did they become more involved in advocacy/social control? Did they have more influence? If not, what were the obstacles? 1.2. Did PC and GF demonstrate full understanding of relevant laws and policies? To what extent did they form useful alliances? allies and Were their proposals for change clearly articulated? Were the reports of watchdog activity considered credible? Did their efforts make a difference – in policy/law/practice, or in action taken in response to watchdog activity? What were the limiting factors? 1.3 Which activities contributed to observable results or change? How did those activities contribute to achievement of the overall objectives of the projects? What characteristics did effective interventions have (in common)? Project documentation: quarterly reports, M&E indicator data, CSO capacity assessments, reports on LOTAIP compliance and other observatory activity USAID, PC and GF staff Staff/leaders of targeted CSOs, coalition members and others Local government officials and community leaders in targeted municipalities Central government officials Media/journalists Academics/experts International DG sector stakeholders (donors, other projects) Document review Key informant interviews with various data sources Focus groups with supported CSOs and coalition members Targeted survey of CSOs Analysis of targeted results, outputs, and outcomes Verification of PC and GF reporting Reporting of perceptions and experience of focus group participants Quantitative analysis of the results of the survey including cross￾tabulation (by gender, province, and project, among others) Critical synthesis and triangulation analysis of opinion data 15 1.4 Which activities had minimal or no observable effect (or negative effect)? What were the main reasons? How did PC and GF deal with obstacles? What could have been done differently? 2. Engagement 2.1. How successful were PC and GF’s interventions at strengthening their engagement and collaborative work with other CSOs, the media, their constituencies, and the government? 2.2. What are the most effective and sustainable mechanisms for civil society organizations to constructively engage with other CSOs, their constituencies, media, and the government? The focus will be on the following mechanisms: discussion forums, dialogues, policy/law proposals, social control mechanisms such as Silla Vacia, and creation/support of networks of CSOs. 2.1 What new links/alliances have been made between CS and others? What new links (including networks) have been made among CSOs themselves? How have pre-existing links been affected by the projects? What positive or negative effect have those links had, on the members and on civil society more broadly? 2.2 What roles do PC and GF play within Ecuadorian CS? What mechanisms have they used to support CSO engagement with their constituencies, media and GOE? How effective were those mechanisms and what were the limiting factors? (Analyze Red Pluralista, Confederación de OSC, and other mechanisms for their achievements to date and potential sustainability) Project documentation: quarterly and final reports, M&E indicator data Reports on international experience and best practices USAID, PC and GF staff Staff/leaders of targeted CSOs, coalition members and others Local government officials and community leaders in targeted areas Central government officials Media/journalists Academics/experts in Ecuador and the region International DG sector stakeholders (donors, other projects) Document review Key informant interviews with various data sources Focus groups with supported CSOs and coalition members Targeted survey of CSOs Analysis of targeted results, outputs, and outcomes Verification of PC and GF reporting Reporting of perceptions and experience of focus group participants Quantitative analysis of the results of the survey including cross￾tabulation (by gender, province, and project, among others) Critical synthesis and triangulation analysis of opinion data 3A. Local Capacity Development 3.1. How effective were PC and GF’s capacity 3.1 To what extent and in which areas did targeted CSOs improve their Project documentation: quarterly and final reports, M&E indicator data, capacity assessments, evaluation of Document review Key informant interviews with various data sources Focus groups with Analysis of targeted results, outputs, and outcomes Verification of PC and 16 development interventions with other CSOs? 3.2. Specifically, what interventions in local capacity development were most effective and why? (grants, training, workshops, university training, etc.)? Analysis of CSO capacity will focus on their level of skill and confidence in engagement with government, understanding of the legal frameworks for CS and citizen participation, and their access to resources (financial, human, and technical). capacity? Did some CSOs change more than others? What difference, if any, has increased capacity made in their work as an organization? How did PC and GF’s interventions affect CSO’s access to resource/information/networks? 3.2 Which activities (or combination) contributed significantly to changes in capacity? Were certain activities linked to certain types of capacity improvement? Were capacity development interventions adapted to the strengths, needs, networks, and culture of targeted groups? If so, how and to what effect? training courses USAID, PC and GF staff Staff/leaders of targeted CSOs, coalition members and others Government officials with close links to CSOs Community leaders and members where CSOs are working Academics/experts on civil society University officials (Loja and Guayaquil) and other CSO trainers/coaches International DG sector stakeholders (donors, other projects) supported CSOs and coalition members Targeted survey of CSOs GF reporting Reporting of perceptions and experience of focus group participants Quantitative analysis of the results of the survey, including cross￾tabulation (by gender, province, and project, among others) Critical synthesis and triangulation analysis of opinion data 3B. Sustainability 3.3. Has USAID/Ecuador’s direct assistance to PC and GF succeeded in building sustainable local capacity of these two groups? 3.4. How far along the road to sustainability are PC and GF? And in what areas do they still need support? Note: Sustainability analysis will focus on perceived and observed technical competency, 3.3 Are PC and GF more sustainable organizations now as compared to before these projects? If so, what has changed? What factors/activities have had a significant impact on their sustainability (organizational, financial, technical/programmatic, etc.) 3.4 Do PC and GF have realistic plans and systems in place for future development? Do they have reasonable prospects for financial sustainability? Project documentation: quarterly and final reports, M&E indicator data, OCA reports, strategic plans, staff training plans, organigrams, financial projections and fundraising plans Reports on international experience and best practices for sustainability Laws, policies and reports related to status of civil society USAID, PC and GF staff Document review Key informant interviews with various data sources Focus groups with staff of each organization (if possible) Site visits to PC and GF offices and affiliates Reporting of perceptions and experience of focus group participants Critical synthesis and triangulation analysis of opinion data 17 credibility (especially with government and other CSOs), public visibility, current funding and future funding prospects, internal governance structures, and external linkages. What are the key risk factors for their continuation at current or similar levels of activity? What type of support would best help them to manage those risks and shore up sustainability? Staff/leaders of CSOs and coalition members Local community members/beneficiaries Central and local government officials Academics/experts in Ecuador and the region International DG sector stakeholders (donors, other projects) 18 ANNEX C: SAMPLE OF DATA COLLECTION TOOLS 19 ILLUSTRATIVE INTERVIEW AND FGD GUIDES A. Preguntas para entrevistas con OSC (CSO Interviews) A. Incidencia y Control Social 1. ¿En general, cómo ve usted la situación de la sociedad civil en Ecuador? (¿De qué manera las organizaciones de la sociedad civil se relacionan actualmente con el gobierno? ¿Qué cambios o tendencias se están dando? 2. ¿Qué tipo de relaciones/interacciones tiene su organización con el gobierno nacional/local? ¿Ha cambiado esta relación en años recientes? Cómo ha cambiado? (¿Es más frecuente, es con diferentes representantes o agencias/departamentos, es más o menos difícil trabajar conjuntamente?) 3. ¿En relación con lo anterior, su organización ha recibido apoyo o ha participado en actividades realizadas por Grupo Faro o Participación Ciudadana? ¿Qué tipo de apoyo? ¿El apoyo que recibió le sirvió para atender alguna necesidad concreta de su organización o alcanzar alguno objetivo concreto? ¿Le ayudó a ser más efectivo en su trabajo? ¿De qué manera? ¿Si no es así, porqué? ¿Cuáles actividades fueron más efectivas? Cuales fueron menos efectivas? ¿Por qué? 4. ¿Se produjo algún cambio en la forma en la que su organización trabaja en temas de incidencia, gracias al apoyo recibido? ¿Si ha cambiado, puede dar ejemplos? 5. ¿Qué otro tipo de acciones ha llevado a cabo Grupo Faro y Participación Ciudadana para fortalecer las interacciones entre la sociedad civil y el gobierno? Conoce de sus actividades para vigilar la gestión pública de manera directa? (Sondear difusión de información pública, entrenamiento a organizaciones ciudadanas, dotación de pequeñas donaciones, monitoreo del informes LOTAIP, auditorias sociales al CPCCS y la Asamblea Nacional, creación de la Confederación, trabajo en las limitaciones legales de la sociedad civil, etc.) 6. ¿Las acciones mencionadas, han generado algún cambio? ¿Si no, cuáles han sido los obstáculos? ¿Cuáles acciones han tenido mayor impacto? ¿Cuáles han tenido menor impacto? 7. ¿Las acciones tomadas por Grupo Faro o Participación Ciudadana han tenido algún efecto negativo en las organizaciones de la sociedad civil, en el gobierno o en el marco legal? ¿Alguna de sus actividades de incidencia tuvo algún efecto negativo reflejado en las organizaciones de la sociedad civil o el marco legal? B. Integración/interacción 8. ¿Cuáles considera usted que son los contactos/aliados/stakeholders más importantes de su organización? (Sondeo – Otras organizaciones civiles, la iglesia, los medios, gobiernos locales, comunidades locales, miembros, donantes etc.)? ¿Cómo colaboran y se ayudan entre sí las OSC en el contexto actual? 9. ¿Dónde acude su organización cuando necesita información o apoyo relacionado con políticas públicas o con la interacción con el gobierno nacional/local? ¿Para buscar información sobre otros temas? ¿Existen otras organizaciones formales o informales que acuden a usted para solicitarle información o apoyo? (ejemplos) 10. ¿Su organización forma parte de alguna coalición o red de algún tipo? ¿Cuál(es)? ¿Esta conexión o participación está relacionada de alguna manera con las actividades de GF o PC? ¿Han recibido algún beneficio de esta asociación? 11. ¿Esa coalición/red de organizaciones (u otras) ha beneficiado a la sociedad civil ecuatoriana en general? ¿Si así ha sido, de qué manera? ¿Ha sido perjudicial para la sociedad civil? ¿Ha afectado de alguna manera el modo en el que la sociedad civil interactúa con el gobierno, los ciudadanos, los medios, la comunidad internacional etc.? 12. ¿Cómo podría tener la sociedad civil una participación más efectiva o una voz más fuerte en otros sectores de la sociedad, como los medios de comunicación, el gobierno y la comunidad? 13. PC ha producido una variedad de productos informativos. ¿Conoce usted alguno de estos productos? (Pulso Ciudadano, boletines semanales, “PC en 5 minutos”, comunicados de prensa, sitio web ciudadanía informada, etc.) ¿Cuál su opinión sobre estos productos? ¿Han sido útiles? ¿Y a quien le han resultado útiles? Si no han sido útiles, ¿porqué? 20 C. Desarrollo de Capacidad Institucional: (Depending on answer to Q. 3 and 17, this section will apply to either GF or PC or both. Repeat the questions for GF in this section if the informant has had significant contact with both PC and GF.) 14. ¿Que tipo de interacciones ha tenido su organización con PC desde el 2010? (sondear: capacitaciones de corto-largo plazo, donaciones coaching, participación en una red/coalición, información, fórums, etc.) 15. ¿El apoyo recibido por parte de PC ha tenido algún impacto en usted personalmente? ¿En su organización? ¿Qué ha cambiado o qué ha mejorado específicamente como resultado de ese apoyo? (Sondeo- tiene nuevos servicios? Hace más incidencia? ¿Nuevos donantes? ¿Más miembros?) 16. ¿Qué tipo de apoyo por parte de PC generó mayor impacto en su organización? ¿Puede dar ejemplos? ¿Por qué fue efectivo? ¿Qué tipo de apoyo generó el menor impacto o no tuvo ningún impacto? ¿Por qué? 17. (Solo si ha habido cambios) ¿Cómo ha impactado esto en su organización? (Additional questions added for sub-grantees for more details about their grant experience.) 18. ¿Considera usted que las actividades apoyadas por PC han beneficiado a otras organizaciones? De qué manera? Considera usted que las actividades apoyadas por PC han afectado a otras organizaciones? De qué manera? ¿Cómo evalúa el proceso aplicado por PC para seleccionar organizaciones de la sociedad civil para participar en las actividades del proyecto? (Sondeo – fue justo, accesible?) ¿Cómo pudo haber sido mejor (el proceso)? 19. ¿Qué pudo haberse hecho de manera diferente para mejorar la capacidad de las OSC? (Repeat same questions for GF in this section if the informant has had significant contact with both PC and GF. D. Sustentabilidad: 20. ¿En una escala de 1 a 5, cómo calificaría la credibilidad de PC?¿ GF? Ha cambiado su nivel de credibilidad en los últimos 3 años? 21. ¿En una escala de 1 a 5, cómo calificaría la capacidad técnica de PC? ¿GF? ¿Ha cambiado su nivel de competencia técnica en los últimos 3 años? 22. ¿En una escala de 1 a 5, cómo calificaría la visibilidad pública de PC? ¿GF? ¿Ha cambiado su nivel de visibilidad en los últimos 3 años? 23. ¿Cómo ve usted el rol de PC dentro de la sociedad civil Ecuatoriana? ¿Y el de GF? 24. ¿Qué efecto ha tenido PC en general sobre la sociedad civil? ¿GF? ¿Le parece importante para la sociedad civil que PC y GF continúen actividades similares en el futuro? (En caso afirmativo), Cómo deben continuar? Qué tipo de actividades deben realizar? ¿Deberían otras organizaciones de la sociedad civil u otros actores asumir un rol más activo y abarcar dichas actividades? Cuáles organizaciones? (brindar ejemplos) 25. ¿Cuáles son los mayores factores de riesgo para la sociedad civil si quiere continuar con el mismo nivel de actividades en estas áreas? 21 II. Preguntas para Representantes del Gobierno Nacional (National Government Questions) A. Incidencia y Control Social 1. En general, ¿cómo ve usted el rol de la sociedad civil ecuatoriana hoy en día? ¿Existen muchas organizaciones de la sociedad civil que interactúan continuamente con el gobierno? ¿Cuáles son? ¿Con que instancias de gobierno interactúan más activamente? (Sondeo – que ministerios, rama judicial, rama ejecutiva, a nivel local) 2. ¿Ha tenido usted alguna relación con Participación Ciudadana? ¿Ha tenido usted alguna relación con Grupo Faro? ¿Si la ha tenido, cómo describiría esa relación? 3. Ha escuchado del trabajo de estas organizaciones o de las actividades que llevan a cabo? ¿Cuál es su lectura de las actividades que han impulsado estas instituciones durante los últimos tres años? ¿Cuáles actividades fueron más útiles para su institución y por qué? ¿Cuáles fueron las menos útiles? 4. ¿Cómo ve los esfuerzos de PC y GF para relacionarse e interactuar con el gobierno? (Sondeo – cometarios sobre la incidencia en temas de interés público, vigilancia ciudadana de acciones del gobierno, capacitación en LOTAIP, etc.) 5. ¿Cuáles han sido los resultados de esa interacción? (Ejemplos positivos y negativos) ¿Cuáles han sido las dificultades, si es que las han habido? 6. Ambas organizaciones, GF y PC, han apoyado a otras organizaciones de la sociedad civil para que interactúen con el gobierno más efectivamente a nivel local y o nacional. ¿Cree usted que esto ha generado alguna respuesta positiva o cambio? ¿Las relaciones entre organizaciones de la sociedad civil y el gobierno han sido afectadas positiva o negativamente? (Ejemplos) B. Integración/interacción 7. ¿Qué redes y coaliciones de OSCs conoce? ¿Qué rol están jugando dichas redes y coaliciones en el campo de la sociedad civil? ¿Qué relación tienen estas redes con instancias de gobierno local y/o nacional? 8. ¿Han generado algún cambio en la forma en que la sociedad civil interactúa con – el gobierno, los medios de comunicación, la comunidad internacional? ¿De qué manera? C. Desarrollo de Capacidad Institucional 9. Desde el 2010 GF y PC han trabajado capacitando a otras OSC a través de talleres, donaciones, acompañamiento o coaching, difundiendo/compartiendo información y proveyendo contactos. ¿Tiene usted conocimiento de alguna de estas actividades o de las organizaciones que participaron en ellas? 10. ¿Cree usted que el apoyo y soporte técnico brindado a las organizaciones de la sociedad civil contribuyeron en algo a fortalecerlas? ¿Por qué? (Ejemplos) 11. ¿Ha notado usted algún cambios en la capacidad de organizaciones de de la sociedad civil para relacionarse con instancias de gobierno en los últimos años? Si los hay, ¿a que le atribuye esos cambios? ¿Ejemplos? D. Sustentabilidad 12. En una escala de 1 a 5, ¿Cómo calificaría la credibilidad de PC al interior de las instancias de gobierno? ¿GF? ¿Ha cambiado su nivel de credibilidad en los últimos 3 años? ¿Por qué? 13. En una escala de 1 a 5, ¿Cómo calificaría la competencia técnica de PC? ¿La de GF? ¿Ha cambiado su nivel de competencia técnica en los últimos 3 años? 14. En una escala de 1 a 5, ¿Cómo calificaría la visibilidad pública de PC? ¿Y de GF? Ha cambiado su nivel de visibilidad en los últimos 3 años? 15. PC ha producido una variedad de productos informativos. ¿Conoce usted alguno de estos productos? (Pulso Ciudadano, boletines semanales, “PC en 5 minutos”, comunicados de prensa, sitio web ciudadanía informada, etc.) ¿Cuál su opinión sobre estos productos? ¿Han contribuido positivamente en algo? ¿Quién los usa? Cree que deben mejorarse? Cómo? 16. ¿Dentro de la sociedad civil ecuatoriana, cuál es el rol que han jugado PC y GF? ¿Cómo ve esos roles en el futuro? 17. ¿Le parece importante que PC y GF continúen ejecutando actividades similares en el futuro? Por qué? 22 18. ¿Cómo ve usted las relaciones e interacción entre las organizaciones de la sociedad civil y el gobierno en el futuro? 19. ¿Cree que las organizaciones civiles podrán cumplir con el nuevo marco legal? ¿Por qué? 23 III. Preguntas para Expertos/Académicos (Questions for Experts) A. Incidencia y Control Social 1. En general, ¿cómo ve usted la situación de la sociedad civil Ecuatoriana? Cómo ve usted el entorno para su trabajo? (¿De qué manera las organizaciones de la sociedad civil se relacionan actualmente con el gobierno? ¿Qué cambios o tendencias se están dando? ¿Cuál ha sido el efecto del decreto 16? Y de otras leyes aprobadas recientemente como la de comunicación o el Código Integral Penal?) 2. ¿Qué sabe usted sobre Participación Ciudadana? ¿Qué sabe sobre Grupo Faro y sus actividades para fortalecer otras organizaciones de la sociedad civil? (Ha tenido usted algún contacto directo con esas iniciativas? ¿Si lo ha tenido, de que manera? 3. ¿Cuál es su lectura de la efectividad de las actividades que han impulsado estas instituciones durante los últimos 3 años? ¿Cuáles actividades fueron más efectivas y por qué? ¿Cuáles fueron las menos efectivas? 4. ¿Cómo ve los esfuerzos de PC y GF para relacionarse e interactuar directamente con el gobierno nacional/local? (Sondeo – comentarios sobre la incidencia en temas de interés público, vigilancia ciudadana de acciones del gobierno, capacitación en LOTAIP, etc.) 5. ¿Han generado acciones concretas o cambio alguna de esas iniciativas? (Ejemplos). ¿Si no generaron cambios, cuáles han sido los obstáculos? ¿Cuáles han tenido mayor impacto? ¿Cuáles han tenido menor impacto? ¿Han sido negativas en algún sentido? 6. Ambas organizaciones, GF y PC, han trabajado apoyando otras organizaciones de la sociedad civil para incidir e interactuar con el gobierno más efectivamente a un nivel local y/o nacional. ¿Cree usted que esto ha generado alguna respuesta positiva o cambio? ¿Cree usted que la efectividad de organizaciones de la sociedad civil para incidir en asuntos de interés público ha sido afectada? Si/no, ¿Por qué? (Ejemplos) ¿Las acciones tomadas por GF o PC han tenido algún efecto negativo para la sociedad civil en su conjunto? en el gobierno? o en el marco legal? B. Integración/interacción 7. Hablando sobre las organizaciones de la sociedad civil que usted más conoce ¿Donde acuden estas organizaciones cuando necesitan información o soporte para resolver un problema o mejorar su trabajo? ¿Cuáles son las organizaciones o contactos más importantes para los grupos de sociedad civil que usted conoce? ¿Existen organizaciones de la sociedad civil que son reconocidas ampliamente porque suelen brindar apoyo a otras OSCs? ¿Cuáles son? (sondeo- son estas PC o GF? ¿Si no, por qué no?) 8. ¿De qué manera colaboran y se ayudan entre si las organizaciones civiles en el contexto actual? 9. ¿Qué rol están jugando las redes y coaliciones en el campo de la sociedad civil? ¿Cuál es el beneficio para las organizaciones que participan en redes/coaliciones? 10. ¿Qué tendencias ha visto usted en cuanto a colaboración y coordinación entre organizaciones de la sociedad civil durante el 2009-14 en Ecuador? ¿Cuáles son los factores que han contribuido a avances o retrocesos durante ese periodo? ¿Ha influido PC o GF en este cambio? 11. ¿Qué sabe usted sobre la Red Pluralista, el Colectivo y la Confederación de OSC? ¿Han contribuido a la sociedad civil del Ecuador? ¿De qué manera? En su opinión, cómo podrían ser más efectivas las redes/agrupaciones de OSC? 12. PC ha producido una variedad de productos informativos. Conoce usted alguno de estos productos? (Pulso Ciudadano, boletines semanales, “PC en 5 minutos”, comunicados de prensa, sitio web ciudadanía informada, etc.) ¿Cuál es su opinión sobre estos productos? ¿Han contribuido positivamente en algo? ¿Quién los usa? ¿A quién han ayudado? 13. ¿Cuál es el rol que PC y GF han jugado dentro de la sociedad civil ecuatoriana? C. Desarrollo de Capacidad Institucional 14. Desde el 2010 GF y PC han trabajado capacitando a otras organizaciones de la sociedad civil a través de talleres, donaciones, coaching, difundiendo/compartiendo información y proveyendo contactos. ¿Tiene usted conocimiento de alguna de estas actividades o de las organizaciones que participaron en ellas? 24 15. ¿Cree usted que el apoyo y soporte técnico brindado a las organizaciones de la sociedad civil contribuyeron en algo a fortalecerlas? ¿Por qué? (Ejemplos). ¿En qué cree usted que se han fortalecido específicamente? ¿En qué han podido mejorar? (Sondeo- tiene nuevos servicios? ¿Nuevos donantes? ¿Más miembros?) ¿Por qué si? ¿Por qué no? 16. ¿Cree usted que ese tipo de apoyo/ayuda tuvo un impacto más amplio en el sector de la sociedad civil en Ecuador? ¿Si lo tuvo, como explica ese efecto? 17. ¿Qué pudo haberse hecho diferente para mejorar la capacidad de las organizaciones de sociedad civil? 18. ¿Qué tipo de ayuda/soporte considera usted que es más efectivo para fortalecer a las OSCs? ¿Por qué? 19. ¿Cuáles experiencias o lecciones aprendidas en el área de fortalecimiento de la sociedad civil y participación ciudadana en Ecuador podrían ser aplicables en otros países? D. Sustentabilidad: 20. En una escala de 1 a 5, ¿Cómo calificaría la credibilidad de PC? GF? ¿Ha cambiado su nivel de credibilidad en los últimos 3 años? 21. En una escala de 1 a 5, ¿Cómo calificaría la competencia técnica de PC? GF? Ha cambiado su nivel de competencia técnica en los últimos 3 años? 22. En una escala de 1 a 5, ¿Cómo calificaría la visibilidad pública de PC? ¿Y de GF? Ha cambiado su nivel de visibilidad en los últimos 3 años? 23. ¿Qué efecto tuvo el apoyo de USAID sobre PC desde el 2003? ¿Que efecto tuvo el apoyo de USAID sobre GF desde el 2010? 24. ¿Cómo van a verse afectadas estas organizaciones y sus actividades con el cierre de operaciones de USAID en el Ecuador? ¿Cree usted que esto tendrá algún efecto dentro de la sociedad civil ecuatoriana en general? Cuál? 25. ¿Le parece importante para la sociedad civil que PC y GF continúen actividades similares en el futuro? Por qué? ¿Pueden otras organizaciones asumir un rol más activo y abarcar ese tipo de actividades? 26. ¿Cuáles serían los mayores factores de riesgo para organizaciones de la sociedad civil si quisieran continuar con actividades en estas áreas? 25 IV. FGD Guide for UTPL course participants, Loja Introductions  Introduce our team and purpose of meeting  Ask each one to give name, organization and choose one word to describe what motivated them to sign up for the course at UTPL (on leadership) Questions 1. What were you expecting from the UTPL course? (building on the single words they mentioned) a. To what extent were your expectations met? (probe for usefulness, quality, timing/duration, participation, materials, etc.) 2. Thinking about the various modules and what you learned – what has had the most effect on the way that you work? What do you do differently now? a. What effect has that had on your organization? (probe for sharing with colleagues, changes in modus operandi, etc.) b. How could the course have been better/had more impact? Should the course be offered to others in future? If so, how and to whom? 3. What other kinds of training and assistance have you or your organization received from GF or PC? What is your view of those activities? 4. What are the top priorities for capacity building of CSOs in the current environment? 5. How do you see the situation of civil society in Loja, and Ecuador more broadly, at present? Strengths? Weaknesses? What are the key challenges or obstacles facing civil society? 6. How are CSOs interacting with local government bodies? Has that interaction or relationship been changing? How? What factors have affected that interaction (probe for effects of GF and PC work as well as RP)? 7. How much are citizens engaging with government at local levels? Is that increasing or decreasing? What factors are affecting that interaction? (probe for various mechanisms, support from government or projects/NGOs, etc.) 8. What kinds of networks are you aware of or involved in? What role are those networks playing? Are they effective/useful? If not, why not? 26 V. FGD Guide for members of Confederation, Quito Introductions  Ask each one to give name and describe the first time they heard about El Colectivo or the Confederation  Introduce our team and purpose of meeting Questions 1. What attracted you to join the Confederation? What do you see as its main objectives? (probe about who and where are members, interaction among members, any changes in scope over time) 2. Has the Confederation been able to make progress towards its objectives? What have been the achievements? (probe about interaction with govt, Rendicion de Cuentas, joint position statements) What about challenges or obstacles? 3. How does the Confederation make decisions? Have you felt included in the process of decision￾making? Why or why not? 4. What kinds of support have you received from the Confederation? What other kinds of support should be offered by the Confederation? 5. What do you think of the joint process of Rendicion de Cuentas, facilitated by the Confederation? (probe for who has participated or plans to participate) Is it useful? If so, why? If not, why not? What about high % of admin costs? Could that have a negative effect? 6. Has the interaction of CSOs with government been changing over the past few years? How? What factors have affected that interaction? Do networks such as the Confederation have a role in contributing to these relationships? 7. How do you see the future of the Confederation? How can it raise funds for its operations? How can more members be attracted? What are the most important areas of activity for the Confederation to pursue? Why? 27 ANNEX D: SURVEY QUESTIONNAIRE 28 CSO Survey Questionnaire 1. What is the full name of your organization? ___________________ 2. What is the acronym of your organization? ______________________ 3. What is your position in the organization?  Executive/senior management (direccion ejecutiva o gerencia de organizacion)  Project manager or coordinator (direccion de proyecto)  Technical specialist or coordinator (tecnico/coordinador/especialista)  Administration, finance or logistics (administracion/finanza/logistica)  Other _______________ 4. What is your gender?  Male  Female 5. Which province do you consider to be your organization’s primary base of operations? (choose one)  Azuay  Bolívar  Cañar  Carchi  Chimborazo  Cotopaxi  El Oro  Esmeraldas  Galápagos  Guayas  Imbabura  Loja  Los Ríos  Manabí  Morona Santiago  Napo  Orellana  Pastaza  Pichincha  Santa Elena  Santo Domingo de los Tsáchilas  Sucumbíos  Tungurahua  Zamora Chinchipe 6. What is the primary sector of focus for your organization at present? (max. 2 choices)  Citizen participation  Transparency/accountability  Media/information sharing  Health  Education  Economic development  Community development 29  Women  Youth  Culture  Justice/human rights  Environment/natural resources  Other __________________ 7. How regularly does your organization engage or have direct contact with the following types of people and organizations: (Scale 1 to 5, never – rarely – sometimes – often – very often  Local government officials  National government officials  Staff or leaders of other CSOs  Community members from your targeted population  Media personnel such as journalists  Participacion Ciudadana  Grupo Faro  Universities  Other ______________________ 8. How has the frequency of your organization’s direct contact with other CSOs changed since 2010? much less frequent less frequent no change more frequent much more frequent 9. Do you believe that CSOs should work to influence or monitor the actions of government?  Yes  No  Not sure 10. Does your organization engage in any activities that aim to influence or monitor the actions of government? (choose one)  Yes, only at level of local government  Yes, only at level of national government  Yes, at both levels  No (skips to 13) 11. How has the frequency of such activities changed since 2010? 1=much less frequent 2=less frequent 3=no change 4=more frequent 5=much more frequent 12. How has the effectiveness of such activities changed since 2010? 1 = much less effective 2 = less effective 3 = no change 4 = more effective 5 = much more effective 13. In which network(s) or coalition(s) is your organization currently a member? (multiple choices allowed)  Red Pluralista  Confederacion Ecuatoriana de OSCs  Asociación Red de ONG de Guayaquil (AROG)  None (skip to Q15) 30  Other ____________________ 14. In which network or coalition is your organization most active?  Red Pluralista  Confederacion Ecuatoriana de OSCs  AROG  None (skip to 17)  Other ____________________ 15. Considering the network or coalition in which you are most active (selected in Q.12 above), how useful is that network or coalition to civil society in general? Not useful somewhat useful moderately useful very useful extremely useful 16. How would you rate the services or benefits your organization has received in the following areas since 2010 from the network/coalition that you selected in Question 12 (Not useful, somewhat useful, moderately useful, very useful, and extremely useful)?  Information  Training  Access to funding  Advocacy support  Relations with other organizations in Ecuador  Relations with other organizations outside Ecuador  Other __________________ 17. Participacion Ciudadana has produced a variety of information products. How useful do you consider the following publications and information websites? (Scale of 1 to 5, 1 - not aware, 2 – not useful, 3 – somewhat useful, 4 – moderately useful, and 5 - very useful)  Revista Pulso Ciudadano newsletter  “PC in 5 Minutes” weekly bulletins  Web site Ciudadanía Informada  Other ______________________ 18. Did your organization apply for a grant from either Participación Ciudadana or Grupo Faro since 2010 (From 2011-2012 Grupo Faro was known as Ciudadanía Activa)?  Yes  No (skip to 24) 19. Did your CSO receive a grant from either PC or GF since 2010?  Yes  No (skip to Question 24) 20. When did you receive the grant: (select one or more)  2012 (Grupo Faro or FEPP)  2013 (Grupo Faro)  2013 (Participación Ciudadana) 21. How useful was the grant to your organization? Not useful somewhat useful moderately useful very useful extremely useful 31 22. How useful was the grant-funded activity to the beneficiaries of your organization? Not useful somewhat useful moderately useful very useful extremely useful 23. How would you rate the grant-giving organization’s management of the grants process? very bad bad satisfactory good excellent 24. Both PC and GF have organized capacity building for CSOs, financial support for CSOs, and support for CSO networks or coalitions between 2010 and 2014, among other activities. From which organization did you receive support during that period?  Only Participacion Ciudadana  Only Grupo Faro  Both PC and GF  From neither organization 25. Please indicate the organization with which you has had more contact and received more services since 2010.  Participacion Ciudadana  Grupo Faro 26. /37.Did your organization participate in training supported or facilitated by PC/GF since 2010?  Yes  No (skip to Q31) 27. /38. What type of training did you received from GF / PC? (Choose all that apply)  Cursos de la Universidad Casa Grande (Guayaquil)  Cursos de la Universidad Técnica Particular de Loja (UTPL)  “Marcando la Hoja de Ruta para la Sociedad Civil” programa de capacitación  Otro____________ 28. /39. Approximately how many days of training did your organization receive from PC/GF since 2010?  1 a 5  6 a 10  11 a 20  Más de 20 29. /40. How would you rate the quality of the training organized by PC/GF? very poor quality poor quality satisfactory quality good quality very good quality 30. /41. How would you rate the impact of that training in your organization? none unimportant moderately important very important extremely important 31. /42. How would you rate the overall quality of services or activities of PC/GF? very poor quality poor quality satisfactory quality good quality very good quality 32 32. /43. How would you rate the overall relevance of services or activities of PC/GF? irrelevant somewhat relevant moderately relevant very relevant extremely relevant 33. /44. In what ways has your organization been affected by the support you received from PC/GF? (Each with a scale of 1-5, no improvement, some improvement, moderately better, much better, absolutely better)  Skills of our staff and volunteers  Knowledge of our staff and volunteers  Our relations with local government  Our relations with national government  Our relations with other civil society organizations  Our work with local community/beneficiaries  The quality of our services  The quantity of our services  Our access to information  Our access to grants and other funding sources  Our internal management  Other _______________________ 34. /45. What level of confidence do you have in the information provided by PC/GF? no confidence little confidence moderate confidence much confidence absolute confidence 35. /46. How important is PC/GF to civil society in Ecuador? Unimportant somewhat important moderately important very important extremely significant 36. /47. What are the two most important services or types of support provided to civil society by PC/GF? (choose max. 2)  Information dissemination  Training  Funding  Advocacy to promote civil society participation in public interest topics  Support to networking and coalitions  Coordination among civil society organizations  Other ________________________ 48. Did your organization participate in any sustainable coaching activity or strategic planning organized by Grupo Faro?  Si  No (Skip to 53) 49. How would you rate the coaching activities’ quality? very poor quality poor quality satisfactory quality good quality very good quality 50. How would you rate the overall relevance of the coaching activities? irrelevant somewhat relevant moderately relevant very relevant extremely relevant 33 51. How would you rate the impact of the coaching activities? none unimportant moderately important very important extremely important 52. How would you rate the impact of the coaching activities for the sustainability of your CSO ? none unimportant moderately important very important extremely important 53. What is the most significant risk faced by your organization at present? (choose one)  Lack of funding  Political situation  Legal framework for civil society  Internal problems  Lack of internal capacity/skills  Other ____________________________ 54. Who do you believe can help you the most in addressing that risk? (choose two)  International agencies/donors  Other actors outside of Ecuador  Confederacion Ecuatoriana  Participacion Ciudadana  Red Pluralista  Grupo Faro  Local government  National government  Universities  Other ___________________________ 34 ANNEX E: DESCRIPTION OF CSO SURVEY PROCESS 35 Summary of CSO Survey Process 1. Selection of target populations The survey process for this evaluation began with the identification of the target population. This was largely defined by USAID in the Request for Proposal (RFTOP), where it was suggested to carry out three mini-surveys among various CSOs in the following words: “The purpose of the mini-surveys for this evaluation is to reach out to a majority of partners and beneficiaries of the PC and GF Projects, particularly those with whom the Evaluation Team will not meet personally during the limited field phase. One mini￾survey should be conducted to small grant recipients while another to other beneficiaries. Members of the CSO networks and coalitions that PC and GF have promoted and supported should also be surveyed.” After learning the details of the two projects to be evaluated, Mendez England & Associates (ME&A) decided to conduct a single survey with multiple targets, rather than three mini-surveys with very small target populations. This decision was based on the realization that there was significant overlap among the three sub-sets of CSOs defined in the RFTOP. In this manner, we sought to avoid burdening respondents with repeated requests to complete separate surveys, and to enhance the data analysis process, since all responses would automatically become part of the same data set and could easily be compared. In order to achieve a broad coverage of the universe of CSOs affected in a significant way by USAID civil society programming, the Evaluation Team (ET) decided to survey all those CSOs that had directly participated in the principal CSO-directed activities of the two projects since late 2010, including those involved as sub-grantees, participants in intensive training, beneficiaries of sustainability coaching, and members of two key networks (the Red Pluralista, and those who had been members of El Colectivo prior to it becoming the Ecuadorian Confederation of CSOs). 2. Selection of methodology The RFTOP suggested that “the survey respondents will be contacted either by phone or face-to-face”. However, in order to reach a greater number of CSOs and ask a larger number of questions, ME&A proposed a web-based survey. As internet access was understood to be high among CSOs that participated in the project, this was identified as the best option in order “to ensure confidential, easily accessible, and fast methodology of collecting and analyzing data” (as stated in the evaluation workplan). To conduct the survey, ME&A chose to use Survey Monkey due to its user-friendly nature for both survey administrators and survey respondents. When choosing the survey methodology, ME&A realized that not all targeted informants would have sufficient access to Internet to readily respond to an online survey, and also anticipated that some CSOs might be nervous about participating in any kind of survey related to these particular projects and implementers, given the context in Ecuador. Therefore, the online survey was supplemented with telephone interviews for those respondents who may have difficulties with Internet access, and by telephone follow-ups to boost response rates. That process is described further below. 3. Design of questions The evaluation workplan (which was finalized once the ET was in country) defined the main subject areas of the survey as follows: “For recipients of sub-grants, the survey will explore how CSOs view the financial support received, how the work of those CSOs has been affected, and how the small grants were administered, among other topics. For those who participated in intensive capacity building activities, the focus of questions will be on changes in structures, activities, alliances, and sustainability linked to participation in the project(s). With respect to members of the major CSO networks and coalitions that the PC and GF Projects have supported, questions will be oriented towards understanding the range of CSO perspectives on the overall utility of such networks and coalitions, and the effect that membership in those bodies has had for the CSO members.” The priority topics for the survey were based on those issues, with some other general questions added as well. 36 The design of specific questions was largely guided by the evaluation matrix contained in the workplan, which defined the evaluation questions in detail. The aim was to try to cover as many areas of investigation as possible, insofar as they related to the projects’ work with and effects on civil society. A number of demographic and other identifying questions were also included. Survey questions were closed-ended, although in some cases respondents had the option to select “other” and explain the details of that choice. The most difficult part of designing the survey questionnaire was the fact that not only were targeted CSOs involved in a variety of different activities, but they could also be placed into three broader categories: those who worked only/primarily with PC, those who worked only/primarily with GF consortium, and those few who worked with both projects. In order to ensure that the survey questions were relevant for each respondent, and to make it possible to differentiate responses relating to each project and implementer, it was decided to create a sub-set of 11 questions that would be answered by respondents in accordance with which project they had MORE interaction with since 2010. One result of the breadth of subjects to be covered by the survey was that the questionnaire ended up rather long – 54 questions in total, some of them with several sub-questions, although no respondent was asked to answer more than 44 questions because of the “split” described above. Many respondents in fact answered less than 44 questions, because there were “skips” built into the survey structure. For example, if a respondent indicated that they had not received a grant, they would automatically be skipped past the questions related to grants. Otherwise, almost all of the questions were mandatory – meaning that if the respondent tried to pass to the next page without answering all questions, they would not be allowed to do so (so, the only way to avoid completion was to close the page). 4. Collection of contact info and testing of survey The next step in the process, which was ongoing while questions were being defined and refined (and translated), was the collection of contact information from the implementing organizations. This involved several exchanges of emails and review of lists of contacts provided by PC, GF and Esquel Foundation. Since those lists were incomplete, the ET had to conduct extensive research on the Internet to find email addresses and phone numbers for many CSO respondents. Once the lists were in place and the questions set up in Survey Monkey, the ET proceeded with testing the survey in two different ways. First, the survey instrument was tested to ensure that the mechanics of the survey were working properly. The other type of testing was with a group of 10 potential respondents from the list of participants in the UTPL course organized by the GF Project (since those were the first targets for whom we had email and phone contacts.) Ten names were selected at random from that list of 129 participants, and emails were sent on 22 April 2014 to explain the purpose of the survey, and ask them to complete the survey at the link provided in the email within a few days. Assurances of confidentiality were included, as well as reference to the independence of the ET. (We decided not to ask respondents to include their own names in the survey, due to potential sensitivities linked to the political context. However, the survey did ask for the name of their CSO, so that follow-ups would be possible.) When no responses were received for two days, telephone calls were attempted and emails were sent again on 24 April, setting a deadline of 26 April for completion. By the time the deadline ran out, only two individuals had completed the survey. At that point, the ET made phone calls to the others on the list, to verify if they had received the email, offer assistance, and enquire about any resistance to completing the survey. We then had our first difficulties with the contact list, but we did manage to reach three of the intended respondents, and learned that at least one had suspected that the email was spam or contained a virus, while the others had simply not paid attention to the email. Based on that learning, we decided to make the subject line of the survey email request more specific. For each group of respondents (grantees of PC, grantees of GF, etc.) we developed more specific subject lines, rather than using a generic subject that might not catch the interest of the recipient. We also decided to include a deadline (3-4 days after sending) for completion of the survey. 5. Emailing of survey 37 Although the test of the survey had generated few responses that could be analyzed, the test had not brought to light any problems with the survey tool. Therefore, the ET decided to proceed with the mass emailing without further testing, due to time pressures. Therefore, the next step was to email the survey on 27-28 April 2014 to the entire list of 221 potential respondents. As with the test, the email included a short introduction to the survey and explanation of its purposes, together with assurances of confidentiality of responses. Respondents were asked to complete the survey by 30 April 2014. At least 25 of the emails were rejected by servers as undeliverable or no longer valid. 6. Follow-up process The chart below shows the response pattern for the survey, indicating the two that responded to the test on 25 April, then the rush of responses on 27-29 April to the first mailing. Once the survey was emailed, the process consisted of twice-daily tracking on Survey Monkey to see how many responses had been received, and to check how many of them were complete (the entire survey finished) in order to identify any problems early. Follow-up phone calls by the ET began on 1-2 May, to verify receipt and investigate any difficulties. 28 people had completed the survey by 3 May, at which point follow-up emails were sent to ALL the targeted population. On 7-8 May, the next round of phone calls began, focused on the other types of respondents, whose organizations generally had closer or more recent links to the projects. 38 The table below gives an idea of the responses that were received from targeted individuals who were reached by telephone: Type of respondent Said they had already sent Promised to send Not wanting to participate UTPL student 13 28 2 UCG student 4 Grantee CSO member 8 9 CSO participants in coaching 1 5 El Colectivo member 2 19 Red Pluralista member 6 3 CSO participant in Marcando la Hoja de Ruta 5 TOTAL 30 73 2 In addition to the above-noted 105 calls, there were at least as many calls made to numbers that did not work, were busy or rang without being answered. It was interesting to note that of all those contacted by phone, none expressed interest in doing the survey by telephone (with the ET member entering the responses on her computer), although this option was offered. Only two respondents indicated (though not in so many words) that were just not interested in doing the survey. With 62 surveys completed, it was decided to suspend follow-ups on 10 May 2014—thirteen days after the survey was mailed to all respondents—and proceed to the analysis stage. 7. Response level Research of the response rates to other surveys complemented by the experience of ME&A and the ET members indicates that there is a vast number of variables that can affect the rate of survey response, including the closeness of the respondents to the activity/service that is being evaluated, invitation wording, perceived benefit from participating in survey, and methodology used (telephone, email, email plus phone calls, etc.) According to one website, “Internal surveys (i.e. employee surveys) generally have a much higher response rate than external surveys (e.g. customer satisfaction surveys). Internal surveys will generally receive a 30- 40% response rate or more on average, compared to an average 10-15% response rate for external surveys. 81 An Ecuadorian statistical expert consulted by the ET following the survey advised that “30% for an Internet survey is high; normally one would expect 10-20%. It depends a great deal on the target group, but even so, 30% is high." In addition, the particular environment in which the survey is carried out can have a significant impact, especially for web-based surveys. Not only are there factors related to access to the Internet (and cost thereof to the respondent), but also one needs to consider the level of comfort and familiarity respondents have with doing surveys on the Internet. In this specific situation, there were more complex factors to consider as well, such as the fact that both projects had ended and that USAID was known to be leaving the country shortly under tense circumstances. These elements could well have affected the motivation of 81 http://www.surveygizmo.com/survey-blog/survey-response-rates/ The same site indicates: “Response rates can also fall below 2% when the respondent population is less-targeted, when contact information is unreliable, or where there is less incentive or little motivation to respond.” 39 respondents to participate (what would they gain?), and some may have been nervous of being linked with USAID through the survey. Taking into account all of the above (including the significant difficulties experienced with contact information), the 28 percent response achieved with this survey should be considered a good response rate. It should be noted that the follow-up phone calls played a crucial role in achieving that rate; without that significant effort, the response rate would probably have been no higher than 20%. 8. Analysis Survey Monkey generates automatic analysis of responses to each question in the form of tables and graphs, and is also capable of doing some cross-tabulation. However, Survey Monkey does not allow for advanced control over cross-tabulations and more in-depth analysis. Therefore, after data collection was complete, the database of responses was converted to SPSS format and sent for further cleaning and analysis by the ME&A data analysis expert. Relevant variables labels were changed to PC or GF accordingly. Special cleaning rules were applied to check for any errors. For facilitation of data analysis, special coding was used to treat multiple choice questions. As well, case status variables were added to identify complete and partial “cases” (surveys either completely or partially filled), taking into account that not all questions were mandatory for all respondents. For cross-tabulations and other data extraction, an SPSS syntax file was created in order to employ automated/replicable procedures during analysis of the data. Descriptive statistics along with the cross￾tabulation were created for all variables and pivot tables were outputted for key variables like grantees and non-grantees, trainings received from GF or PC, etc. 40 ANNEX F: LIST OF DOCUMENTS REVIEWED 41 I. USAID documents  Request for Applications for Strengthening Civil Society in Ecuador (RFA-518-10-000003), 2009  Cooperative Agreements and modifications for both projects, 2010-2014  Democracy and Governance assessment, 2005  Democracy and Governance assessment update, 2009 II. Project Documents A. GF Project (and Grupo Faro)  2010 Proposal for SCS Project by Grupo Faro consortium, and 2012 proposal for revised third year of project  Final report – November 2013  List of sub-grant recipients and evaluation matrix for proposals  Sub-grant reports, indicator tables and related documents  Quarterly Progress Reports (since 2011)  Annual operational plans 2011- 2013  M&E indicator tables  Project budgets and amendments, 2011-13  Organizational Capacity Assessments of Grupo Faro and related documents (2012-2013)  2013 Budget (GF)  2008-2013 Audit Reports (GF)  Table of Income and Sources, 2012-14 (GF)  Investigation and Action Agenda, Grupo FARO 2012-2015.  Sub-grants competition announcements (2012 and 2013)  Premio a la Innovación Cívica, 2013  Legal Manual for the operation of Ecuadorian CSOs. “Manual Legal para el Funcionamiento de las Organizaciones de la Sociedad Civil del Ecuador”, GF, 2013  Gender perspective in the management of CSOs. “Incorporación de la perspectiva de género en la gestión de las organizaciones de la sociedad civil”, GF, 2013  CSO networks: benefits and services. “Redes de OSC: beneficios y servicios”, GF, 2013  UCG and UTPL Courses Diagnostic Report. “Evaluación y Rediseño del Programa de Formación para Líderes de Organizaciones de la Sociedad Civil”, Diagonal, June 2013  Diagnostic report on sustainability coaching activity, 2013  Statute of Ecuadorian Confederation of CSOs, 2013  Sistematización: elaboración de la Planificación Estratégica de la Confederación Ecuatoriana de OSC (description of Confederation’s strategic planning process) B. Strengthening Democracy Project (and Participación Ciudadana)  Monitoring and Evaluation Methodology (annual)  Quarterly Progress Reports since 2003  Annual operational plans since 2009  M&E indicator tables  Budget information, 2010-14  Table of Income and Sources, 2002-2014  Draft Sustainability Plan, 2010  Organizational Capacity Assessments and related documents, 2013  Sub-grants competition announcements, 2013  Training materials “Marcando una hoja de ruta para la sociedad civil”, PC, 2010  Bulletins “Marcando una hoja de ruta para la sociedad civil”, PC, November 2010  Memory “Marcando una hoja de ruta para la sociedad civil, PC, January 2011  Monitoring public management: LOTAIP, Vigilancia de la Gestión Pública, PC, 2012  PC Annual Reports, 2009-2013  Special edition “PC ten years: 2002 – 2012”  Red Pluralista Founding Document, PC, 2013 42  PC Statutes, approved 2005  PC Strategic Plan, 2005-2010  Guía para la Aplicación del Decreto 16, PC, 2013 III. Other Documents  Legislation and Decrees a. Constitución Política del Ecuador b. Ley Orgánica de Transparencia y Acceso a la Información Pública (LOTAIP) c. Ley Orgánica de Participación Ciudadana d. Ley Orgánica del Consejo de Participación Ciudadana y Control Social e. Ley Orgánica de Comunicación f. Ley Orgánica de Educación Superior g. Decreto 699/2007 (international cooperation) h. Decreto 429/2010 (international cooperation) i. Decree 982/2008 (NGO regulation) j. Decree 16/2013 (NGO regulation)  El Poder de una Ciudadanía Informada, PC, 2004  Ecuador 2012 Human Rights Report, US Department of State, 2012  New Directions in Local Capacity Development: Embracing a Systems Perspective, The Capable Partners Learning Agenda on Local Organization Capacity Development, 2013  La Silla Vacía: el nuevo escenario para el diálogo de la sociedad civil y el Estado. Un diagnóstico, Esquel, CEDA, Grupo Faro, Participación Ciudadana, 2011  Informe Final del Índice de la Sociedad Civil en el Ecuador, Fundación Esquel y CIVICUS –Alianza Mundial para la Participación Ciudadana, 2006  The role of Legal Reform in supporting Civil Society: an introductory primer, ICNL – UNDP, 2009.  La relación Estado – Sociedad Civil: develando mitos, 2009  Manifiesto del Colectivo de Organizaciones de la Sociedad Civil (OSC) sobre el proyecto de reglamento para personas jurídicas de derecho privado con finalidad social y sin fines de lucro, January 2011  Pronunciamiento de la Confederación Ecuatoriana de las Organizaciones de la Sociedad Civil sobre el Decreto 16, 2013  Informe Sobre Derechos Humanos, UASB, 2011  Informe Estado del País 1950-2011, Varios Autores, 2011  Informe IECG Ecuador, Fundación Esquel, 2011  Ecuador Human Rights Report, US Department of State, 2012  Cultura política de la democracia en Ecuador, 2012  Temario de Consultas Sobre Sociedad Civil, Fundación Esquel, undated  Ecuador: Clampdown on Civil Society, Human Rights Watch, 2012  Ecuador: Un Régimen Híbrido, Simón Pachano y Sergio García, Fundación Latinoamericana de Ciencias Sociales, 2013  Sistematización: Elaboración de la Planificación Estratégica de la Confederación Ecuatoriana de OSC, Fundación Metis, 2013  How Transparent are Think Tanks about Who Funds Them?, Transparify, May 2014  LAPOP Americas Barometer 2004-2014, draft report shared with USAID/Ecuador in June 2014 IV. Websites  Grupo Faro  Participación Ciudadana http://www.participacionciudadana.org/pc10/  Ciudadanía Informada http://www.ciudadaniainformada.com/  USAID/Ecuador http://www.usaid.gov/where-we-work/latin-american-and-caribbean/ecuador  Confederación Ecuatoriana de OSC http://www.confederacionecuatorianaosc.org/ 43  Websites of various sub-grantees and sub-contractors, including http://guiaosc.org/ , http://cecimec.org/ , http://www.kawsay.org/home.aspx , http://www.ecolex-ec.org/ , http://www.fundapi.org/ , http://www.incluirecuador.org/  Consejo Nacional Electoral http://www.cne.gob.ec/  Consejo de Participación Ciudadana y Control Social http://www.cpccs.gob.ec/  Asociación Red de ONGs de Guayaquil (AROG) http://ongecuador.blogspot.com/2011/05/10-anos￾de-creacion-de-arog.html  Latin American Public Opinion Project (LAPOP) http://www.vanderbilt.edu/lapop/  International Center for Non-Profit Law (ICNL) – page on Ecuador http://www.icnl.org/research/monitor/ecuador.html  Barcelona Center for International Affairs – Ecuador profile and associated documents http://www.cidob.org/es/publicaciones/dossiers/ecuador_elecciones_2013/dossier_especial_ecuado r_elecciones_2013  Transparify http://www.transparify.org/  Human Development Indicators http://www.datosmacro.com/idh/ecuador  Catalogue of International Cooperation (Secretariat of International Cooperation of Ecuador) http://www.cooperacioninternacional.gob.ec/catalogo-de-la-oferta-de-cooperacion-internacional￾no-reembolsable-2/  It’s (almost) all about leadership – INTRAC blog by Rick James http://www.intrac.org/blog.php/54/itacirceurotrades-almost-all-about￾leadership?utm_source=It%27s+%28almost%29+all+about+leadership+- +New+blog+post&utm_campaign=Blog_update_leadership&utm_medium=email  Close the NGOs: Asserting Sovereignty or Eroding Democracy? | North American Congress on Latin America https://nacla.org/blog/2013/12/31/close-ngos-asserting-sovereignty-or-eroding￾democracy  Pretensión de Coerción o Cierre de 31 Organizaciones de Sociedad Civil ~ RedLad Blog http://blog-redlad.blogspot.com/2014/04/pretension-de-coercion-o-cierre-de-31.html  Ciudadanía Informada: Citizens’ Participation - Ecuador — IICD  Participación Cantonal http://participacioncantonal.org/sobre-este-sitio/  La libertad de prensa en Ecuador vive una continua confrontación http://www.eluniverso.com/noticias/2014/05/04/nota/2912431/libertad-prensa-ecuador￾vive-continua-confrontacion  Hoy newspaper, Interview with Ruth Hidalgo, July 2012 http://www.hoy.com.ec/noticias￾ecuador/quinto-poder-utilizado-politicamente￾557252.html?fb_action_ids=4060843074499&fb_action_types=og.likes&fb_source=timeline_og&act ion_object_map={%224060843074499%22%3A10151090018367249}&action_type_map={%224060 843074499%22%3A%22og.likes%22}&action_ref_map  Hoy newspaper, article on freedom of press in Ecuador, May 2014 http://www.hoy.com.ec/noticias￾ecuador/ecuador-esta-en-el-grupo-de-paises-sin-libertad-de-prensa-605638.html  Centro Virtual para la transparencia y la rendición de cuentas de la sociedad civil http://rendircuentas.org/ 44 ANNEX G: HISTORICAL SUMMARY OF PARTICIPACIÓN CIUDADANA CORPORATION AND ITS COLLABORATION WITH USAID 45 Historical Summary of Corporación Participación Ciudadana and its Collaboration with USAID 2002 – 2014 General Objective The main objective of Corporación Participación Ciudadana when founded in 2002 was to contribute to building a democracy based on both an ethical and political commitment between the ruling party and the citizens of Ecuador, through the exercise of citizenship in a responsible, informed and deliberative way. As of 2014, the overall objective had evolved to become the following: Corporación Participación Ciudadana is a non-partisan and pluralistic Ecuadorian nongovernmental nonprofit organization, legally constituted in July 2002 to provide input from civil society to strengthen democracy in Ecuador. It is made up of citizens who promote the deepening of democracy and transparency of public activities and political processes. Creation The origin of PC dates back to 2001, when a group of Ecuadorians, including Valeria Merino (Executive Director of the Latin American Development Corporation), Horacio Sevilla (Former Ambassador of Ecuador to Peru), and Cornelio Marchán (former President of the Esquel Foundation), saw the need to establish a citizen group to oversee electoral processes in the country to ensure transparency of the process and validity of the results. The proximity of the 2002 national elections prompted them to create PC as a corporation, and its legal status was approved on July 12, 2002. In a general meeting of the founding members, César Montúfar, an academic and pro-democracy activist, was chosen as Executive Director. PC opened its offices in Quito and Montúfar immediately began negotiations to launch the process of electoral observation. He contacted the National Democratic Institute (NDI), a US-based NGO, in order to receive advice from that institution. That link eventually led to NDI supporting the first election monitoring by PC in October 2002, with financial support from USAID.82 Leadership PC has had three executive directors during its 12-year history. The first was Cesar Montúfar, who led the organization since its inception until early 2006. PC volunteers and other stakeholders remember him as a charismatic and hands-on leader who was very focused on working with the people. He passed the baton to his Deputy Director and former Legal Director, José Valencia, who headed the organization from early 2006 until mid-2007.83 With a more technical and less citizen-focused political advocacy approach, Valencia began a systematic search of funds other than USAID and sought to strengthen the institutions of PC through the adoption of a procedures manual, a code of ethics and a five-year strategic plan term. In 2008, Ruth Hidalgo took over as Executive Director, at which point President Correa was in power and the political landscape was changing (as outlined in the Context section of this report). Since its creation, and at specific points in time, PC has faced hostility on the part of state institutions, limiting their field of action. Government questioning of PC’s independence (including references to Cesar Montúfar’s current role as an opposition politician, international funding support to PC (including from USAID) and certain activities that have challenged government actions) have placed PC in a vulnerable position, and led its current Executive Director to be criticized in the official media. Despite this situation, PC has made major efforts to maintain high visibility and credibility. Financing PC has consistently received significant funding support from USAID since it was founded, as can be seen in the graph below. The annual funding has gone as high as $1.5 million, though in the last five years it has 82 "We worked with NDI to prepare a proposal for Qualitative Observation and Quick Counts of the Electoral Process. This was initially approved with a value of $160,000, and then received an extension of an additional $250,000. For this expansion, the efforts of USAID office in Quito were crucial.” El Poder de una Ciudadanía Informada: Elecciones 2002, Crónica de una Experiencia, Corporación PC, 2003. 83 Jose Valencia is currently the Ecuadorian Ambassador in South Africa. 46 averaged $790,000. Since 2004, the institution has approached other donors, including various international organizations, with the aim of generating and strengthening international linkages and establishing alternative sources of financing. Nevertheless, the diversification of funding has always been a challenge for PC. PC Funding Amounts and Sources 2002-2014, according to organizational records Institutional Development Between 2003 and 2004, members of the institution proposed to consolidate PC as a nonpartisan civil society group at both national and local levels. Several meetings between PC’s Board of Directors, the technical team, and the first volunteers were held for this purpose. In 2003, the online news agency "Informed Citizenry"84 began operations in Quito and its coverage soon spread to other cities with the support of local correspondents. The website became an important part of PC’s operations, particularly during electoral events. In 2003, PC created a Volunteer Network comprised of 26 teams and 25 Youth Clubs. A volunteer coordinator was appointed in each province of Ecuador, with three canton-level coordinators in Guayas. In 2005, the youth training program called “School of Leadership” was created to improve volunteers’ skills, and in 2006, modules on Democratic Theory, Constitutional Law and Political Communication were included in the training curriculum. However, between 2011 and 2014 the network was weakened, as PC’s monthly funding support for volunteer coordinators’ operational costs was suspended due to overall budget limitations and difficulties to obtain other donors funding, and the level of attention paid by the central office to the network decreased. PC’s satellite office in Guayaquil was closed for the same financial reasons, and the completion of specific projects based in Guayas such as monitoring of compliance with access to information laws by national and provincial institutions, and the youth training project Vive la Democracia. In 2004, PC opened an office in Guayaquil because it was considered essential to have a permanent presence on the Coast of Ecuador. The regional office, led by the lawyer Anunziatta Valdez, a strong 84 www.ciudadaniainformada.org $0 $200,000 $400,000 $600,000 $800,000 $1,000,000 $1,200,000 $1,400,000 $1,600,000 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 F U N D I N G All Other Funders USAID total 47 gender advocate and former member of the Congress,85 focused on gender issues, developed an intensive collaboration with volunteers in Guayas province for electoral observation, with school-age youth through the project "Living Democracy", and with various government institutions for monitoring of compliance with the Access to Information Law. Due to the reduction of financial resources from USAID, difficulties in accessing other funding, and a lower level of activity in Guayas (especially on Access to Information Law), the Guayaquil office was closed by PC in 2013. By 2005, PC had developed (a) a system for monitoring and evaluating its impact; (b) a five-year Strategic Plan; (c) a network of internal communication for better coordination and interrelation between PC’s different branches; and (d) a system for external communication. In 2009, PC began to publish annual reports known as “memorias” to present their work to external stakeholders. Although a strategic planning process was held in 2010, the resulting draft plan was never adopted. The annual operational plan was the guiding document for activities of the organization. In 2013, an Organizational Capacity Assessment was conducted on the basis of self-evaluation by the organization, leading to the identification of several areas for internal strengthening. Activities The early initiatives of PC were largely focused on election and referendum monitoring, civic campaigns, and advocacy and watchdog activities, all with a considerable emphasis on educating and mobilizing citizens. Changes in the political context in 2007 and the new 2008 Constitution prompted a change in PC’s focus and approach. In 2009, PC’s Democracy Strengthening Project shifted its emphasis from “electoral reform and observation to the promotion of greater transparency in other important processes, such as the establishment and functions of new democratic institutions to be organized under the Constitution”. Since 2010, PC has reduced its focus on grassroots volunteer-based action to place a heavier emphasis on direct implementation relying on the technical skills of its staff. PC has developed its advocacy and watchdog skills in relation to the new structures and systems created by the Constitution, and provided training about those systems to government officials, CSOs, CBOs and individuals. The following is a description of the evolution of PC’s work since it was established, based on five main categories of intervention: Citizen Oversight, Citizen Action, Dialogue and Governance, Democracy Education and Informed Citizen. 1. Citizen Oversight Objective: To promote transparency in political activities and processes. 1.1. Citizen Participation Mechanisms: Silla Vacía: Between 2010 and 2013, Silla Vacía proceedings were conducted in 10 municipalities, technical assistance was provided to six cantons, and a total of 5000 citizens were trained on this mechanism. Accountability: Beginning in 2005, PC developed a methodology for a system of ongoing accountability among local government officials such as Prefects, Mayors and Presidents of Parish Boards. In 2006, four Accountability Forums were held. Between 2010 and 2013, forums were held in three prefectures, 18 municipalities and five parishes, and a total of 4820 people participated. Results: Participants are more aware of their rights and now know that elected officials must be accountable to citizens. Obstacles: PC’s activities encountered the same challenges imposed by the political context on any CSOs seeking to contribute to strengthening democracy: the political class viewed it with suspicion and some actors alleged that PC was not representing the people. As well, PC was not able to provide sufficient support to keep these processes alive because of funding and staff limitations. 1.2. Citizen Observatories and Oversight Committees: In 2005, PC monitored the appointment of the judges of the new Supreme Court. In 2006, PC held meetings with representatives of civil society86 to establish agreements to enhance the work of PC’s Observatory of the National Congress. In 2007- 85 Now known as the National Assembly 86 Fundación Esquel, Grupo Faro, CONAMU, Escuela de Leyes de la Universidad de San Francisco, Foro de la Niñez y de la Adolescencia 48 2008, PC oversaw the work of the National Constituent Assembly and in 2008, the Assembly’s Legislative Commission. From 2009-2010 PC oversaw the selection process of the new national authorities established to support citizen participation (including the Citizen Participation and Social Control Council). Unfortunately, PC’s recommendations had limited impact on the process and the ultimate selections. Obstacles: Political pressure from national government to support specific candidates was reportedly a key factor affecting the transparency and neutrality of the process. 1.3. Election Monitoring87: PC observed the elections of 2002, 2004, 2006, 2007, 2008, 2009 and 2013 and conducted “Quick Counts” in 2002, 2004, 2006, 2007, 2008, 2009 and 2013. PC organized the following voter education campaigns: "Responsible and Informed Vote" (2004); "Getting Informed, Comparing and Choosing" (2007)88; "Clean Walls, Honest Candidates" (2008); "The Value of Choosing" (2011); and "Because your Vote has Power" (2013). From its beginning, PC has run campaigns against the use of public resources for political publicity. Before 2006, PC also tracked the process of official review of expenditures of several elections. Results: Through a rigorous and systematic methodological work, PC reached a high level of credibility, transparency and visibility in election monitoring. Large numbers of volunteers have consistently been mobilized by PC for elections. Obstacles: Tracking these processes was challenging, especially before 2007 as the Supreme Electoral Court and provincial courts tried to impede PC’s actions in some elections. Logistics of organizing observation activities across the country were always complex and confronted different challenges, depending on the specific norms determined by the electoral institutions. In 2006, PC proposed to the TSE the design of an Electoral Information System but it was never implemented. 1.4. Promoting Transparency: In 2005, PC supported the drafting and promotion of the Ley Orgánica de Transparencia y Acceso a la Información Pública ((LOTAIP) Organic Law on Transparency and Access to Public Information) through the local NGO network “Access Coalition”, and generally promoted the Right of Access to Public Information. In 2006, it began the "Monitoring the implementation of LOTAIP in 10 public entities in Guayaquil" project. Between 2008 and 2012, PC monitored 30 state institutions’ compliance with the Law, jointly with the Ombudsman. Results: The outcomes were satisfactory: 98% compliance, six institutions developed action plans to improve services, and 12 did assessments of user satisfaction. Obstacles: Lack of consistent political will from the Ombudsman Office to replicate monitoring among other government institutions in accordance with their mandate has limited the impact, especially after the end of PC’s intervention (since 2012), though some institutions are continuing to strive for compliance. 1.5. Legal Reforms and Policy Proposals: PC proposed several law reforms between 2004 and 2006, including reforms to the Organic Law on Electoral Spending and the Law on Electoral Publicity. The final text included several issues raised by PC, such as advertising slots, accountability in spending limits, proportional penalties, and transparency in the reporting of information. PC also worked in 2006 on draft laws on Impeachment and Accountability. In 2008, it proposed constitutional reforms and reforms to the Election Law, and in 2009, it proposed ideas for the Law of Citizen Participation and Social Control, and for the Organic Code of Territorial Organization, Autonomy and Decentralization. In the years that followed, PC proposed amendments to the Code of Democracy (2010), Communications Law (2011), and Access to Information Law (2012). Between 2013 and 2014, PC analyzed Decree 16 and the Regulation for Electoral Campaigning. Obstacles: Since 2007, the spirit of the bills introduced was usually diluted in the debates of the National Assembly. 2. Citizen Action 87 Electoral Calendar 2002–2014. National Elections: October 20 and November 24, 2002; October 15 and November 26, 2006; April 26, 2009; and, February 17, 2013. Sectional Elections: October 17, 2004; April 26, 2009; and, February 23, 2014. National Constituent Assembly Elections: September 30, 2007. Referendum: September 28, 2008. Popular Consultation: April 15, 2007, and May 7, 2011. Source: National Electoral Council. 88 Meetings were held with Grupo Faro y Alianza Equidad to coordinate activities. In 2007, Grupo Faro’s “Agreement of Commitment for Transparency and the Protection of Public Funds for Social Programs in the Electoral Campaign” was adopted. 49 Objective: To encourage citizen participation through the exercise of political rights and the practice of civic values. 2.1. Media Monitoring: PC has systematically monitored the use of public resources for official publicity and personal promotion by public authorities. Between 2004 and 2008, PC published 50 monitoring reports. In 2008, PC sought to promote greater action by control agencies such as the Comptroller General, the National Electoral Council and the Electoral Disputes Tribunal, so that they would exercise their roles more effectively. PC has gathered information on key aspects of the use of media spaces (print, radio and television) for political purposes. It promoted research and discussion on the impact of violence on national television and supported processes of informed debates and search for consensus on how to handle the problem of violence. Results: Between 2009 and 2012, 34 monitoring reports on the use of public resources for official publicity were published. This activity continues to date with NED funding. Obstacles: The government has repeatedly questioned PC’s media monitoring reports, and finally PC had to change its methodology from measuring the amount paid by government to measuring the media time used by the government for self-promotion. This resulted in a reduction of impact since the dollar values were of more interest to the media and the public than the data on air time.89 2.2. Research: In 2006, PC identified specific qualitative indicators to measure the democratic culture of the country, based on studies by the Latin American Public Opinion Project (LAPOP) and local research, which led to publishing of the "What do Ecuadorians Think when Speaking about Democracy?" study and a follow-up study on “Citizen Perceptions on Democracy in the Ecuador of 2012”, published in late 2013. In 2010, it conducted an analysis of CSOs’ activity and interaction with different sectors of the population, other similar organizations and the State in Quito and Guayaquil. PC sought to know: (a) if the new political and constitutional scenario was positive or negative for promoting civil society activities, and (b) how CSOs were affected by its members joining the new government. Results: A baseline survey on the subject of democracy and values; and a qualitative study for the project "Creating a Roadmap for Civil Society" which ran between 2011 and 2012. 2.3. Other activities: Between 2010 and 2011, PC promoted: (a) steady support to the group of public institutions being monitored for compliance with the Access to Information Law; (b) a project called "Monitoring the Work of the Constitutional Court, the National Court of Justice, the Attorney General, and the Management Unit of the Public Defender's Office” which generated a comprehensive report; and (c) a project called "Evaluation of the Soundness of Constitutional Court Sentences” (2008-2009). 3. Dialogues and Governance: Objective: To promote dialogue and generate consensus between political and social actors. In 2004, PC held Citizen Forums to present the work plans of candidates running for mayors and provincial prefects. In 2006, 2009, and 2013, PC organized citizen forums with candidates for the National Assembly. In 2006, PC held the "100 Societal Organizations and 17 Urgent Issues for a Real Democracy" Forum. In 2007, PC developed the "Impact of Civil Society in the National Assembly" program to discuss issues of political reform, economic models, decentralization and autonomy, and human rights, in which over 15,000 people participated. Since 2009, a series of "democratic breakfasts" were developed to encourage open debate on issues related to democracy. 4. Democracy Education Objective: Dissemination of information on civil, political, and social rights as well as citizens’ responsibilities. 4.1. Work with Civil Society: Starting in 2010, PC developed the "Creating a Roadmap for Civil Society" program mentioned above, through which it trained 54 NGOs on issues of: (a) citizen participation and constitutional principles, (b) tools for social oversight, (c) leadership and advocacy, and (d) communication strategies. In 2011, it held a series of lectures on "Obstacles, 89 PC had to proceed in this manner because the government argued that the media outlets gave them substantial discounts, so PC could not rely on established average costs. 50 Challenges, Opportunities and Threats of CSOs" in several provinces. These activities led to the formation of the “Red Pluralista” (Pluralistic Network) of Civil Society Organizations in early 2012.90 In recent years, PC has organized workshops for citizens and community leaders on “Tools for Participation and Social Oversight”, and for journalists and civil society members on “Electoral Advertising and Promotional Segments”, “News Coverage and Constitutional Court Rulings”, “Advertising Bans”, “Electoral Districting”, and “Formula for Allocating Seats”. 4.2. SIEL: PC launched the Integrated Online Education System (SIEL) in 2010, to offer online training on topics such as Access to Information, Alternative Dispute Resolution, and the Law of Citizen Participation, among others. It was intended for a range of users, but in practice it was mainly used by the Volunteer Network. In 2010, courses were offered on Advocacy, Accountability, Political Communication and Leadership, and Communication and Democracy. In 2011, SIEL was used to train users on online study techniques and use of the platform, conflict management, and research techniques. In 2012, a training course on the Access to Information Law was held for officials from public institutions that were part of the targeted group of organizations being monitored by PC to help them comply with the Law’s requirements (70 attendees). Other recent courses were: "Design of Development Assistance Projects" (111 participants); "Political and Social Uses of ICTs" (12 participants). As of May 2014, the website is no longer available. Obstacles: Some informants mentioned that the SIEL was not attractive as a learning mechanism and others commented that, because of limited access to internet in rural areas, its impact was limited. 4.3. Civic Campaigns: The "Delivering on Time" Campaign was launched in 2003, based on partnerships with various public and private entities. 600 promoters were trained and 29 events were held simultaneously throughout the country. The "Citizen Eye" campaign, which promoted respect, tolerance, transparency, freedom, justice, security and solidarity, was implemented in 2010, 2011 and 2012. Another major project was the "Live Democracy" Campaign implemented between 2008 and 2012, which benefited over 2000 young students in Guayas province by promoting political leadership and good citizenship practices. 4.4. Other projects: Between 2005 and 2006, two documentary film competitions known as "Look out for Democracy" were held, and between 2006 and 2008, a traveling exhibition of the documentary films developed under this activity was held in six provinces. Between 2008 and 2011, several initiatives were implemented: (a) a radio project for the promotion of civil, social, and political rights and civic participation through the CORAPE Network;91 (b) a pilot project of Political Training for the PC coordinators Guayas province; (c) a project for Organizational Strengthening of Ecuadorian Immigrants in Loja and Madrid and for Use of Remittances for Agricultural Development Initiatives; and (d) a training for Afro-Ecuadorian women's on rights and participation. 5. Informed Citizenry Objective: To provide quality information to citizens on issues of public interest. 5.1. Digital Newspaper: The digital daily newspaper "Informed Citizenry" has been published on a continuous basis between 2004 and 2014. Internal reports from 2008 to 2012 indicate a high level of visits. PC trained and supported the participation of five local journalists in various provinces to research and submit articles for the website, and has also worked closely with other journalists in recent years. In 2011, PC conducted 5 workshops in several cities that benefited 183 people on topics such as constitutional rights and mechanisms, citizen oversight, access to information rights and investigative journalism. In 2012, PC conducted 5 training workshops for 200 people, including independent journalists, alumni and students of social communication on the "New Challenges for Journalism: Access to Information Law, Legal Framework and Investigative Journalism". In 2013, 90 This network and the training initiative are discussed in depth in the body of the evaluation report, notably in the Engagement and Local Capacity Development sections. 91 CORAPE (Coordinadora de Radio Educativa Popular de Ecuador) is a media network. 51 PC hosted an event on media self-regulation in which international speakers participated, and one workshop on Journalism and Elections for the coverage for the 2013 elections. 52 ANNEX H: CERTIFICATION REGARDING TERRORIST FINANCING CERTIFICATION REGARDING TERRORIST FINANCING IMPLEMENTING E.O. 13224 (Revision 2) By signing and submitting this application, the prospective recipient provides the certification set out below: 1. The Recipient, to the best of its current knowledge, did not provide, within the previous ten years, and will take all reasonable steps to ensure that it does not and will not knowingly provide, material support or resources to any individual or entity that commits, attempts to commit, advocates, facilitates, or participates in terrorist acts, or has committed, attempted to commit, facilitated, or participated in terrorist acts, as that term is defined in paragraph 3. 2. The following steps may enable the Recipient to comply with its obligations under paragraph 1: a. Before providing any material support or resources to an individual or entity, the Recipient will verify that the individual or entity does not (i) appear on the master list of Specially Designated Nationals and Blocked Persons, which list is maintained by the U.S. Treasury’s Office of Foreign Assets Control (OFAC) and is available online at OFAC’s website : http://www.treas.gov/offices/eotffc/ofac/sdn/t11sdn.pdf, or (ii) is not included in any supplementary information concerning prohibited individuals or entities that may be provided by USAID to the Recipient. b. Before providing any material support or resources to an individual or entity, the Recipient also will verify that the individual or entity has not been designated by the United Nations Security (UNSC) sanctions committee established under UNSC Resolution 1267 (1999) (the “1267 Committee”) [individuals and entities linked to the Taliban, Usama bin Laden, or the Al Qaida Organization]. To determine whether there has been a published designation of an individual or entity by the 1267 Committee, the Recipient should refer to the consolidated list available online at the Committee’s website: http://www.un.org/Docs/sc/committees/1267/1267ListEng.htm. c. Before providing any material support or resources to an individual or entity, the Recipient will consider all information about that individual or entity of which it is aware and all public information that is reasonably available to it or of which it should be aware. d. The Recipient also will implement reasonable monitoring and oversight procedures to safeguard against assistance being diverted to support terrorist activity. 3. For purposes of this Certification￾a. “Material support and resources” means currency or monetary instruments or financial securities, financial services, lodging, training, expert advice or assistance, safe houses, false documentation or identification, communications equipment, facilities, weapons, lethal substances, explosives, personnel, transportation, and other physical assets, except medicine or religious materials.” b. “Terrorist act” means- (i) an act prohibited pursuant to one of the 12 United Nations Conventions and Protocols related to terrorism (see UN terrorism conventions Internet site: http://untreaty.un.org/English/Terrorism.asp); or (ii) an act of premeditated, politically motivated violence perpetrated against noncombatant targets by sub national groups or clandestine agents; or (iii) any other act intended to cause death or serious bodily injury to a civilian, or to any other person not taking an active part in hostilities in a situation of armed conflict, when the purpose of such act, by its nature or context, is to intimidate a population, or to compel a government or an international organization to do or to abstain from doing any act. c. “Entity” means a partnership, association, corporation, or other organization, group or subgroup. d. References in this Certification to the provision of material support and resources shall not be deemed to include the furnishing of USAID funds or USAID-financed commodities to the ultimate beneficiaries of USAID assistance, such as recipients of food, medical care, micro￾enterprise loans, shelter, etc., unless the Recipient has reason to believe that one or more of these beneficiaries commits, attempts to commit, advocates, facilitates, or participates in terrorist acts, or has committed, attempted to commit, facilitated or participated in terrorist acts. e. The Recipient’s obligations under paragraph 1 are not applicable to the procurement of goods and/or services by the Recipient that are acquired in the ordinary course of business through contract or purchase, e.g., utilities, rents, office supplies, gasoline, etc., unless the Recipient has reason to believe that a vendor or supplier of such goods and services commits, attempts to commit, advocates, facilitates, or participates in terrorist acts, or has committed, attempted to commit, facilitated or participated in terrorist acts. f. Applicants that have never received a cooperative agreement, grant or contract from the U.S. Government are required to submit a copy of their accounting manual. If a copy has already been submitted to the U.S. Government, the applicant should advise which Federal Office has a copy. This Certification is an express term and condition of any agreement issued as a result of this application, and any violation of it shall be grounds for unilateral termination of the agreement by USAID prior to the end of its term. Signed: _18 March 2014 CERTIFICATION REGARDING TERRORIST FINANCING IMPLEMENTING E.O. 13224 (Revision 2) By signing and submitting this application, the prospective recipient provides the certification set out below: 1. The Recipient, to the best of its current knowledge, did not provide, within the previous ten years, and will take all reasonable steps to ensure that it does not and will not knowingly provide, material support or resources to any individual or entity that commits, attempts to commit, advocates, facilitates, or participates in terrorist acts, or has committed, attempted to commit, facilitated, or participated in terrorist acts, as that term is defined in paragraph 3. 2. The following steps may enable the Recipient to comply with its obligations under paragraph 1: a. Before providing any material support or resources to an individual or entity, the Recipient will verify that the individual or entity does not (i) appear on the master list of Specially Designated Nationals and Blocked Persons, which list is maintained by the U.S. Treasury’s Office of Foreign Assets Control (OFAC) and is available online at OFAC’s website : http://www.treas.gov/offices/eotffc/ofac/sdn/t11sdn.pdf, or (ii) is not included in any supplementary information concerning prohibited individuals or entities that may be provided by USAID to the Recipient. b. Before providing any material support or resources to an individual or entity, the Recipient also will verify that the individual or entity has not been designated by the United Nations Security (UNSC) sanctions committee established under UNSC Resolution 1267 (1999) (the “1267 Committee”) [individuals and entities linked to the Taliban, Usama bin Laden, or the Al Qaida Organization]. To determine whether there has been a published designation of an individual or entity by the 1267 Committee, the Recipient should refer to the consolidated list available online at the Committee’s website: http://www.un.org/Docs/sc/committees/1267/1267ListEng.htm. c. Before providing any material support or resources to an individual or entity, the Recipient will consider all information about that individual or entity of which it is aware and all public information that is reasonably available to it or of which it should be aware. d. The Recipient also will implement reasonable monitoring and oversight procedures to safeguard against assistance being diverted to support terrorist activity. 3. For purposes of this Certification￾a. “Material support and resources” means currency or monetary instruments or financial securities, financial services, lodging, training, expert advice or assistance, safe houses, false documentation or identification, communications equipment, facilities, weapons, lethal substances, explosives, personnel, transportation, and other physical assets, except medicine or religious materials.” b. “Terrorist act” means- (i) an act prohibited pursuant to one of the 12 United Nations Conventions and Protocols related to terrorism (see UN terrorism conventions Internet site: http://untreaty.un.org/English/Terrorism.asp); or (ii) an act of premeditated, politically motivated violence perpetrated against noncombatant targets by sub national groups or clandestine agents; or (iii) any other act intended to cause death or serious bodily injury to a civilian, or to any other person not taking an active part in hostilities in a situation of armed conflict, when the purpose of such act, by its nature or context, is to intimidate a population, or to compel a government or an international organization to do or to abstain from doing any act. c. “Entity” means a partnership, association, corporation, or other organization, group or subgroup. d. References in this Certification to the provision of material support and resources shall not be deemed to include the furnishing of USAID funds or USAID-financed commodities to the ultimate beneficiaries of USAID assistance, such as recipients of food, medical care, micro￾enterprise loans, shelter, etc., unless the Recipient has reason to believe that one or more of these beneficiaries commits, attempts to commit, advocates, facilitates, or participates in terrorist acts, or has committed, attempted to commit, facilitated or participated in terrorist acts. e. The Recipient’s obligations under paragraph 1 are not applicable to the procurement of goods and/or services by the Recipient that are acquired in the ordinary course of business through contract or purchase, e.g., utilities, rents, office supplies, gasoline, etc., unless the Recipient has reason to believe that a vendor or supplier of such goods and services commits, attempts to commit, advocates, facilitates, or participates in terrorist acts, or has committed, attempted to commit, facilitated or participated in terrorist acts. f. Applicants that have never received a cooperative agreement, grant or contract from the U.S. Government are required to submit a copy of their accounting manual. If a copy has already been submitted to the U.S. Government, the applicant should advise which Federal Office has a copy. This Certification is an express term and condition of any agreement issued as a result of this application, and any violation of it shall be grounds for unilateral termination of the agreement by USAID prior to the end of its term. Signed: __________________________________ _____March 18, 2014________ Date