October 2012 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Democracy International. EVALUATION MID-TERM PERFORMANCE EVALUATION OF THE INCREASED TRUST IN ELECTORAL PROCESSES PROJECT (ITEP) MID-TERM PERFORMANCE EVALUATION OF THE INCREASED TRUST IN ELECTORAL PROCESSES PROJECT (ITEP) OCTOBER 2012 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CONTENTS Acronyms ....................................................................................................................................... 1 Executive Summary ......................................................................................................................... 3 Introduction .................................................................................................................................... 8 Background ..................................................................................................................................... 9 Methodology ................................................................................................................................. 11 Findings ........................................................................................................................................ 14 Conclusions .................................................................................................................................. 29 Pre-Election Recommendations ...................................................................................................... 31 Recomendations (To 2014) ............................................................................................................ 32 Annex A: ITEP Evaluation Statement of Work ................................................................................. 33 Annex B: ITEP Evaluation Workplan ............................................................................................... 43 Annex C: Interview Questions List ................................................................................................. 49 Annex D: List of interviewees......................................................................................................... 50 Annex E: Documents List ............................................................................................................... 52 Annex F: Mini-Surveys.................................................................................................................... 54 1 ACRONYMS BRIDGE Building Resources in Democracy, Governance, and Elections CCG Chamber of Control Group CEC Central Election Commission CEPPS Consortium for Elections and Political Process Strengthening COP Chief of Party CRA Civil Registry Agency CSO Civil Society Organization DEC District Election Commission ECWG Election Code Working Group EDR Electoral Dispute Resolution GoG Government of Georgia GYLA Georgian Young Lawyers’ Association IATF Inter-Agency Committee [Task Force] on Free and Fair Elections IFES International Foundation for Electoral Systems IRI International Republican Institute ISFED International Society for Fair Elections and Democracy ITEP Increased Trust in Electoral Processes M&E Monitoring and Evaluation MoU Memorandum of Understanding MP Member of Parliament NDI National Democratic Institute nGnI New Generation, New Initiative NGO Non-governmental organization PEC Precinct Election Commission PMMG Public Movement “Multinational Georgia” PMP Performance Management Plan SAO State Audit Office SOW Scope of Work TI-G Transparency International-Georgia TOT Training of Trainers 2 UNDP United Nations Development Program UNM United National Movement USAID United States Agency for International Development VLVC Voters’ List Verification Commission 3 EXECUTIVE SUMMARY PURPOSE, PROJECT OBJECTIVES, EVALUATION QUESTIONS, AND BACKGROUND This report is a mid-term evaluation of the Increased Trust in Electoral Processes project (ITEP), a four￾year (2010-2014), $8.0 million project implemented by the International Foundation for Electoral Systems (IFES) under the CEPPS Cooperative Agreement. The project has four stated objectives, plus a small fifth objective recently added. This evaluation covers only the first two objectives: (1) improved capacity of election administration; and, (2) improved transparency of electoral processes through improved media coverage and CSO involvement. The intended end result or outcome for these objectives mirrors the project’s title: increased trust in electoral processes. The evaluation focuses on electoral management bodies such as the Central Election Commission (CEC), the Voters’ List Verification Commission (VLVC), the former Chamber of Control Group (CCG), now State Audit Office (SAO); and, to a more limited extent, the Inter-Agency Task Force on Free Elections (IATF), a non-beneficiary. In the non-governmental sector, the evaluation covers four civil society advocacy groups (CSOs) receiving assistance through ITEP. They are the International Society for Fair Elections and Democracy (ISFED), the Georgian Young Lawyers’ Association (GYLA), Transparency International-Georgia (TI-G), and New Generation, New Initiative (nGnI). As the component of assistance to media is just getting underway, those activities were not ripe for evaluation. The evaluation covers four main evaluation questions, all dealing with objectives #1 and #2, stated above. In addition, at the request of Mission staff in the initial meeting with the team, a fifth question was included in the work plan. 1. To what extent are ITEP’s intended results being achieved regarding objectives #1 and #2:  To what extent did the project strengthen the capacity of election administration (for the 2012 election); and  Did the project improve the transparency of Georgian electoral processes? 2. Is the project design appropriate to increase trust in the electoral processes? 3. What has been the commitment of target organizations, e.g., CEC, CCG, and VLVC, to sustain the results of the project:  Did the participants obtain needed skills and knowledge as a result of the project;  Were the skills and knowledge considered useful; and,  Were the skills and knowledge used in the behaviors of individuals and/or organizations? 4. What are the most and the least successful interventions as perceived by main stakeholders (CEC, CSOs, VLVC, CCG)? 5. What can be done to improve election administration and transparency of electoral processes before the October 2012 election? Given the fact that important parliamentary elections will be held in Georgia on October 1, 2012, the report gives special attention to election administration preparations, pre-electoral conditions, and actions that may still be taken before October to increase public confidence in this year’s contest. Georgia has a history of problematic elections, which have led to disputed results and public protests in the past, leaving a legacy of mistrust of electoral processes. This closely watched election is seen as an important test for the maturity of Georgia’s young democracy and its aspirations for Euro-Atlantic integration. 4 METHODOLOGY To answer the main evaluation questions and undertake other analysis, the team relied primarily on the Rapid Appraisal approach, which utilizes different evaluation tools to quickly and systematically obtain information and collect data. Included among these tools were document reviews, key informant interviews, group discussions, and mini-surveys. Fieldwork and data collection were carried out from July 8-27, 2012 in Tbilisi and elsewhere in Georgia. To help team members conduct interviews and group discussions in a systematic and uniform manner, a set of interview questions were prepared . The questions covered topics outlined in the SOW, as well as issues and problems regarding trust in the electoral environment more broadly. At the time of the interviews, the questions were tailored to fit individual informants or groups in accordance with their affiliations, backgrounds, areas of expertise, and other factors. Mini-survey data was tabulated and analyzed. However, methodological limitations and the time span of the evaluation made it difficult in both the public sector and NGO components to measure quantitatively the all-important desired attitudinal end result or outcome – increased trust in electoral processes. FINDINGS The Findings in the report support the appropriateness of the project’s design and the conclusion that the project is performing up to expectations, delivering timely, valued technical assistance through high￾quality field office staff and international consultants. The CEC and VLVC have increased capacity and have improved institutional performance. The SAO has improved its capacity, but it is implementing a political party and campaign finance law that does not meet minimum legal standards of due process and is having a “chilling effect” on freedom of association. The four CSOs that received sub-grants used the funding for well-targeted activities in support of election law reform, civic education outreach, and monitoring of public spending, thereby helping to increase electoral transparency. Even with these accomplishments, dark clouds remain on the electoral horizon. The emergence of the opposition Georgian Dream coalition and the likelihood of a highly competitive election present the first major challenge to the United National Movement (UNM) ruling party, which has been in office since 2004 after the Rose Revolution. Abuses reported from past elections – ‘vote buying;’ misuse of ‘administrative resources’ by government officials, including intimidation; prohibited campaign ‘budget spending’ increases; questionable voting returns in ethnic minority regions; and bias by election officials – may recur in the currently overheated political environment. Much of ITEP’s work is designed to prevent such practices, but the campaign began officially only on August 1. The 11 Findings made are as follows: 1. IFES/ITEP international and local staff expertise and technical assistance was highly valued, internalized, and used by stakeholders (“Learn, Value, Use”). 2. Flexibility in ITEP project design and implementation enabled staff to be responsive to expressed needs of stakeholders in a timely manner. 3. IFES/ITEP training and expert technical assistance have improved CEC election administration capacity (“Learn, Value, Use”). 4. ITEP provided critical, timely expert technical assistance to the VLVC, aiding it with methodology and training for a nationwide canvassing effort to improve the accuracy of the voters’ list (“Learn, Value, Use”). 5. Two linked, major ITEP activities – facilitating the ECWG dialogue and expert technical assistance to the ECWG and its members – were highly valued by participants and informed the electoral code revision process (“Learn, Value, Use”). 6. IFES/ITEP support to the four leading CSOs – ISFED, GYLA, TI-G, nGnI – through sub-grant funding and focused expert technical assistance has helped them play an active and constructive 5 role in the electoral code revision process, campaign finance amendment process, and pre￾election activity. 7. IFES/ITEP TA to the CCG (now SAO) helped build the capacity of that body to carry out and enforce the new political party and campaign financing law, but with unforeseen negative consequences leading to violations of due process of law. 8. ITEP workshops, expert technical assistance, and revision of key forms have helped major stakeholders in their efforts to improve capacity for the process of Election Dispute Resolution. 9. IFES’s inability to carry out the planned program with the CEC Training Center, through no fault of its own, meant ITEP’s reach would not extend to the PECs, the weakest and most problematic link in the electoral chain. 10. The design of the project was appropriate, focusing on key electoral processes trust issues and targeting key stakeholders in government and the CSO community. 11. A number of short-term pre-election activities can be formulated using existing ITEP capability and activities shown in the above Findings. ELECTION ADMINISTRATION ITEP has worked closely with CEC, its principal client, and the recent spin-offs from the commission, VLVC and SAO, which have responsibility for the voters’ list and party and campaign finance law, respectively. IFES provided valuable inputs on these two topics to other major stakeholders that are being put to use, such as staff training and a manual on use of administrative resources by government employees during campaign periods. CEC IFES/ITEP has helped CEC increase its capacity and electoral transparency. The assistance provided has focused on four areas: (a) elections management training (BRIDGE) for all 76 regional election commissions (DECs); (b) strategic communications and public relations training for senior officials of CEC; (c) website upgrades and computer hardware and software for CEC management; and (d) assistance in developing a CEC strategic plan. BRIDGE training reached 378 members of the DECs and was highly regarded by them, and they are putting it to use. The IT system upgrades have increased CEC’s institutional capacity for effective management. The strategic plan exercise was a year-long, participatory exercise that is being used as a ‘living document’ to guide improved performance. Today’s CEC, led by a chairman who reaches out to stakeholders, has gained a more positive reputation that will help increase trust in the electoral process. CEC participated actively in two IFES-led working groups that advanced election law reform and election dispute resolution. Many participating respondents called the Election Code Working Group (ECWG) the best, most valuable part of ITEP, especially the precedent it set. Hopefully, IFES will establish similar working groups shortly after the October election in order to help major stakeholders deal with sensitive, difficult issues that flow from it. VLVC ITEP provided critical, timely technical assistance and training to VLVC against a tight deadline, beginning almost immediately after the responsibility for preparation of the voters’ list was transferred to VLVC from CEC. ITEP provided the ‘know how’ methodology and training for VLVC staff that enabled it to deliver the preliminary voters’ list on schedule by August 1. Respondents unanimously expressed greater confidence in this year’s voters’ list due to the work of VLVC, and public opinion survey data shows that citizens at large have a highly positive opinion of the nationwide canvassing done to update the list. At this stage, increased trust in this important element of the electoral process is measurable. 6 SAO Assistance to SAO is the most problematic activity to date in ITEP, and SAO is having a net negative effect on trust in electoral process. Although ITEP provided valuable technical assistance in preparing the party and campaign financing legislation and developing SAO operating procedures afterwards, the enforcement procedures used by SAO and in the courts do not meet minimum standards of due process of law. The fundamental deficiencies cannot be solved by so-called ‘even-handed’ enforcement or a ‘balanced approach.’ CSOs IFES/ITEP support to the four leading CSOs – ISFED, GYLA, TI-G, nGnI – has helped improve the transparency of electoral processes. Through sub-grant funding and focused expert technical assistance this support has helped these CSOs play an active and constructive role in the electoral code revision process, campaign finance amendment process, and pre-election activity. These CSOs show strong capacity for outreach and monitoring activities, and they play an important role for civil society in participation in election policy discussions and public information. CONCLUSIONS 1. ITEP inputs have improved CEC’s election administration capacity, although the CEC was already operating at a relatively advanced stage. The CEC is on a trajectory that, if continued through the election and afterwards, can serve to increase trust in electoral processes. If left unchecked, however, negative influences beyond the control of CEC, e.g. ‘vote buying,’ campaign budget spending, misuse of administrative resources, and even harassment and intimidation, can undercut progress made. 2. The Electoral Code Working Group (ECWG) showed the value of a respected technical assistance team such as IFES/ITEP and the flexibility shown in project implementation. Bringing skeptical political parties and CSOs to the table for a continuing dialogue was an activity that increased trust. Many participating respondents called the ECWG the best, most valuable part of ITEP, especially the precedent it set. 3. The work of the VLVC to date and the VLVC-prepared preliminary voters’ list will help to reduce or eliminate the voters’ list as a major contentious issue and increase public confidence and trust in the integrity and quality of the upcoming election. 4. The leading CSOs have shown themselves capable of making good use of ITEP funding support for civic engagement in electoral processes. Their influence with the public is less certain. 5. The party and campaign finance statute and the procedures used by SAO to enforce it represent a serious violation of constitutional guarantees of due process of law. These deficiencies cannot be cured by so-called ‘balanced’ or ‘evenhanded’ or ‘non-partisan’ enforcement. Severe damage to an open pre￾election environment caused by SAO actions may seriously damage the legitimacy of this election in the eyes of the public at large, not just Georgian Dream supporters. 6. The results of EDR project activities remain to be tested in an actual election. The real value of the investment made will only become apparent after this fall’s election. However, EDR may prove to be the Achilles heel of this election if the quality of election complaints, their treatment and resolution by PECs and DECs, and court actions do not show a marked improvement from past elections. 7. The PECs were not a specific part of the SOW because CEC had earlier declined IFES assistance, and they were not a part of our structured questioning. However, based on document review and unsolicited comments, the PECs appear to be the weakest element of CEC election administration, with questions raised about their selection process; impartiality; undue influence by certain members; failure to respect accredited observers; and dismissive treatment of election complaints 7 8. Elections in Georgia are still not considered to be the only option in the competition for political power. The overriding interest for Georgian citizens and the international community should be to convince the competing parties to accept the results of an election deemed substantially free and fair. In the event of conflict, IFES/ITEP has the potential to play a leading role (below the diplomatic level) in bringing the parties to the table to seek satisfactory agreed outcomes. PRE-ELECTION ACTION RECOMMENDATIONS First, as we heard many times, do not expect too much impact from any short-term actions, given the limited time available. We also second the most frequent suggestion offered to us by respondents, which was to intensify CSO civic education outreach activities, especially in minority regions and rural areas. Any action open to IFES to strengthen PECs should be initiated, as we see the local level PECs having the greatest vulnerability to problems with voting on election day and with election day results. This holds true for election dispute resolution (EDR) issues as well. Our strongest recommendation is for short-term actions pre-election (and post-election as needed) to strengthen party and public acceptance of election results, assuming that political tension remains high as the electoral process continues to unfold. This will be critical to help avoid civil unrest. POST-ELECTION PROJECT RECOMMENDATIONS The elections may significantly alter the Georgian political playing field and introduce new dynamics. We recommend a joint USAID/IFES stocktaking exercise shortly after the election to see whether new activities should be formulated to deal with new issues, taking advantage of the flexibility in implementation ITEP has shown thus far. The most problematic electoral issue that Georgia needs to address is the structure of the electoral system itself, as carried over from the Soviet era. It has created a structural imbalance favoring the ruling party until now, which has impeded trust. Accordingly, should the election open political space for serious discussion of a more modern, balanced system, IFES should initiate a working group like ECWG to develop a proposal along these lines for Georgia to be presented for public and parliamentary review. We recommend similar action to reengineer the CEC, which should move away from being a party￾based structure to becoming a well-staffed, independent, professional election body. Significant international expertise and technical support would be needed for such an undertaking. If the CEC could achieve this status, it would become desirable to reintegrate functions, such as maintenance of the voters’ list, which have been spun off recently for political reasons. 8 INTRODUCTION PURPOSE/OBJECTIVES The purpose of this mid-term performance evaluation of the Increased Trust in Electoral Processes (ITEP) project, implemented by the International Foundation for Electoral Systems (IFES), is to examine the progress achieved in accomplishing two of its four objectives:  Improving the capacity of election administration; and,  Increasing the transparency of electoral processes through improved media coverage and civil society involvement. The evaluation also focuses on whether ITEP’s design is appropriate to improve the electoral environment in Georgia and the perception among beneficiaries and stakeholders regarding project activities to date. Lastly, the evaluation addresses the Scope of Work (SOW) requirement to provide recommendations on how to improve the project in the remaining years of implementation. The overall evaluation objective is to provide USAID/Georgia with an important accountability and learning tool to help maximize the effectiveness of the ITEP project. As a mid-term evaluation, the study presents a unique opportunity for the Mission to use the results to improve program management, coordination, and implementation and, if needed, take corrective action in advance of the upcoming (October 2012) parliamentary election and/or presidential (2013) and local (2014) elections. In addition, by assessing ITEP’s influence on the electoral environment in Georgia the evaluation will help the Mission improve future project implementation and promote increased trust in the political process. AUDIENCE The audience for the evaluation is USAID/Georgia – in particular, its Office of Democracy and Governance (DG) – as well as the U.S. Embassy and other United States Government (USG) agencies involved with elections issues in Georgia. Evaluation results will also be shared with IFES as USAID’s implementing partner, and local stakeholders such as the Central Election Commission (CEC), State Audit Office (SAO), other donors working in the elections area, and interested NGOs. Finally, evaluation results will be used for reporting to Washington-based stakeholders and in the development of the USAID Performance Plan and Report. 9 BACKGROUND POLITICAL CONTEXT In late 2003, twelve years after Georgia became a sovereign country, the protracted political turmoil that had characterized the post-independence period reached a climax when tens of thousands of demonstrators took to the streets to protest the results of the flawed parliamentary election in November of that year. The results of the demonstrations and continued protests, which brought down the government and became known as the Rose Revolution, marked the beginning of a new day in Georgian politics, with heightened hope for the emergence of a more democratic system of government. Since the Rose Revolution, however, Georgia has been beset by a number of serious governance challenges, including constrained political participation, increasing political polarization, centralization of power within the executive branch, and low levels of citizen engagement and media involvement. In the elections area these challenges are particularly troubling, with widespread mistrust among Georgians regarding the country’s electoral process and procedures in general and specific issues such as inaccurate voter registration lists, inadequate resolution of complaints, and unfair administration of campaign finance procedures. Added fuel for these issues and concerns has come from the emergence of the “Georgian Dream” coalition established by Bidzina Ivanishvili. Recent polls by the National Democratic Institute (NDI) and International Republican Institute (IRI) indicate that this new coalition constitutes the first real threat to the ruling United National Movement (UNM) party, further suggesting the likelihood of a truly competitive race in the upcoming October 2012 parliamentary election.1 Further complicating this situation, in October 2010 the parliament amended the constitution to significantly reduce the power of the next president by ceding certain key functions to the prime minister. The amended constitution will go into effect upon the inauguration of the next president, who will be elected in October 2013. At that time, the government in place will resign and give way to the parliament seated after the October 2012 election. The latter election, therefore, is seen as being likely to have a major impact on Georgia’s political trajectory and future democratic development. In effect, a more free and fair election than those in the past is viewed as essential to consolidate the economic and political reforms that have flowed from the Rose Revolution; and, for this to happen more trust in and sound administration of the electoral process are needed. THE ITEP PROGRAM The U.S. has supported electoral reform in Georgia since the collapse of the Soviet Union, having invested more than $20 million in election assistance programming over the past two decades. ITEP, an $8.0 million, four-year (July 30, 2010 - July 31, 2014) program, continues and builds on these earlier efforts. It focuses on working with key governmental bodies, such as the CEC, CCG, and Voters’ List Verification Commission (VLVC); key NGOs, including the International Society for Fair Elections and Democracy (ISFED), Georgian Young Lawyers Association (GYLA), Transparency International-Georgia (TI-G), and New Generation-New Initiative (nGnI); and, major media and communications interests. ITEP is intended to ameliorate or eliminate the electoral system’s enduring, negative effects on Georgia’s political culture by:  bringing about election law reform; 1 By presidential decree on August 1, 2012, President Saakashvili set October 1 as the date for the election. 10  building the capacity of the election administration, as well as of those bodies responsible for implementing Georgia’s new political finance legislation and verifying the country’s voter registry;  promoting the development of the country’s civic identity;  building the capacity of NGOs to provide oversight, promote accountability, and increase public awareness about election processes and issues; and  providing pre-election training and monitoring as needed to address outstanding issues. Specific activities undertaken by ITEP include:  strengthening the CEC’s organizational capacity for communication and outreach;  providing strategic planning assistance to the CEC;  improving CEC and DEC transparency through website and connectivity enhancements;  providing technical assistance to VLVC for verification of the voters’ registry;  improving the CEC’s election complaint procedures;  raising public awareness of electoral issues;  supporting and assisting media outlets with pre-election coverage;  supporting the ability of CSOs to monitor election processes;  providing technical support for election reform efforts;  providing ongoing technical assistance to the Election Code Working Group (ECWG);  implementing BRIDGE training for members of all 73 DECs; and  providing technical assistance to Georgia’s Chamber of Control. 11 METHODOLOGY APPROACH Democracy International’s evaluation team consisted of: Charles Costello, J.D. (Team Leader); Harold Lippman, Ph.D. (Senior Evaluation Specialist); and David Jijelava, M.A. (Country Expert). As set forth in the Scope of Work (SOW), the team worked to determine how effective the project has been in achieving its intended results and whether or not the project design is still suited to improving the electoral environment in Georgia. The team reviewed the efforts undertaken and results achieved by IFES between July 30, 2010 and the present in two of the project’s four objectives: (1) improved capacity of election administration; and, (2) improved transparency of electoral processes through more and better media coverage and CSO involvement. Consistent with the terms in the Statement of Work of the Task Order, DI in its Technical Proposal submitted a preliminary work plan to USAID before arriving in Georgia. The work plan included the evaluation’s proposed methodology and activities, an outline of the final report, an illustrative list of individuals and organizations the evaluation team planned to interview, and a preliminary list of interview questions. Upon arrival in Georgia, the evaluation team met with USAID to finalize and approve the work plan and to identify additional interviewees. The team’s efforts were organized around and addressed the following four evaluation questions posed in the SOW (Annex A) and captured schematically in the work plan’s Evaluation Matrix (Annex B) as well as in the Q&A Matrix at the beginning of the Findings section. 1. To what extent are ITEP’s intended results being achieved regarding objectives #1 and #2:  To what extent did the project strengthen the capacity of election administration (for the 2012 election); and  Did the project improve the transparency of Georgian electoral processes? 2. Is the project design appropriate to increase trust in the electoral processes? 3. What has been the commitment of target organizations, e.g., CEC, CCG, and VLVC, to sustain the results of the project:  Did the participants obtain needed skills and knowledge as a result of the project;  Were the skills and knowledge considered useful; and,  Were the skills and knowledge used in the behaviors of individuals and/or organizations? 4. What are the most and the least successful interventions as perceived by main stakeholders (CEC, CSOs, VLVC, CCG)? In addition, at the request of Mission staff in their initial meeting with the team a fifth question was included in the work plan: what can be done to improve election administration and transparency of electoral processes before the October 2012 election? To answer these questions the team relied primarily on the Rapid Appraisal approach, which utilizes different evaluation tools to quickly and systematically obtain information and collect data. Included among these tools are document reviews, key informant interviews, group discussions, and mini-surveys. Fieldwork and data collection were carried out from July 8-27, 2012 in Tbilisi and, over a two-day period, in Kutaisi, Akhalkalaki, and Marneuli during the second week of the team’s time in-country. In connection with the latter, the team split into two, two-person sub-teams with one responsible for work in Kutaisi and the other for work in Akhalkalaki and Marneuli. 12 To help team members conduct interviews and group discussions in a systematic and uniform manner, a set of interview questions were prepared (Annex C). The questions covered topics outlined in the SOW, as well as issues and problems regarding trust in the electoral environment more broadly. At the time of the interviews, the questions were tailored to fit individual informants or groups in accordance with their affiliations, backgrounds, areas of expertise, and other factors. With respect to the SOW questions listed above, the lead-in for Question #3 refers to “the commitment of target organizations…to sustain the results of the project,” whereas the three actual questions do not reflect either ‘commitment’ or ‘sustainability.’ They are what we consider ITEP ‘value￾added’ questions. The team’s comparative approach to analyzing the information and data gathered in the field has been further defined by use of the metaphor terms “learn, value, use” in order to raise and answer the evaluation questions. In the absence of sufficient, readily obtainable quantitative data, these somewhat qualitative terms also provided a reliable way to assess the outputs and outcomes of ITEP activities. This applies to Question #1 as well. Among other reasons, we adopted this method because it also tracked the interview and mini-survey questions systematically posed by the team, which were based on the SOW framework. The responses to these questions, in turn, became a key factor for the team’s ability to compare and contrast the data and information from all sources and to be able to respond fully to the questions set forth in the SOW. Question #4 could not be answered satisfactorily based on respondents’ replies for two reasons: (1) they were highly positive about the interventions and declined to make a ‘least successful’ choice, and (2) they were aware of only a limited number of the total ITEP interventions. We attempt to give some answers to this question in our own findings and conclusions based on the fieldwork. TASKS The team interviewed or consulted with representatives of key USAID/Georgia organizational units including the Program Office and the Democracy and Governance Office. It also met with a representative of the U.S. Embassy (POL/ECON) and the Chief of Party and other key staff of IFES. In Tbilisi, the team met with key Georgian government individuals and organizations targeted under ITEP, including the CEC, four District Election Commissions (DECs) in Tbilisi, VLVC, SAO, Inter-Agency Committee [Task Force] on Free and Fair Elections (ITAF), and certain members of the ECWG. Interviews were also arranged with officials representing the majority party and opposition parties in (and/or out of) Parliament, particularly the UNM and the Georgian Dream coalition. The large majority of these meetings consisted of key informant interviews, while in some instances, i.e., with the CEC Chairman and members of his staff and with CEC department heads, the sessions were conducted as group discussions. In Kutaisi, the sub-team had a group meeting with the five members of the DEC and toured their centrally located office. Additional key informant meetings took place with local leaders of GYLA and ISFED and with a journalist for a regional newspaper and a local television news anchor and talk show host. In Marneuli and Akhalkalaki, regions with large ethnic Azeri and Armenian populations, respectively, the sub-team met with the two regional DECs, NGOs, and other stakeholder groups. They visited an independent local television station outside Marneuli and met with the owner. They visited the USAID-supported Civic Engagement Center in Marneuli, where the meeting was held, and discussed local civil society issues with MSI project staff. In Akhalkalaki the sub-team visited the offices of the NGO National Forum, which works especially on minority issues, and met with representatives of UNM and Georgian Dream at their party offices. The team also conducted two mini-surveys (Annex F). One was with a group of 20 randomly selected DEC beneficiaries of ITEP BRIDGE training, which consisted of six close-ended, narrowly focused questions regarding their experience with this activity. Nineteen of the 20 DECs selected responded to the questionnaire. A second mini-survey was conducted with a sample of women that had participated in a two-day roundtable discussion in June 2011 on Political Party Funding and Women’s Participation in 13 Political Life co-sponsored by IFES. Results of this survey could not be ascertained due to the lack of an adequate number of responses. DATA ANALYSIS The evaluation team’s data analysis was grounded in a comparative analysis of ITEP program documents, including annual work plans, quarterly reports, PMP indicator tables, M&E plans, monitoring reports, expert reports, public opinion surveys, and other publications. These source documents identify and describe the causal linkages among program inputs, activities, outputs, and outcomes, as well as anticipated results and underlying assumptions. They also define target populations, identify stated objectives, and establish specific measurable indicators. The team also compared ITEP documents and stakeholder interview information with data related to project performance and outcomes obtained from other source documents, individual and group interviews with stakeholders and beneficiaries, and data obtained in the mini-surveys. By using information obtained from the surveys, interview guides, and common questions, the team systematically compared the answers and comments of different beneficiaries and stakeholders to facilitate inferences on program performance and effects. To identify important trends and uncover latent relationships, the evaluation team analysis disaggregated data as necessary. Outputs and outcomes were analyzed in terms of the project’s implementation schedule, available resources, and related work on the part of other donors. LIMITATIONS The single most important limitation to the team’s approach was that the evaluation took place before the October parliamentary elections; and, therefore, there was no way to assess the effect of program activities on the conduct of an actual election. Moreover, increasing public confidence in the electoral process, as well as improving the capacity of election administration and bolstering transparency of election procedures, are typically long-term efforts, the fruits of which are not fully discernible at the project’s two-year mid-point. The team used evidence available from public opinion research from other sources, such as NDI and IRI, on such questions as trust, although horizontal timelines are known to be short. The team had neither time nor funding to conduct public opinion survey research on its own. As called for in the task order, the team deployed rapidly to Georgia. This rapid deployment had its benefits, but also its costs including real effects on the team’s ability to conduct a complete desk review prior to departure. It also meant the team was not able to arrange as many interviews in advance as otherwise would have been desirable. Indeed, the lack of time required to organize focus group discussions outside Tbilisi was a key determinant in the team’s decision not to pursue these sessions as envisioned prior to departure. Lastly, also consequential was the team’s inability in some instances to obtain access to key ITEP documents (i.e., expert consultant trip reports that contain important details on project technical assistance and support efforts, as well as specific accomplishments and challenges that need attention). Lastly, as the Rapid Appraisal techniques utilized in this evaluation rely on non-representative samples, with the exception of the DEC and women’s roundtable mini-surveys, as well as the fact that no baseline data was collected at the start of the project, the evaluation team was not able to generalize results across the entire population or use “before and after” comparisons. Nevertheless, by using the mixed￾methods approach described above, the evaluation team was able to compare data collected using one method (e.g., document review) to data collected using other methods (e.g., key informant interviews and the mini-surveys). Such triangulation, moreover, minimized bias and strengthened the validity of evaluation findings by acting as a check on the information derived from any one method. 14 FINDINGS We set forth our findings in the narrative text that follows. In order to make clear the relationship between our findings and the four main evaluation questions, in the matrix below we show the questions and answers to them, cross-referenced by number to the findings that relate to them. Using this formulation, some findings appear multiple times in the matrix. Table 1. Findings Matrix Evaluation Questions Finding Number References 1. To What extent are the intended results of the ITEP project being achieved in relation to the objectives 1 and 2 of the project? #1, #2, #3, #4, #5, #6, #7, #8, #10 1.a. To what extent did the project strengthen capacity of election administration (for the 2012 elections)? #1, #2, #3, #4,  #7, #8, #10 1.b. Did the project improve transparency of electoral processes? #1, #2, #4, #5, #7, #8, #10 2. Is the project design appropriate to increase trust towards electoral processes? #1, #2, #5, #10 3. What has been the commitment of target organizations (CCG, CEC, DECs, and Voter's List Commission) to sustain the results of the project? #3, #4, #5, #7, #8 3.a. Did the participants obtain needed skills and knowledge as a result of this project? #1, #4, #5, #7, #8 3.b. Were the skills and knowledge considered useful? #1, #4, #5, #7, #8 3.c. Were the skills and knowledge used in the behaviors of individuals and/or organizations? #1, #4, #5, #7, #8 4. What are the most and the least successful interventions as perceived by main stakeholders (CCG, Voter's List Commission, CEC, NGOs)? Most successful: NA, Least successful: #7, #9 Did ITEP strengthen the capacity of election administration for the 2012 elections? The answer is clearly “yes.” IFES work with CEC such as BRIDGE training and through the dialogue process with various stakeholders in the ECWG and EDR workshops improved election administration capacity. Technical assistance to VLC led to a higher quality voters’ list. Flexibility in project design allowed IFES to respond quickly to changing needs of the electoral authorities. However, assistance to SAO misfired, as the authorities acted improperly to implement the party and campaign finance law. The project also helped to improve transparency of electoral processes. The updating of the voters’ list with a nationwide canvassing effort by VLC gave the public greater confidence in the quality of the list. The ECWG facilitated by IFES provided a space for key actors to discuss issues openly and have a better understanding of how to increase transparency in election laws. The EDR workshops led to improvements in procedures for handling electoral disputes, making that process simpler and more transparent. Again, however, the work with SAO did not yield greater transparency, as enforcement actions were seen as arbitrary and biased. The work of the partner CSOs definitely made the process more transparent by such measures as voter outreach, campaign monitoring, and civic education activities. 15 The project design appeared appropriate in that it focused on key electoral issues, both administrative and political, and worked with key stakeholders in an attempt to address the major challenges. Also, design flexibility allowed IFES to adapt to changing circumstances on the ground. The “commitment of target organizations…to ‘sustain the results’ of the project” was difficult to establish. The work with CSOs did not target sustainability objectives. The work with electoral bodies such as CEC, VLC, DECs, and SAO varied greatly from one entity to another. CEC and the DECs valued the BRIDGE training, and CEC plans to continue its use. VLC learned new technology, but was not established as a permanent body. SAO misused the technical training it received by the manner in which it enforced the law. Participants did obtain some new and needed skills and knowledge through the project. In areas as diverse as website and IT upgrades, strategic planning, election code reform, EDR, teambuilding, voters’ list methodology, budget monitoring, and internal agency procedures, participants benefited from ITEP technical assistance and training. The team applied a “Learn, Value, Use” test to query respondents as to their perceptions about the effectiveness of project activities, and many cited specific examples of using what they had learned as well as its overall value to them. Data from the mini-surveys validated the utility of BRIDGE training for DECs. Respondents generally declined to cite a “most successful” intervention, as they had limited knowledge of the entire range of project activities. Opinions were quite positive as to the ones in which they themselves had participated. Perhaps for the same reasons, respondents uniformly declined to categorize any intervention as “least successful.” The most criticism the team heard from respondents generally about the electoral process centered on the actions of SAO. For this reason, the team on its own account considered that intervention to be the least successful. 1. IFES/ITEP INTERNATIONAL AND LOCAL STAFF EXPERTISE AND TECHNICAL ASSISTANCE WAS HIGHLY VALUED, INTERNALIZED, AND USED BY STAKEHOLDERS (“LEARN, VALUE, USE”). ITEP is a knowledge-based technical assistance project whose main objective is to impart new understandings and skills that participating beneficiaries will use to improve the institutional performance of electoral management bodies and civil society organizations. The new skills can be directly substantive, for example, using computer software methods to improve the voters’ list. They can also be process-oriented, for example, using skills learned for teambuilding or for seeking consensus in dialogue meetings to improve institutional efficiency and working group outcomes. Assistance to the VLC is of the former kind, and assistance to DECs through BRIDGE is of the latter kind. IFES also sponsored two international conferences bringing in outside experts, with more than 100 participants. According to a broad range of respondents’ comments and mini-survey data and measuring both quantitatively and qualitatively by our test of “learn, value, use,” the IFES staff and consultants are providing valuable expertise to their counterparts. An important benefit is that the IFES Chief or Party (COP) and his professional staff have earned such credibility for their impartiality and expertise that ITEP’s “good offices” function with stakeholders can help increase trust among stakeholders they work with in an otherwise highly polarized environment. Examples of ITEP staff expertise, included election management, comparative electoral systems, electoral dispute resolution, Georgian election law, and working group facilitation. Individual informants were unanimous in expressing their opinion that the IFES/ITEP office in Tbilisi is providing high quality inputs to stakeholders and the electoral process. Facilitation of the ECWG and bringing in outside experts to work with its members was often cited. The ITEP office was also considered to be the most helpful, authoritative source for interpretation and opinions on the election code. ITEP’s international consultants received high praise from those who worked with them in-country. Their expertise was considered very relevant and new knowledge gained from them is being put to use, according to many participant informants. ECWG participants said they found the consultancies, which 16 spread know-how across widely different constituency groups, particularly useful and helpful in finding common ground for reasonable discussion of difficult issues. Discussion papers presented by the experts focused conversation on the substance of political issues rather than political posturing by members of the working group. International experts covered electoral system design, campaign finance legislation, and biometric voter identification, among other key areas of interest. Mini-survey data from participants in BRIDGE training for DECs also supports this finding. The same is true as to data from personnel in CEC’s IT and public relations departments, who were queried using an interview protocol as part of structured group interviews. The four CSOs that received sub-grants under ITEP – ISFED, GYLA, TI-G, and nGnI – reported that the technical assistance they received was valued and put to use. For example, TI-G got expert assistance on budget analysis for monitoring government spending, and all four received training on election dispute resolution in a joint working group with the CEC. They also participated in the ECWG and were beneficiaries of the ITEP-funded experts’ inputs on election code reforms. That process empowered them to propose amendments to the new election code when the government introduced the bill in Parliament. To some extent we are able to provide evidence-based, quantitative information to support this finding. This includes triangulated information gleaned from different informants and constituencies in the interviewing process around the same questions. We are satisfied that the information meets the “learn, value, use” test we have utilized to measure outputs as defined in the SOW (although often stated instead as “results”). However, these measuring sticks do not adequately answer the bigger questions about the relevance and importance of what has been “learned, valued and used” in achieving desired project outcomes (results) such as increased public trust in Georgia’s electoral processes. Nor can public opinion survey data in most cases make a reliable direct correlation between any specific project activities and measured changes in citizens’ opinions on these issues over time. Our finding on the value and importance of IFES’s work in ITEP ultimately is based principally on respondents’ judgments cited throughout this report that ITEP is addressing the most important electoral issues in Georgia with high quality staff and consultants, working effectively with the most important constituencies, and offering value-added activities that indeed can serve to improve the quality of electoral processes and civic participation, presumably increasing public trust. Overwhelmingly positive stakeholder responses strongly support such a finding. However, other developments (independent variables) on the political scene occurring outside the scope of the project during this same period may prove to be more powerful and negative in their effect on public trust and confidence in electoral processes, resulting in a net decrease in trust notwithstanding the positive impacts of ITEP. Hopefully, this will not be the outcome, but the ultimate perceived legitimacy of the process in the minds of the public will be heavily influenced by what the voters see and hear up to and through the election. It will depend on whether they perceive the campaign and election results to be ‘fair,’ (a level playing field with no undue advantage to the incumbent party) as well as ‘free’ (open voting and an honest count). This can only be measured meaningfully after the election. 2. FLEXIBILITY IN ITEP PROJECT DESIGN AND IMPLEMENTATION ENABLED STAFF TO BE RESPONSIVE TO EXPRESSED NEEDS OF STAKEHOLDERS IN A TIMELY MANNER. Both ITEP beneficiaries and other donors such as UNDP were emphatic in stating that the flexibility and adaptability shown by IFES in implementing the project yielded important benefits to Georgia’s electoral processes, and benefits greater than those from other more rigidly structured donor projects. The flexibility in design was matched by timely response with experts or other inputs to meet unexpected needs and changing conditions on the ground. This meant a willingness and ability, with support from a USAID Mission also showing flexibility, to move money from one project area to another and to make quick changes to annual work plans as circumstances dictated. 17 This strongly positive response by beneficiaries and other stakeholders answers in large part the question as to whether the project design was appropriate. Respondents clearly stated they valued the ITEP activities that were changed or added with quick turnaround as unforeseen needs became apparent. Examples given by key informants include a package of technical assistance to the newly created VLVC when the responsibility for maintaining the voters’ list was transferred from the CEC and assistance in the development of operating procedures when the responsibility for political party and campaign finance was transferred from the CEC to the CCG, later SAO. Beyond being able to cite specific examples of flexibility and an appreciation of their timeliness and value in helping to advance project objectives, this ‘result’ is an important but intangible benefit not easily captured. But clearly respondents expressed the opinion that it made ITEP (and USAID as its patron) the “go to” project for troubleshooting across a wide range of electoral issues. IFES’s strong “presence” and responsiveness constitutes a valuable asset in the current Georgian context, and this will presumably continue on into 2013 and 2014, according to our informants. 3. IFES/ITEP TRAINING AND EXPERT TECHNICAL ASSISTANCE HAVE IMPROVED CEC ELECTION ADMINISTRATION CAPACITY (“LEARN, VALUE, USE”). a) BRIDGE Training for DECs Developed in 2000, Building Resources in Democracy, Governance, and Elections (BRIDGE) is a highly regarded training curriculum on electoral processes.2 Building on the experience of a similar 2009 UNDP-funded BRIDGE training activity, 378 members (155 female, 223 male)3 representing all 76 DECs and the Adjara Supreme Election Commission came together for a three-and-a-half day workshop. The workshops took place in Batumi and Tbilisi in January/February 2012 and focused on: the electoral cycle; legal frameworks for elections; election management bodies (EMBs); types of electoral systems; and, electoral dispute resolution. ITEP source documents, such as the consultant’s final report4 group interviews with DEC participants, and the results of the evaluation team’s mini-survey on this subject all support a finding that the BRIDGE training helped improve CEC election administration capacity. According to the consultant’s report, 100 percent of the participants scored their “general evaluation of training” at the highest level (a five on a five-point scale). A post-workshops evaluation meeting attended by 25 (10 women) event participants and facilitators affirmed the ratings in the final report. The discussion at the meeting indicated that participants felt the training was timely, necessary and had helped prepare them for the October 2012 parliamentary election. DEC members interviewed in Tbilisi, Kutaisi, Akhalkalaki, and Marneuli who had participated in the BRIDGE training workshops expressed views similar to those found in the ITEP source documents. In Kutaisi, the Commissioners talked about how helpful the role-playing exercises in the training had been. One example mentioned was on the potential importance of how the Commissioners are seated in cases where a difficult issue or question arises. The Chairman of the DEC said he intended to put this newly acquired information to use in the upcoming election. In the mini-survey responses, much the same information and data was obtained as came from the source documents and interviews. The mini-survey questionnaire results affirm and elaborate on the information obtained from the source documents and interviews. For example, 18 of the 19 DECs that responded in the random sample gave the highest rating (five) when asked about the BRIDGE training’s 2BRIDGE training is the result of a partnership of the Australian Electoral Commission, UN Electoral Assistance Division, UNDP, International Foundation of Electoral Systems (IFES), and International IDEA. 3 Of the 378 participants, 155 (41%) were female and 223 (59%) were male. 4 Natia Kashakashvili, Final Report: District Electoral Commission members’ capacity enhancement BRIDGE program. 18 usefulness. The other DEC respondent in this case rated the usefulness at the four level. When asked if the training provided a particular tool, skill, or piece of information they did not have before, all 19 checked the “yes” answer. Asked to elaborate on this item, 12 cited information about elections in various countries; eight identified general workplace and communications skills; and eight listed knowledge directly related to the local electoral process. Similarly, 18 of the 19 respondents checked “yes” when asked if they had used the acquired knowledge and/or skills in their day-to-day work. Ten of the 18 respondents provided concrete details, including: performing managerial and administrative functions, such as planning, assigning responsibilities, and working with documentation; improved teamwork and sharing of opinions; and, improved cooperation with political parties and NGOs. The last two mini-survey questions asked if the BRIDGE training had improved the transparency of the DECs’ electoral processes and their ability to perform their election administration functions. Regarding the election administration item, while all 19 checked the “yes” response, only 12 provided details. Of these 12 DECs, four mentioned an improved workplace with more effective decision-making and teamwork; three listed improved internal structures and division of responsibility; three said they better understood election administration functions and responsibilities; and, two cited an increased capability to create election calendars and plans. On the transparency question, 17 of the 19 DECs said it had increased. Of the 17 that answered affirmatively, nine provided additional details as follows: five mentioned improved relations with media; four cited improved relations with NGOs; and, two listed improved relations with political parties. b) Communication/Outreach Training for CEC Staff Source documents show how ITEP training and technical support have increased CEC’s communication and outreach capabilities. For example, from July through October 2011, an IFES election capacity building advisor provided training on communication strategies to four members of the CEC’s public relations department. The activity focused on English language business communication, using social media, drafting press releases, and writing formal letters and e-mails. In a post-activity online survey, to which participants were able to respond anonymously, they described some of the benefits of this training in their own words:  “The classes have been [of] utmost use to me for developing my business communication skills, as well as for my language skills improvement. They have been useful for my everyday…routine work.”  “The classes… [were more]…interesting and fruitful than expected. Why? I had the great opportunity to communicate with [a] native speaker, to get acquainted with the main principles of press release preparing….In short I improved my skills in listening, writing and speaking….”  “These classes have been very useful for me and my work. The best thing about the classes was that we had a native speaker teacher, it was a good experience for me. I needed to practice my English…and these classes helped me develop my English skills in speaking, writing and listening, and gain more confidence while talking/writing in English.” Another activity, a continuation of the communication strategies training described above, concentrated on how to develop effective messages and build relationships with local and international media. This effort took place in March 2012 and almost all of the participants were women. Topics included the art of effective communication and how to do TV interviews, including professional feedback for participants who did an on-camera simulation that was taped. The training also included a segment on identifying and trying to address inadequacies in the CEC’s communication plan. In an interview, a Public Relations Department manager who had participated in the training described some of its positive effects. She explained that the subject matter covered, such as organizing events and dealing with media inquiries, was entirely new for her employees. She said she could see her employees benefitting from the role-playing exercises and constructive critiques trainers provided on the simulated interviews. More specifically, for example, she observed her employees’ self-confidence being improved 19 in terms of learning how to present themselves with media and the public. She believes that she, as a manager, and CEC, as an institution, also benefitted from the training in her having been able to observe her employees’ strengths and weaknesses. This was important, she added, not only because it helped her better understand, as a manager, who could do what, but also additional training that might be needed. c) Website Upgrades/IT Support In an ongoing effort to improve CEC and DEC transparency, ITEP has supported website and connectivity enhancements. In late 2011, after an August review of CEC’s election administration connectivity capability showed a significant upgrade was needed to manage newly installed infrastructure, IFES procured six servers and complimentary software. The servers include: a virtualization server for human resource management, document administration, and finance and accounting information; a mail exchange server to enable email communication with DECs; a server mirror to provide mirroring of election results in Georgia; a backup server, and database and web application servers related to GIS technology. The servers were delivered in December 2011 and as of June of this year the virtualization and backup servers are operational and the mirror server is set and in standby mode for the upcoming election. However, neither the mail exchange nor GIS-related servers was fully operational. The former is installed but awaiting the necessary data to be inputted, and the latter is in a testing phase. In effect, while the servers and software upgrade is in itself a significant development, at the time of the team’s fieldwork it was still too early for their individual and/or collective impact to be assessed. d) Strategic Planning Assistance Starting in early 2011 and continuing on throughout the year, ITEP provided training and technical assistance in support of the CEC’s successful effort to develop a five-year strategic plan. According to source documents, among the activities most indicative of the support provided was the strategic planning and brainstorming session in May with the CEC working group comprised of 19 participants (five women), including the Chairman and his deputy, the CEC secretary, Commissioners, and Department Heads. The IFES adviser had the participants divide into small groups and asked each one to prepare a list of priority issues in election administration they thought needed to be addressed, along with suggestions for ways to bring about improvements in these areas. This exercise and others like it embody the extraordinarily collaborative nature of the months-long effort that culminated in the Strategic Plan’s finalization late in 2011. In interviews with the evaluation team, CEC beneficiaries of the ITEP training and technical support had positive things to say about their experiences and the results achieved in the development of the Strategic Plan itself. One department head, for example, explained that the training gave him information on planning tools, such as gap analysis and risk assessment, that prompted new thoughts in his and other participants’ minds. Views similar to his were expressed by another department head, who pointed out that the training included new concepts for her, such as the need for a vision in the development of a strategic plan. As important as the benefits of the training and technical support are, the Strategic Plan appears to have become a key part of CEC’s ongoing operational development. For example, action plans to implement its strategic goals – modernization, proactive cooperation with stakeholders, and informed voters – are under consideration or being actively pursued. In addition, as the Chairman indicated at the September 2011 meeting where the Plan was introduced publicly, he sees it as a key element in the effort to raise public confidence in the electoral process prior to the upcoming series of elections. His views in this regard are perhaps best captured in his Message at the outset of the printed version of the Plan: “Improvement of the electoral environment depends significantly on the work of the electoral administration. Four consecutive elections are to be held in the upcoming three years: 2012 Parliamentary Elections and Adjara Highest Council Elections, 2013 Presidential and 2014 Local Self-Government Elections. The Election Administration 20 should respond to this challenge. The present Strategic Plan is part of the process. The Strategic Plan outlines the mission, vision and guiding principles of the Georgian Election Administration as well as its strategic goals.” 4. ITEP PROVIDED CRITICAL, TIMELY EXPERT TECHNICAL ASSISTANCE TO THE VLVC, AIDING IT WITH METHODOLOGY AND TRAINING FOR A NATIONWIDE CANVASSING EFFORT TO IMPROVE THE ACCURACY OF THE VOTERS’ LIST (“LEARN, VALUE, USE”). The accuracy of the voters’ list has been a longstanding controversy in Georgian elections, contributing to public lack of trust in electoral processes. Complaints included a failure to purge dead voters’ names from the list, problems with election-day registration, coverage of internally displaced persons (IDPs), and treatment of so-called ‘labor migrants’ (registered voters living outside Georgia). In 2006, ISFED carried out a first-ever comprehensive voters’ list audit (for the 2006 local elections),5 using scientific sampling techniques. The initial results showed an 8.83% error rate, and after reviewing contested figures, ISFED found an error rate of 5%, still rather high. In 2008 with the support of the CEC, ISFED again carried out a voters’ list audit (for the 2008 extraordinary presidential election), using the 2006 data as baseline. The audit still found a number of serious problems with the lists. For the later parliamentary elections in May 2008, ISFED did a post-election audit of the voters’ list used. Using five different measures, the audit found a high degree of accuracy in the list. Comparing the 2008 results with the 2006 results, ISFED stated “…it is obvious that there is a significant improvement, though still more work has to be done to eradicate the existing problems…[internal migration and emigration, lower registration of ethnic minorities, transparency of the work of CRA and CEC in managing the list].” Nonetheless, concerns about the accuracy of the voters’ list persisted as the country approached the 2012 elections, voiced especially by opposition political parties. The government responded by legislation transferring responsibility for the voters’ list from the CEC to the new VLVC with 21 members, made up of equal numbers of representatives from the governing party, opposition parties and civil society, and chaired by a leader from an opposition party. However, that change did not end the controversy. Leading CSOs, wary of being co-opted in a process they did not control and did not trust completely, declined to participate. According to respondents, some participating NGOs were seen as too ‘government-friendly,’ and the same was said about some opposition parties. And the emerging strong opposition force – Georgian Dream Coalition – refused to participate at all in the VLVC’s work. IFES/ITEP began working with the VLVC upon its spin-off from the CEC in December 2011. IFES provided expert technical assistance to the VLVC in developing its methodology for updating the list, including computer software and training of staff. IFES did ‘training of trainers’ for VLVC who in turn trained the entire field canvassing staff. A CSO leader who attended a two-day retreat for VLVC staff run by IFES praised that activity, saying the methodology they shared with VLVC was adopted and used by the commission. The VLVC undertook a nationwide canvassing effort that used both the ‘list-to￾people’ method and the ‘people-to-list’ method. The VLVC is due to release the voters’ list for the public review and appeal period on August 1. It was assumed by all that the commission would approve the list as presented by the Chairman. The Chairman stated definitively to the team that this work product “would not have been possible without the technical assistance received from IFES.” ITEP 5 Voters’ List Audit, ISFED, 2008. 21 assistance to the VLVC clearly met the standard of “learn, value, use” that we are applying to measure value-added in institutional capacity of electoral institutions. The expected accuracy of the 2012 list to a reasonable degree was not questioned seriously in any of our interviews with election officials, CSOs and political parties (not including Georgian Dream). That is not to say there were no concerns expressed, but the voters’ list issue did not rank high as an election issue. In fact, most interlocutors expressed confidence in the level of accuracy of the list,6 citing different concerns. Respondents working with political parties noted the quality of the training and the canvassing work against a tight deadline, considering the list to be “in pretty decent shape.” The establishment of the VLVC itself was seen more as a political tactic by government and the ruling party than an honest effort to improve the quality of the list, but the actual work done by the VLVC was not seriously challenged. The public shares this opinion. In the NDI survey released in July this year 78% said they were aware of the VLVC canvassing efforts; of those, 78% said the efforts were improving the voters’ list, and 86% said they viewed the work of the VLVC positively. This data, considered reliable, directly addresses the issue of ‘increased trust’, which is at the core of the ITEP project and this evaluation. Both respondent replies and the survey data support this finding. It is noteworthy, however, that the same survey showed that citizens ranked “free elections” down in 7th place on a list of 19 questions asking them to state their most important concerns heading into the 2012 elections. But it is also noteworthy that the 16% level of concern about free elections was up from 10% and 9% in two earlier surveys by NDI. 5. TWO LINKED, MAJOR ITEP ACTIVITIES – FACILITATING THE ECWG DIALOGUE AND EXPERT TECHNICAL ASSISTANCE TO THE ECWG AND ITS MEMBERS – WERE HIGHLY VALUED BY PARTICIPANTS AND INFORMED THE ELECTORAL CODE REVISION PROCESS (“LEARN, VALUE, USE”). This finding, taking the two activities together, proved difficult to substantiate quantitatively in any meaningful way, but by all accounts was seen by participating stakeholders as one of IFES’s most important accomplishments. Participants praised the leadership of the IFES COP and his staff in facilitating an open, respectful dialogue, a rarity in political life in Georgia. They also praised the IFES technical assistance expertise provided to the group and several of its members on electoral code issues. Most of all, they praised the intrinsic value of a candid dialogue, with IFES acting as a respected, impartial “honest broker.” They said the ECWG helped to break down the barriers of mistrust among the parties. This was an elite group of beneficiaries or participants rather than the public at large, but a very influential group; so increasing these political actors’ trust in electoral processes is significant and highly relevant for achieving ITEP’s objectives. Discussions about amending the Election Code, including the critical issue of wide disparities in population among the single-member constituencies (“majoritarian districts”) that constitute one-half of the 150 seats in Parliament, began in late 2010. A consensus of sorts was reached to initiate a more formal process on the topic. Parliament, with its overwhelming UNM majority, was not seen as the best venue to give voice to eligible political parties not represented there and the many civil society organizations strongly interested in the topic. 6 Public Attitudes in Georgia, Results of a June 2012 Survey, National Democratic Institute (NDI/Georgia), Luis Navarro, Ian T. Woodward, July 16, 2012. 22 Stakeholders agreed to establish a working group, the ECWG, to carry out an extensive dialogue about amendments to the code, with IFES/ITEP designated as facilitator for the group’s meetings and the process overall. Over a period of eight months ten meetings of the ECWG were held to discuss reform topics and consider specific proposals from various members. This gave IFES the opportunity to establish working relationships with all seven ‘qualified’ political parties. Beyond its facilitating role, IFES made international experts available on key topics such as biometric voter identification and had expert technical working papers prepared to inform the discussions, such as a suggested design for a revised electoral system. UNM presented a proposal of its own, as well as considering those proposed by others. Ultimately, in the spring of 2011 the group reached a deadlock, with UNM unwilling to support proposed structural changes to the electoral system itself, among others; and the opposition parties at the table broke off the dialogue, leaving only a rump ECWG. The future of amendments to the Election Code before the 2012 election was left uncertain. UNM continued to work on election code reform outside the ECWG. In the fall of 2011 the ruling party presented an entirely new and unexpected election code proposal, eschewing amendments to the existing code altogether. UNM succeeded in gaining support for its proposal from three other opposition parties and moved the bill forward on that basis, leaving the ECWG behind. However, the work of the ECWG was not in vain. Many of the proposals advanced and agreed upon in the ECWG later were offered by parties and CSOs as amendments to the election legislation enacted into law in December 2011. They were accepted and added to the new Election Code in 2012. Although the most substantive and controversial election law changes were deferred at least until after the 2012 election, the IFES-facilitated dialogue in the ECWG brought to the fore a number of amendments for serious, substantive discussion in a reasoned forum. Both smaller political parties and CSOs gained a seat at the table and a voice in discussion of important public policy issues, with considerable success in seeing their proposals adopted. The positive feedback from respondents about the ECWG’s process as a valuable learning exercise and the measurable results of legislative provisions adopted through the participants’ efforts, supports a finding that IFES/ITEP has helped to improve election administration and strengthen the role of civil society in the political arena. Perhaps the most important contribution they cited was to help set a precedent for how stakeholders can work together on tough issues in a process where they as participants “learned, valued, and used.” 6. IFES/ITEP SUPPORT TO THE FOUR LEADING CSOS – ISFED, GYLA, TI-G, NGNI – THROUGH SUB-GRANT FUNDING AND FOCUSED EXPERT TECHNICAL ASSISTANCE HAS HELPED THEM PLAY AN ACTIVE AND CONSTRUCTIVE ROLE IN THE ELECTORAL CODE REVISION PROCESS, CAMPAIGN FINANCE AMENDMENT PROCESS, AND PRE-ELECTION ACTIVITY. Electoral Code Revision Process Throughout the period following the release of the draft election code in 2011, IFES continued to collaborate closely with and support the country’s civil society elements. In September, for example, it hosted a two-day workshop for eight representatives of ISFED, GYLA, and TI-G who had been working on the electoral reform effort. IFES experts provided input on the potential impact of certain provisions and assisted in the group’s article-by-article review and discussion of the draft code’s strengths and weaknesses. From these efforts, the NGO group drafted a joint commentary, which was released at a public presentation attended by more than 70 representatives of political parties, other CSOs, the media, and the international community. According to IFES informants and representatives of ISFED, GYLA, and TI-G, some of the key changes in the draft code the latter recommended were included in the revision of the Election Code on 23 December 8, 2011. Included among these changes were: dropping the controversial proposal to increase the number of Members of Parliament; allowing photo and video equipment to be used at polling sites; and, returning marking requirements at polling stations. The changes also addressed shortcomings in the area of political finance and the rights of election observers and media representatives on election day. Campaign Finance Amendment Process The ongoing effects of IFES expert assistance and technical support that has helped NGOs learn how to build coalitions to act in concert on election issues can also be seen in the work of the new watchdog group, “This Affects You Too!” Led by TI-G, ISFED, and GYLA, this new group of NGOS and media organizations has successfully brought pressure to bear on law makers by increasing awareness of and educating the public about the potential negative effects of certain provisions of recently enacted campaign finance amendments. According to source documents and the team’s interview with a key member of Parliament, the Chairman of the Legal Affairs Committee, agreement was reached to incorporate the watchdog group’s suggestions as part of a new set of amendments to the applicable Law on Political Unions of Citizens. Accordingly, at a March 2012 Committee hearing that watchdog group members and other local and international NGOs had been invited to attend, a number of changes were announced. Included among these changes were:  eliminating blanket terms “directly or indirectly related” in the law;  reducing the amount for fines the CCG can impose;  increasing judicial oversight of CCG decisions;  decreasing the number of entities subject to the campaign finance law; and  inserting language that precludes the CCG from filing requests with the CEC to ban political parties from competing in elections. Pre-Election Activity Raising Public Awareness IFES helped increase public awareness of and involvement in the electoral reform process as a result of funding and technical support provided under ITEP to ISFED. With this support, ISFED led a coalition, comprised of itself, TI-G, GYLA, and new Generation new Initiative (nGnI), in a concerted effort to raise awareness on electoral processes throughout Georgia. In May 2011, the coalition members developed a full list of electoral code proposals based on their individual and collective experience. These proposals, in turn, became the basis for public discussions organized at 20 sites in 16 municipalities spread around the country. The meetings took place over a six-week period and, according to ISFED informants, somewhere around 1,400 Georgians participated. In an interview, a former ISFED program coordinator with detailed knowledge of this activity provided a picture of what took place. She referred to a meeting in Kutaisi, Georgia’s second largest city, which was held at the central library with an estimated 150 participants present. Attendees included local government officials, including DEC members, civil society and media representatives, students, and individual citizens. The discussion in Kutaisi, as well as in the other participating municipalities, revealed common concerns on issues such as the voter list, electoral administration, and misuse of administrative resources by government employees. She said she took notes and wrote down comments and, in consultation with ISFED’s lawyer, used the information from this material to develop a list of recommendations. These recommendations and other issues and concerns that surfaced in the meetings were summarized in a report, “Electoral Code Recommendations from Citizens.” The report was released at an event at the end of July 2011, with more than two dozen representatives of political parties, NGOs, and international organizations present. In the ensuing discussion, those in attendance were informed about citizens’ recommendations regarding the electoral reforms and afterwards the recommendations were distributed widely to policymakers and electoral stakeholders in Tbilisi and elsewhere in Georgia. 24 However, according to the ISFED informant, in the end none of those recommendations found their way into the Election Code that was ultimately approved. Supporting and Assisting Media Also in July, as part of the above-described effort to raise public awareness and interest in the ongoing electoral reform process, ITEP supported ISFED in recording a one-hour television program on electoral code reform entitled, “Public Awareness and Electoral Issues.” The talk-show format discussion was hosted by a well-known journalist and included representatives with opposing viewpoints from the New Rights and Free-Democrats parties and representatives of GYLA, nGnI), and ISFED. The discussants highlighted societal and economic factors that are reflected in the lack of citizen awareness of electoral issues and they also debated key issues of reform, such as the composition of the Voter List. The program aired on TV Maestro on July 31st and on 11 regional channels throughout the ensuing week. Media-related ITEP support is also evidenced in efforts this year on the part the watchdog group (“This Affects You Too!”), which has weighed in on the emerging controversy over full media access during the pre-election run-up. The group submitted a package of proposals to Parliament calling upon it to amend the current law on broadcasting to prevent arbitrary treatment of television channels by cable networks. Referring to their proposals as “must-carry/must-offer,” the group asserted that voters can only make informed choices if they have access to all Georgian television channels. Accordingly, they argued, the country’s cable network providers should be obligated to broadcast (“carry”) all television channels and, likewise, all of the latter should be obliged to “offer” their channels to the different networks operating in the country. According to source documents, as a result of the increased domestic pressure and international attention the watchdog group and other efforts generated, Parliament at least partially responded by amending the electoral code to require all cable operators to broadcast relevant programming on all television channels commencing 60 days prior to election day.7 7. IFES/ITEP TA TO THE CCG (NOW SAO) HELPED BUILD THE CAPACITY OF THAT BODY TO CARRY OUT AND ENFORCE THE NEW POLITICAL PARTY AND CAMPAIGN FINANCING LAW, BUT WITH UNFORESEEN NEGATIVE CONSEQUENCES LEADING TO VIOLATIONS OF DUE PROCESS OF LAW. Prior to this year’s election, party and campaign finance laws existed but were weakly enforced. CEC had the responsibility for monitoring and enforcement. Many opposition political parties and CSOs were pressing for tougher legislation, with the parties hoping to limit the funding advantages of the ruling party (UNM) and the CSOs hoping to improve good governance in electoral practices. Most criticism was directed at the ruling party rather than the weak, underfunded opposition. Circumstances changed dramatically with the emergence in 2011 of the well-funded Georgian Dream, first as an ‘NGO Movement’ and later as a political party umbrella for the Georgian Dream Coalition of several opposition parties. The political patron of Georgian Dream had virtually unlimited personal resources to finance its activities, which created a totally new scenario on the political stage. UNM responded quickly to this threat and with its complete control over the legislative process in Parliament enacted new, stricter party and campaign finance laws. In spite of the obvious political objective of the ruling party to blunt the new financial prowess of the opposition, the fact that the legislation placed tighter limits on funding and broader disclosure 7 On August 16, 2012, after the team had departed Georgia, CEC, ITAF and seven leading CSOs signed a Memorandum of Understanding on agreed interpretation of certain provisions in the law on use of administrative resources. CEC cited IFES’s contributions to this effort. 25 requirements overall helped gain significant civil society support for it. Internationally, this included favorable comment from the Council of Europe’s Venice Commission. IFES/ITEP provided two campaign finance experts to work with CCG staff and CSOs in reviewing and commenting on the draft legislation. One of the proposed changes in governing law was the transfer of responsibility for monitoring and enforcement from CEC to a new entity, the CCG. CEC favored this change, which it saw as a way to narrow its focus more directly to election administration matters and deflect political heat on a hot topic. ITEP’s work plan had included campaign finance topics, originally thought of as part of its work with CEC. IFES shifted to work with CCG on the content of the bill. After passage, an IFES international expert led a 5-day seminar for staff to develop operating procedures. This assistance was seen as a positive contribution by the project, and a senior official at CCG (which was SAO by then) praised the utility and quality of the expertise provided, citing the Handbook of Procedures as a tangible ‘result.’ Responding in part to recommendations from the Venice Commission and ‘usual practices’ in Europe, the CCG function later was folded into the existing State Audit Organization and made into a department of that agency. However, once SAO began to exercise its authorities, the situation with respect to ITEP assistance and further IFES involvement became complicated. SAO took aggressive enforcement action and levied large fines for violations, mainly against Georgian Dream and its donors. Court judgments were immediately executed with seizure of bank accounts and other assets and, in one case, appointment of receivers to take management control of an affected bank. Satellite TV dishes, judged to be part of a vote-buying scheme, were taken away from new TV subscribers set to pay a low monthly fee and thousands more were impounded at customs. Fines were levied for as much as $55 million for violations, an amount unprecedented anywhere else in the world. Using a population ratio comparison, such a fine would amount to $2.3 billion in the United States. According to at least two election law legal experts interviewed, the statutory language for bringing charges is vague and ambiguous, giving prosecutors overly broad discretion in bringing charges. For example, a person allegedly ‘laundering’ political party donations by use of third parties to avoid statutory limits was instead charged with ‘vote-buying.’ SAO completed enforcement actions and decisions were turned over to the courts for judgment within 48 hours, as stipulated by the statute. This gave defendants little time to prepare their defense and handcuffed the judge hearing a case. According to observers or participants, open hearings typically last no more than two hours, with judgments entered immediately with little in the way of supporting opinions by the judges. Appeal periods are similarly circumscribed. SAO, by its own count, has brought 60 enforcement cases to court, and speedy judgment has been entered against the defendants in all 60 cases to date. According to these respondents, the statute and the procedures used to enforce it raise serious questions of constitutional guarantees of due process of law. The requirements for judicial action within a very limited time period also raise serious questions about the independence of the judiciary from interference in its work by the legislative and executive branches. Whether the courts are failing to protect the due process rights of parties charged also becomes an issue. No judge has yet seen fit to enjoin enforcement of the law or dismiss such filings upon pleadings by defendants. No case has been taken up yet either by the Constitutional Court, which could exercise jurisdiction over claims of the law’s unconstitutionality. ITEP is currently not actively engaged with SAO on any of these issues. Although the ‘learn, value, use” test of earlier assistance to CCG/SAO was met in a strict sense, the ultimate value and appropriateness of this activity and any contribution made to greater trust in electoral processes needs to be evaluated in this broader context, given its partial, extremely aggressive application. 8. ITEP WORKSHOPS, EXPERT TECHNICAL ASSISTANCE, AND REVISION OF KEY FORMS HAVE HELPED MAJOR STAKEHOLDERS IN THEIR EFFORTS TO 26 IMPROVE CAPACITY FOR THE PROCESS OF ELECTION DISPUTE RESOLUTION. Resolution of formal election complaints in a satisfactory manner has been very problematic in Georgia. In the 2010 local government elections more than 90% of complaints filed at the PEC level were rejected on technical grounds without any consideration of their substantive merits. Reasons alleged for this extremely high rejection rate include disqualifying deficiencies in the complaint documents; weaknesses in PECs’ administrative capacity to handle complaints; and pro-government bias or undue influence by government inside the PECs and courts. Courts review complaint appeals upon petition, but none of the complaints taken to court on appeal resulted in a ruling favorable to petitioners. Procedures used for adjudicating disputes and the pattern of outcomes in adjudication can have a very large impact on whether losing political parties and the public-at-large trust and accept the legitimacy of electoral processes. Of course, this is especially true in hotly contested, close elections where election dispute resolution can directly affect MP race outcomes in ‘majoritarian’ districts or the number of seats awarded using the formula for proportional representation. At this point in the pre-election period it appears that the October 2012 elections will be very competitive, making acceptance of the outcomes of resolution of election disputes crucial to increasing trust in electoral processes between competing parties and in the eyes of the public. IFES/ITEP worked closely with interested stakeholders such as CEC, political parties and CSOs to try to improve EDR. They supported a detailed working group review with four CSOs of the entire process that led to agreement with CEC on changes to filing forms and procedures to be used, all with the objective of simplifying the complaint process and making it more transparent. IFES helped participants prepare a Manual on Electoral Dispute Resolution, which is being widely distributed. IFES carried out “training of trainers” (ToT) workshops for 14 political parties covering 89 attendees. Thus, the party activists trained thereafter will have a better technical grasp of the EDR process, and the Manual should be a valuable reference resource. We did not have a chance to survey this population of direct and indirect beneficiaries in order to apply the “learn, value, use” test. As results from EDR project activities are still to be tested in an actual election, informants’ replies indicate that the approach to dealing with the problem appears sound and that the inputs/outputs can serve to improve EDR if used appropriately and fairly. As most of the complaints arise and are dealt with at the PEC level, we cannot safely predict whether the political party representatives’ new skills in making well prepared complaints will be matched by objective, well-trained PEC members. This is one of the areas of training for PECs handled by the CEC Training Center with some assistance from UNDP, not IFES/ITEP. We have expressed concern about the quality and reliability of PEC electoral management elsewhere in this report. PECs for this election have not yet been formed, and training will face a narrow time window of opportunity. If the vast majority of election complaints again are dismissed out of hand, it will have a negative impact on public trust in the electoral process. 9. IFES’S INABILITY TO CARRY OUT THE PLANNED PROGRAM WITH THE CEC TRAINING CENTER, THROUGH NO FAULT OF ITS OWN, MEANT ITEP’S REACH WOULD NOT EXTEND TO THE PECS, THE WEAKEST AND MOST PROBLEMATIC LINK IN THE ELECTORAL CHAIN. Both in its Technical Proposal and first year work plan IFES proposed to work with CEC and its revamped Election Systems Development Reforms and Training Center, not only with central staff and DECs but also with PECs. PECs are the local level, six-member temporary election commissions formed to run polling stations around the country. However, the CEC Chairman, who also had election support from UNDP, decided that donor’s aid and internal capacity were sufficient and declined the IFES offer. However, document review, especially domestic and international observer reports, shows serious problems with the capacity, impartiality and professionalism of PECs. This is especially true in rural areas and minority regions. According to many civil society and political party respondents, PECs represent 27 the “Achilles heel” of CEC. One prominent CSO opined that many PECs are under the influence of local government officials, and all 63 municipalities in Georgia plus Tbilisi are controlled by UNM. Many respondents questioned the impartiality of the ‘professionals’ who are selected by the DECs to work in PECs, saying they often have ties to the ruling party or are schoolteachers beholden to their school principals who also sit on the DEC. The team had little opportunity to pursue this issue itself, given that ITEP is not working with PECs. Nonetheless, we include this finding due to its relevance to public trust in electoral processes and the fact that the issue surfaced without our prompting. 10. THE DESIGN OF THE PROJECT WAS APPROPRIATE, FOCUSING ON KEY ELECTORAL PROCESSES AND TRUST ISSUES, AND TARGETING KEY STAKEHOLDERS IN GOVERNMENT AND THE CSO COMMUNITY. In order to pursue the desired result of “increased trust in electoral processes,” the project design focused on technical capacity of election management bodies, greater transparency in their operations and decision-making, and the ability of civil society organizations to provide outreach to voters around the country and to act as watchdogs for the unfolding of the electoral process. This was an appropriate development hypothesis and took into account the social and political context in Georgia. The completed activities were consistent with these objectives. CEC and VLVC received technical assistance and training to increase technical capability but also to improve working relationships with stakeholders and the public through such activities as BRIDGE training for DECs, the work of ECWG, and the EDR working group with CSOs. Civic education campaigns, especially those run by impartial NGOs, rather than government bodies, are considered a “best practice” to motivate citizens to vote and help them make informed choices at the polls. Both government and NGO activities to show that proper procedures in election administration will be followed helps to increase public trust in electoral processes. Examples of this were the outreach forums by ISFED on election law reform, TI-G reporting on election period budget expenditures, and the CSO coalition campaign “This Affects You.” Unfortunately, the project design assumption that IFES would work with PECs to improve their performance capability could not be carried out. It would have been advisable to have CEC agreement before project planning and contractor work plans proceeded on that assumption. A worthwhile activity, to improve party and campaign finance laws and their administration, misfired. The objective was appropriate, especially in the Georgian context, and the technical assistance provided to CCG and SAO was well targeted. However, the way in which the law has been applied and the actions of the enforcement agencies have decreased trust in the electoral process and set the stage for post-election unrest. This could not reasonably have been foreseen at the outset, and the activities were later discontinued; but it created a “bad practice” for USAID, IFES, and ITEP parties to follow. The flexibility built into the project design leaves ITEP well-positioned to develop useful new activities, which may change previous plans, to respond to conditions after the 2012 election is over. Elections in 2013 and 2014 still lie ahead, and ITEP can apply “lessons learned” through the first half of the project to those elections. 11. A NUMBER OF SHORT-TERM PRE-ELECTION ACTIVITIES CAN BE FORMULATED USING EXISTING ITEP CAPABILITY AND ACTIVITIES SHOWN IN THE ABOVE FINDINGS. In response to the Mission’s special request to try to determine what actions could be taken to improve election administration and transparency before the October 1 election, the team reviewed all its 28 materials and findings as well as relevant comments by respondents in interviews covering the four evaluation questions. This became a fifth question the team probed. Although the evaluation scope did not include any special focus on activities in ethnic minority and rural areas, when asked what short-term actions would be beneficial our respondents most frequently mentioned more civic education and voter outreach in those areas of the country. In Finding #9 we lament the fact that IFES training such as BRIDGE for CEC, extended only to DECs and not to the polling station-level PECs, which we consider most problematic. CEC handled training of PECs through its training center with some UNDP assistance. Short-term training to strengthen PEC’s competency could be made available to PECs if CEC gives its approval. This is especially true in ethnic minority and rural areas. In Finding #8 we discuss the value of the work done by ITEP with CEC and CSO stakeholders to improve the quality of EDR. As EDR may become critical in a hotly contested election, further short￾term activities could increase CEC capacity and that of CSOs as well to resolve disputes in a professional manner. The campaign finance law and the manner of its enforcement are doing serious damage to public trust in the fairness of the electoral process, as shown in Finding #7. Among the CSO beneficiaries of ITEP, GYLA has unique capability to engage in advocacy or educational activity for the public about the negative impact of the law on a free and fair electoral process. Voters need a better understanding of how the law affects the ability of parties to campaign effectively and how it has a “chilling effect” on the exercise of individuals’ associational rights guaranteed in the Constitution. We have found that the single most important factor to increase public trust in electoral processes will be broad public acceptance of election results flowing from a well-conducted, open, and free election. Supporters of competing political parties and the leaders of those parties themselves need to be able to make an informed judgment about the quality of this year’s electoral process. Elections produce winners and losers. Both sides need to be ready to accept the results of a good election. CSOs in particular can carry out campaigns to educate voters about the criteria used to judge whether an election is free and fair and why qualified domestic and international observer groups play an important impartial role. Reliable parallel vote tabulations (PVT) are a key factor in validating official voting results. 29 CONCLUSIONS 1. The CEC under the leadership of its current Chairman has reached out effectively to political parties, CSOs, and other stakeholders. ITEP inputs have improved CEC’s election administration capacity, although the CEC was already operating at a relatively advanced stage. The CEC is on a trajectory that, if continued through the election and afterwards, can serve to increase trust in electoral processes. If left unchecked, however, negative influences beyond the control of CEC, e.g. ‘vote buying,’ campaign budget spending, misuse of administrative resources, and even harassment and intimidation, can undercut progress made. CEC impartiality and reliability after the election is untested and much less certain based on past performance. 2. The ECWG showed the value of a respected technical assistance team such as IFES/ITEP and the flexibility shown in project implementation. Bringing skeptical political parties and CSOs to the table for a dialogue with ten lengthy meetings over an 8-month period, with IFES as facilitator, was an activity that increased trust. The combination of substantive topics, outside experts, and good facilitation in a neutral site led to civilized argumentation and served to help overcome the polarization that poisons most debate in Georgia. Many participating respondents called the ECWG the best, most valuable part of ITEP, especially the precedent it set. 3. The work of the VLVC to date and the VLVC-prepared preliminary voters’ list will help to reduce or eliminate the voters’ list as a major contentious issue and increase public confidence and trust in the integrity and quality of the upcoming election. ITEP assistance should continue until the final voters’ list is ready. 4. The leading CSOs have shown themselves capable of making good use of ITEP funding support for civic engagement in electoral processes. Their influence with the public is less certain. The high level of donor financing, including through ITEP, enables them to play an active role but at times makes them appear to be the voice of the international donor community rather than truly Georgian. Their long￾term financial sustainability with domestic support receives too little attention, as short-term donor electoral interests take precedence. They are in danger of becoming an elite, self-anointed ‘voice of the people’ in dealings with the government and the political parties. 5. The party and campaign finance statute and the procedures used by SAO to enforce it represent a serious violation of constitutional guarantees of due process of law. The requirements for judicial action within a very limited time period also represent a serious violation of the independence of the judiciary. The courts are failing to protect the due process rights of parties charged. These deficiencies cannot be cured by so-called ‘balanced’ or ‘evenhanded’ or ‘non-partisan’ enforcement by SAO, prosecutors, and the courts. Anyone being charged and fined will be subjected to the same violation of due process under this statutory framework. Thus, severe damage to an open, free and fair pre-election environment caused by SAO actions and failures in the courts may seriously damage the legitimacy of this election in the eyes of the public at large, not just Georgian Dream supporters. It appears too late to take meaningful corrective action before October 1, although we understand after the team’s departure SAO, under heavy pressure, decided to at least suspend enforcement action of fines levied for the duration of the campaign 6. The results of EDR project activities remain to be tested in an actual election. The real value of the investment made will only become apparent after this fall’s election. However, EDR may prove to be the Achilles heel of this election if the quality of election complaints, their treatment and resolution by PECs and DECs, and court actions do not show a marked improvement from past elections. With the voters’ list, balloting, and vote tabulation elements of the electoral process appearing to be in reasonably good shape, EDR could be vital to determining outcomes for representation in Parliament. 30 7. The PECs were not a specific part of the SOW because CEC had earlier declined IFES assistance, and they were not a part of our structured questioning. However, based on document review and unsolicited comments, the PECs appear to be the weakest element of CEC election administration, with questions raised about their selection process; impartiality; undue influence by certain members; failure to respect accredited observers; and dismissive treatment of election complaints. PECs and supervisory DEC members also will make decisions on election irregularity complaints. 8. Elections in Georgia are still not considered to be the only option in the competition for political power. Although IFES/ITEP has worked hard to help ensure sufficient transparency in this election, the competitive nature and high stakes of the election, damage already done to the pre-electoral environment, and the still-low level of trust in electoral processes all portend trouble and unrest in the post-election period. The overriding interest for Georgian citizens and the international community should be to convince the competing parties to accept the results of an election deemed substantially free and fair. IFES/ITEP has the potential to play a leading role (below the diplomatic level) in bringing the parties to the table to seek satisfactory agreed outcomes. 31 PRE-ELECTION RECOMMENDATIONS 1. Intensify voter education and outreach activities in minority areas. This was the most frequent suggestion made to us by respondents, given its importance and partial feasibility in the time available before the elections. 2. Short-term actions by IFES to help ensure that PECs are adequately prepared to function well should be prepared and offered. Such actions should ensure that PECs in minority regions, especially in rural areas, have language-qualified staff and have received additional training to test their readiness. 3. Short-term actions to strengthen the credibility of EDR work by PECs, DECs, and courts would help to ensure the integrity of the electoral process and reinforce ITEP’s strong work to date in this area. IFES should request CEC permission to work directly with PECs in vulnerable rural areas and outside major cities. 4. Similar action on the part of IFES/ITEP in support of partner CSOs, especially GYLA, should be undertaken to try to stop due process violations by SAO and courts in enforcement of the campaign and party finance laws. 5. Because public support for announced results will be the single most important criterion to demonstrate increased trust in electoral processes, the IFES-supported CSOs and domestic election observation groups with which they overlap should carry out in the limited time available a short-term civic education checklist program on “criteria for a free and fair election and when an election merits your respect and support.” 32 RECOMMENDATIONS (TO 2014) The elections may significantly alter the Georgian political playing field and introduce new dynamics. A more balanced party representation in Parliament, the upcoming presidential election in 2013 with constitutional changes affecting the powers of that office and those of the prime minister, and issues arising from the conduct of this year’s election, may lead Georgian stakeholders to want new approaches to democratic development and elections. Accordingly, we offer the following recommendations: 1. A joint USAID/IFES stocktaking exercise shortly after the election to see whether new activities should be formulated to deal with new issues, taking advantage of the flexibility in implementation ITEP has shown thus far. 2. The most problematic electoral issue that Georgia needs to address is the structure of the electoral system itself, as carried over from the Soviet era. It has created a structural imbalance favoring the ruling party until now, which has impeded trust. Accordingly, should the election open political space for serious discussion of a more modern, balanced system, IFES should initiate a working group like ECWG to develop a proposal along these lines for Georgia to be presented for public and parliamentary review. 3. Similar action to reengineer the CEC, which should move away from being a party-based structure to becoming a well-staffed, independent, professional election body. Significant international expertise and technical support would be needed for such an undertaking. If the CEC could achieve this status, it would become desirable to reintegrate functions, such as maintenance of the voters’ list that have been spun off recently for political reasons. 4. As the statutory authorization for the VLVC expires on election day, the future of the VLVC as an entity and the responsibility for the voters’ list going forward should be clarified following this election and well before the 2013 election. 5. As a part of the above-mentioned reform efforts, support to CSOs (and future work with media groups) should be focused on increasing public trust and confidence in Georgia’s electoral structure and institutions, with the intention of making the process more participatory and nonpartisan. 6. To help promote the continuing viability and long-term sustainability of CSOs involved in promoting civic engagement in electoral processes, future work under ITEP and/or other donor community activities should provide expert assistance and technical support to promote their financial independence and institutional stability. 7. To prevent the problems that have surfaced in the run-up to the upcoming election from potentially polluting the 2013 and 2014 elections, the legal framework and enforcement process for party and campaign financing should be corrected as soon as possible after the October 2012 election cycle has been completed. 33 ANNEX A: ITEP EVALUATION STATEMENT OF WORK SECTION C - DESCRIPTION / SPECIFICATIONS/STATEMENT OF WORK MID-TERM EVALUATION OF THE INCREASED TRUST IN ELECTORAL REFORMS PROJECT UNDER THE CONSORTIUM FOR ELECTIONS AND POLITICAL PROCESSES Summary The performance evaluation is a mid-term evaluation of ITEP project that must be carried out in summer 2012. The evaluation must determine how effective (or otherwise) the project has been to date; how it is perceived by beneficiaries; will examine the extent to which the intended results have occurred and how relevant are the project approaches in attaining the end result – increasing trust in electoral processes. Evaluation must also provide recommendations on how to improve the project and whether course corrections are necessary in remaining years of implementation. I. Background Political context Georgia’s political environment today is characterized by an unbalanced party system, with relatively low and decreasing levels of political competition, high and increasing levels of political polarization and distrust, the eclipse of policy discussion by debate over the rules of the game, and the over-personalization of politics and of parties. Because of the distrust in the fairness of the system, political parties have refused to constructively engage in existing political institutions and the resulting increased cynicism in the population has discouraged broader political participation. Polarization between the ruling party, which currently holds a significant advantage, and the numerous opposition groups, has intensified and spilled over into civil society and media organizations as well, and has negatively affected the broader political and public discourse in Georgia. In developing democracies such as Georgia, where the political sphere is highly polarized, the execution of elections as well as the perception of the fairness of election results may strengthen or weaken the state. Smooth turnover of political power through elections is a challenge in many transition democracies like Georgia. Transfer of political power through free and fair elections has been a major challenge in Georgia for the last 17 years. Lack of trust in the electoral system and distrust in decisions made by the Central Election Commission (CEC) resulted in the Rose Revolution in 2004. Distrust by opposition parties led to popular protests after the presidential election that was held in January 2008. Furthermore, dissatisfaction in the electoral process surrounding parliamentary elections in May 2008 motivated opposition parties to abandon newly-won seats in the Parliament and eventually resulted in the street protests of mid-2009. One issue feeding the continued polarization in the country is the fact that the vocal opposition believes the electoral system in Georgia is fundamentally flawed. They point to abuses of voter lists, uncertainty about fair composition of electoral administration, unbalanced districting, abuse of administrative resources and even the 34 fundamental design of Georgia’s electoral system as contributing to their belief that the cards are stacked in favor of the current ruling majority. The fact that one party dominates both the national legislature and all municipal governments reinforces the general perception that the system is not fair and that it is difficult, if not impossible, for alternative voices to be elected and therefore heard in government. Recent actions of the state agencies against new opposition leader Bidzina Ivanishvili reinforced this perception. Several laws were adopted to target newly emerged leader and his business companies. The newly created political finance monitoring department at Chamber of Control (CCG) of Georgia scrutinized all expenses of a new coalition “Georgian Dream” established by Bidzina Ivanishvili and as a result he and companies associated with him were fined by seven million GEL during three month period. In October 2010 Parliament adopted amendments to the constitution, which will significantly reduce powers of next President in favor of Prime Minister and the government. The new constitution will go into force upon the inauguration of the next president, who is scheduled to be elected in October 2013. As soon as the new constitution goes into force in 2013, the government existing at the time will resign to give way to the Parliament, elected in 2012, to compose the new government. That is why upcoming 2012 and 2013 elections are crucial for Georgia’s future democratic development. Increased Trust in Electoral Reforms (ITEP) ITEP has started on July 30, 2010 with the aim of raising the level of confidence among the public and political participants in the electoral process over the next three election cycles (2012 parliamentary election, 2013 presidential and 2014 local elections). This project involves close collaboration with the CEC on matters of capacity and transparency, technical support to Georgian NGOs and media outlets, work with political parties on election law reform, engagement with local stakeholders to help improve Georgia’s voter registry, implementation of a curriculum used to educate university students on civic responsibility, and efforts to improve voter access for the disenfranchised. ITEP’s objective is to address the enduring negative effects related to the electoral system on the country’s enabling environment by focusing on election law reform; building the capacity of the election administration as well as of those bodies responsible for implementing Georgia’s new political finance legislation and verifying the country’s voter registry; raising the country’s civic identity; building the capacity of NGOs to provide oversight, encourage accountability and increase public awareness about election-related issues and processes; and providing pre￾election training and monitoring as needed to address outstanding issues. The project has four objectives, achieving of which would lead to the end results of increasing trust in electoral processes:  Objective 1: Improve Capacity of Election Administration  Objective 2: Improved Transparency of Electoral Processes through Improved Media Coverage and CSO Involvement  Objective 3: Protection of Voters’ Rights  Objective 4: Increased Understanding about Election-Related Issues and Civic Responsibility The evaluation must only look at the objectives 1 and 2. Results achieved to date by ITEP: 35  The CEC drafted five-year strategic plan that covers the upcoming cycle of elections. In doing so, the CEC solicited public input and began developing an implementation plan for each of the strategic plan’s three objectives.  The ITEP-facilitated Election Support Working Group (ECWG) has provided an opportunity for regular discussion and debate about how to improve the elections complaints and appeals process. The group developed a draft Election Dispute Resolution (EDR) handbook for use at the precinct election commission (PEC) level.  The CEC and four NGOs established a working group on EDR which provides an opportunity for regular discussion and debate about how to improve the complaints and appeals process.  ITEP organized two conferences on cooperation with OSCE/ODHIR and CEC for over 100 representatives of political parties, civil society organizations, the media, and the international community, increased their knowledge of issues related to campaign and political finance and women’s participation in elections and political life.  ITEP supported 10 ECWG meetings over the course of 2010 – 2011, where the group discussed reform of legislation governing topics including voter registration, election administration, political party financing, and dispute resolution.  ITEP provided expert technical assistance to the ECWG on the issues of the use of biometric identification technologies for elections, electoral systems, and political finance. The process was observed by election administration experts, representatives of civil society and international organizations, embassies, and media.  In the fall 2011 the commentary in Georgian and English languages on the draft Election Code was distributes to Georgian and foreign government officials, international organizations, media outlets, and political parties, and was discussed publically. Changes recommended by NGOs such as Georgian Young Lawyers Association (GYLA), Transparency international (TI) and International Society for Fear Elections and Democracy (ISFED) supported by ITEP, were included in the December 8, 2011 revision of the Election Code which was adopted later in December 2011. These include: dropping the controversial proposal to increase the number of MPs and allowing photo and video equipment to be used at polling sites. The changes also addressed other legislative shortcomings related to political finance and the rights of election observers and media representatives on the elections day.  ITEP provided technical support to newly established voter’s list commission and to the Chamber of Control of Georgia (CCG). II. Purpose of the Evaluation and Its Intended use The purpose of this mid-term performance evaluation is to: a) determine how effective (or otherwise) the project has been in achieving its intended results; b) examine if the project design is appropriate to improve electoral environment in Georgia; and c) to provide recommendations on how to improve the project and whether course corrections are necessary in remaining years of implementation. The timeframe to be covered by the evaluation is from the start of the project in July 2010 through the initiation of this evaluation on/about June 2012. The results of the evaluation must be used by USAID/Georgia for improving ongoing interventions in the area of elections in particular in light of the upcoming 2012 and 2013 elections, in order to focus on the activities that are most meaningful and critical for improving the environment for political competition in Georgia. The audience of the evaluation must be the USAID and in particular its Democracy and Governance (DG) 36 office, as well as other USG agencies involved in elections area. In addition, the results of the evaluation must be useful for USAID’s current implementing partner to improve its interventions. With similar purpose, the results of the study must be shared with other stakeholders locally – CEC, CCG, other donors working in this area, and interested NGOs. Finally, evaluation results must also be used for reporting purposes to Washington-based stakeholders. It will be useful in developing the Performance Plan and Report for FY 2012 (sometime in the fall). III. Evaluation Questions The evaluation must be based on the series of evaluation questions to elicit the results of ITEP project to date. The evaluation report must provide evidence-based answers to the following questions: 1. To what extent the intended results of the ITEP project are being achieved in relation to the objectives 1 and 2 of the project (page 2): a. To what extend did the project strengthen capacity of election administration (for 2012 elections)? b. Did the project improve transparency of electoral processes? 2. Is the project design appropriate to increase trust towards electoral processes? 3. What has been the commitment of target organizations (CCG, CEC, DECs and Voter’s list commission) to sustain the results of the project? a. Did the participants obtain needed skills and knowledge as a result of this project? b. Were the skills and knowledge considered useful? c. Were the skills and knowledge used in the behaviors of individuals and/or organizations? 4. What are the most and the least successful interventions as perceived by main stakeholders (CCG, Voter’s list Commission, CEC, NGOs)? IV. Suggested Evaluation methods Democracy International, Incorporated must suggest the use of various data collection and analysis methods, both quantitative and qualitative, including document review, key informant interviews, focus group discussions, survey instruments, and others. For survey or mini survey (if proposed) number of respondents and their selection process should be explained and justified. The same is true for key informants as well. Selected respondents must be representative of women, youth and vulnerable groups, where appropriate. Democracy International, Inc. must develop a detailed evaluation design (to be included in the proposal), including data collection plan and data collection tools. This plan must be presented to the Mission during the in-brief in more details and adjusted later based on the Mission’s comments. The evaluation design must include the evaluation matrix (an illustrative evaluation matrix for this study is given below). The contractor must also explain in details limitations and weaknesses of the methodology. 37 DII must also describe a data analysis plan that details the analysis of information collected; what procedures will be used to analyze qualitative data collected through key informant and other stakeholder interviews; and how the evaluation will analyze and use quantitative data. Tentative Evaluation Matrix: Research Question Data Source Methodology 1. To what extent are the intended results of the ITEP project are being achieved in relation to the objectives 1 and 2 of the project? a. To what extent did the project strengthen capacity of election administration (for the 2012 elections)? Central Election Commission (CEC), District Election Commission (DECs), the Center of Electoral Systems Development, Reforms and Trainings, the Voters’ List Verification Commission (VLVC), the Civil Registry Agency (CRA) of Georgia, the Election Support Working Group (ECWG), other Government of Georgia (GoG) agencies Project documentation Document review to clarify intended results for capacity as defined in project documentation and compare results to the criteria Informant interviews to gauge officials’ capacity to effectively manage the electoral process, knowledge of election administration, perception of the ITEP program, and ability to conduct elections with transparency, impartiality, integrity, and professionalism, among other topics Focus groups with election administration officials b. Did the project improve transparency of electoral processes? Civil society organizations, including the International Society for Fair Elections and Democracy (ISFED), the Georgian Young Lawyers’ Association (GYLA), Transparency International Georgia (TI-G), New Generation New Initiative (NGNI), the Public Movement “Multinational Georgia” Media CSOs that receive assistance from ITEP, members of the newly formed special media working group Document review to clarify intended results for transparency as defined in project documentation and compare results to the criteria Informant interviews to assess beneficiaries’ perception of the ITEP program and determine the extent to which the ITEP program has improved the participation of civil society and media in the electoral process, including monitoring equality of opportunities among election contestants, participating in 38 Project documentation changes to the Electoral Code, and reporting on election-relation issues Focus groups with civil society organizations and media representatives to understand the degree to which media and civil society organizations are improving transparency in the electoral process 2. Is the project design appropriate to increase trust towards electoral processes? Project documentation, including results framework, monitoring and evaluation plan, development hypothesis, etc. Program staff members Election administration USG agencies Document review to look at the appropriateness of the design. This will include presenting the best practices on increasing trust in electoral processes and examining whether and how such best practices were adapted to use in the Georgian context in this project Informant interviews Focus groups with election officials and CSOs 3. What has been the commitment of target organizations (CCG, CEC, DECs, and Voter’s list commission) to sustain the result of the project? a. Did the participants obtain needed skills and knowledge as a result of this project? The CCG, the CEC, DECs, the Center of Electoral Systems Development, Reforms and Trainings, the VLVC, the CRA, the ECWG, ISFED, GYLA, TI-G, NGNI, the Public Movement “Multinational Georgia,” other stakeholders as appropriate Project documentation Document review Informant interviews Focus groups with election officials and CSOs Minisurveys b. Were the skills and knowledge considered useful? The CCG, the CEC, DECs, the Center of Electoral Systems Development, Reforms and Trainings, the VLVC, the CRA, the ECWG, ISFED, GYLA, TI-G, NGNI, the Public Movement “Multinational Georgia,” other stakeholders as appropriate Informant interviews Focus groups with election officials and CSOs Minisurveys 39 c. Were the skills and knowledge used in the behaviors of individuals and/or organizations? The CCG, the CEC, DECs, the Center of Electoral Systems Development, Reforms and Trainings, the VLVC, the CRA, the ECWG, ISFED, GYLA, TI-G, NGNI, the Public Movement “Multinational Georgia,” other stakeholders as appropriate Project documentation Informant interviews Focus groups with election officials and CSOs Document review, including implementer monitoring reports and any other documentary evidence 4. What are the most and the least successful interventions as perceived by main stakeholders (CCG, Voter’s list commission, CEC, NGOs)? The CCG, the CEC, DECs, the Center of Electoral Systems Development, Reforms and Trainings, the VLVC, the CRA, the ECWG, ISFED, GYLA, TI-G, NGNI, the Public Movement “Multinational Georgia,” other stakeholders as appropriate Informant interviews Focus groups with election officials and CSOs V. Deliverables and timeline The deliverables associated with this contract must be completed and accepted by USAID/Caucasus by September 3, 2012. The team must provide both an in-brief upon arrival and an out brief prior to departure. Estimated timeframe Documents review: July 1-5, 2012 In country work: July 6 – July 27, 2012 Draft evaluation design and a work plan submitted to USAID: July 6, 2012 In brief at USAID: July 9, 2012 Out brief at USAID and outline of the final report: July 25 or 26, 2012 Draft final reports due to USAID: August 13, 2012 USAID Comments due to the contractor: August 17, 2012 Final report: August 29, 2012 The last date of finalizing report: September 10, 2012 If the DII arrives earlier USAID will shift deliverables accordingly. The deliverables for this evaluation are: 1. Detailed evaluation design and the work plan: The contractor must include in the proposal the proposed research design and what methods they will use to get answers for each evaluation question. The evaluation design must include a detailed evaluation matrix (including the key questions, methods and data sources used to address each question and the data analysis plan for each question), draft questionnaires and other data collection instruments or their main features, known limitations to the evaluation design, and a dissemination plan. The evaluation design must also include specific sub￾questions for 40 each evaluation questions. This information must be discussed in detail during the in￾brief meeting with USAID and must be finalized within five (5) days of receiving USAID’s comments. The final design requires COR approval. The work plan must include the anticipated schedule and logistical arrangements and delineate the roles and responsibilities of members of the evaluation team. 2. In brief with the mission – within three (3) days of arrival in country, the contractor must present draft design plan and a work plan. 3. Out brief – one (1) day prior to departure, the contractor must present an outline (in bullets, possibly in power point or as a handout) of the evaluation report with general findings, conclusions, and anticipated recommendations. 4. Outline of the evaluation report (to be presented at the out brief) including findings, conclusions and recommendations. 5. Draft Report – The contractor must submit a draft report within seven (7) working days of completing the out brief with USAID. This document must explicitly respond to the requirements of the SOW, should answer the evaluation questions, be logically structured, and adhere to the standards of the USAID Evaluation Policy of January 2011 and the criteria to ensure the quality of the evaluation report. The report must not exceed 25 pages, excluding executive summary and annexes. 6. Final Report – The contractor must incorporate USAID’s comments and submit the final report to USAID/Georgia within five (5) working days following receipt of comments on the draft report. Final evaluation report should include an executive summary, introduction, background of the local context and the projects being evaluated, major findings, conclusions and recommendations. The report must not exceed 25 pages, excluding executive summary and annexes. The contractor must make the final evaluation reports publicly available through the Development Experience Clearinghouse at http://dec.usaid.gov within 30 calendar days of final approval of the formatted report with USAID consent. Recommendation section of the evaluation reports must be omitted from the public version due to Procurement sensitivity. 7. All records from the evaluation (e.g. interview transcripts and summaries, etc.) must be provided to the evaluation COR. All qualitative data collected by the evaluation team must be provided in an electronic file in easily readable format agreed upon with the COR. The data must be organized and fully documented for use by those not fully familiar with the project or the evaluation. USAID must retain ownership of the survey and all datasets developed. Reporting Guidelines The format for the evaluation report is as follows: 1. Executive Summary—concisely state the purpose, background of the project, main evaluation questions, methods, findings, conclusions, recommendations and any lessons learned; must be sufficiently detailed, yet brief, to serve a stand-alone product (3-5 pp) 2. Introduction—state the purpose, audience, and outline of the evaluation (1 pp) 3. Background—provide a brief overview of the project and the study implemented (1-2 pp) 4. Methodology— the evaluation methodology must be explained in the report in detail. 41 Limitations to the evaluation must be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology. Greater detail can be included in the appendices (2-3 pp); 5. Findings/Conclusions/Recommendations—explicitly answer each evaluation question; the report must distinguish between findings (the facts), conclusions (interpretation of the facts), and recommendations (judgments related to possible future programming) (10-15 pp); however it should be clear what is the link between them; 6. Issues—provide a list of key technical and/or administrative issues, if any; may include Disclosure of Conflict of Interest; and Statement of Differences: when applicable, evaluation reports must include statements regarding any significant unresolved differences of opinion on the part of funders, implementers and/or members of the evaluation team; 7. Lessons Learned (if not covered in findings, conclusions and recommendations) (2–3 pp); 8. Annexes—annexes must include this statement of work and its modifications (if any), a glossary of terms, and a clear documentation of evaluation methods, schedules, bibliographical documentation, meetings, interviews and focus group discussions, and any focus group scripts or questionnaires, checklists and discussion guides used (the presentation should be succinct, pertinent and readable). The evaluation design must also be attached to the report. The report format must be presented in Microsoft Word and use 12-point type font throughout the body of the report, using page margins” top/bottom and left/right. The body of the report must ideally be within 20-25 pages, excluding the executive summary, table of contents, references and annexes. The final report must follow USAID branding and marking requirements. Per the USAID evaluation policy, draft and final evaluation reports must be evaluated against the following criteria to ensure the quality of the evaluation report.1  The evaluation report must represent a thoughtful, well-researched and well organized effort to objectively evaluate what worked in the projects, what did not and why.  Evaluation reports must address all evaluation questions included in the statement of work.  The evaluation report must include the statement of work as an annex.  Evaluation methodology must be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report.  Evaluation findings must assess outcomes and impact on males and females.  Limitations to the evaluation must be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.).  Evaluation findings must be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence.  Sources of information shall be properly identified and listed in an annex.  Recommendations must be supported by a specific set of findings.  Recommendations must be action-oriented, practical and specific, with defined responsibility for the action. 42 VI. Logistics USAID Mission will not be responsible for arranging logistics for the evaluation team. Democracy International, Inc. must suggest how they plan to arrange translation, transportation and logistical support to the evaluation team. The COR, through the Mission’s Democracy and Governance (DG) office must put the contractor in contact with its implementing partner and might provide help with a small number of meetings (such as meeting with USG agencies). To the extent possible, relevant reports and other project documentation must be provided by the Mission to the contractor prior to travel to Georgia. These documents are:  Project Descriptions as is stated in the award;  Implementing partners Quarterly Reports;  Initial list of in-country contacts;  PMP indicator tables;  M&E plans submitted and approved by USAID;  Implemented monitoring reports;  Other deliverables (expert report, publications) produced by partner. Prior to arriving to Georgia, the contractor must decide to interview USAID/E&E and USAID/DCHA officials in the United States on programing in political processes generally, and Georgia specifically, in addition to any other Washington-based experts as appropriate. The Mission must not be involved in arranging these meetings. While in Georgia, the contractor must conduct meetings both in Tbilisi and in regions outside Tbilisi, in cities agreed with COR. The Mission’s DG Team must brief the evaluation team on their perceptions of political dynamics. VII. Other Requirements The evaluation team must be familiar with USAID’s Human Subject Protection Policy and USAID’s Evaluation Policy (http://www.usaid.gov/evaluation). The evaluation team must provide adequate training for its survey staff on survey methodology, USAID’s survey regulations, other relevant regulations, and data collection plan. The contractor has the responsibility to safeguard the rights and welfare of human subjects involved in the survey research supported by USAID. USAID has adopted the Common Federal Policy for the Protection of Human Subjects, Part 225 of Title 22 of the Code of Federal Regulations (http://www.usaid.govb/policy/ads/200/200mbe.pdf). Recipient organizations must familiarize themselves with the USAID policy and provide “assurance” that they will follow and abide by the procedures of the Policy. All records from the evaluation (e.g., interview transcripts or summaries) must be provided to the COR. All quantitative data collected by the evaluation team must be provided in an electronic file in easily readable format agreed upon with the COR. The data must be organized and fully documented for use by those not fully familiar with the project or the evaluation. USAID must retain ownership of the survey and all datasets developed. All modifications to the SOW, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by the COR. 43 ANNEX B: ITEP EVALUATION WORKPLAN Work Plan for USAID/CAUCACUS Mid-Term Evaluation of the Increased Trust in Electoral Reforms Project (ITEP) Submitted by Charles Costello and Harold Lippman Democracy International July 11, 2012 Approach/Methodology Democracy International’s evaluation team is comprised of: Charles Costello (team leader); Harold Lippman (evaluation expert); David Jijelava (local expert); and, Mikheil Svanidze (logistics coordinator). As set forth in the scope of work, the team will: a) determine how effective the project has been in achieving its intended results; b) examine if the project design is appropriate to improve the electoral environment in Georgia; and c) provide recommendations on how to improve the project and whether course corrections are necessary in the remaining years of implementation. The assessment will review the efforts undertaken and results achieved by IFES – ITEP’s implementer – between July 30, 2010 and the present in two of the project’s four primary objectives: (1) improve capacity of election administration; and, (2) improved transparency of electoral processes through improved media coverage and CSO involvement. To accomplish these ends, the team will primarily rely on Rapid Appraisal, an approach that utilizes several evaluation methods to quickly and systematically collect data, including: document review, key informant interviews, group discussions, and mini-surveys. Fieldwork and data collection will be conducted in Tbilisi and, over a two-day period, in Kutaisi, Akhalkalaki, and Marneuli during the second week of the team’s time in-country. In connection with the latter, the team will split into two, two￾person sub-teams with one responsible for work in Kutaisi and the other for work in Akhalkalaki and Marneuli. Interview guides will be prepared to help team members conduct interviews and group discussions in a systematic and uniform manner. The guides will include interview topics that address the four key evaluation questions as well as general topics regarding trust in the electoral process more broadly. The team will tailor interviews to informants depending on their professional affiliation, background, expertise, and other factors. (See Annex C Questions List) The evaluation team will arrange group discussions of 90 minutes duration, comprised of between eight and 12 participants, in Kutaisi, Marneuli and Akhalkalaki. The groups will consist of a combination of local CSOs following interviews with DEC representatives who have participated in an ITEP-sponsored training activity. During the group discussions, the Team Leader or Evaluation Expert will act as moderator and introduce topics for discussion based on the interview guides mentioned above. The team will also conduct a survey (instrument provided separately to USAID Mission) with 20 randomly selected DEC beneficiaries of ITEP training (BRIDGE) that will feature a limited number of close-ended, narrowly-focused questions to generate quantitative data that can be collected and analyzed quickly. Tasks 44 The team will interview and/or consult with representatives of key USAID/Georgia organizational units including the Program Office and the Democracy and Governance Office. It will also meet with a representative of the U.S. Embassy (POL/ECON) and the Chief of Party and other key staff of IFES. The team will interview key GoG individuals and organizations targeted under ITEP, including the Central Election Commission (CEC), eight District Election Commissions (DECs) in Tbilisi, Kutaisi, Akhalkalaki and Marneuli, Voters’ List Verification Commission (VLVC), Civil Registry Agency (CRA) of Georgia, Inter-Agency Committee [Task Force] on Free and Fair Elections (ITAF), and certain members of the Election Code Working Group (ECWG), Interviews will also be arranged with officials representing the majority party and opposition parties in (and/or out of) Parliament, particularly the United National Movement (UNM) and the Georgian Dream coalition. The team’s evaluation efforts will include targeted NGOs and other stakeholder organizations in Tbilisi, Kutaisi, Akhalkalaki, and Marneuli including the International Society for Fair Elections and Democracy (ISFED), Georgian Young Lawyers’ Association (GYLA), Transparency International Georgia (TI-G), New Generation New Initiative (NGNI), and Public Movement “Multinational Georgia.” Representatives of media CSOs have not yet received ITEP assistance through ITEP sub-grants, but four of them will also be included in the team’s visit and interview list in these locations. The team’s efforts (See Annex B for schedule) will be organized around and address the following four evaluation questions posed in the Scope of Work and addressed schematically in the attached Evaluation Matrix (Annex A): 1. To what extent are the intended results of the ITEP project being achieved in relation to objectives #1 and #2: (a) to what extent did the project strengthen the capacity of election administration (for the 2012 elections); (b) did the project improve transparency of electoral processes? 2. Is the project design appropriate to increase trust in electoral processes? 3. What has been the commitment of target organizations, e.g. CEC, DECs ITAF, VLC and CCG to sustain the results of the project: (a) did the participants obtain needed skills and knowledge as a result of this project; (b) were the skills and knowledge considered useful; and (c) were the skills and knowledge used in the behaviors of individuals and/or organizations? N.B. ITAF, VLC and CCG were not direct beneficiaries of ITEP in the same sense as CEC and DECs; they participated in ITEP activities rather than receiving training or direct technical assistance. 4. What are the most and the least successful interventions as perceived by main stakeholders ( CEC, CSOs, VLC, ITAF, CCG)? Data Analysis The evaluation team’s data analysis will be grounded in a comparison and analysis of ITEP program documents, including annual work plans, quarterly reports, PMP indicator tables, M&E plans, monitoring reports, and other deliverables (e.g., expert reports, publications). These source documents define and describe the causal linkages among program inputs, activities, outputs, and outcomes, as well as anticipated results and underlying assumptions. They also define target populations, identify specific objectives, and establish certain measurable indicators. The team will also compare IFES documents and stakeholder interview information with data related to project performance and outcomes obtained from other source documents, individual and group interviews with stakeholders and beneficiaries, and the data obtained in the mini-survey. By using information obtained from the survey instrument, interview guides and common questions, the team will systematically compare the answers and comments of different beneficiaries and stakeholders to 45 facilitate inferences on program performance and effect. To identify important trends and uncover latent relationships, the evaluation team will disaggregate data as necessary. Outputs and outcomes will be considered relative to the project’s implementation schedule, available resources, and the work of other donors. Methodological Limitations and Weaknesses The single most important limitation to the team’s approach is that the evaluation will take place before the October parliamentary elections; therefore, the timing affords no way to assess the effect of program activities on the conduct of an actual election. Moreover, increasing public confidence (trust) in the electoral process, as well as improving the capacity of election administration and bolstering transparency of electoral processes, are typically long-term efforts, the fruits of which are not fully discernible at the project’s mid-point, which covers only a two-year period. The team will use evidence available from public opinion research from other sources, e.g. NDI, on such questions as trust, although horizontal timelines are known to be short. The team has neither time nor funding to conduct scientific public opinion survey research on its own. Available time and resources are also major limitations. The evaluation’s accelerated timeframe to arrive in-country, lack of timely receipt of essential source documents prior to departure, and the team’s resulting inability to perform its full pre-departure desk review research, led to more constrained advance methodological planning It also meant the team was not able to arrange as many key informant interviews and group interview in advance as otherwise would be desirable. Lastly, as the Rapid Appraisal techniques utilized in this evaluation rely on non-representative samples, with the exception of the DEC mini-survey, as well as the fact that no baseline data was collected at the start of the project, the evaluation team will not be able to generalize results across the entire population or use “before and after” comparisons. Nevertheless, by using a mixed-methods approach, the evaluation team can compare data collected using one method (e.g., document review) to data collected using other methods (e.g., key informant interviews and a mini-survey). Triangulation minimizes bias and strengthens the validity of evaluation findings by acting as a check on the findings from any one method. Deliverables Briefings The in-brief at USAID was July 10, 2012. The out-brief and submission of report outline is July 26 or 27, 2012. Report The team will deliver a draft final report by August 13, 2012. The document will follow the format suggested in the scope of work: 1. Executive Summary 2. Introduction 3. Background 4. Methodology 5. Findings/Conclusions/Recommendations 6. Issues 46 7. Lessons Learned (if not covered in #5) 8. Annexes Mission comments to Democracy International are due August 17, 2012. The final draft report (including annexes) will be submitted by Democracy International by August 29, 2012. The last date for finalizing the report will be September 10, 2012. Annex A: Evaluation Matrix Research Question Data Source Methodology 1. To what extent are the intended results of the ITEP project being achieved in relation to objectives 1 and 2 of the project? a. To what extent did the project strengthen capacity of election administration (for the 2012 elections)? Central Election Commission (CEC), District Election Commission (DECs), the Center of Electoral Systems Development, Reforms and Trainings, the Voters’ List Verification Commission (VLVC), the Civil Registry Agency (CRA) of Georgia, the Election Code Working Group (ECWG), the Inter-Agency Task Force on Free and Fair Elections (IATF), and other Government of Georgia (GoG) agencies Project documentation Document review to clarify intended results for capacity as defined in project documentation and compare results to the criteria Informant interviews to gauge officials’ capacity to effectively manage the electoral process, knowledge of election administration, perception of the ITEP program, and ability to conduct elections with transparency, impartiality, integrity, and professionalism, among other topics Group meetings with election administration officials DEC mini-survey b. Did the project improve the transparency of electoral processes? Civil society organizations, including the International Society for Fair Elections and Democracy (ISFED), the Georgian Young Lawyers’ Association (GYLA), Transparency International Georgia (TI-G), New Generation New Initiative (NGNI), the Public Movement “Multinational Georgia” Media CSOs that receive assistance from ITEP, members of the newly formed special media working group Project documentation Document review to clarify intended results for transparency as defined in project documentation and compare results to the criteria Informant interviews to assess beneficiaries’ perception of the ITEP program and determine the extent to which the ITEP program has improved the participation of civil society and media in the electoral process, including monitoring equality of opportunities among election contestants, participating in changes to the Electoral Code, and reporting on election-relation 47 issues Group meetings with civil society organizations and meetings with media representatives to understand the degree to which media and civil society organizations are improving transparency in the electoral process 2. Is the project design appropriate to increase trust towards electoral processes? Project documentation, including results framework, monitoring and evaluation plan, development hypothesis, etc. Program staff members Election administration USG agencies Project beneficiaries, stakeholders and other participants in project activities Document review to look at the appropriateness of the design. This will include discussion of good practices on increasing trust in electoral processes and examining whether and how such practices were adapted to use in the Georgian context in this project Informant interviews Group interviews with election officials and CSOs 3. What has been the commitment of target organizations (CCG, CEC, DECs, and Voter’s List Commission) to sustain the results of the project? a. Did the participants obtain needed skills and knowledge as a result of this project? The CCG, the CEC, DECs,, the VLC, the CRA, the ECWG, ISFED, GYLA, TI-G, NGNI, the Public Movement “Multinational Georgia,” other stakeholders as appropriate N.B. Many of the above participants are not actual beneficiaries of ITEP. Project documentation Document review Informant interviews Group meetings with election officials and CSOs Mini-survey b. Were the skills and knowledge considered useful? The CCG, the CEC, DECs, the VLC, the CRA, the ECWG, ISFED, GYLA, TI-G, NGNI, the Public Movement “Multinational Georgia,” other stakeholders as appropriate N.B. Many of the above participants are not actual beneficiaries of ITEP Informant interviews Group meetings/discussions with election officials and CSOs Mini-survey c. Were the skills and knowledge used in the The CCG, the CEC, DECs, the VLC, the CRA, the ECWG, Informant interviews 48 behaviors of individuals and/or organizations? ISFED, GYLA, TI-G, NGNI, the Public Movement “Multinational Georgia,” other stakeholders as appropriate N.B. Many of the above participants are not actual beneficiaries of ITEP Project documentation Group meetings/discussions with election officials and CSOs Document review, including implementer monitoring reports and any other documentary evidence 4. What are the most and the least successful interventions as perceived by main stakeholders (CCG, Voter’s list commission, CEC, NGOs)? The CCG, the CEC, DECs,, the VLC, the CRA, the ECWG, ISFED, GYLA, TI-G, NGNI, the Public Movement “Multinational Georgia,” other stakeholders as appropriate Informant interviews Group meetings with election officials and CSOs 49 ANNEX C: INTERVIEW QUESTIONS LIST Questions for ITEP Beneficiary Organization Interviews (all questions re. Project timeframe 7/30/10 – present) Background Job title Functions/responsibilities Length of time in position Project Activities 1. What Project-sponsored activities was the respondent involved in or knows about? Training – courses/classes, worksite observation Study/observation trips Other 2. What knowledge, skills, and/or abilities did the respondent gain or knows was gained from these activities? 3. In connection with anticipated outcomes of project objective #1 (improved capacity of election administration) and #2 (improved transparency of electoral processes through improved media coverage and CSO involvement), how has the respondent used or knows about the use of the results of training or other project activities? For example, is the respondent able to do something relevant to project objectives he/she was not able to do before? 4. Reflecting on his or her ITEP experience is there something he or she thinks can be done to improve election administration and transparency of electoral processes before the October 2012 election? 5. Is there something in his or her that could be done to improve the project? 50 ANNEX D: LIST OF INTERVIEWEES Nermin Nisic, Keti Maisuradze, IFES, CoP/Project Manager Fron Nahzi, EWMI G-PAC, CoP Khatuna Khvichia, Lela Kerashvili, USAID Amy Diaz, US Embassy POL/ECON Andrea Keerbs, IRI CoP Ian Woodward, Maka Meshveliani, NDI Eka Siradze, ISFED Nino Lomjaria, ISFED Nini Dolidze, ISFED Nina Khatiskatsi, TI-G Zviad Devdariani, CIDA Zurab Kharatishvili, CEC Chair Arnold Stepanian, PMMG Vladimer Tsutskiridze, DEC Mtatsminda Tea Tsulukiani, Georgian Dream Iakob Asanidze, Aleksandre Giorgadze, Elguja Chkhitunidze, Tsitsino Khmaladze, Marneuli DEC Akhalkalaki UNM, Georgian Dream Representatives Lela Taliuri, GYLA, Project Coordinator Ketevan Chachava, nGnI, executive director Mamuka Katsitadze, VLC chair Tamar Kintsurashvili, IATF Tinatin Bokuchava, SAO (CCG) Giorgi Chkheidze, EWMI-JILEP CoP Luis Navarro, NDI CoP Teona Gamtsemlidze, UNDP Lasha Shurghaia, Vasil Chiabrishvili, Davit Petviashvili, Gvantsa Gugava, Krtsanisi DEC Akaki Minashvili, UNM MP Marneuli Roundtable discussion with NGOs: Mariam Samkharadze, USAID Democratic Involvement Center Zaur Dargalli, USAID Democratic Involvement Center Sabir Mekhtiev, Sadakhlo Revival Union Sabina Talibova, Democratic Union “Dove” Olga Endeladze, Marneuli Democrat Women’s Union Leila Mamedova, Union of Georgia’s Azerbaijani Youth Asad Ali-Yeve, Grand Union Anar Gasanov, Grand Union Tamar Mjavanadze, PMMG-Marneuli Elmira Yusubova, Youth Organization Union Shalva Shubladze, Marneuli TV director Hasmik Marangozian, Paata Panjakidze, Zuriko Melikidze, Rafik Karapetian, Akhalkalaki DEC Akhalkalaki Roundtable Discussion With NGOs: Harutuyn Aivazian, Civic Forum Javakheti Seda Melkumyan, ECMI consultant, Civic Education for Lawful State Ketino Khutsishvili, ISFED Ninotsminda Makhare Matsukatov, Akhalkalaki Business Centre 51 Vagarshak Shakhbekian, Correspondent of OSCE Dali Agdgomeladze, Women for the Future of Javakheti, GFSIS Representative Shorena Tetvadze, Akhalkalakis Adult Education, Language House in Georgia, UNA. Misha Katoyan, Youth Initiative Centre. Kutaisi Kutaisi DEC members Radio Dzveli Kalaki, Ketevan Berdzenishvili, Head of the Information Department Georgian Young Lawyers' Association - Kutaisi, Nodar Jikia, Programs Director Mega TV, Nani Toshkhua, Director TV Rioni, Tamar Gvinianidze, Director International Society for Fair Elections and Democracy, Teona Gogishvili, Regional Coordinator Newspaper “Imeretis Moambe,” Lela Khachapuridze, Journalist 52 ANNEX E: DOCUMENTS LIST DOCUMENTS Canham, Steven, Georgia Voter List Verification Framework: Developed at the request of Voter List Verification Commission, March 2, 2012 CEPPS/IFES, ITEP Annual Work Plan, Year One CEPPS/IFES, ITEP Annual Work Plan, Year Two CEPPS/IFES, ITEP Monitoring and Evaluation Plan for Year One CEPPS/IFES, ITEP Performance Management Plan, 2010-2014, November 15, 2010 CEPPS/IFES, ITEP Performance Management Plan for Year Two CEPPS/IFES, Attachment B, Program Description CEPPS/IFES, Quarterly Report: July 1, 2010 – September 30, 2010, Georgia: Increased Trust in the Electoral Process CEPPS/IFES Quarterly Report: October 1, 2010 – December 31, 2010, Georgia: Increased Trust in the Electoral Process CEPPS/IFES Quarterly Report: January 1, 2011 – March 31, 2011, Georgia: Increased Trust in the Electoral Process CEPPS/IFES Quarterly Report: April 1, 2011 – June 30, 2011, Georgia: Increased Trust in the Electoral Process CEPPS/IFES Quarterly Report: July 1, 2011 – September 30, 2011, Georgia: Increased Trust in the Electoral Process CEPPS/IFES Quarterly Report: October 1, 2011 – December 31, 2011, Georgia: Increased Trust in the Electoral Process CEPPS/IFES Quarterly Report: January, 1, 2012 – March 31, 2012, Georgia: Increased Trust in the Electoral Process Dahl, Robert, Georgia’s Parliamentary Electoral System: Options for Advancing Voter Equality, IFES, March 2011 European Commission for Democracy Through Law (Venice Commission) Organic Law on Political Unions of Citizens, November 7, 2011 IFES, A Nationwide Survey of Electoral Issues in Georgia, July, 2011 IFES, Brief overview of IFES activities in 2012 for the Council of Europe, January 31, 2012 IFES, ESWG Activities and Follow-up, June 6, 2012 Kashakashvili, Natia, Final Report: District Electoral Commission members’ capacity enhancement BRIDGE Program, December 14, 2011 – March 1, 2012 Kippen, Grant, Recommendations on improving the EDR process in Georgia, IFES, September 30, 2011 LEPL Center for Electoral Systems Development, Reforms and Trainings, Handbook for Media: Election of Parliament of Georgia 2012, Central Election Commission of Georgia LEPL Center for Electoral Systems Development, Reforms and Trainings, Handbook for Observers: Election of Parliament of Georgia 2012, Central Election Commission of Georgia LEPL Center for Electoral Systems Development, Reforms and Trainings, Handbook for PEC Members: Election of Parliament of Georgia 2012, Central Election Commission of Georgia National Democratic Institute (NDI), Statement of the National Democratic Institute Pre-Election Delegation to Georgia, Tbilisi, Georgia, June 29, 2012 Ohman, Magnus, Comments on proposals for political finance reform in Georgia, IFES, August 4, 2011 Ohman, Magnus, Recommendations for reform of political finance sanctions in Georgia, IFES, August 4, 2011 Ohman, Magnus, Recommendations to the Chamber of Control of Georgia Regarding its Work in Overseeing Political Finance, IFES, April 18, 2012 53 Public Movement ‘Multinational Georgia,’ Elections to the Parliament of Georgia, 2008 Saakashvili, Mikheil “Amendments to the Organic Law of Georgia on Political Unions of Citizens, May 8, 2012 USAID/Central Election Commission of Georgia, Memorandum of Understanding Regarding the Increased Trust in the Electoral Process Project, April 26, 2012 USAID/Georgia, List of Indicators, 2.3.2 Elections and Political Processes, November 8, 2011 USAID, Evaluation: Learning from Experience, USAID Evaluation Policy, January 2011 U.S. Helsinki Commission Briefing, Elections in the Caucasus: Armenia, Azerbaijan and Georgia, May 23, 2012 WEBSITES Central Election Commission of Georgia, Bulletin of the Central Election Commission of Georgia, July 1 – August 15, 2012 at www.cesko.ge Daily Reports on Georgia, at www.civil.ge National Security Council of Georgia, Bulletin of the Inter-Agency Task Force on Free and Fair Elections, July 1 – August 15, 2012 at www.nsc.gov.ge 54 ANNEX F: MINI-SURVEYS QUESTIONNAIRE FOR DISTRICT ELECTION COMMISSION (DEC) MEMBERS (Email survey) Name of the respondent Position at DEC Number of years working at DEC Date of filling the questionnaire 1. When and for how many days did you receive IFES BRIDGE training? Month: |___|___| Year: |___|___|___|___| Number of days: |___|___| 2. On a scale of 1 to 5, where “1” means “not useful at all” and “5” means “very useful,” please circle the number that corresponds with how useful the IFES BRIDGE training was in helping you acquire information relevant to the performance of your DEC responsibilities? Not useful at all Not useful Average Useful Very useful Don’t know 1 2 3 4 5 99 3. Did the training provide you a particular tool/skill/piece of information you did not have before? [Circle appropriate number] 1 Yes 2 No 99 Don’t know If yes, in a few words, please explain what the tool/skill/piece of information was? 55 4. Have you used the tool/skill/piece of information in performing your DEC responsibilities? [Circle appropriate number] 1 Yes 2 No 99 Don’t know If yes, please explain briefly 5. Has the BRIDGE training improved your DEC’s ability to perform its election administration functions? [Circle appropriate number] 1 Yes 2 No 99 Don’t know If yes, please explain briefly 6. Has the IFES BRIDGE training improved the transparency of your DECs electoral processes? [Circle appropriate number] 1 Yes 2 No 99 Don’t know If yes, please explain briefly 56 SAMPLED DECS FOR EMAIL QUESTIONNAIRE # # of DEC 1 11 2 17 3 26 4 28 5 29 6 35 7 39 8 41 9 43 10 44 11 46 12 49 13 50 14 51 15 52 16 58 17 76 18 79 19 80 20 81 57 ROUNDTABLE ON POLITICAL PARTY FUNDING AND WOMEN’S PARTICIPATION IN POLITICAL LIFE On June 29 and 30, IFES co-sponsored a two-day roundtable on political party funding and women’s participation in political life with the Organization for Security and Cooperation in Europe/Office of Democratic Institutions and Human Rights (OSCE/ODIHR), the Central Election Commission, and Netherlands Institute for Multiparty Diplomacy (NIMD). Sixty-five (32 women) participants attended the first day’s roundtable on political finance and 69 (57 women) participants attended the subsequent roundtable on women’s participation in political life. Participants in attendance included representatives of political parties, local civil society organizations, the media and the international community. The discussions on June 29 focused on the provision of state subsidies for political parties as well as the establishment of independent and effective supervision mechanisms. IFES consultant Joe Stoltz, Director of Audits at the Federal Election Commission (FEC), delivered two presentations at these sessions which focused on party financing, the groundwork needed for compliance, and enforcement mechanisms. The discussions on June 30 focused on women’s participation in elections and included sessions on international standards for women’s participation and on voluntary measures for political parties to increase women’s participation. The roundtable participants offered recommendations on the key aspects of political party financing and women’s participation in political life to ensure fair, level and competitive playing field for the functioning of political parties. Intro e-mail explains event Same overall first question 1-5 scale Did the roundtable discussion provide you with knowledge or information you did not have before? [Circle appropriate number] 1 Yes 2 No 99 Don’t know If yes, in a few words, please explain what the knowledge or information was? Have you used this knowledge or information in the performance of your professional responsibilities and/or personal life? [Circle appropriate number] 1 Yes 2 No 99 Don’t know 58 If yes, in a few words, please explain what the tool/skill/piece of information was? SURVEY ANALYSIS – DEC BRIDGE TRAINING METHODOLOGY IFES has conducted BRIDGE trainings in all 73 electoral districts of Georgia. The evaluation teams had in-depth interviews with DECs in Tbilisi, Kutaisi, Marneuli, and Akhalkalaki. Based on the received feedback and program information collected from IFES, the evaluation team drafted the questionnaire and randomly selected 20 DECs for a small survey. Random numbers were drawn using MS Excels “RANDBETWEEN” function. In cases when Excel generated duplicate numbers, we kept one number and substituted the other with the next number. For example if Excel generated two "15s" we would change them into 15 and 16, to ensure that numbers are not duplicated. Following DECs were sampled for the questionnaire: We received answers from 19 out of the 20 selected DECs. Only DEC#79 didn’t provide answers in time. On the map below you can see highlighted DECs which were sampled. Municipality DEC# Sagarejo 11 Telavi 17 Tetritskaro 26 Dusheti 28 Kazbegi 29 Khashuri 35 Aspindza 39 Ninotsminda 41 Oni 43 Ambrolauri 44 Lentekhi 46 Terjola 49 Sachkhere 50 Zestafoni 51 Bagdati 52 Tskaltubo 58 Poti 70 Batumi 79 Keda 80 Kobuleti 81 59 Figure 1: Randomly selected DECs for the survey Note: there are no functioning DECs in South Ossetia and Abkhazia, covered with grey OVERALL TRENDS Most of the perceived effects of the BRIDGE training can be divided into three groups: 1) Those related to improved communication inside the teams (solving disputes, sharing opinions, making team decisions) as well as with external parties (with media, NGOs, political parties and other electoral bodies); 2) Related to management (planning, decision making, responsibility division, election projects/calendars); 3) Theoretical knowledge, understanding of foreign and local electoral systems. Overall, all three aspects received a significant share of attention. However, not all district representatives gave detailed answers. 8 of the 19 (42%) didn’t give relevant answers to at least half of the questions. Two — Kobuleti and Ninotsminda — gave no examples at all, while five (Tetritskaro, Aspindza, Terjola, Zestaponi, Tskaltubo) gave only one or two examples in response to the four questions, which asked for details. GENERAL EFFICIENCY When asked generally about efficiency of the trainings, average score was almost the maximum -4.95. 18 DECs gave the top grade (5 — Very Useful) while one, Lentekhi, gave 4 (Useful). ACQUIRED KNOWLEDGE AND SKILLS The respondents were asked to specify, whether they did receive particular knowledge/skills, which they didn’t have before, or not. All 19 DECs answered positively. Of them: 60 ‐ 63% (12 out of 19) of DECs mentioned information about elections in various countries (10 times — electoral systems, 2 — election administration models, 2 — in more general terms) ‐ 42% (8 out of 19) of DECs identified general working and communication skills as a benefit received from the trainings (6 times — team work; 1 each: conflict resolution, handling emotions, fast decision making, improved self-belief, applying theory in practice) ‐ 42% (8 out of 19) of DECs listed knowledge directly related to the local electoral process (6 — solving electoral disputes; 2 — election administration rules; 2 — administrator’s code of conduct; 1 each: planning election cycle, working with the voters’ list, working with the district budget) Two DECs mentioned “legal framework”, by which either Georgian legal framework or foreign examples could have been meant. Among the less specific answers: 3 mentioned improved qualification or increased knowledge, one said that the trainings helped to “see facts in a different light”; 3 times the process, rather than the results, was assessed (2 — very interesting, 1 — effective teaching methods). KNOWLEDGE AND SKILLS USED IN PRACTICE The question asked, whether the trainees had used the acquired knowledge and skills in day-to-day work. 18 of the 19 answered positively, however, only 10 gave some concrete and relevant details. The only negative answer came from the Zestafoni DEC. Of those who answered positively, but gave no relevant examples: two gave no details at all (Terjola, Poti); one said that you may not be able to use the acquired information on daily basis, although it does “increase your knowledge” (Aspindza); two spoke about future, rather than current practice (Kobuleti — will use in parliamentary elections, Kazbegi — “I am prepared for the parliamentary elections”); one simply restated, that they “did use” the knowledge (Ninotsminda). Twice, values, rather than practices, were mentioned (neutrality, transparency). As for relatively specific answers: 5 district representatives named managerial activities (4 times — planning, 2— distribution of responsibilities, 1 — “management and administration”) and one of them also mentioned “working with documentation”; in 3 cases, improved team-work or opinion-sharing was mentioned. Baghdati representative said that the trainings allowed them to solve all the electoral disputes, which occurred at the 2012 local elections, so that they received no complaints. Tetritskaro representative mentioned improved cooperation with political parties and NGOs. Two DECs only were as specific as to say that they had used the knowledge “while making decisions”. ELECTION ADMINISTRATION CAPABILITIES The question asked, whether a district electoral commission had improved its ability to administer elections, as a result of the Bridge trainings. All 19 answers were positive. However, 7 district representatives failed to give any specific details: 3 of them — Keda, Sachkhere and Terjola — left the field blank, while 4 gave only generic answers (trainings “are useful”, working quality/knowledge/skills did increase). Of the remaining 12, 4 again named improvement of general working skills (team-work, effective decision-making, administrative/managerial skills, planning and “mobility”). Improved internal structure and responsibility division was mentioned by 3 others. 3 said they understood functions and responsibilities of electoral administration better; 2 mentioned ability to create election project/calendar/plan; others — ability to solve election disputes, to encourage voter participation and to improve cooperation between various levels of election administration (1 each). 61 TRANSPARENCY In the last question, district representatives were asked to assess, whether transparency of their commission’s work did increase and how. 17 said it did increase, while one (Aspindza) chose “don’t know” and one more (Terjola) gave the negative answer. Of those who answered positively, 8 failed to give any relevant details: 1 (Zestaponi) left the field blank, 6 only gave a very general answer (“did increase”, “is important”, “is transparent”), and one (Kobuleti) mentioned “other countries’ experience”. The Aspindza representative explained that their commission’s work had been transparent anyway, although some “nuances” certainly were affected by the trainings. The remaining 9 district representatives gave following answers: improved cooperation and relations with media, increased role of media (5 out of 9); improved relations with NGOs (4) and regular consultations with NGOs (1); improved relations with political parties (2) and regular consultations with political parties (1); providing information openly, as specified by law (3), and timely (1). 62 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523